Medical LLM model reasoning method based on knowledge graph and related equipment

Abstract

The invention provides a medical LLM model reasoning method based on a knowledge graph and related equipment. Wherein the knowledge graph comprises a plurality of nodes and edges connected with the nodes, the nodes represent entities, and the edges represent relationships among the entities. The method comprises the following steps: acquiring a target dialogue text input by a user, and searching a sub-graph related to entity mention contained in the target dialogue text from a knowledge graph; the sub-graph comprises a target node representing a target entity and a plurality of neighbor nodes connected with the target node; obtaining attribute values of a plurality of attributes of the target entity from multiple rounds of dialogue texts which are input by the user and contain the target dialogue text; constructing a dialogue graph corresponding to the target dialogue text based on the sub-graph and the attribute values of the plurality of attributes of the target entity; and according to the dialogue graph and the target dialogue text, constructing cue words, inputting the cue words into an LLM model, and performing logical reasoning by the LLM model based on the cue words.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of large model technology, and in particular, to a medical LLM model reasoning method based on knowledge graph and related equipment. Background Art

[0002] Large Language Models (LLM), or pre-trained models, are language models built with deep neural networks containing more than tens of billions of parameters. They use self-supervised learning methods to master many language phenomena through pre-training on large and diverse public datasets.

[0003] Furthermore, in order to apply the LLM model to a specific application scenario, the pre-trained LLM model can be further fine-tuned using a data set related to the specific application scenario. The fine-tuned LLM model can be used to perform multi-round dialogue tasks related to the specific application scenario.

[0004] However, the fine-tuned LLM model still has many defects when performing multi-round dialogue tasks. For example, it still lacks professional knowledge related to the application scenario, especially long-tail knowledge, and it is difficult to remember all the detailed information contained in the multi-round dialogue history. This may eventually lead to hallucinations in the dialogue content generated by the LLM model, that is, the content is unreliable and inaccurate, and cannot meet the actual usage needs of users. Even the erroneous inferences given by the LLM model may cause serious harm to users. Summary of the invention

[0005] In view of this, one or more embodiments of the present specification provide a medical LLM model reasoning method and related equipment based on a knowledge graph.

[0006] In a first aspect, this specification provides an LLM model reasoning method based on a knowledge graph, wherein the knowledge graph comprises a plurality of nodes and edges connecting the nodes, wherein the nodes represent entities, and the edges represent relationships between entities; the method comprises:

[0007] Obtain a target conversation text input by a user, and search the knowledge graph for a subgraph related to the entity mention contained in the target conversation text; the subgraph includes a target node representing the target entity, and a plurality of neighbor nodes connected to the target node;

[0008] Acquire attribute values ​​of several attributes of the target entity from multiple rounds of dialogue texts input by the user including the target dialogue text;

[0009] Based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity, construct a dialogue graph corresponding to the target dialogue text;

[0010] Prompt words are constructed according to the dialogue diagram and the target dialogue text, and the prompt words are input into the LLM model, and the LLM model performs logical reasoning based on the prompt words.

[0011] In a second aspect, the present specification provides an LLM model reasoning device based on a knowledge graph, wherein the knowledge graph comprises a plurality of nodes and edges connecting the nodes, wherein the nodes represent entities, and the edges represent relationships between entities; the device comprises:

[0012] A first acquisition unit is used to acquire a target conversation text input by a user, and to search the knowledge graph for a subgraph related to the entity mention contained in the target conversation text; the subgraph includes a target node representing the target entity, and a plurality of neighbor nodes connected to the target node;

[0013] A second acquisition unit, configured to acquire attribute values ​​of several attributes of the target entity from the multi-round dialogue texts input by the user including the target dialogue text;

[0014] A dialogue graph construction unit, configured to construct a dialogue graph corresponding to the target dialogue text based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity;

[0015] The reasoning unit is used to construct prompt words according to the dialogue diagram and the target dialogue text, and input the prompt words into the LLM model, so that the LLM model performs logical reasoning based on the prompt words.

[0016] Accordingly, this specification also provides a computing device, comprising: a memory and a processor; the memory stores computer programs / instructions that can be executed by the processor; when the processor executes the computer program / instructions, it executes the LLM model reasoning method based on the knowledge graph described in the first aspect above.

[0017] Accordingly, this specification also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the LLM model reasoning method based on the knowledge graph as described in the first aspect above is executed.

[0018] Accordingly, this specification also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the LLM model reasoning method based on the knowledge graph as described in the first aspect above is executed.

[0019] In summary, after obtaining the target conversation text input by the user, the present application can, on the one hand, search the knowledge graph for the subgraph related to the target entity corresponding to the entity mention contained in the target conversation text to assist in enhancing the professional knowledge of the LLM model. On the other hand, considering that several attributes of the entity (such as symptoms, duration, etc.) are crucial to the reasoning of the LLM model, the attribute values ​​of several attributes of the entity can be obtained from the user's multi-round conversation text to capture the fine-grained information in the user's conversation. Then, based on the obtained subgraph and the attribute values ​​of several attributes of the entity, a clear and detailed conversation graph is constructed, and then the conversation graph that contains both professional knowledge and fine-grained information in the user conversation is used to assist the LLM model in logical reasoning, thereby enhancing the reasoning performance of the LLM model and ensuring the correctness of its reasoning results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a medical dialogue schematic diagram provided by an exemplary embodiment;

[0021] Figure 2 is a schematic diagram of a system architecture provided by an exemplary embodiment;

[0022] Figure 3 It is a flowchart of an LLM model reasoning method based on a knowledge graph provided by an exemplary embodiment;

[0023] Figure 4 It is a schematic diagram of a process of constructing a dialogue diagram provided by an exemplary embodiment;

[0024] Figure 5 It is a schematic diagram of generating a prompt text based on a dialogue diagram provided by an exemplary embodiment;

[0025] Figure 6 is another schematic diagram of generating prompt text based on a dialogue diagram provided by an exemplary embodiment;

[0026] Figure 7 is another schematic diagram of generating a prompt text based on a dialogue diagram provided by an exemplary embodiment;

[0027] Figure 8 It is a structural schematic diagram of an LLM model reasoning device based on a knowledge graph provided by an exemplary embodiment;

[0028] Fig. 9 It is a structural diagram of a computing device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0030] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0031] It should be noted that the “plurality” mentioned in this application refers to two or more than two.

[0032] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] As mentioned above, after fine-tuning the pre-trained LLM model using a data set related to a specific application scenario, the fine-tuned LLM model can perform multi-round dialogue tasks related to the specific application scenario, but it still has many defects when performing multi-round dialogue tasks and cannot meet the actual usage needs of users.

[0034] For example, taking the specific application scenario as a medical consultation scenario, the fine-tuned LLM model can be a medical LLM model. Based on the medical LLM model, a complete medical dialogue system (Medical Dialogue System, MDS) can be further constructed to realize human-computer interactive consultation in the medical field, including analyzing the patient's input and making corresponding disease diagnosis, treatment suggestions and drug recommendations, etc., which will not be elaborated here.

[0035] Among them, the multi-round dialogue between the user and the medical LLM model can be recorded as H = (p1, d1, ..., p N-1 , d N-1 , p N), where P = (p1, ..., p N ) is a multi-round dialogue input by the user, D = (d1, ..., d N-1 ) is a multi-round dialogue output by the medical LLM model. The final generation goal of the medical LLM model can be to generate dialogue d N , dialogue d N The final disease diagnosis results, treatment suggestions and drug recommendations can be included.

[0036] However, the current medical LLM model still has many flaws. Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a medical dialogue provided by an exemplary embodiment. Figure 1 As shown, the patient and the medical LLM model can have multiple rounds of dialogue, and finally the medical LLM model needs to give the recommended medicine for the patient. Figure 1 The content shown can be analyzed and it can be concluded that the current medical LLM model still has the following shortcomings:

[0037] (1) Co-reference resolution: It is difficult to determine whether the medical LLM model can understand the specific medical meaning of "this situation" in the figure. For example, "blood pressure 145" represents "hypertension". In other words, the medical LLM model may not be able to connect the specific symptoms described by the patient with the professional medical meaning, thus affecting the subsequent diagnosis.

[0038] (2) Forgetting of fine-grained information in multi-round dialogue history: After multiple rounds of dialogue, it is difficult for the medical LLM model to remember the fine-grained medical information contained in the dialogue history.

[0039] (3) Lack of expertise: Although the medical LLM model has been fine-tuned and trained on a large amount of medical data, it is usually difficult for the model to remember the complex expertise in the medical field, especially the long-tail knowledge. Figure 1 As shown in the figure, since the medical LLM model lacks the professional knowledge that "bronchitis patients are prohibited from using or using labetalol with caution", it only knows that labetalol is a commonly used drug for hypertension, and thus directly outputs the wrong medication advice to the patient that "you can consider taking labetalol to control blood pressure", which seriously endangers the patient's health.

[0040] (4) Hallucination phenomenon: The above shortcomings will lead to hallucinations in the generated content of the medical LLM model, that is, outputting unreasonable, unreliable, and inaccurate consultation results to patients (including wrong disease diagnosis results, wrong treatment suggestions, and wrong drug recommendations, etc.), which seriously endangers the health of patients.

[0041] Based on this, this specification provides a technical solution that explicitly captures the fine-grained information contained in multiple rounds of conversations by constructing a conversation graph to prevent the LLM model from forgetting, and enhances professional knowledge with the help of a knowledge graph to avoid hallucinations in the LLM model, thereby comprehensively improving the reasoning ability of the LLM model and ensuring the reliability and correctness of the reasoning results.

[0042] During implementation, the target dialogue text input by the user can be obtained first, and a subgraph related to the target entity corresponding to the entity mention contained in the target dialogue text can be found from the knowledge graph. Among them, the subgraph can include a target node representing the target entity, and several neighbor nodes connected to the target node. Further, the attribute values ​​of several attributes of the target entity can be obtained from the multi-round dialogue text input by the user containing the above target dialogue text. Further, based on the obtained subgraph and the attribute values ​​of several attributes of the target entity, a dialogue graph corresponding to the target dialogue text is constructed. Further, a corresponding prompt word can be constructed according to the dialogue graph and the above target dialogue text, and the prompt word can be input into the LLM model, and the LLM model performs logical reasoning based on the prompt word.

[0043] In the above technical solution, after obtaining the target dialogue text input by the user, on the one hand, the subgraph related to the target entity corresponding to the entity mention contained in the target dialogue text is searched from the knowledge graph to assist in enhancing the professional knowledge of the LLM model. On the other hand, considering that several attributes of the entity (such as symptoms, duration, etc.) are crucial to the reasoning of the LLM model, the attribute values ​​of several attributes of the entity can be obtained from the user's multi-round dialogue text to capture the fine-grained information in the user's dialogue. Then, based on the obtained subgraph and the attribute values ​​of several attributes of the entity, a clear and detailed dialogue graph is constructed, and then the dialogue graph containing both professional knowledge and fine-grained information in the user dialogue is used to assist the LLM model in logical reasoning, thereby enhancing the reasoning performance of the LLM model and ensuring the correctness of its reasoning results. Especially in the medical field, the reliability of the LLM model in medical dialogues can be improved to avoid harm to the user's health due to incorrect diagnosis by the LLM model.

[0044] See also Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a system architecture provided by an exemplary embodiment. Figure 2 As shown, one or more embodiments provided in this specification may be Figure 2 The present invention is specifically implemented in the system architecture shown or a similar system architecture.

[0045] like Figure 2As shown, the computing device is equipped with an LLM model, which can be an LLM service model obtained by fine-tuning the pre-trained LLM basic model based on data related to the target application scenario, and can be used to perform multi-round dialogue tasks related to the target application scenario. In an illustrated embodiment, a user can conduct multi-round dialogues with the LLM model through an input device (such as a keyboard or a touch screen) provided by the computing device to consult the LLM model related to the target application scenario and obtain corresponding answers, etc.

[0046] In one illustrated embodiment, the target application scenario may be a medical consultation scenario, a life assistant scenario, a sentiment analysis scenario, etc., and this specification does not specifically limit this.

[0047] like Figure 2 As shown, the user can input the target dialogue text through the computing device, and accordingly, the computing device can obtain the target dialogue text input by the user. For example, taking the target application scenario as a medical consultation scenario, the target dialogue text can be Figure 1 The instructions do not make any specific limitations on the information shown, such as "My wife is 30 weeks pregnant and her blood pressure is 145 now", "About 3 days ago", or "I also have bronchitis, will this have any impact", etc.

[0048] Furthermore, if Figure 2 As shown, the computing device may perform entity extraction on the acquired target conversation text to extract entity mentions contained in the target conversation text. For details, please refer to the following Figure 3 The description of the corresponding embodiment will not be elaborated here. Figure 1 Taking the target dialogue text "My wife is 30 weeks pregnant and her blood pressure is 145 now" as an example, the extracted entity mention may be "pregnancy".

[0049] Furthermore, if Figure 2 As shown in the figure, considering that the entity mentions contained in the target dialogue text input by the user may contain abbreviations, omissions, aliases and other irregularities, the entity mentions need to be mapped to (or linked to) a unified and standard entity in the knowledge graph. Among them, the knowledge graph usually contains multiple nodes and edges connecting nodes. Nodes represent entities, and edges represent the relationship between entities.

[0050] In an illustrated embodiment, a target entity corresponding to the entity mention contained in the knowledge graph can be determined by entity linking (EL). Furthermore, a subgraph related to the target entity can be found from the knowledge graph. For details, please refer to the following Figure 3The description of the corresponding embodiment will not be elaborated here. Figure 1 Taking the target dialogue text "My wife is 30 weeks pregnant and her blood pressure is 145 now" as an example, the entity mention contained in it is "pregnancy", and the target entity in the entity link to the knowledge graph in the medical field may also be "pregnancy" or "pregnancy", which is not specifically limited in this specification.

[0051] Furthermore, if Figure 2 As shown in the figure, considering the attribute values ​​of the entity, such as the duration of pregnancy, the duration of symptoms, the specific value of blood pressure and the specific location of pain, etc., as reasoning conditions, they will greatly affect the reasoning results of the LLM model. Therefore, detailed information related to the target entity can also be obtained from the multi-round dialogue texts input by the user containing the target dialogue text. For details, please refer to the following Figure 3 The description of the corresponding embodiments will not be elaborated in detail here.

[0052] Furthermore, if Figure 2 As shown in the figure, the corresponding dialogue graph can be constructed based on the subgraph related to the target entity contained in the knowledge graph and the detailed information related to the target entity obtained from the multi-round dialogue text input by the user. For details, please refer to the following Figure 3 The description of the corresponding embodiments will not be elaborated in detail here.

[0053] Furthermore, if Figure 2 As shown, after constructing the dialogue graph, key prompt information can also be mined from the dialogue graph. Finally, the above target dialogue text, dialogue graph and prompt information can be input into the LLM model for reasoning to obtain and output the corresponding reasoning results to the user, that is, output the response dialogue corresponding to the above target dialogue text, and so on.

[0054] So far, this application has constructed a conversation graph to explicitly capture the fine-grained information contained in multiple rounds of conversations on the one hand to prevent the LLM model from forgetting, and on the other hand to enhance professional knowledge with the help of knowledge graphs to avoid hallucinations in the LLM model, thereby comprehensively improving the reasoning ability of the LLM model and ensuring the reliability and correctness of the reasoning results.

[0055] In one illustrated embodiment, the computing device equipped with the LLM model may be, for example, a smart wearable device, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., and this specification does not specifically limit this.

[0056] See also Figure 3 , Figure 3 This is a flowchart of an LLM model reasoning method based on a knowledge graph provided by an exemplary embodiment. This method can be applied to Figure 2In the computing device shown. Figure 3 As shown, the method may specifically include the following steps S301 to S304.

[0057] Step S301, obtain the target dialogue text input by the user, and search the knowledge graph for a subgraph related to the entity mention contained in the target dialogue text; the subgraph includes a target node representing the target entity, and a number of neighbor nodes connected to the target node.

[0058] First, the computing device may obtain the target dialogue text input by the user, which may be the dialogue text of any round input by the user during the multi-round dialogue between the user and the LLM model. Then, the computing device may perform entity extraction on the target dialogue text to extract the entity mentions contained in the target dialogue text.

[0059] It should be noted that this specification does not specifically limit the specific implementation method of entity extraction.

[0060] In an illustrated embodiment, the target dialogue text and the preset entity extraction instruction can be input into the LLM model, and the LLM model extracts the entity mentions contained in the target dialogue text according to the entity extraction instruction. In an illustrated embodiment, the LLM model can be pre-tuned on the information extraction dataset, and this specification does not specifically limit this.

[0061] Exemplarily, the formula corresponding to the entity extraction may be: C = LLM (I NER , p m ), where C is the entity mention set consisting of several extracted entity mentions, I NER Extract instructions for entities, p m The dialogue text of the mth round entered by the user (ie, the target dialogue text).

[0062] It should be noted that this specification does not refer to the entity extraction instruction I NER The specific content is not particularly limited.

[0063] In one illustrated embodiment, the entity extracts instruction I NER It may include extraction hints and several examples. NER It can be as follows:

[0064] You are a named entity recognition annotator in the medical field. Given a medical conversation text, you need to return the disease entities in it in the form of a list. The reference format is as follows: Output format: ["entity 1", "entity 2", ...]

[0065] Input Example 1:

[0066] Patient: I just went to the hospital for a check-up and found out that my hepatitis B surface antigen HBSAg is positive (+). What does this mean and what medicine can I take to get better?

[0067] Output example 1: [“Hepatitis B”]

[0068] Input Example 2:

[0069] Patient: I have a cold but no fever, runny nose and swollen eyes.

[0070] Doctor: Hello, how long has this situation lasted?

[0071] Patient: I have been coughing for 3 days.

[0072] Doctor: Is there phlegm when you cough? Is it white or yellow?

[0073] Patient: White sputum.

[0074] Patient: I sweat a lot.

[0075] Doctor: Well, what medicine have you taken in the past few days?

[0076] Patient: I didn’t take any medicine.

[0077] Doctor: You have a viral cold.

[0078] Patient: Do you want an IV?

[0079] Doctor: I recommend taking [MASK], [MASK], [MASK]

[0080] Output example 2: ["cold", "viral cold"]

[0081] Input Example 3:

[0082] Patient: The doctor at the pharmacy said it should be chronic urticaria. For more than half a year, the area where the belt presses on my lower abdomen is often itchy. The more I scratch it, the more it itches. When I scratch it, a red patch will appear. It also occurs occasionally in other areas. I have allergic rhinitis and my eyes are also very sensitive. I think they are all related. Now I take loratadine and it is fine. In addition, the doctor at the pharmacy told me to take Baishan Xiatare tablets, vitamin E, and calcium tablets for several months. He said it can be cured?

[0083] Doctor: Hello, the treatment of chronic urticaria requires continuous medication for 4 weeks, then slowly reducing the dosage and continuing for another 4 weeks. Oral loratadine is enough, and other drugs are auxiliary.

[0084] Output example 3: ["Chronic grass hives", "Allergic rhinitis"]

[0085] Furthermore, after extracting entity mentions from the target conversation text input by the user, the entity mentions in the target conversation text can be unambiguously and correctly linked to the entities in the knowledge graph (or knowledge base) through entity linking, that is, the target entity contained in the knowledge graph corresponding to the entity mention is determined.

[0086] It should be noted that this specification does not specifically limit the specific implementation method of entity linking.

[0087] In an illustrated embodiment, the entity mention extracted from the target conversation text can be mapped to the target entity that is a synonym of the entity mentioned in the knowledge graph through a synonym word list in the knowledge graph. Alternatively, the entity mention extracted from the target conversation text can be synonymously matched with the entities represented by multiple nodes in the knowledge graph based on the synonym word list; if the entity mention is a synonym that matches the target entity represented by the target node in the knowledge graph, the entity mention can be linked to the corresponding target entity in the knowledge graph.

[0088] In an illustrated embodiment, entity linking can also be converted into a classification task for knowledge graph entities and processed using a trained mapping classification model to determine the target entity contained in the knowledge graph that corresponds to the entity mention.

[0089] In one illustrated embodiment, heuristic methods such as the longest substring can also be used to match entity mentions with entities represented by multiple nodes contained in the knowledge graph to determine the target entity contained in the knowledge graph that corresponds to the entity mention, etc. This specification does not specifically limit this.

[0090] For example, the entity mention extracted from the target conversation text may be "nocturia", and the target entity in the corresponding knowledge graph may be "frequent urination". For example, the entity mention extracted from the target conversation text may be "pregnancy", and the target entity in the corresponding knowledge graph may be "pregnancy", or "pregnancy", etc., which is not specifically limited in this specification.

[0091] Furthermore, after determining the target entity corresponding to the entity mention contained in the knowledge graph, a subgraph related to the target entity can be found from the knowledge graph. The subgraph may include a target node representing the target entity and several neighbor nodes connected to the target node.

[0092] Step S302: acquiring attribute values ​​of several attributes of the target entity from the multi-round dialogue texts input by the user including the target dialogue text.

[0093] Furthermore, considering the state of the entity, that is, the attribute values ​​of the various attributes of the entity, such as the duration of pregnancy, the duration of symptoms, the specific value of blood pressure and the specific location of pain, etc. as reasoning conditions will greatly affect the reasoning results of the LLM model, and users often do not express all their detailed information clearly in one round of conversation. Therefore, the present application can further obtain detailed information related to the above-mentioned target entity (that is, the attribute values ​​of several attributes of the target entity) from multiple rounds of conversation texts input by the user containing the above-mentioned target conversation text.

[0094] It should be noted that this specification does not specifically limit the specific implementation method of obtaining the attribute values ​​of several attributes of the target entity.

[0095] In one illustrated implementation, attribute values ​​of several attributes of the target entity may be obtained by slot filling.

[0096] Specifically, several attributes of the target entity may be determined according to the entity type of the target entity, and then several preset slots corresponding to the several attributes may be predefined. Then, the slot values ​​of the several preset slots corresponding to the target entity may be obtained from the multi-round dialogue texts input by the user including the target dialogue text by means of slot filling.

[0097] For example, assuming that the target entity is "low back pain", and correspondingly, its entity type is symptom, then its several attributes (slots) may include duration and location, etc., and correspondingly, several attribute values ​​(slot values) may be three days and left side, etc., and this specification does not make any specific limitations on this.

[0098] In an illustrated embodiment, when obtaining the slot values ​​of several preset slots corresponding to the target entity from the multi-round dialogue texts input by the user including the target dialogue text by slot filling, it can specifically include: inputting the multi-round dialogue texts input by the user including the target dialogue text and the preset slot filling instructions into the LLM model, and the LLM model extracting the slot values ​​of several preset slots corresponding to the target entity from the multi-round dialogue texts according to the slot filling instructions, and outputting several triples corresponding to the slot values ​​of the several preset slots. Each triple may include the target entity, any preset slot and the slot value of the preset slot, which can be recorded as (entity, slot, slot value).

[0099] For example, the formula corresponding to the above slot filling can be as follows:

[0100] SV={(c,s,v)|c∈C}=LLM(I SV ,p m-k:m+k ,d m-k:m+k )

[0101] Where SV is a triple set consisting of several triples obtained, c is any target entity, C is the entity set containing the target entity, (c, s, v) is any triple, s represents the preset slot, v represents the slot value of the slot, I SV is the slot filling instruction, p m-k:m+k is the conversation text from the mkth round to the m+kth round input by the user (i.e., the 2kth round of conversation text before and after the target conversation text of the mth round), d m-k:m+k The dialogue text from the mkth round to the m+kth round output by the LLM model, where k can be an integer greater than or equal to 1.

[0102] It should be understood that the attribute values ​​of several attributes of the target entity may be mentioned in the user's current target dialogue text and the previous historical dialogue text, but it is also very likely that the user will give relevant descriptions in subsequent dialogues. Therefore, for the target entity in the current target dialogue text, it is necessary to expand the scope of its slot filling to the previous and subsequent rounds of dialogue. In this way, even if the attribute values ​​(i.e., slot values) of several attributes of the target entity cannot be obtained temporarily in the current target dialogue text and the historical dialogue text, in subsequent rounds of dialogue, the slots can still be filled for the target entity to obtain the slot values, thereby achieving the acquisition of sufficient fine-grained information as much as possible without missing any details provided by the user.

[0103] In addition, it should be noted that this specification does not refer to the slot filling instruction I SV The specific content is not particularly limited.

[0104] In one illustrated embodiment, the slot filling instruction I SV It may include slot filling instructions and several examples. SV It can be as follows:

[0105] You are an experienced doctor. Based on the context of the medical conversation, identify the status of the given disease / symptom and return the result in JSON format.

[0106] Main status: Candidate types are from ["patient self-reported positive", "patient self-reported negative", "doctor diagnosed positive", "doctor diagnosed negative", "unknown"]

[0107] [Description of main status types]

[0108] Past medical history: Candidate types from ["yes", "no"]

[0109] [Description of past medical history type]

[0110] Other relevant information: Other information about the given disease / symptom mentioned in the conversation, such as duration and body part. Store the information in a list format.

[0111] Step S303: constructing a dialog graph corresponding to the target dialog text based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity.

[0112] Furthermore, based on the acquired subgraphs in the knowledge graph related to the target entity and the attribute values ​​of several attributes of the target entity (such as the above-mentioned triples), a conversation graph corresponding to the target conversation text can be constructed. In this way, the constructed conversation graph contains not only the professional knowledge in the knowledge graph, but also the fine-grained information provided by the user in multiple rounds of conversation.

[0113] In an illustrated embodiment, the target entity in the conversation graph can be connected to several neighbor nodes included in the subgraph (i.e., neighbor nodes in the shared knowledge graph), and the target entity is connected to several slot values ​​included in the above-mentioned triples through edges whose edge types are corresponding preset slots. In an illustrated embodiment, the conversation graph also includes a user node, and the user node is connected to the target entity through an edge whose edge type is the entity type of the target entity.

[0114] It should be understood that, given that users often do not express much information in the first round of conversation, an initial empty graph can be constructed based on the user's first round of conversation text, and a user node can be created in the empty graph. As multiple rounds of conversations gradually unfold between the user and the LLM model, the computing device can obtain more entities and richer information from the conversation, thereby gradually expanding and enriching the content of the conversation graph.

[0115] See also Figure 4 , Figure 4 FIG. 1 is a flow chart of constructing a dialogue diagram provided by an exemplary embodiment. Figure 4 As shown, based on the multi-round dialogue between the user and the LLM model, the dialogue graph constructed may include a user node at the center and multiple nodes connected to the user node. The target entity may be Figure 4 The neighbor nodes in the subgraph can be Figure 4 "Aspirin" and "Berberine" shown.

[0116] like Figure 4As shown, the edge type of the edge connecting the user node and "cold" can be "disease", that is, the entity type of the entity "cold"; the edge type of the edge connecting "pregnancy" and "16 weeks" can be "duration", that is, the preset slot corresponding to the entity "pregnancy"; in addition, the edge type of the edge connecting "cold" and "aspirin" can be "treatment method", and the edge type of the edge connecting "cold" and "berberine" can be "treatment method", and so on. The details will not be elaborated here.

[0117] In addition, in some possible implementations, in addition to performing entity extraction on the dialogue text input by the user, entity extraction and subsequent entity linking and slot filling can also be performed on the dialogue text output by the LLM model in multiple rounds of dialogue, so as to further enrich the information in the dialogue diagram, etc. This specification does not make specific limitations on this.

[0118] Step S304, constructing prompt words according to the dialogue diagram and the target dialogue text, and inputting the prompt words into the LLM model, and the LLM model performs logical reasoning based on the prompt words.

[0119] Furthermore, after constructing the dialogue graph, prompt words can be constructed based on the dialogue graph and the target dialogue text input by the user, and the prompt words can be input into the LLM model. The LLM model performs logical reasoning based on the prompt words and outputs corresponding reasoning results, that is, outputs dialogue text that responds to the target dialogue text.

[0120] For example, taking the target application scenario as a medical consultation scenario, the logical reasoning performed by the LLM model may include logical reasoning related to multi-round dialogue tasks corresponding to the medical consultation scenario, such as symptom analysis, disease diagnosis, treatment suggestions and drug recommendations, etc. This specification does not make specific limitations on this.

[0121] In an illustrated embodiment, after constructing the dialogue graph, the computing device can also generate prompt information related to the target dialogue text based on the dialogue graph. Accordingly, prompt words can be constructed based on the dialogue graph, the prompt information and the target dialogue text, and the prompt words are input into the LLM model, and the LLM model performs logical reasoning based on the prompt words. In this way, by further combining the key information mined from the dialogue graph as prompts, the LLM model can be assisted in performing logical reasoning more comprehensively and efficiently, further improving the reasoning performance of the model.

[0122] Below, the solution for generating prompt information based on the dialogue diagram provided by the present application will be described with reference to specific examples.

[0123] First of all, it should be noted that this specification does not specifically limit the specific implementation method of generating prompt information based on the dialogue diagram.

[0124] In an illustrated embodiment, the prompt information may include prompt text for describing the logical relationship between the target entity and at least some of the neighbor nodes.

[0125] Accordingly, when prompt information is generated based on a conversation graph, it may specifically include: determining the edge type of the edge connecting the target entity contained in the conversation graph and a number of neighbor nodes (e.g., first-order neighbor nodes), then selecting at least some of the neighbor nodes whose edge types match the user's intention from the several neighbor nodes, and then generating prompt text for describing the logical relationship between the target entity and the at least some of the neighbor nodes.

[0126] In an illustrated embodiment, when selecting at least some of the neighboring nodes whose edge types match the user's intention from a number of neighboring nodes, it can specifically include: inputting the edge types (such as treatment methods, related symptoms and precautions, etc.) of the edges corresponding to the number of neighboring nodes to the LLM model and prompting it to make a selection; if the LLM model selects a target type (such as "treatment method") from the number of edge types based on the inferred user intention, at least some of the neighboring nodes connected to the target entity through the edge of the target type can be screened out from the number of neighboring nodes. Furthermore, based on a number of triples consisting of the target entity, the edge of the target type and at least some of the neighboring nodes, a corresponding prompt text for describing the logical relationship between the target entity and the at least some of the neighboring nodes can be generated.

[0127] For example, see Figure 5 , Figure 5 It is a schematic diagram of generating prompt text based on a dialogue diagram provided by an exemplary embodiment. Figure 5 Still Figure 4 Take the dialogue graph shown as an example, in which the target entity contained therein may be "cold", and several neighbor nodes in the subgraph connected to "cold" may include "aspirin", "berberine", "keep warm" and "no alcohol". Among them, the edge type of the edge connecting the neighbor nodes "aspirin" and "berberine" to the target entity "cold" is "treatment method"; the edge type of the edge connecting the neighbor nodes "keep warm" and "no alcohol" to the target entity "cold" is "precautions". Based on this, the two edge types of "treatment method" and "precautions" can be input into the LLM model and prompted to make a selection. Figure 5 As shown, if the LLM model selects "treatment method", the corresponding prompt text can be generated based on the triples (cold, treatment method, aspirin) and (cold, treatment method, berberine), such as "cold can be treated with the drugs aspirin and berberine".

[0128] In an illustrated embodiment, the prompt information may also include a prompt text for describing a conflict between the information included in the dialog diagram and other information. The so-called other information may be professional knowledge information related to the target application scenario, or known historical experience information, etc., which is not specifically limited in this specification.

[0129] Accordingly, when generating prompt information based on a conversation graph, it may specifically include: obtaining a pre-built prompt path, which may include multiple nodes connected in sequence; further, determining whether the conversation graph contains a path that matches the prompt path; if so, querying the knowledge graph for information related to the multiple nodes included in the path; further, in response to a conflict between the information in the queried knowledge graph and the information included in the path, generating a prompt text for describing the conflict.

[0130] Exemplarily, the prompt path may be, for example: pregnancy←user→high blood pressure→drug, pregnancy←user→cold→drug, etc. This specification does not specifically limit this.

[0131] For example, see Figure 6 , Figure 6 It is another schematic diagram of generating prompt text based on a dialogue diagram provided by an exemplary embodiment. Figure 6 Still Figure 4 Take the dialogue diagram shown in the figure as an example, assuming that the preset prompt path is: pregnancy ← user → cold → medicine. Figure 6 As shown, a path matching the prompt path (pregnancy←user→cold→aspirin) is found from the conversation graph, and then the knowledge graph can be queried for information related to multiple nodes contained in the path, such as querying the drug instructions for aspirin and precautions for catching a cold during pregnancy. Exemplarily, if the information queried from the knowledge graph may include that pregnant women cannot take aspirin, a prompt text describing the conflict can be generated based on the conflict between the information in the knowledge graph and the information contained in the path, such as "pregnant women cannot take aspirin."

[0132] In one illustrated embodiment, conflicts between information included in a conversation graph and other information may be difficult to obtain simply by querying a knowledge graph, or when it involves reasoning about more fine-grained medical information, conflicts between information included in a conversation graph and other information cannot be directly discovered.

[0133] Based on this, after determining that the dialogue graph contains a path that matches the preset prompt path, a question text corresponding to the path can be generated based on the information contained in the path, that is, the matched path is converted into a natural language question.

[0134] Furthermore, the generated question text can be input into the LLM model, and the LLM model can infer the possible conflict between the information contained in the path and other information based on the question text, and output a prompt text for describing the conflict. In this way, the LLM model can be encouraged to perform additional thinking to try to infer the conflict between the information contained in the path and other information.

[0135] Alternatively, the generated question text may be input into an Internet search engine to query the search engine for conflicts between the information included in the path and other information, and a prompt text describing the conflict may be generated based on the query result.

[0136] For example, see Figure 7 , Figure 7 It is another schematic diagram of generating prompt text based on a dialogue diagram provided by an exemplary embodiment. Figure 7 Still Figure 4 Take the dialogue diagram shown in the figure as an example, assuming that the preset prompt path is: duration←pregnancy←user→cold, such as Figure 7 As shown in , a path matching the prompt path (16 weeks ← pregnancy ← user → cold) is found from the conversation graph. Figure 7 As shown, a question text corresponding to the path can be generated based on the information contained in the path, such as "What are the precautions for catching a cold after 16 weeks of pregnancy?" Further, the question text can be input into the LLM model for reasoning, or input into an Internet search engine for query, so as to obtain a prompt text for describing the conflict between the information contained in the path and other information, such as "Try to avoid using drugs to prevent any impact on the fetus", etc., which is not specifically limited in this specification.

[0137] In one illustrated embodiment, the present application may predefine multiple prompt paths, and concurrently perform path matching and prompt text generation based on the multiple prompt paths, etc., to improve the efficiency of prompt text generation, thereby enriching the dialogue strategy of the LLM model, etc. This specification does not make specific limitations on this.

[0138] In summary, this application constructs a dialogue graph through a set of pipeline frameworks, which not only explicitly captures the fine-grained information in the dialogue flow (i.e., multi-round dialogues), prevents the large model from forgetting, but also masters the professional knowledge in the knowledge graph. This dialogue graph can not only be used for retrieval-augmented generation (RAG), generating electronic medical records, etc., but is also adapted to the medical dialogue system framework in the era of large models, and can provide better medical services for patients. In addition, the method provided in this application only prompts the large model through RAG and context learning based on the knowledge graph, without the need to adapt to other hardware or software methods, "plug and play", low deployment cost, and a wide range of applications. In addition to being applicable to the above-mentioned medical consultation scenarios, it can also be applied to various scenarios such as life assistants and sentiment analysis to provide users with high-quality services.

[0139] Corresponding to the implementation of the above method flow, the embodiment of this specification also provides an LLM model reasoning device based on knowledge graph. The device can be applied to Figure 2 See the computing device shown. Figure 8 , Figure 8 FIG. 1 is a schematic diagram of a structure of an LLM model reasoning device based on a knowledge graph provided by an exemplary embodiment. Figure 8 As shown, the device 80 includes:

[0140] The first acquisition unit 801 is used to acquire a target conversation text input by a user, and to find a subgraph related to the entity mention contained in the target conversation text from the knowledge graph; the subgraph includes a target node representing the target entity, and a plurality of neighbor nodes connected to the target node;

[0141] A second acquisition unit 802 is used to acquire attribute values ​​of several attributes of the target entity from the multi-round dialogue texts input by the user including the target dialogue text;

[0142] A dialog graph construction unit 803, configured to construct a dialog graph corresponding to the target dialog text based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity;

[0143] The reasoning unit 804 is used to construct prompt words according to the dialogue diagram and the target dialogue text, and input the prompt words into the LLM model, so that the LLM model performs logical reasoning based on the prompt words.

[0144] In an illustrated embodiment, the first acquisition unit 801 is specifically configured to:

[0145] Inputting the target dialogue text and a preset entity extraction instruction into the LLM model, and extracting the entity mentions contained in the target dialogue text by the LLM model according to the entity extraction instruction;

[0146] Through entity linking, a target entity contained in the knowledge graph and corresponding to the entity mention is determined, and a subgraph related to the target entity is found from the knowledge graph.

[0147] In an illustrated embodiment, the second obtaining unit 802 is specifically configured to:

[0148] Determining a number of attributes of the target entity according to the entity type of the target entity;

[0149] A plurality of preset slots constructed for the plurality of attributes of the target entity are obtained, and slot values ​​of the plurality of preset slots corresponding to the target entity are obtained from multiple rounds of dialogue texts input by the user containing the target dialogue text through slot filling.

[0150] In an illustrated embodiment, the second obtaining unit 802 is specifically configured to:

[0151] Inputting the multi-round dialogue texts input by the user including the target dialogue text and the preset slot filling instructions into the LLM model, the LLM model extracting the slot values ​​of the preset slots corresponding to the target entity from the multi-round dialogue texts according to the slot filling instructions, and outputting a plurality of triples corresponding to the slot values ​​of the preset slots;

[0152] Each triplet includes the target entity, a preset slot and a slot value of the preset slot.

[0153] In an illustrated embodiment, the dialogue graph construction unit 803 is specifically used to:

[0154] Based on the subgraph and the plurality of triples, a conversation graph corresponding to the target conversation text is constructed; wherein the target entity in the conversation graph is respectively connected to the plurality of neighbor nodes contained in the subgraph, and the target entity is connected to the plurality of slot values ​​contained in the plurality of triples via edges whose edge types are corresponding preset slots; the conversation graph also includes a user node, and the user node is connected to the target entity via an edge whose edge type is the entity type of the target entity.

[0155] In an illustrated embodiment, the reasoning unit 804 is specifically configured to:

[0156] Based on the dialogue graph, generating prompt information related to the target dialogue text;

[0157] Prompt words are constructed according to the dialogue diagram, the prompt information and the target dialogue text, and the prompt words are input into the LLM model.

[0158] In an illustrated embodiment, the prompt information includes a prompt text for describing a logical relationship between the target entity and at least some of the neighboring nodes; the reasoning unit 804 is specifically used to:

[0159] Determining an edge type of an edge connecting the target entity and the plurality of neighbor nodes included in the conversation graph;

[0160] At least some of the neighbor nodes whose edge types match the user intention are selected from the plurality of neighbor nodes, and prompt text is generated for describing the logical relationship between the target entity and the at least some of the neighbor nodes.

[0161] In an illustrated embodiment, the prompt information includes a prompt text for describing a conflict between the information included in the conversation diagram and other information; the reasoning unit 804 is specifically used to:

[0162] Obtaining a pre-built hint path, wherein the hint path includes a plurality of nodes connected in sequence;

[0163] Determine whether the conversation graph contains a path matching the prompt path; if so, query the knowledge graph for information related to a plurality of nodes contained in the path;

[0164] In response to a conflict between the queried information in the knowledge graph and the information included in the path, a prompt text is generated to describe the conflict.

[0165] In an illustrated embodiment, the prompt information includes a prompt text for describing a conflict between the information included in the conversation diagram and other information; the reasoning unit 804 is specifically used to:

[0166] Obtaining a pre-built hint path, wherein the hint path includes a plurality of nodes connected in sequence;

[0167] Determine whether the dialogue graph contains a path matching the prompt path; if so, generate a question text corresponding to the path based on information contained in the path;

[0168] The question text is input into the LLM model, and the LLM model infers the conflict between the information included in the path and other information based on the question text, and outputs a prompt text for describing the conflict; or,

[0169] The question text is input into an Internet search engine to search for conflicts between the information included in the path and other information, and a prompt text for describing the conflict is generated based on the query result.

[0170] In one illustrated embodiment, the LLM model is an LLM service model obtained by fine-tuning a pre-trained LLM basic model based on data related to a target application scenario.

[0171] In one illustrated embodiment, the target application scenario includes a medical consultation scenario, the logical reasoning includes logical reasoning related to a multi-round dialogue task corresponding to the medical consultation scenario, and the knowledge graph is a knowledge graph in the medical field.

[0172] The implementation process of the functions and effects of each unit in the above-mentioned device 80 is specifically described in the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 80 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for realizing network communication functions.

[0173] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.

[0174] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, a vehicle-mounted computer, or a combination of any of these devices.

[0175] Corresponding to the above method embodiment, the embodiment of this specification also provides a computing device. Fig. 9 , Fig. 9FIG. 1 is a schematic diagram of a computing device provided by an exemplary embodiment. Fig. 9 As shown, the computing device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 may be connected via a bus or other means. Fig. 9 As shown, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program that can be run by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiment. When the processor 1001 runs the stored computer program, it can execute the various steps of the LLM model reasoning method based on the knowledge graph in the embodiment of this specification, including: obtaining the target dialogue text input by the user, and finding a subgraph related to the entity mention contained in the target dialogue text from the knowledge graph; the subgraph includes a target node representing the target entity, and several neighbor nodes connected to the target node; obtaining the attribute values ​​of several attributes of the target entity from the multi-round dialogue text input by the user containing the target dialogue text; based on the subgraph and the attribute values ​​of the several attributes of the target entity, constructing a dialogue graph corresponding to the target dialogue text; constructing a prompt word according to the dialogue graph and the target dialogue text, and inputting the prompt word into the LLM model, and the LLM model performs logical reasoning based on the prompt word, and so on.

[0176] For a detailed description of the various steps of the above-mentioned knowledge graph-based LLM model reasoning method, please refer to the previous content and will not be repeated here.

[0177] Corresponding to the above method embodiment, the embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored. When these computer programs are executed by a processor, they execute the various steps of the LLM model reasoning method based on the knowledge graph in the embodiment of this specification. Please refer to the description of the above embodiment for details, which will not be repeated here.

[0178] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0179] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0180] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0181] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0182] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0183] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0184] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A LLM model reasoning method based on knowledge graph, characterized in that: The knowledge graph includes a plurality of nodes and edges connecting the nodes, the nodes represent entities, and the edges represent the relationships between entities; the method includes: Obtain a target conversation text input by a user, and search the knowledge graph for a subgraph related to the entity mention contained in the target conversation text; the subgraph includes a target node representing the target entity, and a plurality of neighbor nodes connected to the target node; Acquire attribute values ​​of several attributes of the target entity from multiple rounds of dialogue texts input by the user including the target dialogue text; Based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity, construct a dialogue graph corresponding to the target dialogue text; Prompt words are constructed according to the dialogue diagram and the target dialogue text, and the prompt words are input into the LLM model, and the LLM model performs logical reasoning based on the prompt words.

2. The method according to claim 1, characterized in that The step of searching the knowledge graph for a subgraph related to the entity mention contained in the target conversation text includes: Inputting the target dialogue text and a preset entity extraction instruction into the LLM model, and extracting the entity mentions contained in the target dialogue text by the LLM model according to the entity extraction instruction; Through entity linking, a target entity contained in the knowledge graph and corresponding to the entity mention is determined, and a subgraph related to the target entity is found from the knowledge graph.

3. The method according to claim 1, characterized in that The acquiring the attribute values ​​of the plurality of attributes of the target entity from the multi-round dialogue texts input by the user including the target dialogue text comprises: Determining a number of attributes of the target entity according to the entity type of the target entity; A plurality of preset slots constructed for the plurality of attributes of the target entity are obtained, and slot values ​​of the plurality of preset slots corresponding to the target entity are obtained from multiple rounds of dialogue texts input by the user containing the target dialogue text through slot filling.

4. The method according to claim 3, characterized in that The step of obtaining slot values ​​of the plurality of preset slots corresponding to the target entity from the multiple rounds of dialogue texts input by the user and including the target dialogue text by slot filling includes: Inputting the multi-round dialogue texts input by the user including the target dialogue text and the preset slot filling instructions into the LLM model, the LLM model extracting the slot values ​​of the preset slots corresponding to the target entity from the multi-round dialogue texts according to the slot filling instructions, and outputting a plurality of triples corresponding to the slot values ​​of the preset slots; Each triplet includes the target entity, a preset slot and a slot value of the preset slot.

5. The method according to claim 4, characterized in that The step of constructing a dialog graph corresponding to the target dialog text based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity includes: Based on the subgraph and the plurality of triples, a conversation graph corresponding to the target conversation text is constructed; wherein the target entity in the conversation graph is respectively connected to the plurality of neighbor nodes contained in the subgraph, and the target entity is connected to the plurality of slot values ​​contained in the plurality of triples via edges whose edge types are corresponding preset slots; the conversation graph also includes a user node, and the user node is connected to the target entity via an edge whose edge type is the entity type of the target entity.

6. The method according to claim 1, characterized in that The step of constructing prompt words according to the dialogue diagram and the target dialogue text, and inputting the prompt words into the LLM model includes: Based on the dialogue graph, generating prompt information related to the target dialogue text; Prompt words are constructed according to the dialogue diagram, the prompt information and the target dialogue text, and the prompt words are input into the LLM model.

7. The method according to claim 6, characterized in that The prompt information includes a prompt text for describing a logical relationship between the target entity and at least some of the neighboring nodes among the plurality of neighboring nodes; The generating of prompt information related to the target dialogue text based on the dialogue graph includes: Determining an edge type of an edge connecting the target entity and the plurality of neighbor nodes included in the conversation graph; At least some of the neighbor nodes whose edge types match the user intention are selected from the plurality of neighbor nodes, and prompt text is generated for describing the logical relationship between the target entity and the at least some of the neighbor nodes.

8. The method according to claim 6, characterized in that The prompt information includes a prompt text for describing a conflict between the information included in the dialogue graph and other information; and the generating of the prompt information related to the target dialogue text based on the dialogue graph includes: Obtaining a pre-built hint path, wherein the hint path includes a plurality of nodes connected in sequence; Determine whether the conversation graph contains a path matching the prompt path; if so, query the knowledge graph for information related to a plurality of nodes contained in the path; In response to a conflict between the queried information in the knowledge graph and the information included in the path, a prompt text is generated to describe the conflict.

9. The method according to claim 6, characterized in that The prompt information includes a prompt text for describing a conflict between the information included in the dialogue graph and other information; and the generating of the prompt information related to the target dialogue text based on the dialogue graph includes: Obtaining a pre-built hint path, wherein the hint path includes a plurality of nodes connected in sequence; Determine whether the dialogue graph contains a path matching the prompt path; if so, generate a question text corresponding to the path based on information contained in the path; The question text is input into the LLM model, and the LLM model infers the conflict between the information included in the path and other information based on the question text, and outputs a prompt text for describing the conflict; or, The question text is input into an Internet search engine to search for conflicts between the information included in the path and other information, and a prompt text for describing the conflict is generated based on the query result.

10. The method according to any one of claims 1 to 9, characterized in that: The LLM model is an LLM service model obtained by fine-tuning the pre-trained LLM basic model based on data related to the target application scenario.

11. The method according to claim 10, characterized in that The target application scenario includes a medical consultation scenario, the logical reasoning includes logical reasoning related to a multi-round dialogue task corresponding to the medical consultation scenario, and the knowledge graph is a knowledge graph in the medical field.

12. An LLM model reasoning device based on knowledge graph, characterized in that: The knowledge graph includes a plurality of nodes and edges connecting the nodes, the nodes represent entities, and the edges represent the relationships between entities; the device includes: A first acquisition unit is used to acquire a target conversation text input by a user, and to search the knowledge graph for a subgraph related to the entity mention contained in the target conversation text; the subgraph includes a target node representing the target entity, and a plurality of neighbor nodes connected to the target node; A second acquisition unit, configured to acquire attribute values ​​of several attributes of the target entity from the multi-round dialogue texts input by the user including the target dialogue text; A dialogue graph construction unit, configured to construct a dialogue graph corresponding to the target dialogue text based on the subgraph and the attribute values ​​of the plurality of attributes of the target entity; The reasoning unit is used to construct prompt words according to the dialogue diagram and the target dialogue text, and input the prompt words into the LLM model, so that the LLM model performs logical reasoning based on the prompt words.

13. A computing device, characterized in that: include: Memory and processor; The memory has stored thereon computer programs / instructions executable by the processor; When the processor runs the computer program / instructions, it performs the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

15. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 11 is implemented.

Images