A knowledge graph enhanced prompt word generation method, device, equipment and medium

Abstract

The application discloses a knowledge graph enhanced prompt word generation method and device, equipment and medium, and relates to the technical field of large language models. The method comprises the following steps: constructing knowledge retrieved from a knowledge graph into the form of a mind map; converting the knowledge between different organizational forms in one or two rounds of conversion through a large language model; finally generating a plurality of candidate knowledge prompt words in the form of a mind map; scoring the candidate prompt words; selecting the optimal prompt word corresponding to each training data to form a pseudo prompt data set; fine-tuning a large language model as a knowledge prompt optimizer by using the pseudo prompt data set; and generating a knowledge prompt word suitable for the large language model according to the input and related knowledge by using the fine-tuned knowledge prompt optimizer, so as to fully stimulate the reasoning ability of the large language model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model, in particular to a knowledge graph enhanced prompt word generation method and device, equipment and medium. BACKGROUND

[0002] Large language models have been widely applied to question and answer tasks in various vertical fields. However, due to the lack of factual knowledge of large language models, hallucinations, i.e. generating incorrect or unrealistic answers, may occur. The prompt technology based on knowledge graph enhancement refers to adding knowledge information related to the input retrieved from an external knowledge graph in the prompt word, so that the large language model generates output based on the prompt word combined with external knowledge, thereby reducing hallucinations. There are various construction modes and optimization techniques for prompt words, and the large language model is very sensitive to prompt words. The subtle modification of the structure and content of the prompt word will significantly affect the results generated by the large language model. Therefore, how to effectively organize the knowledge retrieved from the knowledge graph to generate knowledge graph enhanced prompt words and maximize the ability of large language models to complete question and answer tasks in vertical fields is a technical problem that needs to be solved at present.

[0003] The existing organization method of the knowledge part in the prompt word includes converting the knowledge into an unordered triple sequence form, a knowledge path form, and a natural language text form. However, the existing organization method of the knowledge part in the prompt word can only present a relatively single reasoning process, which has limitations in stimulating the reasoning ability of the large language model. SUMMARY

[0004] The purpose of the present application is to provide a knowledge graph enhanced prompt word generation method, device, equipment and medium, which can improve the reasoning ability of the large language model in the question and answer task in the vertical field.

[0005] To achieve the above purpose, the present application provides the following solutions.

[0006] In a first aspect, the present application provides a knowledge graph enhanced prompt word generation method, comprising:

[0007] obtaining a sample input text;

[0008] performing entity recognition on the sample input text to obtain a set of key entity mentions; the set of key entity mentions includes a plurality of key entity mentions; a key entity mention is a target domain keyword;

[0009] performing entity linking on the set of key entity mentions and all target domain entities in a target domain knowledge graph to determine a set of entity corresponding groups; each set of entity corresponding groups includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity;

[0010] determine, based on the target domain knowledge graph, a directed path set corresponding to the sample input text by using the entity corresponding group set; two end point entities in the directed path set are target domain entities in the entity corresponding group set;

[0011] For each first training sample in the training data set, a first large language model is used to generate a knowledge candidate prompt word set corresponding to the first training sample; the knowledge candidate prompt word set includes several mind maps; each first training sample includes a sample input text, a directed path set composed of knowledge paths corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0012] For each sample input text, a final candidate knowledge prompt word corresponding to the sample input text is determined according to the knowledge candidate prompt word set corresponding to the first training sample in which the sample input text is located;

[0013] A second large language model is trained using a pseudo prompt training data set to obtain a knowledge prompt optimizer; the pseudo prompt training data set includes several second training samples, each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0014] A knowledge prompt word is generated using the knowledge prompt optimizer.

[0015] In a second aspect, the present application provides a knowledge graph enhanced prompt word generation device, comprising:

[0016] A text acquisition module is configured to acquire a sample input text;

[0017] An entity recognition module is configured to perform entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes several key entity mentions; a key entity mention is a target domain keyword;

[0018] An entity linking module is configured to perform entity linking on the key entity mention set and all target domain entities in a target domain knowledge graph to determine an entity corresponding group set; each entity corresponding group set includes several entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity;

[0019] A path retrieval module is configured to determine, based on the target domain knowledge graph, a directed path set corresponding to the sample input text by using the entity corresponding group set; two end point entities in the directed path set are target domain entities in the entity corresponding group set;

[0020] The knowledge candidate prompt word set generation module is configured to: for each first training sample in the training data set, generate a knowledge candidate prompt word set corresponding to the first training sample by using a first large language model; the knowledge candidate prompt word set comprises a plurality of mind maps; each first training sample comprises sample input text, a knowledge path set composed of a set of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0021] The final candidate knowledge prompt word determination module is configured to: for each sample input text, determine a final candidate knowledge prompt word corresponding to the sample input text according to a knowledge candidate prompt word set corresponding to a first training sample in which the sample input text is located;

[0022] The training module is configured to: train a second large language model by using a pseudo prompt training data set to obtain a knowledge prompt optimizer; the pseudo prompt training data set comprises a plurality of second training samples, each second training sample comprising sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0023] The knowledge prompt word generation module is configured to: generate a knowledge prompt word by using the knowledge prompt optimizer.

[0024] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the knowledge graph enhanced prompt word generation method.

[0025] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the knowledge graph enhanced prompt word generation method.

[0026] According to the embodiments provided in the present application, the following technical effects are disclosed:

[0027] The application provides a knowledge graph enhanced prompt word generation method, device and equipment and medium. Key entity mentions in input are recognized, and each key entity mention is linked to a specific target domain entity according to embedding similarity between the key entity mention and the target domain entity in a target domain knowledge graph. Then, all directed paths between target domain entities are retrieved on the target domain knowledge graph. Next, the retrieved knowledge paths are converted between different forms through one-round and two-round prompt large language models. Finally, multiple candidate knowledge prompt words in the form of mind maps are generated, and the candidate prompt words are used to prompt the large language model to output answers. The embedding similarity between the output answers and standard answers is used to select the prompt word with the highest score to form a pseudo prompt training data set. Finally, the pseudo prompt training data set is used to fine-tune another large language model to obtain a knowledge prompt optimizer. The trained knowledge prompt optimizer can generate effective knowledge prompt words in the form of mind maps according to input and retrieved relevant knowledge paths. The application organizes the knowledge part in the prompt in the form of mind maps, surpasses the prompt ability of knowledge paths and natural language texts, and makes the reasoning of the large language model closer to the thinking mode of human beings, and greatly stimulates the reasoning ability of the large language model in the vertical field question and answer task. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0029] Figure 1 The application environment diagram of the knowledge graph enhanced prompt word generation method in an embodiment of the present application;

[0030] Figure 2 The flowchart of the knowledge graph enhanced prompt word generation method provided by an embodiment of the present application;

[0031] Figure 3 The specific process diagram of the knowledge graph enhanced prompt word generation method provided by an embodiment of the present application;

[0032] Figure 4 The functional module diagram of the knowledge graph enhanced prompt word generation device provided by an embodiment of the present application;

[0033] Figure 5 The structural diagram of the computer equipment provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts are within the scope of the present application.

[0035] The above purposes, features and advantages of the present application will be more apparent and understandable. The present application will be further described in detail below with reference to the drawings and specific embodiments.

[0036] The prompt word based on knowledge graph enhancement generally consists of four parts: instruction, knowledge, input and output format. The instruction generally describes the task to be performed by the large language model, the knowledge generally contains relevant knowledge information retrieved from the knowledge graph, the input generally refers to the question or content that the large language model is expected to answer, and the output format generally indicates the output requirement. The generation of the prompt word based on knowledge graph enhancement generally includes three steps: first, extracting key entities from the input, and then retrieving relevant knowledge from the external knowledge graph. Among them, the existing organization methods of the knowledge part in the prompt word include converting the knowledge into an unordered triple sequence form, a knowledge path form and a natural language text form. The unordered triple sequence form refers to simply listing the retrieved knowledge in the form of triples (subject, predicate, object); the knowledge path form refers to connecting the directed paths between entities on the knowledge graph with arrows “->”, such as “entity1->relation1->entity2->relation2->...->entityn”; and the natural language text form converts the original knowledge retrieved into natural language text using heuristic rules or text generation models.

[0037] The unordered triple sequence form directly ignores the structure of the graph and does not contain the reasoning process at all; the knowledge path form only considers part of the structure of the graph and ignores the structural relationship between paths, only having a single reasoning mode from beginning to end; and the converted natural language text form loses the structure of the graph and implicitly contains a single reasoning process. Therefore, the existing organization methods of the knowledge part in the prompt word can only present a relatively single reasoning process, which has limitations in stimulating the reasoning ability of the large language model. At the same time, since the large language model is very sensitive to the prompt word, slight changes in the structure or content of the prompt word can lead to great differences in the output of the large language model, and the content of the retrieved knowledge changes with the input. Therefore, the knowledge part in the prompt word formed by only a single round of conversion method cannot be effective for all questions.

[0038] The knowledge graph enhancement prompt word generation method provided in the embodiments of the present application can be applied to, for example, Figure 1The application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the target domain knowledge graph. The data storage system can be separately arranged, or integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the sample input text to the server 104, and after the server 104 receives the sample input text, for the sample input text, the server 104 performs entity recognition on the sample input text to obtain a set of key entity mentions; the set of key entity mentions includes several key entity mentions; the key entity mention is a target domain keyword; determine the entity corresponding group set according to the set of key entity mentions and all target domain entities in the target domain knowledge graph; each entity corresponding group set includes several entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity; based on the target domain knowledge graph, the entity corresponding group set is used to determine the knowledge path set composed of directed paths corresponding to the sample input text; the two end point entities in the knowledge path set composed of directed paths are both target domain entities in the entity corresponding group set; for each first training sample in the training data set, a first large language model is used to generate a knowledge candidate prompt word set corresponding to the first training sample; the knowledge candidate prompt word set includes several mind maps; each first training sample includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text; for each sample input text, the set of final candidate knowledge prompts corresponding to the sample input text is determined according to the set of knowledge candidate prompts of the first training sample corresponding to the sample input text; the second large language model is trained using the pseudo prompt training data set to obtain a knowledge prompt optimizer; the pseudo prompt training data set includes several second training samples, and each second training sample includes a sample input text, a final candidate knowledge prompt corresponding to the sample input text, and a standard answer corresponding to the sample input text; the knowledge prompt optimizer is used to generate a knowledge prompt word. The server 104 can feed back the obtained knowledge prompt word to the terminal 102.

[0039] Among them, the terminal 102 can be but not limited to various desktop computers, notebook computers, smart phones and tablet computers. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0040] In an exemplary embodiment, as shown in Figure 2 and Figure 3 A knowledge graph enhanced prompt word generation method is provided, which is executed by a computer device, specifically by a terminal or a server, etc. Computer device alone, or by the terminal and the server together, in the embodiment of the present application, the method is applied to Figure 1The server 104 in the server 104 is taken as an example for illustration, including the following steps 201 to 208.

[0041] Wherein:

[0042] Step 201: Obtain a sample input text.

[0043] Step 202: Perform entity recognition on the sample input text to obtain a set of key entity mentions; the set of key entity mentions includes a plurality of key entity mentions; a key entity mention is a target domain keyword.

[0044] Step 203: Perform entity linking on the set of key entity mentions and all target domain entities in a target domain knowledge graph to determine a set of entity corresponding groups; each set of entity corresponding groups includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity.

[0045] Step 204: Based on the target domain knowledge graph, determine a set of knowledge paths corresponding to the sample input text using the set of entity corresponding groups; both end point entities in the set of knowledge paths are target domain entities in the set of entity corresponding groups.

[0046] Step 205: For each first training sample in the training data set, use a first large language model to generate a set of knowledge candidate prompt words corresponding to the first training sample; the set of knowledge candidate prompt words includes a plurality of mind maps; each first training sample includes a sample input text, a set of knowledge paths composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text.

[0047] Step 206: For each sample input text, determine a final candidate knowledge prompt word corresponding to the sample input text according to the set of knowledge candidate prompt words corresponding to the first training sample in which the sample input text is located.

[0048] Step 207: Train a second large language model using a pseudo prompt training data set to obtain a knowledge prompt optimizer; the pseudo prompt training data set includes a plurality of second training samples, each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text.

[0049] Step 208: Generate a knowledge prompt word using the knowledge prompt optimizer.

[0050] The above steps 201 to 208 are implemented to organize the knowledge part in the prompt in the form of a complex mind map, which exceeds the prompt ability of the knowledge path and natural language text, and makes the reasoning of the large language model closer to the human thinking way and more fully stimulates the reasoning ability of the large language model. In addition, the present application can also generate multiple candidate knowledge prompt words in the form of a mind map by prompting the large language model to convert the knowledge between different organization forms for one or two rounds to construct a pseudo prompt training data set, which not only fully considers the influence of the difference between the knowledge content and structure on the output of the large language model, but also fully utilizes the implicit knowledge of the large language model, that is, the alignment of the retrieved knowledge is realized, and the knowledge completion can also be completed.

[0051] In another exemplary embodiment of the present application, after obtaining the sample input text, step 202 specifically includes: performing entity recognition on the sample input text by using a trained named entity recognition model to obtain the set of key entity mentions. The trained named entity recognition model is a model trained with the sample input text as input and the set of key entity mentions corresponding to the sample input text as label.

[0052] For a sample input text, the set of key entity mentions M is obtained by using a trained named entity recognition model corresponding to the domain of the sample input text to recognize the key entity mentions in the sample input text. The named entity recognition model can be W2NER, Pyramid, GCDT, etc.

[0053] In another exemplary embodiment of the present application, step 203 can be replaced by steps 301 to 303.

[0054] Step 301: Each key entity mention in the set of key entity mentions and each target domain entity in the target domain knowledge graph are encoded respectively to obtain the entity mention embedding code corresponding to each key entity mention and the target domain entity embedding code corresponding to each target domain entity.

[0055] Step 302: For the entity mention embedding code corresponding to each key entity mention, the similarity between the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity is calculated.

[0056] Step 303: The linking entity of the key entity mention is determined according to the similarity between the entity mention embedding code corresponding to all key entity mentions and the target domain entity embedding code corresponding to each target domain entity, and each key entity mention and the linking entity of the key entity mention constitute an entity corresponding group.

[0057] Using the BERT (Bidirectional Encoder Representations from Transformers) embedding model, each key entity mention in the key entity mention set M and each target domain entity in the target domain knowledge graph G is encoded respectively to obtain the entity mention embedding code Emb corresponding to each key entity mention M The target domain entity embedding code Emb corresponding to each target domain entity G . Calculate the key entity mention embedding and all target domain entity embedding codes Emb in the target domain knowledge graph G G According to the similarity between the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity calculated above, each key entity mention embedding mention is linked to the target domain entity closest to it. When the cosine similarity algorithm is used to calculate the similarity between the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity, the target domain entity with the largest cosine similarity value is closest to the key entity mention, and finally the entity corresponding group set E is obtained.

[0058] In another exemplary embodiment of the present application, step 204 may be replaced by the following steps 401 to 402 .

[0059] Step 401: construct entity pairs from each of the target domain entities in the entity corresponding group set, use the first target domain entity and the second target domain entity in the entity pair as endpoint entities, and use the target domain knowledge graph to search for the target domain entities in the entity pair to obtain all directed paths from the first target domain entity to the second target domain entity;

[0060] Step 402: All directed paths are screened according to the number of entities in each directed path and the set number, and directed paths corresponding to the sample input text are determined to form a knowledge path set.

[0061] All the target domain entities in the entity corresponding group set E are combined into entity pairs (e i , e j ), e i ∈E,e j ∈E,e i ≠e j , retrieve the first target domain entity e on the target domain knowledge graph G i To the second target domain entity e j The directed paths with no more than k hops constitute the knowledge path set P = {p1...p i ...p n}, wherein each directed path p i , i∈[1, n], n is the total number of directed paths in the knowledge path set, and is a knowledge path represented by entities and relationships connected by "->" to represent the reasoning process. It should be noted that no more than k hops means that the number of entities of the directed path from the first target domain entity to the second target domain entity does not exceed the set number.

[0062] In another exemplary embodiment of the present application, in step 205, the first large language model is used to generate the knowledge candidate prompt set corresponding to the first training sample, specifically including steps 501-504.

[0063] Step 501: input the sample input text and the directed path set corresponding to the sample input text into the first large language model to obtain a natural language explanation, a reasoning path, and a first mind map;

[0064] Step 502: input the sample input text and the natural language explanation into the first large language model to obtain a second mind map;

[0065] Step 503: input the sample input text and the reasoning path into the first large language model to obtain a third mind map;

[0066] Step 504: input the sample input text and the first mind map into the first large language model to obtain a fourth mind map; the first mind map, the second mind map, the third mind map, and the fourth mind map constitute the knowledge candidate prompt set corresponding to the first training sample.

[0067] Preferably, in step 206, the final candidate knowledge prompt corresponding to the sample input text is determined according to the knowledge candidate prompt set corresponding to the first training sample in which the sample input text is located, which can specifically include steps 601-604.

[0068] Step 601: for each mind map in the knowledge candidate prompt set corresponding to the first training sample in which the sample input text is located, determine the knowledge graph enhancement prompt corresponding to the mind map according to the mind map and the first training sample;

[0069] Step 602: input the knowledge graph enhancement prompt corresponding to the mind map into the first large language model to obtain the output answer corresponding to the mind map;

[0070] Step 603: input the output answer corresponding to each of the mind maps and the standard answer corresponding to the sample input text into the BERT embedding model respectively to obtain the output answer embedding code and the standard answer embedding code corresponding to each of the mind maps.

[0071] Step 604: determining a final mind map according to the standard answer embedding code and the output answer embedding code corresponding to each of the mind maps; the final mind map is a final candidate knowledge prompt word corresponding to the sample input text.

[0072] Step 604, specifically comprising: calculating the similarity corresponding to each of the mind maps according to the standard answer embedding code and the output answer embedding code corresponding to each of the mind maps; determining the mind map corresponding to the maximum similarity as the final mind map.

[0073] Steps 205 and 206 are used to construct a pseudo prompt training data set, specifically:

[0074] The training data set Train = {data = (q, P, a)}, where q is a sample input text, P is a knowledge path set composed of directed paths corresponding to the sample input text q obtained by the above step 204, and a is a standard answer corresponding to the sample input text q. For each first training sample data e Train, prompt the first large language model to convert the knowledge path in different forms, finally obtain a plurality of candidate prompt words in the form of mind map knowledge, and score, keep the highest score candidate prompt word as the final candidate knowledge prompt word corresponding to the sample input text, finally, obtain the pseudo prompt training data set Train' = {data' = (q, K best , a)}. The specific process is as follows:

[0075] (1) generate a candidate knowledge prompt word set K in the form of mind map:

[0076] For each first training sample data, there are two types of conversion methods, respectively through one round of large language model and through two rounds of large language model, combined together a total of four, the specific process is as follows:

[0077] a. Through one round of large language model: input the sample input text q and the knowledge path set composed of directed paths P into the first large language model, prompt the first large language model to generate a natural language explanation k_explain, a reasoning path k_path connecting entities and relationships with "->", and a first mind map k_mind to answer the sample input text q;

[0078] b. After two rounds of large language models: input the sample input text q and the natural language explanation k explain, the reasoning path connecting entities and relationships with "->", and the first mind map k mind respectively to obtain three input combinations, which are: sample input text q and natural language explanation k explain, sample input text and reasoning path k path, sample input text q and the first mind map k mind. Input the above three input combinations into the first large language model respectively, prompting the first large language model to generate the second mind map, the third mind map, and the fourth mind map k mind' that answers the sample input text q respectively.

[0079] The first mind map k mind, the second mind map, the third mind map, and the fourth mind map k mind' generated by the above four ways form a knowledge candidate prompt word set K in the form of a mind map corresponding to the first training sample data.

[0080] Next, taking the medical field as an example, the sample input text and its corresponding natural language explanation, reasoning path, and mind map are illustrated as follows:

[0081] 1) The sample input text is generally the input text formed by the user's question in a certain field. A specific example of a sample input text in the medical field is {Adverse reactions and precautions of propranolol}.

[0082] 2) The natural language explanation generally includes detailed answer explanations. A specific example of the natural language explanation of the above sample input text in the medical field is: Propranolol is a beta blocker, mainly used for the treatment of hypertension and angina. Since beta blockers can affect the normal function of the sympathetic nervous system, causing a decrease in heart rate, patients need to be vigilant about bradycardia, an adverse reaction, when using propranolol, especially when taking it for the first time. In addition, propranolol can also cause orthostatic hypotension, which is manifested as a decrease in blood pressure and can cause symptoms such as dizziness and syncope. Although gastric spasm and allergic reactions are also adverse reactions that propranolol can cause, they are not as important as bradycardia and orthostatic hypotension, and are not particularly important for first-time use. As for pulmonary embolism, although beta blockers can increase the risk of thrombosis, there is no clear evidence that propranolol increases the risk of pulmonary embolism, so it is generally not considered an adverse reaction that needs special attention when taking it for the first time.

[0083] 3) Inference paths generally show the complete reasoning process, which consists of inference paths. Each inference path is of the following form: 'Entity Name 1' -> 'Relationship Name 1' -> 'Entity Name 2' -> 'Relationship Name 2' -> ... -> 'Entity Name n', with paths separated by ';'. Note that entity names or relationship names should be kept as short as possible and do not include options.

[0084] Specific examples of reasoning paths corresponding to the sample input text in the medical field are: 'propranolol'->'treatment'->'hypertension'; 'propranolol'->'treatment'->'angina pectoris'; 'propranolol'->'may cause'->'postural hypotension'; 'postural hypotension'->'features'->'decreased blood pressure'; 'postural hypotension'->'associated symptoms'->'dizziness'; 'postural hypotension'->'associated symptoms'->'syncope'; 'propranolol'->'may cause'->'bradycardia'; 'propranolol'->'may cause'->'stomach cramps'; 'propranolol'->'may cause'->'allergic reaction'; 'propranolol'->'may cause'->'pulmonary embolism'.

[0085] 4) Thinking Guide Figure 1 It is generally a hierarchical tree structure, corresponding to the thinking process. The specific example of the mind map corresponding to the sample input text in the medical field mentioned above is as follows:

[0086] Propranolol

[0087]

[0088] (2) Rating:

[0089] Each candidate knowledge prompt word in the candidate knowledge prompt word set K in the form of a mind map corresponding to each first training sample data and the sample input text q in the first training sample data are filled into the prompt word template with pre-set instructions and output format to form a complete knowledge graph enhanced prompt word.

[0090] The knowledge graph enhanced prompt word is input into the large language model to obtain the output answer a'. The BERT embedding model is used to obtain the embedded encoding of the output answer a' and the standard answer a respectively. The BERTScore scores of the output answer a' and the standard answer a are calculated, and the candidate knowledge prompt word is scored. The candidate knowledge prompt word P' with the highest score is retained to form the pseudo prompt training dataset Train'={data'=(q, K best , a)}, data' represents the second training sample.

[0091] BERTScore is a metric used to measure text similarity. It's based on the BERT model and evaluates the similarity between two sentences by calculating the cosine similarity between their embeddings in the BERT model. BERTScore has precision, recall, and F1 scores. In this application, the candidate knowledge prompt words are scored using precision. The BERTScore score calculation process is as follows:

[0092] Standard answer: x = <x1,...,x k >

[0093] Output answer:

[0094] The vector of each token (token is each word in the sentence of the standard answer or output answer) is obtained through the BERT embedding model:

[0095] Standard answer embedding code: <x1,...x k >

[0096] Output answer embedding code:

[0097] Token in standard answer a: x i And output the answer a' in token: The cosine similarity is: But since the token vector has been normalized, the cosine similarity is i∈[1,k],j∈[1,m].

[0098] The complete precision score is obtained by matching each token in the output answer a' with the most similar token in the standard answer a. The formula is as follows:

[0099]

[0100] In another example of the present application, the following similarity calculation method may also be used, as follows:

[0101] (1) Similarity calculation based on word2vec: Get the vector of each word in the output answer and the standard answer respectively, and calculate the average of the vectors of all words in the output answer and the standard answer respectively. Use the respective average values ​​as the embedding of the output answer and the embedding of the standard answer, and then calculate the similarity based on the standard vector of the output answer and the embedding of the standard answer.

[0102] (2) Similarity calculation based on SentenceTransformer: the embedding of the output answer and the embedding of the standard answer are obtained using SentenceTransformer, and the cosine similarity of the embedding of the output answer and the embedding of the standard answer is calculated using the following formula.

[0103]

[0104] where x1 is the embedding of the output answer and x2 is the embedding of the standard answer.

[0105] In addition to the cosine similarity algorithm and the BERTScore score, the Euclidean distance algorithm and the Manhattan distance algorithm can also be used to calculate the similarity.

[0106] The pseudo prompt training data set Train'={data'=(q,K best , a)} is used to fine-tune the second large language model so that it can serve as a knowledge prompt optimizer to generate effective knowledge prompt words in the form of a mind map according to the sample input text and its corresponding directed path set. That is, a better mind map form of knowledge prompt that integrates the retrieved knowledge with the question can be generated (better means that the knowledge prompt can guide the large language model to answer the question more accurately and truly, that is, to stimulate the performance of the large language model).

[0107] The application also provides an application scenario of the knowledge graph enhanced prompt word generation method. Specifically, the knowledge graph enhanced prompt word generation method provided in the embodiment can be applied in a medical question and answer scenario. The medical question and answer scenario includes an input text acquisition link, a knowledge graph enhanced prompt word generation link, and an answering link. Sample medical input text enters the knowledge graph enhanced prompt word generation link from the input text acquisition link, obtains corresponding knowledge graph enhanced prompt words through human-computer cooperation, and enters the downstream answering link. The knowledge graph enhanced prompt word generation method provided in the embodiment belongs to the knowledge graph enhanced prompt word generation link. In the process of the knowledge graph enhanced prompt word generation link for the sample medical input text, entity recognition can be performed on the sample medical input text to obtain a key entity mention set. The key entity mention set includes a plurality of key entity mentions. The key entity mention is a medical field keyword. Entity corresponding group sets are determined according to the key entity mention set and all medical field entities in the medical field knowledge graph. Each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a medical field entity. Based on the medical field knowledge graph, the entity corresponding group sets are used to determine a knowledge path set composed of directed paths corresponding to the sample medical input text. Both end point entities in the knowledge path set composed of directed paths are medical field entities in the entity corresponding group set. For each first training sample in the training data set, a first large language model is used to generate a knowledge candidate prompt word set corresponding to the first training sample. The knowledge candidate prompt word set includes a plurality of mind maps. Each first training sample includes a sample medical input text, a knowledge path set composed of directed paths corresponding to the sample medical input text, and a standard answer corresponding to the sample medical input text. For each sample medical input text, a final candidate knowledge prompt word corresponding to the sample medical input text is determined according to the knowledge candidate prompt word set corresponding to the first training sample in which the sample medical input text is located. A second large language model is trained using a pseudo prompt training data set to obtain a knowledge prompt optimizer. The pseudo prompt training data set includes a plurality of second training samples. Each second training sample includes a sample medical input text, a final candidate knowledge prompt word corresponding to the sample medical input text, and a standard answer corresponding to the sample medical input text. The knowledge prompt optimizer is used to generate a knowledge prompt word.

[0108] Based on the same inventive concept, the embodiment of the present application also provides a knowledge graph enhanced prompt word generation device for implementing the knowledge graph enhanced prompt word generation method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more knowledge graph enhanced prompt word generation device embodiments provided below can be referred to the limitations of the knowledge graph enhanced prompt word generation method in the above text, which will not be repeated here.

[0109] In one example embodiment, as shown in Figure 4 A knowledge graph enhanced prompt word generation apparatus is provided, comprising:

[0110] A text acquisition module T1 is configured to acquire a sample input text.

[0111] An entity recognition module T2 is configured to perform entity recognition on the sample input text to obtain a set of key entity mentions; the set of key entity mentions includes a plurality of key entity mentions; a key entity mention is a target domain keyword.

[0112] An entity linking module T3 is configured to perform entity linking on the set of key entity mentions and all target domain entities in a target domain knowledge graph to determine a set of entity corresponding groups; each set of entity corresponding groups includes a plurality of entity corresponding groups; each entity corresponding group includes a key entity mention and a target domain entity.

[0113] A path retrieval module T4 is configured to determine, based on the target domain knowledge graph, a set of knowledge paths corresponding to the sample input text by using the set of entity corresponding groups; two end point entities in the set of knowledge paths are both target domain entities in the set of entity corresponding groups.

[0114] A knowledge candidate prompt word set generation module T5 is configured to, for each first training sample in a training data set, generate a set of knowledge candidate prompt words corresponding to the first training sample by using a first large language model; the set of knowledge candidate prompt words includes a plurality of mind maps; each first training sample includes a sample input text, a set of knowledge paths composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text.

[0115] A final candidate knowledge prompt word determination module T6 is configured to, for each sample input text, determine a final candidate knowledge prompt word corresponding to the sample input text according to a set of knowledge candidate prompt words corresponding to a first training sample in which the sample input text is located.

[0116] A training module T7 is configured to train a second large language model by using a pseudo prompt training data set to obtain a knowledge prompt optimizer; the pseudo prompt training data set includes a plurality of second training samples; each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text.

[0117] A knowledge prompt word generation module T8 is configured to generate a knowledge prompt word by using the knowledge prompt optimizer.

[0118] In an exemplary embodiment, a computer device is provided, which can be a server or a terminal, and an internal structure diagram thereof can be as shown in FIG. 1. Figure 5 The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store knowledge graph enhanced prompt word generation data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a knowledge graph enhanced prompt word generation method.

[0119] Those skilled in the art can understand that Figure 5 The structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In an exemplary embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0121] In an exemplary embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. The non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0124] The database involved in the embodiments provided in the present application can include at least one of various efficient graph storage databases. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0125] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0126] The principles and implementation modes of the present application are described by applying specific examples herein. The above description of the embodiments is only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A knowledge graph enhanced prompt word generation method, characterized in that: The method for generating knowledge graph enhanced prompt words includes: Get sample input text; Performing entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a plurality of key entity mentions; the key entity mentions are target domain keywords; Perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity; Based on the target domain knowledge graph, the entity corresponding group set is used to determine the directed path composition knowledge path set corresponding to the sample input text; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity corresponding group set; For each first training sample in the training data set, a first large language model is used to generate a set of candidate knowledge prompt words corresponding to the first training sample; the set of candidate knowledge prompt words includes a plurality of mind maps; each first training sample includes a sample input text, a knowledge path set consisting of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text; For each sample input text, determining a final candidate knowledge prompt word corresponding to the sample input text according to a set of candidate knowledge prompt words corresponding to the first training sample where the sample input text is located; The second language model is trained using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text; The knowledge prompt optimizer is used to generate knowledge prompt words.

2. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Perform entity recognition on the sample input text to obtain a key entity mention set, specifically including: The trained named entity recognition model is used to perform entity recognition on the sample input text to obtain the key entity mention set.

3. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine the entity corresponding group set, specifically including: Encode each key entity mention in the key entity mention set and each target domain entity in the target domain knowledge graph respectively to obtain an entity mention embedding code corresponding to each key entity mention and a target domain entity embedding code corresponding to each target domain entity; For each entity mention embedding code corresponding to the key entity mention, calculating the similarity between the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity; The linked entity of the key entity mention is determined according to the similarity between the entity mention embedding codes corresponding to all the key entity mentions and the target domain entity embedding codes corresponding to each target domain entity, and each key entity mention and the linked entity of the key entity mention constitute an entity correspondence group.

4. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Based on the target domain knowledge graph, the entity corresponding group set is used to determine the directed paths corresponding to the sample input text to form a knowledge path set, specifically including: Form entity pairs with each target domain entity in the entity corresponding group set, use the first target domain entity and the second target domain entity in the entity pair as endpoint entities, use the target domain knowledge graph to search the target domain entities in the entity pair, and obtain all directed paths from the first target domain entity to the second target domain entity; According to the number of entities in each directed path and the set number, all directed paths are screened to determine the directed paths corresponding to the sample input text to form a knowledge path set.

5. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Generating a set of candidate knowledge prompt words corresponding to the first training sample using the first language model specifically includes: Inputting a sample input text and a directed path corresponding to the sample input text into a knowledge path set into a first language model to obtain a natural language interpretation, a reasoning path, and a first mind map; Inputting the sample input text and the natural language interpretation into the first language model to obtain a second mind map; Inputting the sample input text and the reasoning path into the first language model to obtain a third mind map; The sample input text and the first mind map are input into the first large language model to obtain a fourth mind map; the first mind map, the second mind map, the third mind map and the fourth mind map constitute a knowledge candidate prompt word set corresponding to the first training sample.

6. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Determining the final candidate knowledge prompt word corresponding to the sample input text according to the knowledge candidate prompt word set corresponding to the first training sample of the sample input text specifically includes: For each mind map in the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located, determining the knowledge graph enhancement prompt word corresponding to the mind map according to the mind map and the first training sample; Inputting the knowledge graph enhancement prompt word corresponding to the mind map into the first language model to obtain an output answer corresponding to the mind map; Inputting the output answer corresponding to each mind map and the standard answer corresponding to the sample input text into the BERT embedding model respectively, to obtain the output answer embedding code and the standard answer embedding code corresponding to each mind map; The final mind map is determined according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each mind map; the final mind map is the final candidate knowledge prompt word corresponding to the sample input text.

7. The method for generating knowledge graph enhanced prompt words according to claim 6, characterized in that: Determining a final mind map according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each mind map specifically includes: Calculating the similarity corresponding to each mind map according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each mind map; The mind map corresponding to the similarity with the largest value is determined as the final mind map.

8. A knowledge graph enhanced prompt word generation device, characterized in that: The knowledge graph enhanced prompt word generation device includes: The text acquisition module is used to: obtain sample input text; An entity recognition module is configured to perform entity recognition on the sample input text to obtain a key entity mention set, wherein the key entity mention set includes a plurality of key entity mentions, and the key entity mentions are keywords in the target domain; An entity linking module is configured to perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity; A path retrieval module is configured to: determine, based on the target domain knowledge graph, a directed path composition knowledge path set corresponding to the sample input text using an entity correspondence group set; wherein both endpoint entities in the directed path composition knowledge path set are target domain entities in the entity correspondence group set; The knowledge candidate prompt word set generation module is configured to: for each first training sample in the training data set, generate a knowledge candidate prompt word set corresponding to the first training sample using a first large language model; the knowledge candidate prompt word set includes a plurality of mind maps; each first training sample includes a sample input text, a knowledge path set consisting of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text; A final candidate knowledge prompt word determination module is configured to: for each sample input text, determine a final candidate knowledge prompt word corresponding to the sample input text based on a set of knowledge candidate prompt words corresponding to a first training sample containing the sample input text; a training module, configured to train the second language model using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text; The knowledge prompt word generation module is used to: generate knowledge prompt words using the knowledge prompt optimizer.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and runnable on the processor, characterized in that the processor executes the computer program to implement the knowledge graph enhanced prompt word generation method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the knowledge graph enhanced prompt word generation method described in any one of claims 1-7.

Images