Knowledge graph enhanced cue word generation method and device, equipment and medium

Abstract

The invention discloses a knowledge graph enhanced cue word generation method and device, equipment and a medium, and relates to the technical field of large language models.The method comprises the steps that knowledge retrieved from a knowledge graph is constructed into a mind map form; the method comprises the following steps of: performing one-round or two-round conversion on knowledge among different organization forms through a prompt large language model, finally generating a plurality of candidate knowledge cue words in a mind map form, scoring the candidate cue words, selecting optimal cue words corresponding to each piece of training data to form a false cue data set, and performing false cue processing on the false cue data set. A large language model is finely adjusted by using the pseudo prompt data set to serve as a knowledge prompt optimizer, the finely-adjusted knowledge prompt optimizer can generate knowledge prompt words adaptive to the large language model according to input and related knowledge, and the inference ability of the large language model is fully stimulated.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a method, device, equipment and medium for generating knowledge graph enhanced prompt words. Background Art

[0002] Large language models have been widely used in question-answering tasks in various vertical fields. However, due to the lack of factual knowledge in large language models, hallucinations will occur, that is, wrong or untrue answers will be generated. The prompt technology based on knowledge graph enhancement refers to adding knowledge information related to the input retrieved from the external knowledge graph to the prompt word, so that the large language model can generate output based on the prompt word combined with external knowledge, thereby reducing hallucinations. There are many construction patterns and optimization techniques for prompt words, and large language models are very sensitive to prompt words. Slight changes in the structure and content of prompt words will significantly affect the results generated by the large language model. Therefore, how to effectively organize the knowledge retrieved from the knowledge graph, generate knowledge graph enhanced prompt words, and maximize the ability of the large language model to complete vertical field question answering is a technical problem that needs to be solved at present.

[0003] Existing methods for organizing the knowledge part of prompt words include converting knowledge into disordered triple sequence form, knowledge path form and natural language text form. However, existing methods for organizing the knowledge part of prompt words can only present a relatively simple reasoning process, which has limitations in stimulating the reasoning ability of large language models. Summary of the invention

[0004] The purpose of this application is to provide a knowledge graph enhanced prompt word generation method, device, equipment and medium, which can improve the reasoning ability of large language models in vertical field question and answer tasks.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for generating knowledge graph enhanced prompt words, comprising:

[0007] Get sample input text;

[0008] Performing entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a number of key entity mentions; the key entity mentions are target domain keywords;

[0009] Perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity;

[0010] Based on the target domain knowledge graph, the directed path composition knowledge path set corresponding to the sample input text is determined using the entity correspondence group set; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity correspondence group set;

[0011] For each first training sample in the training data set, a first large language model is used to generate a knowledge candidate prompt word set corresponding to the first training sample; the knowledge candidate prompt word set includes a plurality of mind maps; each first training sample includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0012] For each sample input text, determining a final candidate knowledge prompt word corresponding to the sample input text according to a set of candidate knowledge prompt words corresponding to a first training sample where the sample input text is located;

[0013] The second largest language model is trained using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each of which includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0014] The knowledge prompt optimizer is used to generate knowledge prompt words.

[0015] In a second aspect, the present application provides a knowledge graph enhanced prompt word generation device, comprising:

[0016] The text acquisition module is used to: acquire sample input text;

[0017] An entity recognition module is used to: perform entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a number of key entity mentions; the key entity mentions are target domain keywords;

[0018] An entity linking module is used to: perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity;

[0019] A path retrieval module is used to: determine the directed path composition knowledge path set corresponding to the sample input text based on the target domain knowledge graph using the entity correspondence group set; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity correspondence group set;

[0020] The knowledge candidate prompt word set generation module is used to: for each first training sample in the training data set, generate a knowledge candidate prompt word set corresponding to the first training sample using a first large language model; the knowledge candidate prompt word set includes a plurality of mind maps; each of the first training samples includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0021] The final candidate knowledge prompt word determination module is used to: for each sample input text, determine the final candidate knowledge prompt word corresponding to the sample input text according to the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located;

[0022] A training module is used to: train the second language model using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each of which includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0023] The knowledge prompt word generation module is used to: generate knowledge prompt words using a knowledge prompt optimizer.

[0024] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned knowledge graph enhanced prompt word generation method.

[0025] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned knowledge graph enhanced prompt word generation method.

[0026] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0027] The present application provides a method, device, equipment and medium for generating knowledge graph enhanced prompt words, identifying key entity mentions in input, and linking each key entity mention to a specific target domain entity according to the embedding similarity between the key entity mention and the target domain entity in the target domain knowledge graph; then, the directed paths between all target domain entities are retrieved on the target domain knowledge graph; then, the retrieved knowledge paths are converted between different forms through one round and two rounds of prompting the large language model, and finally multiple candidate knowledge prompt words in the form of mind maps are generated, and these candidate prompt words are used to prompt the large language model to output answers, and the prompt words with the highest scores are selected according to the embedding similarity between the output answer and the standard answer to form a pseudo prompt training data set, and finally, the pseudo prompt training data set is used to fine-tune another large language model to obtain a knowledge prompt optimizer. The trained knowledge prompt optimizer can generate knowledge prompt words in the form of valid mind maps according to the input and the retrieved related knowledge paths. This application uses a complex form of mind map to organize the knowledge part of the prompt, which goes beyond the prompting capabilities of knowledge paths and natural language texts, making the reasoning of large language models closer to the way humans think, and stimulating the reasoning ability of large language models in vertical field question-answering tasks to a greater extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 This is an application environment diagram of a knowledge graph enhanced prompt word generation method in one embodiment of the present application;

[0030] Figure 2 A schematic diagram of a flow chart of a method for generating knowledge graph enhanced prompt words provided in one embodiment of the present application;

[0031] Figure 3 A schematic diagram of a specific process of a method for generating knowledge graph enhanced prompt words provided in one embodiment of the present application;

[0032] Figure 4 A schematic diagram of the functional modules of a knowledge graph enhanced prompt word generation device provided in one embodiment of the present application;

[0033] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] The prompt words based on knowledge graph enhancement usually consist of four parts: instructions, knowledge, input, and output format. Instructions generally describe the tasks that the large language model needs to perform, knowledge generally includes relevant knowledge information retrieved from the knowledge graph, input generally refers to the questions or content that the large language model is expected to answer, and the output format generally describes the requirements for the output. The generation of prompt words based on knowledge graph enhancement is generally divided into three steps: first, extract key entities from the input, and then retrieve relevant knowledge from the external knowledge graph. Among them, the existing organization methods of the knowledge part in prompt words include converting knowledge into an unordered triple sequence form, a knowledge path form, and a natural language text form. The unordered triple sequence form refers to simply listing the retrieved knowledge in the form of triples (subject, predicate, object); the knowledge path form refers to connecting the directed paths between entities on the knowledge graph with arrows "->", such as "entity 1->relation 1->entity 2->relation 2->...->entity n"; and the natural language text form uses heuristic rules or text generation models to convert the retrieved original knowledge into natural language text.

[0037] The unordered triple sequence form directly ignores the structure of the graph and does not include the reasoning process at all; the knowledge path form only considers part of the structure of the graph, ignores the structural relationship between paths, and only has a single reasoning mode limited from beginning to end; the converted natural language text form loses the structure of the graph and implicitly includes a single reasoning process. Therefore, the existing organization method of the knowledge part in the prompt word can only present a relatively single reasoning process, which has limitations in stimulating the reasoning ability of the large language model. At the same time, because the large language model is very sensitive to the prompt word, a slight change in the structure or content of the prompt word may lead to a huge difference in the output of the large language model, and the retrieved knowledge content changes with the change of the input. Therefore, the knowledge part in the prompt word constructed using only a single round of conversion method cannot be effective for all problems.

[0038] The knowledge graph enhanced prompt word generation method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the target domain knowledge graph. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the sample input text to the server 104. After the server 104 receives the sample input text, the server 104 performs entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes several key entity mentions; the key entity mention is a target domain keyword; the entity corresponding group set is determined according to the key entity mention set and all the target domain entities in the target domain knowledge graph; each entity corresponding group set includes several entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity; based on the target domain knowledge graph, the entity corresponding group set is used to determine the directed path composition knowledge path set corresponding to the sample input text; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity corresponding group set; for each in the training data set A first training sample, using a first language model to generate a knowledge candidate prompt word set corresponding to the first training sample; the knowledge candidate prompt word set includes a number of mind maps; each first training sample includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text; for each sample input text, the final candidate knowledge prompt word corresponding to the sample input text is determined according to the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located; a pseudo prompt training data set is used to train the second language model to obtain a knowledge prompt optimizer; the pseudo prompt training data set includes a number of second training samples, each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text; and a knowledge prompt optimizer is used to generate knowledge prompt words. The server 104 can feed back the obtained knowledge prompt words to the terminal 102.

[0039] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, and tablet computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0040] In an exemplary embodiment, Figure 2 and Figure 3 As shown, a method for generating knowledge graph enhanced prompt words is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1The server 104 in the example is used for explanation, and the steps include the following steps 201 to 208.

[0041] in:

[0042] Step 201: Obtain sample input text.

[0043] Step 202: Perform entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a number of key entity mentions; the key entity mentions are target domain keywords.

[0044] Step 203: Entity linking is performed on the key entity mention set and all target domain entities in the target domain knowledge graph to determine an entity corresponding group set; each entity corresponding group set includes a number of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity.

[0045] Step 204: Based on the target domain knowledge graph, determine the directed path composition knowledge path set corresponding to the sample input text using the entity correspondence group set; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity correspondence group set.

[0046] Step 205: For each first training sample in the training data set, a first large language model is used to generate a knowledge candidate prompt word set corresponding to the first training sample; the knowledge candidate prompt word set includes a number of mind maps; each first training sample includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text.

[0047] Step 206: For each sample input text, determine the final candidate knowledge prompt word corresponding to the sample input text according to the set of candidate knowledge prompt words corresponding to the first training sample where the sample input text is located.

[0048] Step 207: Use a pseudo-prompt training data set to train the second largest language model to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each second training sample includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text.

[0049] Step 208: Use the knowledge prompt optimizer to generate knowledge prompt words.

[0050] Implementing the above steps 201 to 208, using the complex form of mind maps to organize the knowledge part of the prompt, surpassing the prompting ability of knowledge paths and natural language texts, making the reasoning of the large language model closer to the way humans think, and stimulating the reasoning ability of the large language model to a greater extent. In addition, the present application can also generate multiple candidate knowledge prompt words in the form of mind maps to construct a pseudo-prompt training data set by prompting the large language model to convert knowledge between different organizational forms for one or two rounds, which not only fully considers the impact of differences in knowledge content and structure on the output of the large language model, but also makes full use of the implicit knowledge of the large language model, that is, it realizes the alignment of retrieved knowledge and can also complete knowledge completion.

[0051] In another exemplary embodiment of the present application, after obtaining the sample input text, step 202 specifically includes: using the trained named entity recognition model to perform entity recognition on the sample input text to obtain the key entity mention set. The trained named entity recognition model is a model trained with the sample input text as input and the key entity mention set corresponding to the sample input text as labels.

[0052] For a sample input text, a named entity recognition model trained in the field corresponding to the sample input text is used to identify key entity mentions in the sample input text, and a key entity mention set M is obtained. The named entity recognition model can be W2NER, Pyramid, GCDT, etc.

[0053] In another exemplary embodiment of the present application, step 203 may be replaced by the following steps 301 to 303 .

[0054] Step 301: Encode each key entity mention in the key entity mention set and each target domain entity in the target domain knowledge graph respectively to obtain an entity mention embedding code corresponding to each key entity mention and a target domain entity embedding code corresponding to each target domain entity.

[0055] Step 302: For each entity mention embedding code corresponding to the key entity mention, calculate the similarity between the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity.

[0056] Step 303: Determine the linked entity of the key entity mention according to the similarity between the entity mention embedding codes corresponding to all the key entity mentions and the target domain entity embedding codes corresponding to each target domain entity, and each key entity mention and the linked entity of the key entity mention constitute an entity correspondence group.

[0057] Using the BERT (Bidirectional Encoder Representations from Transformers) embedding model, each key entity mention in the key entity mention set M and each target domain entity in the target domain knowledge graph G are encoded respectively to obtain the entity mention embedding code Emb corresponding to each key entity mention M The target domain entity embedding code Emb corresponding to each target domain entity G . Calculate the embedding of key entity mentions and the embedding encoding of all target domain entities in the target domain knowledge graph G G According to the similarity of the entity mention embedding code corresponding to the key entity mention calculated above and the target domain entity embedding code corresponding to each target domain entity, each key entity mention embedding mention is linked to the target domain entity closest to it. When the cosine similarity algorithm is used to calculate the similarity of the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity, the target domain entity with the largest cosine similarity value is closest to the key entity mention, and finally the entity corresponding group set E is obtained.

[0058] In another exemplary embodiment of the present application, step 204 may be replaced by the following steps 401 to 402 .

[0059] Step 401: construct entity pairs from each of the target domain entities in the entity corresponding group set, use the first target domain entity and the second target domain entity in the entity pair as endpoint entities, use the target domain knowledge graph to search for the target domain entities in the entity pair, and obtain all directed paths from the first target domain entity to the second target domain entity;

[0060] Step 402: All directed paths are screened according to the number of entities in each directed path and the set number, and the directed paths corresponding to the sample input text are determined to form a knowledge path set.

[0061] All the target domain entities in the entity correspondence group set E are combined into entity pairs (e i , e j ), e i ∈E,e j ∈E,e i ≠e j , retrieve the first target domain entity e on the target domain knowledge graph G i To the second target domain entity e j The knowledge path set P = {p1...p i ...p n}, where each directed path p i , i∈[1,n], n is the total number of directed paths in the knowledge path set composed of directed paths, all of which are knowledge paths that use “->” to connect entities and relationships to represent the reasoning process. It should be noted that no more than k hops means that the number of entities in the directed path from the first target domain entity to the second target domain entity does not exceed the set number.

[0062] In another exemplary embodiment of the present application, in step 205, a set of candidate knowledge prompt words corresponding to the first training sample is generated using a first large language model, which specifically includes the following steps 501 to 504.

[0063] Step 501: input a sample input text and a directed path corresponding to the sample input text into a knowledge path set and input the set into a first language model to obtain a natural language interpretation, a reasoning path and a first mind map;

[0064] Step 502: inputting the sample input text and the natural language interpretation into the first language model to obtain a second mind map;

[0065] Step 503: inputting the sample input text and the reasoning path into the first language model to obtain a third mind map;

[0066] Step 504: input the sample input text and the first mind map into the first large language model to obtain a fourth mind map; the first mind map, the second mind map, the third mind map and the fourth mind map constitute a knowledge candidate prompt word set corresponding to the first training sample.

[0067] Preferably, in step 206, the final candidate knowledge prompt word corresponding to the sample input text is determined according to the set of candidate knowledge prompt words corresponding to the first training sample where the sample input text is located, which may specifically include the following steps 601 to 604.

[0068] Step 601: for each mind map in the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located, determine the knowledge graph enhancement prompt word corresponding to the mind map according to the mind map and the first training sample;

[0069] Step 602: inputting the knowledge graph enhanced prompt words corresponding to the mind map into the first language model to obtain an output answer corresponding to the mind map;

[0070] Step 603: inputting the output answer corresponding to each of the mind maps and the standard answer corresponding to the sample input text into the BERT embedding model respectively, and obtaining the output answer embedding code and the standard answer embedding code corresponding to each of the mind maps;

[0071] Step 604: Determine a final mind map according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each of the mind maps; the final mind map is the final candidate knowledge prompt word corresponding to the sample input text.

[0072] Step 604 specifically includes: calculating the similarity corresponding to each mind map according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each mind map; and determining the mind map corresponding to the similarity with the largest value as the final mind map.

[0073] Step 205 and step 206 are used to construct a pseudo-prompt training data set, specifically:

[0074] The training data set Train = {data = (q, P, a)}, where q is the sample input text, P is the knowledge path set consisting of the directed paths corresponding to the sample input text q obtained in step 204, and a is the standard answer corresponding to the sample input text q. For each first training sample data∈Train, the first language model is prompted to transform the knowledge path in different forms, and finally multiple candidate prompt words of mind map form knowledge are obtained, and the candidate prompt words with the highest scores are retained as the final candidate knowledge prompt words corresponding to the sample input text. Finally, the pseudo prompt training data set Train' = {data' = (q, K best , a)}. The specific process is as follows:

[0075] (1) Generate a candidate knowledge prompt word set K in the form of a mind map:

[0076] For each first training sample data, there are two types of conversion methods, one is through one round of large language model and the other is through two rounds of large language model. There are four combinations in total. The specific process is as follows:

[0077] a. After a round of large language model: the sample input text q and the knowledge path set P composed of directed paths are input into the first large language model, prompting the first large language model to generate a natural language explanation k_explain to answer the sample input text q, a reasoning path k_path connecting entities and relationships with “->”, and the first mind map k_mind;

[0078] b. After two rounds of large language model: the sample input text q is combined with the natural language explanation k_explain, the reasoning path connecting entities and relationships with “->”, and the first mind map k_mind, and three input combinations are obtained. The three input combinations are: sample input text q and natural language explanation k_explain, sample input text and reasoning path k_path, sample input text q and the first mind map k_mind. The above three input combinations are input to the first large language model respectively, prompting the first large language model to generate the second mind map, the third mind map, and the fourth mind map k_mind' that answer the sample input text q respectively.

[0079] The first mind map k_mind, the second mind map, the third mind map and the fourth mind map k_mind' generated by the above four methods form a knowledge candidate prompt word set K in the form of a mind map corresponding to the first training sample data.

[0080] Taking the medical field as an example, the following is an illustration of sample input text and its corresponding natural language interpretation, reasoning path, and mind map:

[0081] 1) Sample input text is generally input text formed by users in response to questions in a certain field. A specific example of sample input text in the medical field is {adverse reactions and precautions of taking propranolol}.

[0082] 2) Natural language interpretation generally includes detailed explanations of answers. A specific example of natural language interpretation of the sample input text in the medical field is: Propranolol is a beta-blocker, mainly used to treat hypertension and angina pectoris. Since beta-blockers may affect the normal function of the sympathetic nervous system and cause a slow heart rate, patients need to be alert to the adverse reaction of bradycardia when using propranolol, especially when taking it for the first time. In addition, propranolol may also cause postural hypotension, which is manifested as a drop in blood pressure and may cause symptoms such as dizziness and syncope. Although gastric spasm and allergic reactions are also adverse reactions that propranolol may cause, compared with bradycardia and postural hypotension, they are not particularly important when taking it for the first time. As for pulmonary embolism, although beta-blockers may increase the risk of thrombosis, in the absence of clear evidence that propranolol increases the risk of pulmonary embolism, it is generally not regarded as an adverse reaction that should be paid special attention to when taking it for the first time.

[0083] 3) The reasoning path generally shows the complete reasoning process, which consists of reasoning paths. Each reasoning path is in the following form: 'entity name 1'->'relationship name 1'—>'entity name 2'->'relationship name 2'->...->'entity name n', and the paths are separated by ';'. Note that the entity name or relationship name should be as short as possible and do not include options.

[0084] Specific examples of reasoning paths corresponding to the sample input texts in the above medical field are: 'propranolol'->'treatment'->'hypertension'; 'propranolol'->'treatment'->'angina pectoris'; 'propranolol'->'may cause'->'postural hypotension'; 'postural hypotension'->'features'->'drop in blood pressure'; 'postural hypotension'->'associated symptoms'->'dizziness'; 'postural hypotension'->'associated symptoms'->'syncope'; 'propranolol'->'may cause'->'bradycardia'; 'propranolol'->'may cause'->'stomach cramps'; 'propranolol'->'may cause'->'allergic reaction'; 'propranolol'->'may cause'->'pulmonary embolism'.

[0085] 4) Mindset Figure 1 It is generally a hierarchical tree structure, corresponding to the thinking process. The specific example of the mind map corresponding to the sample input text in the medical field is as follows:

[0086] Propranolol

[0087]

[0088] (2) Rating:

[0089] Each candidate knowledge prompt word in the candidate knowledge prompt word set K in the form of a mind map corresponding to each first training sample data and the sample input text q in the first training sample data are filled into a prompt word template with pre-set instructions and output format to form a complete knowledge graph enhanced prompt word.

[0090] The knowledge graph enhanced prompt word is input into the large language model to obtain the output answer a'. The BERT embedding model is used to obtain the embedded encoding of the output answer a' and the standard answer a respectively, and the BERTScore scores of the output answer a' and the standard answer a are calculated. The candidate knowledge prompt word is scored, and the candidate knowledge prompt word P' with the highest score is retained to form the pseudo prompt training data set Train'={data'=(q,K best , a)}, data' represents the second training sample.

[0091] BERTScore is an indicator for measuring text similarity. It is based on the BERT model and evaluates the similarity of two sentences by calculating the cosine similarity between the embedded codes in the BERT model. BERTScore has precision, recall and F1 score. The score of candidate knowledge prompt words in this application adopts the precision indicator. The BERTScore score calculation process is as follows:

[0092] Standard answer: x = <x1,...,x k >

[0093] Output answer:

[0094] The vector of each token (token is each word in the sentence of the standard answer or output answer) is obtained through the BERT embedding model:

[0095] Standard answer embedding code: <x1,...x k >

[0096] Output answer embedding encoding:

[0097] Token in standard answer a: x i And output the answer a' in token: The cosine similarity is: But since the token vector has been normalized, the cosine similarity is i∈[1,k],j∈[1,m].

[0098] The complete precision score is obtained by matching each token in the output answer a' with the most similar token in the standard answer a. The formula is as follows:

[0099]

[0100] In another example of the present application, the following similarity calculation method may also be used, as follows:

[0101] (1) Similarity calculation based on word2vec: Obtain the vector of each word in the output answer and the standard answer respectively, and calculate the average of the vectors of all words in the output answer and the standard answer respectively. Use the respective average values ​​as the embedding of the output answer and the embedding of the standard answer, and then calculate the similarity based on the standard vector of the output answer and the embedding of the standard answer.

[0102] (2) Similarity calculation based on SentenceTransformer: Use SentenceTransformer to obtain the embedding of the output answer and the embedding of the standard answer respectively, and use the following formula to calculate the cosine similarity between the embedding of the output answer and the embedding of the standard answer.

[0103]

[0104] Among them, x1 is the embedding of the output answer, and x2 is the embedding of the standard answer.

[0105] In addition to the cosine similarity algorithm and BERTScore score, the Euclidean distance algorithm and Manhattan distance algorithm can also be used to calculate similarity.

[0106] Use pseudo-prompt training data set Train' = {data' = (q, K best , a)}, the instruction fine-tunes the second largest language model so that it can act as a knowledge prompt optimizer, and generate effective knowledge prompt words in the form of a mind map based on the knowledge path set composed of the sample input text and its corresponding directed path. That is, it can generate a better knowledge prompt in the form of a mind map that integrates the retrieved knowledge based on the question and the knowledge path related to the question (better means that the knowledge prompt can guide the large language model to answer the question more accurately and truthfully, that is, stimulate the performance of the large language model).

[0107] The present application also provides an application scenario, which applies the above-mentioned knowledge graph enhanced prompt word generation method. Specifically: the knowledge graph enhanced prompt word generation method provided in this embodiment can be applied in a medical question and answer scenario. The medical question and answer scenario includes an input text acquisition link, a knowledge graph enhanced prompt word generation link, and an answer link; the sample medical input text enters the knowledge graph enhanced prompt word generation link from the input text acquisition link, obtains the corresponding knowledge graph enhanced prompt word through human-computer collaboration, and enters the downstream answer link. The knowledge graph enhanced prompt word generation method provided in this embodiment belongs to the knowledge graph enhanced prompt word generation link. Specifically, in the process of generating a link for knowledge graph-enhanced prompt words for sample medical input text, entity recognition can be performed on the sample medical input text to obtain a key entity mention set; the key entity mention set includes several key entity mentions; the key entity mentions are medical field keywords; an entity corresponding group set is determined based on the key entity mention set and all medical field entities in the medical field knowledge graph; each entity corresponding group set includes several entity corresponding groups, and each entity corresponding group includes a key entity mention and a medical field entity; based on the medical field knowledge graph, the entity corresponding group set is used to determine the directed path composition knowledge path set corresponding to the sample medical input text; the two endpoint entities in the directed path composition knowledge path set are both medical field entities in the entity corresponding group set; for each first training sample in the training data set, the first largest language model is used to generate A set of candidate knowledge prompt words corresponding to the first training sample; the set of candidate knowledge prompt words includes several mind maps; each first training sample includes a sample medical input text, a knowledge path set composed of directed paths corresponding to the sample medical input text, and a standard answer corresponding to the sample medical input text; for each sample medical input text, a final candidate knowledge prompt word corresponding to the sample medical input text is determined according to the set of candidate knowledge prompt words corresponding to the first training sample where the sample medical input text is located; a second language model is trained using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes several second training samples, each second training sample includes a sample medical input text, a final candidate knowledge prompt word corresponding to the sample medical input text, and a standard answer corresponding to the sample medical input text; and a knowledge prompt optimizer is used to generate knowledge prompt words.

[0108] Based on the same inventive concept, the embodiment of the present application also provides a knowledge graph enhanced prompt word generation device for implementing the above-mentioned knowledge graph enhanced prompt word generation method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more embodiments of the knowledge graph enhanced prompt word generation device provided below can refer to the limitations of the knowledge graph enhanced prompt word generation method above, and will not be repeated here.

[0109] In an exemplary embodiment, Figure 4 As shown, a knowledge graph enhanced prompt word generation device is provided, comprising:

[0110] The text acquisition module T1 is used to: acquire sample input text;

[0111] The entity recognition module T2 is used to: perform entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a number of key entity mentions; the key entity mentions are target domain keywords;

[0112] The entity linking module T3 is used to: perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a number of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity;

[0113] The path retrieval module T4 is used to: determine the directed path composition knowledge path set corresponding to the sample input text based on the target domain knowledge graph using the entity correspondence group set; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity correspondence group set;

[0114] The knowledge candidate prompt word set generation module T5 is used to: for each first training sample in the training data set, generate a knowledge candidate prompt word set corresponding to the first training sample using the first large language model; the knowledge candidate prompt word set includes a plurality of mind maps; each first training sample includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0115] The final candidate knowledge prompt word determination module T6 is used to: for each sample input text, determine the final candidate knowledge prompt word corresponding to the sample input text according to the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located;

[0116] The training module T7 is used to: train the second largest language model using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each of which includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text;

[0117] The knowledge prompt word generation module T8 is used to: generate knowledge prompt words using a knowledge prompt optimizer.

[0118] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store knowledge graph enhanced prompt word generation data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a knowledge graph enhanced prompt word generation method is implemented.

[0119] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0121] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0122] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0123] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0124] The database involved in each embodiment provided in this application may include at least one of various efficient graph storage databases. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but is not limited thereto.

[0125] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0126] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A knowledge graph enhanced prompt word generation method, characterized in that: The knowledge graph enhanced prompt word generation method comprises: Get sample input text; Performing entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a number of key entity mentions; the key entity mentions are target domain keywords; Perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity; Based on the target domain knowledge graph, the directed path composition knowledge path set corresponding to the sample input text is determined using the entity correspondence group set; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity correspondence group set; For each first training sample in the training data set, a first large language model is used to generate a knowledge candidate prompt word set corresponding to the first training sample; the knowledge candidate prompt word set includes a plurality of mind maps; each first training sample includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text; For each sample input text, determining a final candidate knowledge prompt word corresponding to the sample input text according to a set of candidate knowledge prompt words corresponding to a first training sample where the sample input text is located; The second largest language model is trained using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each of which includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text; The knowledge prompt optimizer is used to generate knowledge prompt words.

2. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Perform entity recognition on the sample input text to obtain a key entity mention set, which specifically includes: The trained named entity recognition model is used to perform entity recognition on the sample input text to obtain the key entity mention set.

3. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine the entity corresponding group set, specifically including: Encode each key entity mention in the key entity mention set and each target domain entity in the target domain knowledge graph respectively to obtain an entity mention embedding code corresponding to each key entity mention and a target domain entity embedding code corresponding to each target domain entity; For each entity mention embedding code corresponding to the key entity mention, calculating the similarity between the entity mention embedding code corresponding to the key entity mention and the target domain entity embedding code corresponding to each target domain entity; The linked entity mentioned by the key entity is determined according to the similarity between the entity mention embedding codes corresponding to all the key entity mentions and the target domain entity embedding codes corresponding to each target domain entity, and each key entity mention and the linked entity mentioned by the key entity constitute an entity correspondence group.

4. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Based on the target domain knowledge graph, the directed paths corresponding to the sample input text are determined to form a knowledge path set using the entity correspondence group set, specifically including: The target domain entities in the entity corresponding group set are formed into entity pairs, and the first target domain entity and the second target domain entity in the entity pair are used as endpoint entities. The target domain entities in the entity pair are retrieved using the target domain knowledge graph to obtain all directed paths from the first target domain entity to the second target domain entity; According to the number of entities in each directed path and the set number, all directed paths are screened to determine the directed paths corresponding to the sample input text to form a knowledge path set.

5. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Generating a set of candidate knowledge prompt words corresponding to the first training sample using the first language model specifically includes: Input a knowledge path set composed of a sample input text and a directed path corresponding to the sample input text into a first language model to obtain a natural language interpretation, a reasoning path and a first mind map; Inputting the sample input text and the natural language interpretation into the first language model to obtain a second mind map; Inputting the sample input text and the reasoning path into the first language model to obtain a third mind map; The sample input text and the first mind map are input into the first large language model to obtain a fourth mind map; the first mind map, the second mind map, the third mind map and the fourth mind map constitute a knowledge candidate prompt word set corresponding to the first training sample.

6. The method for generating knowledge graph enhanced prompt words according to claim 1, characterized in that: Determining the final candidate knowledge prompt word corresponding to the sample input text according to the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located specifically includes: For each mind map in the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located, determine the knowledge graph enhancement prompt word corresponding to the mind map according to the mind map and the first training sample; Inputting the knowledge graph enhancement prompt words corresponding to the mind map into the first language model to obtain an output answer corresponding to the mind map; Inputting the output answer corresponding to each of the mind maps and the standard answer corresponding to the sample input text into the BERT embedding model respectively, to obtain the output answer embedding code and the standard answer embedding code corresponding to each of the mind maps; The final mind map is determined according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each of the mind maps; the final mind map is the final candidate knowledge prompt word corresponding to the sample input text.

7. The method for generating knowledge graph enhanced prompt words according to claim 6, characterized in that: Determining the final mind map according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each mind map specifically includes: Calculate the similarity corresponding to each mind map according to the embedded code of the standard answer and the embedded code of the output answer corresponding to each mind map; The mind map corresponding to the similarity with the largest value is determined as the final mind map.

8. A knowledge graph enhanced prompt word generation device, characterized in that: The knowledge graph enhanced prompt word generating device comprises: The text acquisition module is used to: acquire sample input text; An entity recognition module is used to: perform entity recognition on the sample input text to obtain a key entity mention set; the key entity mention set includes a number of key entity mentions; the key entity mentions are target domain keywords; An entity linking module is used to: perform entity linking on the key entity mention set and all target domain entities in the target domain knowledge graph to determine a set of entity corresponding groups; each entity corresponding group set includes a plurality of entity corresponding groups, and each entity corresponding group includes a key entity mention and a target domain entity; A path retrieval module is used to: determine the directed path composition knowledge path set corresponding to the sample input text based on the target domain knowledge graph using the entity correspondence group set; the two endpoint entities in the directed path composition knowledge path set are both target domain entities in the entity correspondence group set; The knowledge candidate prompt word set generation module is used to: for each first training sample in the training data set, generate a knowledge candidate prompt word set corresponding to the first training sample using a first large language model; the knowledge candidate prompt word set includes a plurality of mind maps; each of the first training samples includes a sample input text, a knowledge path set composed of directed paths corresponding to the sample input text, and a standard answer corresponding to the sample input text; The final candidate knowledge prompt word determination module is used to: for each sample input text, determine the final candidate knowledge prompt word corresponding to the sample input text according to the knowledge candidate prompt word set corresponding to the first training sample where the sample input text is located; A training module is used to: train the second language model using a pseudo-prompt training data set to obtain a knowledge prompt optimizer; the pseudo-prompt training data set includes a plurality of second training samples, each of which includes a sample input text, a final candidate knowledge prompt word corresponding to the sample input text, and a standard answer corresponding to the sample input text; The knowledge prompt word generation module is used to: generate knowledge prompt words using a knowledge prompt optimizer.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the knowledge graph enhanced prompt word generation method described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the knowledge graph enhanced prompt word generation method described in any one of claims 1-7.

Images