Metacognition and multi-angle prompt-based large language model knowledge graph completion method

Abstract

The invention discloses a large language model knowledge graph completion method based on meta-cognition and multi-angle prompt, relates to the technical field of knowledge graph completion, and solves the problem of magical property caused when knowledge graph completion is solved by adopting prompt engineering in the prior art. The knowledge graph completion method comprises the following steps of performing data extraction on a knowledge graph data set in a medical database, constructing an original knowledge graph structure of medical data according to the extracted data, performing initial embedding on the original knowledge graph structure, and enhancing embedding generation; optimizing the large language model by adopting a meta-cognition prompting method and a multi-angle prompting method; and enriching the enhanced and embedded original mapping knowledge domain structure by adopting the optimized large language model, wherein the enriched mapping knowledge domain contains new relationships and entities. The method is suitable for completing the knowledge graph in the medical database in the medical field.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph completion, and in particular to a large language model knowledge graph completion method based on metacognition and multi-angle prompts. Background Art

[0002] The demand for knowledge graphs in the medical field is growing, especially in auxiliary diagnosis and treatment, medical knowledge management, etc. For example, in auxiliary diagnosis and treatment, knowledge graphs can provide structured and associated knowledge sharing and utilization capabilities, especially in the field of medical diagnosis where knowledge is updated rapidly, which can relieve the memory and analysis pressure of clinicians. Through knowledge graphs, doctors can quickly obtain all-round information such as patients' disease history, family history, medication status, etc., providing strong support for diagnosis and treatment. In addition, knowledge graphs can also assist in imaging diagnosis, and through intelligent image analysis, improve the accuracy of the conversion between images, semantics / knowledge, and diagnostic reports. In terms of medical knowledge management, knowledge graphs can integrate scattered medical data and literature to help researchers discover new treatments and drugs. It can integrate heterogeneous data from different sources, such as electronic medical records, medical literature, clinical guidelines, and patient-generated data, to form a more comprehensive and accurate knowledge representation.

[0003] Knowledge graph (KG) constructs a semantic network by describing entities and their relationships, providing semantic support for tasks such as information retrieval and intelligent question answering. However, knowledge graphs usually face the problem of incomplete data, and knowledge graph completion (KGC) is committed to filling these knowledge gaps and ensuring the completeness and accuracy of knowledge graphs.

[0004] Common methods for knowledge graph completion can be divided into two categories: structured and description-based completion methods. Structured methods focus on using the topological structure of the graph to infer missing relationships, such as predicting unknown connections through embedding models or path reasoning; while descriptive methods use text descriptions of entities and relationships and natural language processing technology to reason from a semantic perspective. Although description-based knowledge graph construction (KGC) has made significant progress in learning text and structured knowledge, its effectiveness is still limited by text quality and structural incompleteness. For example, in Figure 1In the article, Gary Roger Rydstrom is described as an American sound designer and director who has won seven Academy Awards for sound effects and directed the Oscar-nominated film Meet Joe Black. However, relying solely on the description of the relation "award" may cause a large language model to incorrectly infer that Rydstrom won an Oscar for directing, thereby causing a bias in understanding entity types. In addition, for long-tail entities, there are also challenges in learning structured patterns from existing graphs. These limitations make it difficult for KGC to achieve ideal performance in practical applications when dealing with insufficient and incomplete knowledge graphs.

[0005] In recent years, the application of prompt engineering in knowledge graph completion has received widespread attention. Prompt engineering guides large language models to solve specific problems through carefully designed prompts, providing a new path to improve the effect of knowledge graph completion. In this context, reasoning strategies such as Chain of Thought (CoT) and Tree of Thought (ToT) have also emerged. Among them, Chain of Thought enhances the model's logical reasoning ability in solving complex problems through a step-by-step reasoning process, while Tree of Thought further guides the model to explore different reasoning possibilities from multi-branch paths.

[0006] The above models and strategies have shown significant effects in specific environments, especially in improving the model's explicit reasoning ability, such as arithmetic, common sense, and symbolic reasoning. However, the effectiveness of these methods in deepening understanding is relatively limited, because reasoning tasks usually involve systematically connecting different concepts, and true understanding requires mastering the underlying semantics as well as broader contextual information. Conventional prompt engineering methods may cause "hallucination" in large language models during the process of guiding reasoning, that is, the model generates incorrect or unreasonable reasoning results when the information is insufficient or misleading. Summary of the invention

[0007] The present invention aims to solve the hallucinogenic problem caused by the existing use of prompt engineering to solve the knowledge graph completion.

[0008] To achieve the above object, the present invention provides the following solutions: The present invention provides a large language model knowledge graph completion method based on metacognition and multi-angle prompts, and the knowledge graph completion method comprises the following steps: Step S1: extract data from the knowledge graph dataset in the medical database, construct the original knowledge graph structure of the medical data based on the extracted data, and after initially embedding the original knowledge graph structure, enhance the generation of embedding; Step S2: using metacognitive prompting method and multi-angle prompting method to optimize the large language model; Step S3: Use the optimized large language model to enrich the original knowledge graph structure after enhanced embedding, and the enriched knowledge graph contains new relations and entities.

[0009] Furthermore, in a preferred embodiment, the above-mentioned knowledge graph dataset can be selected as MIMIC-III dataset, FB15k237 or WN18RR.

[0010] Furthermore, there is another preferred embodiment, the above step S1 is specifically as follows: Step S11: extract triple data from the MIMIC-III dataset, establish a mapping from head entity and relationship to tail entity, and construct the original knowledge graph structure of medical data; Step S12: Use the TransE algorithm to perform initial embedding on the original knowledge graph structure; Step S13: Input the triple data into the Transformer encoder, generate context-aware embedding, and then calculate it through the Transformer to obtain the final dynamic relation embedding; Step S14: Calculate the neighbor embedding of any entity; Step S15: Generate a fused relation embedding based on the final dynamic relation embedding and neighbor embedding to complete the enhanced embedding.

[0011] Furthermore, there is a preferred embodiment in which the above-mentioned original knowledge graph structure is a triple structure, including entities and relationships.

[0012] Furthermore, in a preferred embodiment, the above-mentioned metacognitive prompting method is used to guide the large language model to solve a specific problem; The above multi-angle prompting method is used to enrich the knowledge graph structure through entities, relations, passives and keywords.

[0013] Furthermore, there is another preferred embodiment, the above step S2 is specifically as follows: The metacognitive prompting method is embedded into the large language model, wherein the metacognitive prompting method includes: understanding the input text, making a preliminary judgment and giving a confidence through the prompt words, revising or confirming the answer according to the confidence, giving a final decision and explaining the reasoning process; The multi-angle prompting method is embedded into the large language model. The multi-angle prompting method includes: entity part: using a sentence to describe the comprehensive information of each entity; relationship part: explaining and predicting the relationship information between each entity, converting the relationship information into verbs and expressing it in passive voice; structure part: extracting keywords from the description generated by the entity part, and calculating the similarity between entities based on the keyword set S , reorder the entities according to their similarity scores and recreate new triples in the form of {head, relation, tail}.

[0014] Furthermore, there is a preferred embodiment in which the similarity for:

[0015] in, and Represents the set of head and tail entity keywords respectively, and Respectively represent the weights of each keyword in the intersection and union.

[0016] The present invention also provides a large language model knowledge graph completion system based on metacognition and multi-angle prompts, which is used to implement the large language model knowledge graph completion method based on metacognition and multi-angle prompts described above. The system includes an extraction and construction module, a large language model prompt module, and an enhancement and improvement module; The extraction and construction module is used to extract data from the knowledge graph dataset in the medical database, and to construct the original knowledge graph structure of the medical data based on the extracted data. It is also used to enhance the generation of embedding after the initial embedding of the original knowledge graph structure. The large language model prompt module is used to enrich and optimize the original knowledge graph structure of medical data through metacognitive prompting methods and multi-angle prompting methods; The enhancement and improvement module is used to enhance and improve the enriched and optimized knowledge graph structure through the KGC model.

[0017] The large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the present invention can be fully implemented using computer software. Therefore, correspondingly, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes any one of the large language model knowledge graph completion methods based on metacognition and multi-angle prompts described above.

[0018] The present invention also provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes any one of the above-mentioned large language model knowledge graph completion methods based on metacognition and multi-angle prompts.

[0019] The beneficial effects of the present invention are: The present invention provides a large language model knowledge graph completion method based on metacognition and multi-angle prompts, which extracts data from the knowledge graph data set in the medical database, constructs the original knowledge graph structure of the medical data according to the extracted data, and enhances the generation of embedding after the original knowledge graph structure is initially embedded; then the large language model is optimized by the metacognitive prompt method and the multi-angle prompt method, and the optimized large language model is used to enrich the original knowledge graph structure after the enhanced embedding, and the rationality of the reasoning process is evaluated by the metacognitive prompt to ensure the accuracy of the model output; the multi-angle prompt guides the model to understand the complex relationship in the knowledge graph from multiple perspectives, so that the knowledge graph completion task in the medical database can be better completed. Furthermore, the present invention introduces the metacognitive prompt method into the knowledge graph completion part of the natural language processing model, thereby improving the model function, making it go beyond simple system reasoning, and prompting it to participate in the introspective evaluation of the depth and relevance of the reaction, thereby solving the hallucinogenicity problem of the large language model.

[0020] 2. While the present invention uses a metacognitive prompting method to solve the hallucination problem of a large language model, it cannot solve the problem of incomplete entity information and ambiguous relationships. Therefore, the present invention designs a chain reasoning, that is, by adding a multi-angle prompting method to prompt the large language model, and uses its knowledge to expand the entity description to solve the problem of incomplete entity information.

[0021] Furthermore, the present invention enriches the knowledge graph structure through entities, relationships, passives and keywords, namely: the entity method is mainly a further description of entity information. The relationship method is mainly an expansion of the relationship information in the {entity 1, relationship, entity 2} structure. The passive method refers to converting the description information of {entity 1, relationship, entity 2} into a passive sentence, and changing the structure to {entity 2, passive relationship, entity 1}. The keyword method mainly extracts the entity description information into corresponding keywords. It enables better capture of the association between relationships, thereby promoting more effective reverse prediction, thereby solving the problem of relationship ambiguity.

[0022] Furthermore, the present invention combines the additional entity information generated by metacognitive prompts with the large language model, measures the similarity between entities by summarizing keywords, and creates a new triple model to further enrich the knowledge graph, thereby solving the problem of sparse graph links.

[0023] The present invention is applicable to knowledge graph completion in medical databases in the medical field. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flowchart of the descriptive knowledge graph completion method mentioned in the background technology; Figure 2 It is a flowchart of improving and enriching the knowledge graph structure through entities, relationships, passives and keywords according to the present invention; Figure 3 It is a flowchart of the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the present invention; Figure 4 is the link prediction experiment result of the present invention; Figure 5 is the triple classification experiment result of the present invention; Figure 6 It is a performance comparison chart of the knowledge graph completion method proposed in the present invention and the LMKE knowledge graph completion method on different language models; Figure 7 This is a performance comparison chart of the knowledge graph completion method proposed in the present invention and the SimKGC knowledge graph completion method on different language models. DETAILED DESCRIPTION

[0026] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings. The following implementation will help those skilled in the art to further understand the present invention, but will not limit the present invention in any form.

[0027] Implementation method 1. This implementation method aims at solving the hallucinogenic problem caused by the existing use of prompt engineering to complete the knowledge graph, which makes it impossible to complete the knowledge graph in the medical database. Therefore, a large language model knowledge graph completion method based on metacognition and multi-angle prompts is proposed. This method can be used to complete the knowledge graph in the medical database and realize the perfect management of auxiliary diagnosis and treatment, medical knowledge management and other aspects.

[0028] The knowledge graph completion method comprises the following steps: Step S1: extract data from the knowledge graph dataset in the medical database, construct the original knowledge graph structure of the medical data based on the extracted data, and after initially embedding the original knowledge graph structure, enhance the generation of embedding; Step S2: using metacognitive prompting method and multi-angle prompting method to optimize the large language model; Step S3: Use the optimized large language model to enrich the original knowledge graph structure after enhanced embedding, and the enriched knowledge graph contains new relations and entities.

[0029] This implementation is applicable to the medical field in practical applications. In the medical knowledge graph, entities include diseases, symptoms, drugs, etc., and relationships may include causes, symptom manifestations, treatment methods, etc. This implementation generates detailed descriptions of diseases, such as causes, symptoms, preventive measures, etc., as well as the mechanism of action and side effects of drugs, through an optimized large language model, thereby completing the missing information in the knowledge graph. For example, for a rare disease, the optimized large language model can generate detailed information about its symptoms, diagnostic methods, treatment plans, etc., to provide doctors with auxiliary diagnosis and treatment support.

[0030] Implementation Method 2: See Figure 2 This embodiment is described. This embodiment specifically describes the steps of the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the first embodiment above. Step S1: extract data from the knowledge graph dataset in the medical database, construct the original knowledge graph structure of the medical data based on the extracted data, and after initially embedding the original knowledge graph structure, enhance the generation of embedding; Take the MIMIC-III dataset as an example: Specifically: Step S11: extract triple data from the MIMIC-III dataset, establish a mapping from head entity and relationship to tail entity, and construct the original knowledge graph structure of medical data; Step S12: Use the TransE algorithm to perform initial embedding on the original knowledge graph structure; Step S13: Input the triple data into the Transformer encoder, generate context-aware embedding, and then calculate it through the Transformer to obtain the final dynamic relation embedding; Step S14: Calculate the neighbor embedding of any entity; Step S15: Generate a fused relation embedding based on the final dynamic relation embedding and neighbor embedding to complete the enhanced embedding.

[0031] In practical application, it is more specific as follows: The MIMIC-III dataset (containing patient electronic health records) was selected to construct the initial medical knowledge graph. The knowledge graph contains entities (such as diseases, drugs, symptoms, treatment methods) and relations (such as "treatment", "cause").

[0032] Knowledge graph construction and initial embedding: The knowledge graph G consists of M triplets (h, r, t), where: h∈ : Head entity, representing concepts in the medical field, such as the disease "hypertension".

[0033] r∈R: Relation, indicating the association between entities, such as "treatment".

[0034] t∈ε: tail entity, such as drug “Drug A”.

[0035] Define the entity set ε={e1,e2,…,e n}, the relation set R={r1,r2,…,r m}, where n and m are the number of entities and relations, respectively.

[0036] Use the TransE algorithm to initially embed the knowledge graph and generate: The entity embedding vector set ε′={e1′,e2′,…,e n ′}, The relation embedding vector set R′={r1′,r2′,…,r m ′}.

[0037] The formula is as follows: ∣∣e h ′+r′−e t ′∣∣ 2 , so that h,r,t∈G Division of task data sets: based on task relationship s , the atlas data is divided into: Reference Set S r : Extract K entity pairs from known positive triples; Query set Q r : Extract Z entity pairs from the remaining positive triples; Negative sample set F r :Randomly generate negative samples (h, r s , t′), where t′∈ε.

[0038] Enhanced embedding generation: Dynamic relational embedding computation Input the triple (h, r, t) to the Transformer encoder to generate context-aware embedding: X={x i1 ,x i2 ,x i3}, where x i1 =e h ′, x i2 =r′, x i3 =e t ' After L layers of Transformer calculations, the final dynamic relationship embedding is obtained: Rel dynamic =C l (x i1 ,x i2 ,x i3 ) Neighborhood-aware embedding computation for any entity e i , and compute its neighbor embeddings:

[0039] Among them, n j for e i The j-th one-hop neighbor of .

[0040] The fused enhanced embedding combines the task relation embedding Reldynamic and the neighbor embedding Relh−inb to generate a fused relation embedding:

[0041] Among them, λ is a learnable parameter, which is dynamically adjusted according to the neighbor information loss.

[0042] An optimized large language model is used to generate multi-angle question prompts and the confidence score of the model output is calculated; the confidence scores of the candidate tail entities t′ output by the model are sorted to generate a priority list.

[0043] Similarity calculation and optimization: Define the semantic similarity between the query triple and the reference triple:

[0044] The model is optimized by minimizing the following loss function:

[0045] Among them, γ is the safety margin distance.

[0046] Missing triple completion: For the missing triple (h, r, ?), traverse the candidate entity set ε and calculate the completion score of each candidate entity:

[0047] Select the entity with the highest score as the tail entity to complete the completion: (h , r , t′)→(hypertension, treatment, drug A) Finally, complete and evaluate the entire graph: Traverse all missing relations in the knowledge graph and repeat the completion process. Evaluate the accuracy and recall of the completion results by comparing them with the true labels.

[0048] For FB15k-237 and WN18RR datasets: Specifically: 1. Data loading and preprocessing: The structures of FB15k-237 and WN18RR contain a large amount of triples. The entity information is loaded from the JSON file and stored as a quickly accessible index structure, which also supports filtering entities that appear in a specific test set. At the same time, triple data is loaded from JSON to establish a mapping from head entity, relationship to tail entity.

[0049] 2. Knowledge graph construction: Use triples to generate an undirected graph, where entities are nodes and relationships are edges connecting entities at both ends. At the same time, the constructed graph is stored in the form of an adjacency list.

[0050] 3. Graph structure analysis: Through the neighbor query method, the direct neighbors of an entity are obtained and the number of neighbors is returned. At the same time, all related entities within the n-hop range of an entity are found through breadth-first search (BFS) to facilitate fast query.

[0051] Through the above three steps, data can be extracted from the two knowledge graph datasets FB15k-237 and WN18RR and the original knowledge graph structure of medical data can be constructed.

[0052] Step S2: using metacognitive prompting method and multi-angle prompting method to optimize the large language model; The metacognitive prompting method is used to guide the large language model to solve a specific problem: Specifically, the metacognitive prompt method is embedded into the large language model. The metacognitive prompt method includes: understanding the input text, making a preliminary judgment and giving confidence through the prompt words, revising or confirming the answer according to the confidence, giving the final decision and explaining the reasoning process. The specific process is shown in Table 1 below: Table 1

[0053] As can be seen from Table 1, the metacognitive prompting method is a process used to guide the large language model to solve a specific problem, namely, four stages: (1) understanding the input text; (2) making preliminary judgments and evaluating them; (3) correcting or confirming the answer; and (4) making a final decision and explaining the reasoning process.

[0054] The multi-angle prompting method is used to enrich the knowledge graph structure through entities, relations, passives and keywords.

[0055] Specifically, a multi-angle prompting method is embedded into a large language model. The multi-angle prompting method includes: entity part: using a sentence to describe the comprehensive information of each entity; relation part: explaining and predicting the relationship information between each entity, converting the relationship information into verbs and expressing it in passive voice; structure part: extracting keywords from the description generated by the entity part, and calculating the similarity between entities based on the keyword set, reordering the entities according to the similarity scores, and recreating new triples in the form of {head, relation, tail}. The specific process is shown in Table 2 below: Table 2

[0056] As can be seen from Table 2, MMPP-E, MMPP-R, MMPP-P and MMPP-K represent the improvement methods in terms of entities, relations, passives and structures, respectively. That is, the multi-angle prompting method is used to improve and enrich the knowledge graph structure through entities, relations, passives and keywords.

[0057] Among them, MMPP-E (entity part) pays more attention to the importance of entities and uses shorter and more concise description sentences to focus on the comprehensive information of each entity. Figure 3 MMPP-E (the entity part) generates a brief but fully informative description of the entity: "Meet Joe Black (The Sixth Sense of Life and Death) is a 1998…". This approach highlights the key information of the entity and makes the description concise and representative.

[0058] Different from MMPP-E (entity part), MMPP-R (relation part) is more inclined to the connection between triples, such as Figure 2As shown in , MMPP-R (relation part) generates a semantically rich description for the triple {Gary Roger Rydstrom, occupation, Sound Designer}, emphasizing the specific meaning of the relation word "occupation". In this process, MMPP-R (relation part) not only predicts the missing information, but also suggests other possible associations. For example: for the triple {Gary Roger Rydstrom, occupation, Sound Designer}, MMPP-R (relation part) will point out that "Gary Roger Rydstrom" is a "Sound Designer"; while in the description of {Gary Roger Rydstrom, award, Oscars}, it mentions that he has won the Academy Award for Best Sound Effects 7 times. Through this kind of relational description, MMPP-R (relation part) can avoid the incorrect inference of the large language model that "Gary Roger Rydstrom" won the Oscar for Best Director for directing the movie "Meet Joe Black".

[0059] In addition, MMPP-P (relation part) also requires the large language model to represent relations in passive voice and rewrite the description of triples into passive form. For example, Figure 2 The word “award” in the sentence can be converted to “be awarded to”, thus generating the description “The Oscar for Best Sound was awarded to Gary Roger Redstrom”. This strategy improves the understanding ability of large language models and optimizes the backward prediction effect.

[0060] As for MMPP-K (structure part), it is an auxiliary mechanism for MMPP-E (entity part). The main idea of ​​MMPP-K (structure part) is to extract keywords from the simple description generated by MMPP-E (entity part) and calculate the similarity between entities based on the keyword set. :

[0061] in and Represents the set of head and tail entity keywords respectively, through To determine the keywords that both have in common, to identify all the different keywords that both have in common. and Represents the weight of each keyword in the intersection and union respectively. Through this algorithm, the similarity scores are sorted and the top Yes, recreate new triplets in the form of {head, relation, tail} and add them to the training set to enrich the overall graph structure.

[0062] To sum up, the multi-angle prompt method proposed in this embodiment only enhances the knowledge graph completion task from the three parts of entity, relationship and structure, and does not provide a detailed description of each entity. This is because a detailed description of each entity will make each problem concept unclear. This is because the sentences describing the entity are lengthy and contain too many unnecessary details, resulting in information overload, which not only increases the complexity of the model's inference, but may also reduce the accuracy of its output. Therefore, the multi-angle prompt method proposed in this embodiment only enhances the knowledge graph completion task from the three parts of entity, relationship and structure, and describes the entity information with fewer and more refined sentences, so as to minimize the possibility of information overload.

[0063] Implementation method three, see Figure 3 This embodiment is described. This embodiment provides a large language model knowledge graph completion system based on metacognition and multi-angle prompts. The system is used to implement the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the above embodiment. The system includes an extraction and construction module, a large language model prompt module and an enhancement and improvement module; The extraction and construction module is used to extract data from the knowledge graph dataset in the medical database and construct the original knowledge graph structure of the medical data based on the extracted data; The large language model prompt module is used to enrich and optimize the original knowledge graph structure of medical data through metacognitive prompting methods and multi-angle prompting methods. It is also used to enhance the generation of embedding after the initial embedding of the original knowledge graph structure. The enhancement and improvement module is used to enhance and improve the enriched and optimized knowledge graph structure through the KGC model.

[0064] In practical application, if Figure 3 As shown in the figure, the overall framework is divided into three modules from left to right: extraction and construction, large language model prompts, and promotion and improvement. In the extraction and construction module, data is first extracted from the two data sets FB15k237 and WN18RR and the original knowledge graph structure is constructed. Then, in the large language model prompt module, multiple large language models are optimized using metacognitive prompts and multi-angle prompts. Finally, on this basis, the knowledge graph is expanded to include new relations and entities, and the enhanced knowledge graph is tested through the KGC model in the promotion and improvement module.

[0065] Implementation method 4: The large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the above implementation method can be implemented entirely by computer software. Therefore, correspondingly, this implementation method provides a large language model knowledge graph completion system based on metacognition and multi-angle prompts, and the system includes a storage device, which is used to perform the following steps: Step S1: extract data from the knowledge graph dataset in the medical database, construct the original knowledge graph structure of the medical data based on the extracted data, and after initially embedding the original knowledge graph structure, enhance the generation of embedding; Step S2: using metacognitive prompting method and multi-angle prompting method to optimize the large language model; Step S3: Use the optimized large language model to enrich the original knowledge graph structure after enhanced embedding, and the enriched knowledge graph contains new relations and entities.

[0066] Implementation method five: This implementation method provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in any one of the above implementation methods is executed.

[0067] Implementation method six: This implementation method provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the large language model knowledge graph completion method based on metacognition and multi-angle prompts as described in any one of the above implementation methods.

[0068] A computer device is provided in this embodiment. The hardware device of this part is a general model and is not shown in the form of a diagram. The system includes a processor and a memory, wherein the processor and the memory can be connected via a bus or other means. The memory is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, and corresponding program instructions / modules. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, so as to realize the large language model knowledge graph completion method and steps based on metacognition and multi-angle prompts in the above method embodiment.

[0069] Implementation method 7: See Figures 4 to 7 This embodiment is to describe the experimental description of the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the above embodiment: Link prediction experiments and triple classification experiments were conducted on two widely used medical databases. The two datasets are FB15k237 and WN18RR. They are shown in Table 3 below: Table 3

[0070] The following metrics are used in the model performance evaluation. In the link prediction task, Mean Rank, MR (mean ranking), Mean Reciprocal Rank, MRR (mean reciprocal ranking) and Hits@n, H@n (hit rate) are used; in the triple classification task, the traditional accuracy (acc) is used to measure the performance of the model.

[0071] In the selection of large language models, the latest large language models are selected: Llama-2 (Llama-2-7b-chat)), ChatGLM2-6B, ChatGPT (gpt-3.5-turbo-0613) and GPT4 (gpt-4-0613).

[0072] Among them, GPT4 is used as the large language model for our main data. For all models, the temperature is set to 0.2 and the maximum length is set to 256. Other hyperparameters are set according to the corresponding models. In terms of equipment, all experiments are conducted on NVIDIA (GeForce RTX 4090D). Considering the computing power of the device, the experiments on the LMKE model and SimKGC are subject to certain limitations, so we changed the batch-size to 64 and 256 respectively to adapt to the model.

[0073] Get as Figure 4 The link prediction experiment results shown in Figure 4 The best result is indicated by "", and the second best result is indicated by an underscore. ↓ indicates that the lower the value, the better the effect, and ↑ indicates that the higher the value, the better the effect.

[0074] from Figure 4 It can be seen that the large language model knowledge graph completion method based on metacognition and multi-angle prompts proposed in this embodiment performs better than the original results of other benchmark models on the FB15k237 dataset, which shows that introducing the metacognitive prompt method into the natural model does improve the model performance to a certain extent, mainly due to the metacognitive prompt method's stronger reasoning ability, more comprehensive semantic understanding and flexibility of dynamic adjustment.

[0075] On the WN18RR dataset, the large language model knowledge graph completion method based on metacognition and multi-angle prompts proposed in this paper performs best. Thanks to the advantages of the multi-angle prompt method in deep utilization of semantic information, expansion of structural information, and effective modeling of antonym relations, these factors make it more outstanding in processing a large number of abstract concepts and complex semantic relations in the WN18RR dataset. In contrast, the adaptability and information richness of MMPP-MP and MMPP-K on this type of dataset are relatively insufficient.

[0076] In summary, the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in this embodiment has achieved significant improvements compared to other existing benchmark models, especially in the H@3 index, with a maximum improvement of 0.74%. On the WN18RR dataset, our improvement is even more obvious, with the H@10 index score reaching 81.47%, an increase of 1.05%. This proves the effectiveness of the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in this embodiment in link prediction.

[0077] Get as Figure 5 The triple classification experimental results shown are from Figure 5 It can be seen that the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in this embodiment achieved a good result of 99.831% on the WN18RR dataset.

[0078] Furthermore, this embodiment uses a large language model knowledge graph completion method based on metacognition and multi-angle prompts for testing on both benchmark models. Figure 6 and Figure 7 As shown in the figure, the results show that GPT-4 performs best among all large language models, thanks to its larger scale and rich knowledge base. In addition, Llama-2 and GPT-3.5 also perform relatively well, which shows that no matter what large language model is used, our method can effectively improve the LMKE-based KGC task and verify the effectiveness of the designed prompts.

[0079] In summary, the large language model knowledge graph completion method based on metacognition and multi-angle prompts described in the above implementation aims to minimize the "hallucinogenic" tendency of the large language model and improve its semantic understanding ability through innovative designs in terms of entity, relationship, reverse reasoning and keyword improvement prompts. We evaluated the link prediction and triple classification performance of the knowledge graph completion method on the FB15k237 and WN18RR datasets. The experimental results show that compared with the existing models, the knowledge graph completion method proposed in this implementation shows a significant performance improvement.

[0080] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may be modified and varied in various ways.

Claims

1. A large language model knowledge graph completion method based on metacognition and multi-angle prompts, characterized in that: The method is: S1: Extract data from the knowledge graph dataset in the medical database, construct the original knowledge graph structure of the medical data based on the extracted data, and after the initial embedding of the original knowledge graph structure, enhance the generation of embedding; S2: Use metacognitive prompting and multi-angle prompting methods to optimize the large language model; S3: The optimized large language model is used to enrich the original knowledge graph structure after enhanced embedding. The enriched knowledge graph contains new relations and entities.

2. The large language model knowledge graph completion method based on metacognition and multi-angle prompts according to claim 1 is characterized in that: The knowledge graph dataset is the MIMIC-III dataset, FB15k237, or WN18RR.

3. The large language model knowledge graph completion method based on metacognition and multi-angle prompts according to claim 2 is characterized in that: S1 is specifically: S11: Extract triple data from the MIMIC-III dataset, establish a mapping from head entity and relationship to tail entity, and construct the original knowledge graph structure of medical data; S12: Use the TransE algorithm to perform initial embedding on the original knowledge graph structure; S13: Input the triple data into the Transformer encoder to generate context-aware embedding, and then calculate it through the Transformer to obtain the final dynamic relation embedding; S14: Calculate the neighbor embedding of any entity; S15: Generate fused relation embedding based on the final dynamic relation embedding and neighbor embedding to complete the enhanced embedding.

4. The large language model knowledge graph completion method based on metacognition and multi-angle prompts according to claim 1 is characterized in that: The original knowledge graph structure is a triple structure, including entities and relationships.

5. The large language model knowledge graph completion method based on metacognition and multi-angle prompts according to claim 1 is characterized in that: The metacognitive prompting method is used to guide the large language model to solve a specific problem; The multi-angle prompting method is used to enrich the knowledge graph structure through entities, relations, passives and keywords.

6. The large language model knowledge graph completion method based on metacognition and multi-angle prompts according to claim 5 is characterized in that: S2 is specifically: The metacognitive prompting method is embedded into the large language model, wherein the metacognitive prompting method includes: understanding the input text, making a preliminary judgment and giving a confidence through the prompt words, revising or confirming the answer according to the confidence, giving a final decision and explaining the reasoning process; The multi-angle prompting method is embedded into the large language model. The multi-angle prompting method includes: entity part: using a sentence to describe the comprehensive information of each entity; relationship part: explaining and predicting the relationship information between each entity, converting the relationship information into verbs and expressing it in passive voice; structure part: extracting keywords from the description generated by the entity part, and calculating the similarity between entities based on the keyword set , reorder the entities according to their similarity scores and recreate new triples in the form of {head, relation, tail}.

7. The large language model knowledge graph completion method based on metacognition and multi-angle prompts according to claim 6 is characterized in that: Similarity for: in, and Represents the set of head and tail entity keywords respectively, and Represent the weights of each keyword in the intersection and union respectively.

8. A large language model knowledge graph completion system based on metacognition and multi-angle prompts, characterized by: The system includes an extraction and construction module, a large language model prompt module, and an enhancement and improvement module; The extraction and construction module is used to extract data from the knowledge graph dataset in the medical database, and to construct the original knowledge graph structure of the medical data based on the extracted data. It is also used to enhance the generation of embedding after the initial embedding of the original knowledge graph structure. The large language model prompt module is used to enrich and optimize the original knowledge graph structure after enhanced embedding through metacognitive prompting methods and multi-angle prompting methods; The enhancement and improvement module is used to enhance and improve the enriched and optimized knowledge graph structure through the KGC model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the large language model knowledge graph completion method based on metacognition and multi-angle prompts as described in any one of claims 1 to 7.

10. A computer device, characterized in that: The device includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the large language model knowledge graph completion method based on metacognition and multi-angle prompts as described in any one of claims 1-7.

Images