Knowledge graph enhanced expert information retrieval prompt project self-generation method and device

Abstract

The invention discloses a knowledge graph enhanced expert information retrieval prompt project self-generation method and device, and the method comprises the steps: extracting a keyword of a request input by a user through employing a pre-trained large language model, and carrying out the intention understanding analysis of the keyword based on a vertical domain expert knowledge graph, and obtaining a knowledge graph retrieval result; the search intention of the user is redescribed by utilizing a knowledge graph retrieval result, and a plurality of prompt statements for task execution are generated in combination with the calling capability of a large language model tool; scoring the accuracy of the plurality of prompt statements, and inputting a scoring result into a large language model agent for query and task execution response to obtain a query result containing expert information feature data; and generating a prompt statement capable of guiding multi-modal expert information fusion expression according to expert information feature data contained in the query result. According to the method, the vertical domain service capability of the general large language model is effectively expanded, and the method has an important value for promoting the application of a generative artificial intelligence technology in the field of experts.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence retrieval information enhancement, and specifically to a method and device for self-generating an expert information retrieval prompt engineering enhanced by a knowledge graph. Background Art

[0002] Currently, the pre-trained large language models are mainly based on general retrieval capabilities, which supplement the expert domain knowledge background reserves, and the understanding and query accuracy of expert information is poor. The method of manually writing prompt words to enhance the output results of the large model is inefficient and has high requirements on personnel's query skills, making it difficult to form large-scale engineering applications.

[0003] The mainstream knowledge augmented retrieval method is mainly based on RAG (Retrieval Augmented Generation), which segments long texts according to fixed rules and uses an encoder model to convert text paragraphs into vector form. When a user initiates a query request, the query request will be converted into a vector according to the same encoder model, and then the distance between vectors will be calculated using the Euclidean norm, inner product or cosine similarity to find the most relevant text fragments, thereby reducing the hallucination output generated when general large models perform expert information retrieval. However, the method of matching knowledge fragments based on RAG technology has problems such as slow matching query speed and limited knowledge base coverage. In the production application process, the expert information involved in the business is often not only related to a single scenario, and the knowledge augmented retrieval method of traditional RAG technology has limited applicability. Summary of the invention

[0004] The present invention provides a method and device for self-generating expert information retrieval prompt engineering enhanced by a knowledge graph. The technical problem it solves is the difficulty of accurate retrieval of expert field information by a large language model. A knowledge graph-enhanced prompt engineering method is designed to automatically generate prompt sentences through structural design of the knowledge graph, retrieval enhancement and intent description scoring, to assist the large language model in accurately understanding the user's intent, and to call special tools to obtain data, knowledge and other information in the expert field.

[0005] The specific technical solutions of the present invention are as follows:

[0006] A method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph, comprising the following steps:

[0007] Step 1: Search intent understanding: Use the pre-trained large language model to extract the keywords of the user's input request, perform intent understanding analysis on the keywords based on the vertical domain expert knowledge graph, and obtain the knowledge graph retrieval results;

[0008] Step 2: Restate the search intent: Use the knowledge graph retrieval results obtained in step 1 to restate the user's search intent, and combine the large language model tool calling capabilities to generate multiple prompt statements for task execution;

[0009] Step 3, expert information retrieval: score the accuracy of the multiple prompt sentences generated in step 2, and input the score results into the large language model agent for query and task execution response, and obtain the query results containing expert information feature data;

[0010] Step 4, retrieval result generation: Based on the expert information feature data contained in the query result of step 3, a prompt sentence is generated that can guide the fusion expression of multimodal expert information to support the realization of the final retrieval result output.

[0011] Furthermore, step 1 specifically includes:

[0012] Construct a two-layer graph structure including knowledge nodes and knowledge fragment nodes, where knowledge nodes are first-level nodes, including entity concepts extracted from information sources; knowledge fragment nodes express knowledge information in the form of structured text, extracted by information source segmentation, or introduced through the description of chart-based expert knowledge;

[0013] After the large language model extracts keywords from the user input request, it first performs a global match on the first-level nodes of the knowledge graph in a traversal search manner to obtain matching first-level nodes, and then performs a local search on the knowledge fragments associated with the matching first-level nodes in the knowledge graph to obtain the knowledge graph retrieval results, which include the nodes and path information in the graph that match the user request.

[0014] Furthermore, the information sources include technical standards, reports and documents.

[0015] Furthermore, step 2 specifically includes:

[0016] After global and local retrieval using the knowledge graph, the knowledge text and path information between nodes of the graph that match the user's request are fed into the large language model to generate multiple prompt statements to re-describe the user's request, so that the large language model agent can accurately understand the expert field and smoothly implement tasks, including data query, extraction, and analysis.

[0017] Furthermore, in step 3, the accuracy of the multiple prompt sentences generated in step 2 is scored, specifically including:

[0018] The original user intention and the restated user intentions are sent to the large language model for similarity scoring. At the same time, a vector vocabulary model is built and trained based on the expert information corpus. The distance score of the restated intention is scored through the vector vocabulary model. Finally, the similarity score of the large language model and the distance score of the vector vocabulary model are weighted to obtain the final scoring results of each restated intention.

[0019] Furthermore, step 4 specifically includes:

[0020] The large language model agent performs the expert information query task based on the intention description with the highest score in step 3. The expert information query task includes data query, knowledge query, and data mining. The large language model agent returns information including numerical values, tables, and knowledge. The large language model generates prompt sentences that can be displayed visually to guide expression based on the expert information feature data.

[0021] Furthermore, the expert information feature data includes data modality, data volume and professional data visualization targets.

[0022] A knowledge graph-enhanced expert information retrieval prompt engineering self-generation device, characterized by comprising:

[0023] The search intention understanding module uses the pre-trained large language model to extract the keywords of the user's input request, performs intention understanding analysis on the keywords based on the vertical domain expert knowledge graph, and obtains the knowledge graph retrieval results;

[0024] A search intention restatement module, which uses the knowledge graph retrieval results to restate the user's search intention and generates multiple prompt statements for task execution in combination with the large language model tool call capability;

[0025] The expert information retrieval module scores the accuracy of the generated multiple prompt sentences and inputs the scoring results into the large language model agent for query and task execution response, thereby obtaining query results containing expert information feature data;

[0026] The retrieval result generation module generates prompt sentences that can guide the fusion expression of multimodal expert information based on the expert information feature data contained in the query results, supporting the realization of the final retrieval result output.

[0027] The advantages of the present invention are:

[0028] (1) Retrieval information enhancement based on knowledge graph is a knowledge expression method that can flexibly express expert domain information. It can cover two types of information, expert knowledge concepts and unstructured text, through the unique structure of two-level knowledge expression. It has advantages in enhancing expert knowledge coverage and retrieval matching capabilities.

[0029] (2) A large language model prompt engineering method for expert knowledge information retrieval is proposed. The ability to understand user request intentions is enhanced through knowledge graphs, and search prompt sentences are autonomously generated through self-scoring and text generation capabilities. This improves the effectiveness of general large language models in professional fields while reducing the technical cost of manual prompt word design. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is an implementation principle diagram of a knowledge graph-enhanced expert information retrieval prompt engineering self-generation method of the present invention;

[0031] Figure 2 It is a brief example of the knowledge graph used to label expert information in the present invention;

[0032] Figure 3 It is a flow chart of a method for self-generating a knowledge graph-enhanced expert information retrieval prompt engineering according to the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The expression modes of expert information include knowledge, numerical values, unstructured, semi-structured, and structured data forms. In order to give accurate feedback on the content of user requests, multiple processes such as table data query, related data analysis, and knowledge expression are often required. To this end, it is necessary to combine the query and processing needs of professional information, and gradually enhance the ability to understand the user's request intent through the automatic generation of phased prompt word engineering, so as to improve the accuracy of information feedback of large language models.

[0035] The search intent understanding of the present invention will perform global and local double retrieval based on the knowledge graph, and then use the large language model to restate the expression; score the restated content and input it into the large language model agent for query; finally, generate information feedback prompts based on the data characteristics of the query results.

[0036] See also Figure 1 and Figure 3 The main implementation method of the present invention includes the following steps:

[0037] Step 1: Search intent understanding: Use the pre-trained large language model to extract the keywords of the user's input request, perform intent understanding analysis on the keywords based on the vertical domain expert knowledge graph, and obtain the knowledge graph retrieval results;

[0038] Step 2: Restate search intent: Use the knowledge graph retrieval results obtained in step 1 to restate the user's search intent, and combine the large language model tool calling capabilities to generate multiple prompt statements for task execution;.

[0039] Step 3: Expert information retrieval: Score the accuracy of the multiple prompt sentences generated in step 2, and input the scoring results into the large language model agent for query and task execution response, and obtain the query results containing expert information feature data;.

[0040] Step 4: Retrieval result generation: Based on the expert information feature data (such as data modality, data volume, and professional data visualization goals) contained in the query results of step 3, generate prompt statements that can guide the fusion expression of multimodal expert information to support the final retrieval result output.

[0041] Furthermore, in step 1, the knowledge graph is used to accurately understand the user's search intent, including:

[0042] Construct a two-layer graph structure consisting of knowledge nodes and knowledge fragment nodes, where knowledge nodes are first-level nodes, containing entity concepts extracted from information sources (such as technical standards, reports, and documents); knowledge fragment nodes express knowledge information in the form of structured text, extracted by information source segmentation, or introduced through the description of chart-based expert knowledge;

[0043] After the large language model extracts keywords from the user input request, it first performs a global match on the first-level nodes of the knowledge graph in a traversal search manner to obtain matching first-level nodes, and then performs a local search on the knowledge fragments associated with the matching first-level nodes in the knowledge graph to obtain the knowledge graph retrieval results, which include the nodes and path information in the graph that match the user request.

[0044] Considering that the number of matched nodes obtained after knowledge graph retrieval may be large, it is necessary to set thresholds for the matching number of first-level nodes and knowledge fragment nodes. To further speed up the knowledge graph retrieval, if there are multiple matching first-level nodes, only the knowledge fragment nodes on the path of the globally matched first-level nodes will be calculated during the local retrieval process, such as Figure 2As shown in the figure, the circle and square represent the first-level node and knowledge fragment node respectively. Assuming that both nodes 3 and 4 match the user request in the global search, only fragment D needs to be locally searched among all the associated knowledge fragments C, D, E, and F. For example, when a user asks "What are the characteristics of electricity consumption in a 5G data center?", the keywords "5G data center" and "electricity consumption characteristics" are first extracted from the question through the big model, and matched to the two first-level nodes (5G data center) and (load) in the knowledge graph.

[0045] Furthermore, the step 2 restates the results obtained by the enhanced retrieval based on the knowledge graph in step 1, specifically including:

[0046] After global and local retrieval using knowledge graph retrieval, the knowledge text matching the user's request and the path information between the nodes of the graph will be sent to the large language model together to generate multiple prompt sentences to re-describe the user's request, so that the large language model agent can accurately understand the expert field data query, extraction, analysis and other tasks to smoothly implement; for example, when a user asks "What are the characteristics of electricity consumption in 5G data centers?", the two first-level nodes extracted in step 1 can be used to restate the user's intention multiple times, one of which is: "What are the load characteristics of electricity consumption in 5G data centers?".

[0047] Furthermore, step 3 will autonomously score the restated user intention, including:

[0048] The original user intent and the restated user intent are sent to the large language model for similarity scoring. At the same time, a vector word library model is constructed and trained based on the expert information corpus. The distance score of the restated intent is scored through the vector word library model. Finally, the similarity score of the large language model and the distance score of the vector word library model are weighted to obtain the final scoring results of each restated intent. For example, the user intents of the previous multiple restates are scored and sorted, and the best intent prompt is selected; in addition, according to the search for a path between two first-level nodes, three knowledge fragment nodes [History and current status of 5G technology], [Geographical distribution of 5G base stations in a certain area] and [Typical load types] are retrieved, and three text materials (unstructured knowledge) are stored respectively. The algorithm will also score and sort these three text fragments, and finally exclude [Geographical distribution of 5G base stations in a certain area], and select the remaining two texts as prompt words to form the fragment.

[0049] Furthermore, step 4 specifically includes:

[0050] The large language model agent will perform expert information query tasks based on the intention description with the highest score in step 3, including data query, knowledge query, data mining, etc. The large language model agent returns information including values, tables, knowledge, etc. The large language model will generate prompt sentences for visual display based on information modality, data volume and other features to guide expression. For example, in the previous example, a path between two first-level nodes is searched: (5G data center)--Generate--> [5G center power consumption meter] <--Belongs to--(power consumption meter)--Related--> (load) <--Belongs to--[hourly load characteristics]--Related--> [typical load type]. Among them, three knowledge fragment nodes, [5G center power consumption meter], [hourly load characteristics] and [typical load type], are retrieved, which are database table and field introduction (semi-structured), calculation formula (unstructured) and text material (unstructured) knowledge. Therefore, after one retrieval, three different modal knowledge related to the question can be obtained and integrated. In addition, the system will automatically select prompt word types to integrate unstructured text knowledge, user questions, and queried structured data, so that the general large model can answer professional field questions based on data and knowledge.

[0051] The embodiment of the present invention also provides a knowledge graph-enhanced expert information retrieval prompt engineering self-generation device, comprising:

[0052] The search intention understanding module uses the pre-trained large language model to extract the keywords of the user's input request, performs intention understanding analysis on the keywords based on the vertical domain expert knowledge graph, and obtains the knowledge graph retrieval results;

[0053] A search intention restatement module, which uses the knowledge graph retrieval results to restate the user's search intention and generates multiple prompt statements for task execution in combination with the large language model tool call capability;

[0054] The expert information retrieval module scores the accuracy of the generated multiple prompt sentences and inputs the scoring results into the large language model agent for query and task execution response, thereby obtaining query results containing expert information feature data;

[0055] The retrieval result generation module generates prompt sentences that can guide the fusion expression of multimodal expert information based on the expert information feature data contained in the query results, supporting the realization of the final retrieval result output.

[0056] The present invention can use the pre-built knowledge graph, general large language model, and large language model agent to restate the user request, generate prompt sentences to guide the large language model to complete expert information retrieval and result output, effectively expanding the vertical service capabilities of the general large language model, and is of great value in promoting the application of generative artificial intelligence technology in the expert field.

[0057] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A knowledge graph-enhanced expert information retrieval prompt engineering self-generation method, characterized by The following steps are involved: Step 1: Search intent understanding: Use the pre-trained large language model to extract the keywords of the user's input request, perform intent understanding analysis on the keywords based on the vertical domain expert knowledge graph, and obtain the knowledge graph retrieval results; Step 2: Restate the search intent: Use the knowledge graph retrieval results obtained in step 1 to restate the user's search intent, and combine the large language model tool calling capabilities to generate multiple prompt statements for task execution; Step 3, expert information retrieval: score the accuracy of the multiple prompt sentences generated in step 2, and input the score results into the large language model agent for query and task execution response, and obtain the query results containing expert information feature data; Step 4, retrieval result generation: Based on the expert information feature data contained in the query result of step 3, a prompt sentence is generated that can guide the fusion expression of multimodal expert information to support the realization of the final retrieval result output.

2. The method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph according to claim 1, characterized in that: Step 1 specifically includes: Construct a two-layer graph structure including knowledge nodes and knowledge fragment nodes, where knowledge nodes are first-level nodes, including entity concepts extracted from information sources; knowledge fragment nodes express knowledge information in the form of structured text, extracted by information source segmentation, or introduced through the description of chart-based expert knowledge; After the large language model extracts keywords from the user input request, it first performs a global match on the first-level nodes of the knowledge graph in a traversal search manner to obtain matching first-level nodes, and then performs a local search on the knowledge fragments associated with the matching first-level nodes in the knowledge graph to obtain the knowledge graph retrieval results, which include the nodes and path information in the graph that match the user request.

3. The method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph according to claim 2, characterized in that: The information sources include technical standards, reports and literature.

4. The method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph according to claim 1, characterized in that: Step 2 specifically includes: After global and local retrieval using the knowledge graph, the knowledge text and path information between nodes of the graph that match the user's request are fed into the large language model to generate multiple prompt statements to re-describe the user's request, so that the large language model agent can accurately understand the expert field and smoothly implement tasks, including data query, extraction, and analysis.

5. The method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph according to claim 1, characterized in that: In step 3, the accuracy of the multiple prompt sentences generated in step 2 is scored, including: The original user intention and the restated user intentions are sent to the large language model for similarity scoring. At the same time, a vector vocabulary model is built and trained based on the expert information corpus. The distance score of the restated intention is scored through the vector vocabulary model. Finally, the similarity score of the large language model and the distance score of the vector vocabulary model are weighted to obtain the final scoring results of each restated intention.

6. The method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph according to claim 1, characterized in that: Step 4 specifically includes: The large language model agent performs the expert information query task based on the intention description with the highest score in step 3. The expert information query task includes data query, knowledge query, and data mining. The large language model agent returns information including numerical values, tables, and knowledge. The large language model generates prompt sentences that can be displayed visually to guide expression based on the expert information feature data.

7. The method for self-generating expert information retrieval prompt engineering enhanced by knowledge graph according to claim 1, characterized in that: The expert information feature data includes data modality, data volume and professional data visualization targets.

8. A knowledge graph-enhanced expert information retrieval prompt engineering self-generation device, characterized in that: include: The search intention understanding module uses the pre-trained large language model to extract the keywords of the user's input request, performs intention understanding analysis on the keywords based on the vertical domain expert knowledge graph, and obtains the knowledge graph retrieval results; A search intention restatement module, which uses the knowledge graph retrieval results to restate the user's search intention and generates multiple prompt statements for task execution in combination with the large language model tool call capability; The expert information retrieval module scores the accuracy of the generated multiple prompt sentences and inputs the scoring results into the large language model agent for query and task execution response, thereby obtaining query results containing expert information feature data; The retrieval result generation module generates prompt sentences that can guide the fusion expression of multimodal expert information based on the expert information feature data contained in the query results, supporting the realization of the final retrieval result output.

Images