Intelligent question and answer method based on knowledge graph and large language model

Abstract

The invention provides an intelligent question and answer method based on a knowledge graph and a large language model, and the method comprises the steps: constructing a knowledge graph concept layer from top to bottom, and obtaining a knowledge relation definition through the knowledge graph concept layer; extracting data features of the open domain data based on the knowledge relationship definition, and constructing a knowledge graph data layer from bottom to top according to the data features; performing knowledge fusion on the data features through entity disambiguation and co-reference resolution to obtain fused data; and constructing a secondary intention classification model, performing intention prediction on a user problem through the secondary intention classification model, and performing intelligent reply based on knowledge relationship definition and a knowledge graph formed by fusion data. According to the method, the knowledge graph and the large language model are combined, so that the answering accuracy and the intelligent level of the question and answer system are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer science and educational technology, and in particular to an intelligent question-answering method based on a knowledge graph and a large language model. Background Art

[0002] The development of the internet has made information acquisition easier, but it has also led to an explosive growth of information. As more and more people use the internet, more and more information is uploaded online, including news, articles, images, videos, and more. This information is constantly updated and expanded, requiring us to spend more time and effort to find the information we need. Against this backdrop, search engines were designed as a more efficient way to obtain information. However, traditional search engines are increasingly unable to meet people's needs. They simply match user-entered keywords with web page content, failing to truly understand the user's semantic information. Furthermore, query results are primarily in the form of web links, forcing users to click on relevant links and search for the information they need.

[0003] Question-answering systems based on artificial intelligence technologies, including natural language processing, cognitive image processing, and learning, can help people understand their thoughts quickly and accurately. However, traditional search engines can only return web links related to the user's search keywords. Therefore, it is necessary to design an intelligent question-answering method based on knowledge graphs and large language models. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent question-answering method based on knowledge graph and large language model, so as to improve the answer accuracy and intelligence level of the question-answering system by combining knowledge graph and large language model.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] An intelligent question-answering method based on a knowledge graph and a large language model includes the following steps:

[0007] Construct the knowledge graph concept layer in a top-down manner, and obtain the knowledge relationship definition through the knowledge graph concept layer;

[0008] Extract data features of open domain data based on knowledge relationship definitions, and construct the knowledge graph data layer in a bottom-up manner based on the data features; open domain data includes: structured data, semi-structured data and unstructured data; data features include: entity features, relationship features and attribute features;

[0009] Through entity disambiguation and coreference resolution, the data features are fused to obtain fused data.

[0010] Construct a secondary intent classification model, use it to predict the intent of user questions, and provide intelligent responses based on the knowledge graph composed of knowledge relationship definitions and fused data; intent prediction includes: intent and information recognition stage and intent query stage.

[0011] Optionally, a knowledge graph concept layer is constructed in a top-down manner, and the knowledge relationship definition is obtained through the knowledge graph concept layer. Specifically, authoritative data in the computer field is used as a data source to construct the knowledge graph concept layer, and the data source is abstracted and summarized based on the three-element principle of knowledge graph construction to obtain the knowledge relationship definition; the knowledge relationship definition includes: entity definition, attribute definition and relationship definition.

[0012] Optionally, the expression of the knowledge graph data layer is: G = {E, R, F}; where G is the knowledge graph, E is the entity set, R is the relationship set, and F is the fact set.

[0013] Optionally, data features of open domain data are extracted based on knowledge relationship definitions, and a knowledge graph data layer is constructed in a bottom-up manner based on the data features, including:

[0014] Extract entity features and attribute features of semi-structured data and unstructured data respectively based on knowledge relationship definition;

[0015] The Prompt prompt project of the large language model extracts entity features, relationship features and attribute features of semi-structured data and unstructured data respectively.

[0016] Optionally, the built-in extraction expression of the large language model is as follows:

[0017] P i =Bu(in e ,es,tx i );

[0018] A i =Pa(LLM(P i ));

[0019] Among them, P i is the i-th input of the large language model, Bu(·) is the input synthesizer, in e is the extraction instruction, es is the extraction case, tx i is the i-th text to be extracted, LLM(·) is the large language model function, Pa(·) is the result parser, A i is the extraction result of the i-th article.

[0020] Optionally, entity disambiguation is to determine the entity or concept referred to by a word or phrase in a text; and coreference resolution is to identify whether different words or pronouns in a text refer to the same entity or concept.

[0021] Optionally, the intent and information identification phase includes:

[0022] Collect and organize questions and answers related to user issues from FAQs and classify their intent;

[0023] Define the TF-IDF intent classification model and the Naive Bayes intent classification model;

[0024] Based on the TF-IDF intent classification model and the Naive Bayes intent classification model, combined with the JIEBA word segmentation, the user question is segmented into phrases to obtain the segmentation results;

[0025] The segmentation result is matched with the entity feature vocabulary through similarity calculation to obtain the first intent recognition result.

[0026] Optionally, the expression of the TF-IDF intent classification model is:

[0027]

[0028] TF_IDF I =TF i,j ×IDF i ;

[0029] Among them, TF i,j is the frequency of word i in question j, n i,j is the number of times word i appears in question j, k is the total number of words, IDF i is the inverse document frequency of word i in question j, D is the total number of questions, |j:t i ∈d j | is the number of questions containing word i, TF_IDF I is the importance of word i in question j.

[0030] Alternatively, the Naive Bayes intent classification model is expressed as: Among them, f(x) is the prediction result, x i is the i-th feature of question x, n is the number of features, y k is the number of intent classifications of the question, and P(·) is the probability.

[0031] Optionally, the intent query stage is: keyword matching of user questions is performed through the domain AC tree to obtain a second intent recognition result.

[0032] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the intelligent question-answering method based on the knowledge graph and the large language model provided by the present invention comprises: constructing a knowledge graph concept layer in a top-down manner, and obtaining knowledge relationship definitions through the knowledge graph concept layer; extracting data features of open domain data based on the knowledge relationship definitions, and constructing a knowledge graph data layer in a bottom-up manner according to the data features; performing knowledge fusion on the data features through entity disambiguation and coreference resolution to obtain fused data; constructing a secondary intent classification model, predicting the intent of user questions through the secondary intent classification model, and making intelligent replies based on the knowledge graph composed of the knowledge relationship definitions and the fused data. The present invention greatly improves the answer accuracy and intelligence level of the question-answering system by combining the knowledge graph with the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 This is a flow chart of the intelligent question-answering method of the present invention;

[0035] Figure 2 This is a flowchart of the intelligent question-answering framework of the present invention. DETAILED DESCRIPTION

[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0037] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0038] like Figure 1 As shown, the present invention provides an intelligent question answering method based on knowledge graph and large language model, comprising the following steps:

[0039] Step 100: Construct a knowledge graph concept layer in a top-down manner, and obtain knowledge relationship definitions through the knowledge graph concept layer;

[0040] Specifically, authoritative data in the computer field is used as the data source. The authoritative data in the computer field used in this embodiment is the electronic textbook for the computer composition principle course. The data is saved after acquisition. The data source is of high quality and can ensure the authority and accuracy of the knowledge points. Then, the acquired computer composition principle data is analyzed, and the concepts, relationships and attributes of the knowledge graph are determined according to the composition and characteristics of the data. The computer composition principle data is abstracted and summarized using the three-element principle of knowledge graph construction, so as to perform ontology modeling of computer composition principle and complete the construction of the knowledge graph concept layer. Among them, the knowledge graph is a structured semantic knowledge base, which saves data in the form of triples of <entity, relationship, entity> to express the relationship between things in the real world. The overall intelligent question-answering process is as follows: Figure 2 shown.

[0041] It should be noted that the knowledge graph ontology is a formalized, standardized description of concepts and their relationships within a domain, laying the theoretical framework for the decomposition of elements and attributes in the data layer. The knowledge graph concept layer primarily includes the hierarchical structure and hierarchical relationship definitions of knowledge classes such as entities, relations, and attributes, which are used to constrain the specific knowledge forms in the data layer. According to the three-element principle of knowledge graph construction, it is necessary to clarify the three elements of "entity-relationship-entity attribute" or "entity-relationship-entity".

[0042] In this embodiment, a course knowledge point ontology can be constructed within the computer organization principle. The course knowledge point ontology is divided into five parts: course knowledge, knowledge system, knowledge unit, knowledge set, and knowledge point. The course knowledge is, for example, "Computer Organization Principles." The knowledge system can be divided into "software" and "hardware." Knowledge units include textual content such as "central processing unit" and "peripheral devices." Knowledge sets include textual content such as "arithmetic unit" and "controller." Knowledge points refer to the smallest indivisible unit within the course, such as textual content such as "computing speed" and "machine word length."

[0043] The objects and data attributes of the knowledge point ontology provide the prerequisites for constructing the knowledge graph. First, the data attributes are defined into eight attribute categories: knowledge name, knowledge content, English expression, learning objectives, learning difficulty, predecessor courses, subsequent courses, and related resources. "Learning objectives" and "learning difficulty" are derived from the course objectives in the course objective ontology, while "predecessor courses" and "subsequent courses" are determined by the relationship between hardware series courses. Then, the object attributes of the knowledge point ontology are defined. Knowledge points may have relationships such as contain, belong to, related, synonymous, predecessor, and successor, expressed as contain, belongTo, relation, synonymy, precedence, and follow-up.

[0044] Step 200: Extracting data features of open domain data based on knowledge relationship definitions, and constructing a knowledge graph data layer in a bottom-up manner based on the data features; open domain data includes: structured data, semi-structured data, and unstructured data; data features include: entity features, relationship features, and attribute features;

[0045] Specifically, the knowledge graph data layer is based on fact triples and is used to store specific data information. The expression of the knowledge graph data layer is:

[0046] G = {E, R, F};

[0047] Among them, G is the knowledge graph, E is the entity set {e1,e2,……,e n}, e n It is the nth most basic component element in the knowledge graph and is represented by a node in the knowledge graph; R is the relationship set {r1, r2, ..., r n}, r n is the nth edge in the knowledge graph, indicating the connection between different entities; F is the fact set {f1,f2,……,f n}, f n For the nth fact, each fact f is defined as a triple (h, r, t)∈f, where h represents the head entity, r represents the relationship, and t represents the tail entity.

[0048] Specifically, the knowledge graph data layer mainly pre-processes the acquired data to extract entities, relationships, and attributes, and then fuses the obtained knowledge features into entities and relationships according to the constructed concept layer, and then stores the fused information in the form of triples in the knowledge graph database.

[0049] Furthermore, graph databases are storage units based on nodes and edges, and are data management systems designed for efficient storage and querying of graph data. Unlike relational databases like MySQL and MongoDB, graph databases are non-relational databases. They directly store the dependencies between nodes and add corresponding labels or attributes to visualize the stored content. The ability to quickly search and query node relationships is a significant performance advantage of graph databases. Graph databases are based on graph theory and consist of nodes and edges, where nodes represent entities or instances and edges represent relationships between nodes. The two main graph models used are property graphs and RDF graphs.

[0050] The graph database used in this example is the Neo4j graph database, which offers advantages such as being embedded, high-performance, and lightweight. In the Neo4j graph database, the built-in Cypher language enables operations such as adding, deleting, modifying, and querying nodes, relationships, and attributes. Furthermore, in complex scenarios, Cypher can also enable graph queries and related graph computations.

[0051] More specifically, the specific implementation method of extracting data features of open domain data based on knowledge relationship definition and combining with the knowledge graph data layer is as follows:

[0052] First, under the guidance of the concept layer framework, entity features and attribute features corresponding to concept layer entities and entity attributes are extracted from unstructured data and semi-structured data, and stored in the knowledge graph database in the form of triples; then, the entity features, relationship features, and attribute features of semi-structured data and unstructured data are extracted respectively through the Prompt prompt project of the large language model. Among them, Prompt contains multiple elements, such as instructions, background information, and input text. Instructions are short sentences that tell the model to perform a specific task. Background information is the input text or relevant information provided by few-shot learning. Input text is text that needs to be processed by the large language model. The built-in extraction expressions of the large language model are as follows:

[0053] P i =Bu(in e ,es,tx i );

[0054] A i =Pa(LLM(P i ));

[0055] Among them, P i is the i-th input of the large language model, Bu(·) is the input synthesizer, in e is the extraction instruction, es is the extraction case, tx i is the i-th text to be extracted, LLM(·) is the large language model function, Pa(·) is the result parser, Ai is the extraction result of the i-th article.

[0056] Step 300: Perform knowledge fusion on data features through entity disambiguation and coreference resolution to obtain fused data;

[0057] Specifically, entity disambiguation refers to determining which entity or concept a word or phrase in a text specifically refers to, especially when the word may have multiple meanings. In this embodiment, the "register" in the computer organization principle can refer to different things, such as a general register, a special register (such as a program counter, a stack pointer), etc. In the example "the program counter is a special register used to store the address of the next instruction", the "register" refers to the program counter, not other general registers. The task of entity disambiguation is to determine which type of register the "register" specifically refers to.

[0058] Co-reference resolution refers to identifying whether different words or pronouns in a text refer to the same entity or concept. In this embodiment, there are two cases: the first case is that in the principle of computer organization, for "central processing unit" and "CPU", the two express the same meaning, and a synonymous relationship is used to establish the relationship between the two. The second case is that in the example "the central processing unit is responsible for executing instructions, it is the core component of the computer", "it" refers to the "central processing unit" in the previous sentence. The task of co-reference resolution is to identify whether "it" and "central processing unit" are the same entity.

[0059] Step 400: Construct a secondary intent classification model, use the secondary intent classification model to predict the intent of user questions, and provide intelligent replies based on the knowledge graph composed of knowledge relationship definitions and fusion data; intent prediction includes: intent and information recognition stage and intent query stage.

[0060] Specifically, in the intent and information recognition stage, questions and answers related to user questions are collected and organized from FAQs and intent classification is performed; then the TF-IDF intent classification model and the Naive Bayes intent classification model are defined; then the phrases in the user questions are segmented based on JIEBA word segmentation, and similarity calculation and matching are performed with the entity feature vocabulary, finally achieving understanding of user intent and question entities, and obtaining the first intent recognition result.

[0061] Furthermore, the TF-IDF intent classification model is used to measure the importance of a word to a document and convert text data into a vector representation. The expression used is:

[0062]

[0063] TF_IDF I =TF i,j×IDF i ;

[0064] Among them, TF i,j is the frequency of word i in question j, indicating the frequency of the word appearing in the user's question, n i,j is the number of times word i appears in question j, k is the total number of words, IDF i is the inverse document frequency of word i in question j, indicating the rarity of the word in the entire corpus, D is the total number of questions, |j:t i ∈d j | is the number of questions containing word i, TF_IDF I is the importance of word i in question j.

[0065] Specifically, the Naive Bayesian intent classification model is used in document classification. It assumes that text features (words, phrases, etc.) are independent of each other. It then calculates the conditional probability of each category based on the question features and question intent classification labels in the training data. Finally, it uses Bayes' theorem to calculate the probability that the user question belongs to each intent category and selects the category with the highest probability as the prediction result. The expression used is:

[0066]

[0067] Among them, f(x) is the prediction result, x i is the i-th feature of question x, y k is the number of intent classifications of the question, P(·) is the probability, and n is the number of features.

[0068] More specifically, during the intent query phase, key feature words are designed for each type of question intent, generating a keyword category dictionary. A domain AC tree is then constructed using these feature words for keyword matching, identifying user intent and generating a second intent recognition result. Finally, the two intent recognition results are intersected to determine a response strategy.

[0069] The beneficial effects of the present invention are as follows:

[0070] 1) Knowledge graphs provide effective support for information retrieval, reasoning, and semantic analysis by presenting knowledge points, concepts, and their relationships in a graph structure.

[0071] 2) Through the powerful semantic understanding and information extraction capabilities of the large language model, relevant knowledge can be automatically extracted from unstructured and semi-structured data and mapped into the knowledge graph, ensuring the comprehensiveness and accuracy of the knowledge graph and greatly improving the level of intelligence;

[0072] 3) In the process of constructing the concept layer, the authority and systematicness of the knowledge points are guaranteed through high-quality electronic course materials;

[0073] 4) In the process of building the data layer, the content of the knowledge graph is greatly enriched by integrating structured, semi-structured and unstructured data sources;

[0074] 5) Combining the TF-IDF and Naive Bayes intent classification models can accurately identify user intent, improve the efficiency of knowledge graph construction, and enhance the accuracy of intelligent question answering and user experience.

[0075] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0076] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. An intelligent question answering method based on knowledge graph and large language model, characterized by: The steps include: Constructing a knowledge graph concept layer in a top-down manner, and obtaining knowledge relationship definitions through the knowledge graph concept layer; Extracting data features of open domain data based on the knowledge relationship definition, and constructing a knowledge graph data layer in a bottom-up manner according to the data features; The open domain data includes: structured data, semi-structured data and unstructured data; the data features include: entity features, relationship features and attribute features; Performing knowledge fusion on the data features through entity disambiguation and coreference resolution to obtain fused data; A secondary intent classification model is constructed, and the intent of user questions is predicted by the secondary intent classification model, and intelligent replies are made based on the knowledge relationship definition and the knowledge graph composed of the fused data; the intent prediction includes: an intent and information recognition stage and an intent query stage.

2. The intelligent question answering method based on knowledge graph and large language model according to claim 1 is characterized in that: A knowledge graph concept layer is constructed in a top-down manner, and knowledge relationship definitions are obtained through the knowledge graph concept layer. Specifically, authoritative data in the computer field is used as a data source to construct the knowledge graph concept layer, and the data source is abstracted and generalized based on the three-element principle of knowledge graph construction to obtain the knowledge relationship definitions; The knowledge relationship definition includes: entity definition, attribute definition and relationship definition.

3. The intelligent question answering method based on knowledge graph and large language model according to claim 1 is characterized in that: The expression of the knowledge graph data layer is: G = {E, R, F}; where G is the knowledge graph, E is the entity set, R is the relationship set, and F is the fact set.

4. The intelligent question answering method based on knowledge graph and large language model according to claim 1 is characterized in that: Extracting data features of open domain data based on the knowledge relationship definition, and constructing a knowledge graph data layer in a bottom-up manner based on the data features, including: extracting the entity features and the attribute features of the semi-structured data and the unstructured data respectively based on the knowledge relationship definition; The entity features, the relationship features and the attribute features of the semi-structured data and the unstructured data are extracted respectively through the Prompt prompt project of the large language model.

5. The intelligent question answering method based on knowledge graph and large language model according to claim 4 is characterized in that: The built-in extraction expression of the large language model is as follows: P i =Bu(in e ,es,tx i ); A i =Pa(LLM(P i )); Among them, P i is the i-th input of the large language model, Bu(·) is the input synthesizer, in e is the extraction instruction, es is the extraction case, tx i is the i-th text to be extracted, LLM(·) is the large language model function, Pa(·) is the result parser, A i is the extraction result of the i-th article.

6. The intelligent question answering method based on knowledge graph and large language model according to claim 1 is characterized in that: The entity disambiguation is to determine the entity or concept referred to by a word or phrase in the text; the coreference resolution is to identify whether different words or pronouns in the text refer to the same entity or concept.

7. The intelligent question answering method based on knowledge graph and large language model according to claim 1 is characterized in that: The intent and information recognition phase includes: Collect and organize questions and answers related to the user's questions from the FAQ and classify their intent; Define the TF-IDF intent classification model and the Naive Bayes intent classification model; Based on the TF-IDF intent classification model and the naive Bayes intent classification model, combined with JIEBA word segmentation, the user question is segmented into phrases to obtain a segmentation result; The segmentation result is matched with the entity feature vocabulary through similarity calculation to obtain a first intent recognition result.

8. The intelligent question answering method based on knowledge graph and large language model according to claim 7 is characterized in that: The expression of the TF-IDF intent classification model is: Among them, TF i,j is the frequency of word i in question j, n i,j is the number of times word i appears in question j, k is the total number of words, IDF i is the inverse document frequency of word i in question j, D is the total number of questions, |j:t i ∈d j | is the number of questions containing word i, TF_IDF I is the importance of word i in question j.

9. The intelligent question answering method based on knowledge graph and large language model according to claim 7, characterized in that: The Naive Bayesian intent classification model is expressed as: Among them, f(x) is the prediction result, x i is the i-th feature of question x, n is the number of features, y k is the number of intent classifications of the question, and P(·) is the probability.

10. The intelligent question answering method based on knowledge graph and large language model according to claim 1, characterized in that: The intention query stage is: performing keyword matching on the user question through the domain AC tree to obtain a second intention recognition result.

Images