Knowledge graph generation-assisted teaching question answering method and system based on LLM

Abstract

This application belongs to the field of intelligent question-answering technology, and specifically discloses a knowledge graph generation-assisted teaching question-answering method and system based on LLM, wherein the method includes: filtering the questions to be answered input by the user to determine whether the questions to be answered conform to the scientific and reasonable question-answering subject area; in the knowledge graph corresponding to the pre-constructed question-answering subject area, using the LangChain agent to search for the questions to be answered that conform to the question-answering subject area, and determine the relevant knowledge of the questions to be answered; inputting the relevant knowledge into the large language model to generate the answers to the questions to be answered; wherein, the questions to be answered that conform to the question-answering subject area are searched, and at least one of the following search methods is used: search based on Cypher query statements, SBERT question mapping model and vector similarity. This application combines LangChain and LLM to produce professional answers to questions in the input question-answering subject area, and provides multi-angle search capabilities to improve the quality and speed of the system's response to user queries.

Description

Technical Field

[0001] The present application belongs to the field of intelligent question answering technology, and more specifically, relates to a knowledge graph generation-assisted teaching question answering method and system based on a large language model (LLM). Background Art

[0002] The rapid development of information technology has significantly improved people's living efficiency and profoundly impacted all aspects of society. While this informatization initiative has brought enormous opportunities, it has also posed new challenges. In education, in particular, digital transformation, centered around artificial intelligence (AI), is actively advancing, aiming to optimize the management and processing of educational data through modern information technology tools and platforms. Decision-making in traditional education models often relies on subjective assessments and human intervention, which presents challenges in the information age. To adapt to these changes, the education system must leverage modern information technology to promote automation and personalization in the teaching process, thereby providing more personalized educational services. The application of AI in education can not only reduce teachers' traditional teaching and management workload but also encourage them to focus more on academic research and innovation, ultimately improving the overall quality of education. Summary of the Invention

[0003] The embodiments of the present application provide a knowledge graph generation-assisted teaching question-answering method and system based on LLM.

[0004] In the first aspect, an embodiment of the present application provides a knowledge graph generation-assisted teaching question-answering method based on LLM, including: filtering questions to be answered input by the user to determine whether the questions to be answered conform to a scientific and reasonable question-answering subject area; in the knowledge graph corresponding to the pre-constructed question-answering subject area, using the LangChain agent to search for questions to be answered that conform to the question-answering subject area, and determine the relevant knowledge of the questions to be answered; inputting the relevant knowledge into the large language model to generate answers to the questions to be answered; wherein, the questions to be answered that conform to the question-answering subject area are searched using at least one of the following search methods: retrieval based on Cypher query statements; retrieval based on the SBERT question mapping model; retrieval based on vector similarity.

[0005] In a second aspect, the present application also provides a knowledge graph generation-assisted teaching question-answering system based on LLM, including:

[0006] The question filtering module is used to filter the questions to be answered by the user and determine whether the questions to be answered are in line with the scientific and reasonable question answering subject area;

[0007] A professional question-answering module based on a knowledge graph is used to search for questions that match the Q&A topic domain in a pre-built knowledge graph, determine relevant knowledge for the questions, input relevant knowledge into a large language model, and generate answers to the questions. Questions that match the Q&A topic domain are searched using at least one of the following methods: Cypher query; search based on the SBERT question mapping model; or search based on vector similarity.

[0008] The professional question-and-answer module based on the local knowledge base is used to retrieve relevant knowledge of the question to be answered based on the local knowledge base if the relevant knowledge of the question to be answered is not retrieved in the knowledge graph; the relevant knowledge is input into the large language model to generate the answer to the question to be answered.

[0009] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0010] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0011] In a fifth aspect, an embodiment of the present application further provides a computer program product, which, when running on a processor, enables the processor to execute the method described in the first aspect or any possible implementation of the first aspect.

[0012] The embodiment of the present application provides a knowledge graph generation-assisted teaching question-answering method and system based on LLM. The method and system use the LangChain agent to retrieve relevant professional knowledge in the knowledge base, and pass this knowledge and the question text as input to the LLM to generate the answer text. By combining LangChain and LLM, professional answers are produced. Three retrieval methods are used: retrieval based on Cypher query statements, retrieval based on the SBERT question mapping model, and retrieval based on vector similarity. These methods provide multi-angle retrieval capabilities for structured data retrieval, natural language query adaptability, and fuzzy query processing capabilities, respectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in this application or related technologies, the following is a brief introduction to the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0014] Figure 1 This is one of the flow charts of the knowledge graph generation-assisted teaching question-answering method based on LLM provided in the embodiment of the present application;

[0015] Figure 2 This is one of the flowcharts of the retrieval based on the SBERT question mapping model provided in the embodiment of the present application;

[0016] Figure 3 This is the second flowchart of the retrieval based on the SBERT question mapping model provided in the embodiment of the present application;

[0017] Figure 4 This is a schematic diagram of the process of creating a knowledge graph provided in an embodiment of the present application;

[0018] Figure 5 This is the second flow chart of the knowledge graph generation-assisted teaching question-answering method based on LLM provided in the embodiment of the present application;

[0019] Figure 6 This is a schematic diagram of the architecture of the knowledge graph generation-assisted teaching question-answering system based on LLM provided in an embodiment of the present application;

[0020] Figure 7 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0022] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0023] Figure 1 This is one of the flow charts of the knowledge graph generation-assisted teaching question-answering method based on LLM provided in the embodiment of the present application, such as Figure 1 As shown in FIG, the method is applied to a knowledge graph generation-assisted teaching question answering system based on LLM, and the method includes at least the following steps:

[0024] S101. Filter the questions to be answered input by the user to determine whether the questions to be answered are in line with scientific and reasonable question-answering subject areas.

[0025] Specifically, by filtering user-entered questions, we select those that fall within the scientifically sound Q&A topic area for subsequent Q&A. Question filtering limits the Q&A system's response scope and accurately identifies student questions that are relevant and scientifically sound, thereby reducing the risk of the Large Language Model (LLM) generating inaccurate information and optimizing the accuracy of answer quality.

[0026] Taking the topic of computer science as an example, S101's primary goal is to accurately determine whether student questions involve computer science or the C++ language, ensuring that the system only responds to questions within its capabilities. This mechanism effectively ensures the quality and relevance of the system's responses by refusing to answer questions that are unrelated to computer science or beyond the system's expertise. It also significantly reduces the risk of providing erroneous or misleading information and mitigates the likelihood that students will overestimate the system's cognitive capabilities.

[0027] On the technical level, the core implementation of S101 relies on text classification technology, which uses deep learning models to identify and classify the content of questions, thereby realizing automatic screening and classification of questions.

[0028] In some embodiments, S101 specifically includes: inputting the question to be answered into the text classification model based on the improved BERT for question filtering, and determining whether the question to be answered is consistent with the question-answering topic area; the text classification model based on the improved BERT includes an input layer, a Transformer encoder layer, a fully connected layer, a Softmax layer and an output layer.

[0029] Specifically, a text classification model based on the improved Bidirectional Encoder Representations from Transformers (BERT) is adopted. A fully connected layer and a softmax layer are added as classifiers on the basis of the BERT model. The BERT model extracts the feature vector of the question to be answered, and the classifier predicts the type of the question to be answered.

[0030] Taking the question answering subject area of ​​computers as an example, the training process of the text classification model based on the improved BERT is as follows:

[0031] Dataset construction: Use Python's Requests library and BeautifulSoup library to collect student questions on the education and teaching website, including 2,000 computer-related questions and 4,000 computer-unrelated questions, to build a dataset.

[0032] Data preprocessing: The input question is decomposed into tokens by the tokenizer and converted into corresponding token IDs. At the same time, in order to maintain word order information, position encoding is added and special tags (such as the sentence start tag CLS and the sentence end tag SEP) are inserted.

[0033] Pre-training: The pre-processed input question is fed into a pre-trained text classification model based on an improved BERT. The BERT model uses a multi-layer Transformer structure to generate high-quality feature vectors by learning the deep semantics and contextual dependencies of the text. The feature vector passes through a fully connected layer that includes a linear transformation, dropout, and an activation function. The output dimension of this layer matches the number of categories of the classification task, for example, it is divided into two categories: computer-related and non-computer-related. The output of the fully connected layer is then converted into a probability distribution through a Softmax function. The probability of each category represents the model's confidence that the input text belongs to that category. Ultimately, the category with the highest probability is selected as the model's predicted output, thereby determining whether the input question is computer-related.

[0034] S102. In the pre-constructed knowledge graph corresponding to the question-answering subject area, the LangChain agent is used to search for questions to be answered that match the question-answering subject area, and determine the relevant knowledge of the questions to be answered.

[0035] S103. Input relevant knowledge into the large language model to generate answers to the questions to be answered.

[0036] Specifically, for questions that fall within the Q&A topic area, the LangChain agent is used to retrieve relevant knowledge in the corresponding knowledge graph, and the retrieved relevant knowledge is input into the large language model to generate answers to the questions.

[0037] LangChain is an open-source framework designed to integrate large language models (such as GPT-3) to build and deploy natural language processing (NLP) applications, particularly those that rely on complex language understanding and generation techniques. The framework provides a series of tools and application programming interfaces (APIs) that enable developers to easily integrate large language models into their projects. This integration enables diverse functionality, such as text generation, question-answering systems, automated text summarization, language translation, and various other language understanding tasks. Furthermore, LangChain not only promotes widespread adoption of technology but also provides a platform for researchers and developers to experiment and validate various NLP techniques.

[0038] In an agent system built using the LangChain framework, the data flow for processing student queries typically follows the following pattern: First, the agent data flow is initiated when the system receives student input. The agent then issues a request to the LLM, containing the student's question and a set of well-defined natural language instructions, known as agent prompts, guiding the model to generate the desired response. Upon receiving these inputs, the LLM generates further action instructions based on its internal algorithms. The agent adjusts its subsequent behavior based on these instructions and may request additional contextual information. Once the necessary information has been collected, the agent calls the LLM again, where the model utilizes the new input data to generate the final answer, which is then transmitted back to the student. This interactive data flow ensures the adaptability and dynamism of the information processing process, allowing the system to iterate and refine the information as necessary before providing an answer.

[0039] If the system goal is to build a chatbot, a conversational agent can be used as the intermediary for the chat model. LangChain, the library implementing the agent functionality, is based on Python code and uses prompt engineering to enhance model performance. Alternatively, the GPT-3.5-turbo model, a large language model, can be used as the core inference engine for the LangChain agent. This choice is based on its advanced performance in language understanding and generation tasks, as well as its excellent integration with LangChain.

[0040] The embodiment of the present application uses the LangChain agent to retrieve relevant professional knowledge in the knowledge base, and passes this knowledge and the question text as input to the LLM to generate the answer text. By combining LangChain and LLM, a professional answer is produced.

[0041] Furthermore, by analyzing the prompting mechanisms that conversational agents rely on when handling conversations, we selected three different query strategies for application. Langchain agents are autonomous and can independently utilize the tools provided to perform actions. These three query strategies include: Cypher query-based retrieval; retrieval based on the Twin BERT (Sentence-BERT, SBERT) question mapping model; and retrieval based on vector similarity.

[0042] ① Retrieval based on Cypher query statements.

[0043] Neo4j retrieval based on the Cypher graph query language uses the Cypher graph query language as the primary tool for data retrieval. As a declarative query language specifically developed for graph database environments, Cypher allows users to intuitively express retrieval requirements for data in graph databases by constructing schema descriptions. Currently, the LangChain framework supports proxy entities extracting required information from Neo4j databases by generating Cypher query statements. Prompt text is designed to guide the query generation process during Cypher query construction. Furthermore, by imposing specific constraints, the model is only allowed to construct Cypher query statements that can be logically inferred from the given training samples.

[0044] When students submit their queries, the system combines their questions with agent prompts and sends them to the LLM. Using a Cypher query tool, the tool constructs a Cypher query statement to perform a search operation on the Neo4j graph database. The retrieved knowledge is then passed back to the agent. The agent then submits another request to the LLM, incorporating the newly acquired contextual information. Given that this new context contains the necessary information to construct an answer, the LLM constructs the final answer and instructs the agent to provide it to the student.

[0045] When using the Cypher query language to search a database, it's crucial to ensure that student input strictly corresponds to the nodes and relationships defined in the knowledge graph. This places high demands on the format of student input. While this method offers excellent accuracy, its applicability is limited, particularly when students lack a thorough understanding of the knowledge graph's structure or specialized terminology. Therefore, two other search methods are employed as supplementary methods.

[0046] ② Retrieval based on the SBERT question mapping model

[0047] Figure 2 This is one of the flowcharts of the retrieval based on the SBERT question mapping model provided in the embodiment of the present application, such as Figure 2 As shown, the search method specifically includes:

[0048] S201. Input the questions to be answered that meet the question answering topic area into the BERT-CRF head entity detection model for head entity detection, and output the head entity in the questions to be answered.

[0049] Specifically, in the construction and application of knowledge graphs, structured information extraction aims to automatically identify and extract entity, relationship, and attribute information from semi-structured and unstructured text information and convert it into a structured format. The core process includes: entity recognition, relationship extraction, and attribute extraction. In the embodiments of this application, a large model is used to directly generate structured information.

[0050] This process begins with head entity detection. Questions that match the Q&A topic are fed into the BERT-CRF head entity detection model, which then outputs the head entity in the question. The BERT-CRF head entity detection model combines the strengths of the BERT pre-trained model and the Conditional Random Fields (CRF) model. The BERT model is capable of learning deep semantic information from text, while the CRF model excels at sequence labeling, generating the most likely label sequence for each position based on contextual information.

[0051] In some embodiments, the BERT-CRF head entity detection model includes a data processing layer, a BERT-based feature processing layer and a CRF-based sequence annotation layer; the data processing layer performs word segmentation and BIO (Begin-Inner-Other) annotation on the question to be answered, and outputs a BIO sequence; the BERT-based feature processing layer extracts the contextual features of each word based on the BIO sequence and generates a corresponding word feature vector; the CRF-based sequence annotation layer performs label prediction based on the word feature vector, obtains the globally optimal label sequence based on the transition probability between labels, and decodes to determine the head entity in the question to be answered.

[0052] Specifically, at the data processing layer, the original dataset consists of user questions and their corresponding answer triples. To enable the model to more accurately identify head entity boundaries, the dataset needs to be transformed. Optionally, BIO annotations can be used to clearly identify the location of the corresponding head entity in the question, effectively converting the head entity detection problem into a character-level sequence annotation problem.

[0053] For an input question, each character in the sentence is assigned a label to distinguish head entity characters from non-head entity characters. The first character of the head entity in the sentence is labeled "B", the remaining characters are labeled "I", and all characters that do not belong to the head entity are uniformly labeled "O". For example, for the question "When was Hanzheng Street in Wuhan built?", the converted format is "OOBIII IO OOOOOOOO".

[0054] BERT, a cutting-edge natural language processing pre-training model, uses its deep, bidirectional Transformer architecture to learn context-aware text embeddings rich in semantic information. The BERT framework consists of two steps: pre-training and fine-tuning. During the pre-training phase, BERT is trained on a large-scale, unlabeled, general-domain corpus for two self-supervised tasks: Masked Language Model (MLM) modeling and Next Sentence Prediction (NSP).

[0055] In the MLM task, some token strings in the input sequence are randomly masked and then predicted by the model. In this way, the results encoded by the model also contain contextual information.

[0056] In the NSP task, two sentences are input to the model at the same time, and the model is asked to determine whether the two sentences are continuous. In the fine-tuning stage, BERT is initialized using pre-trained parameter weights, and then all parameters are fine-tuned using labeled data from downstream tasks. In the embodiment of the present application, in order to make full use of the head entity and context representation, the pre-trained BERT is fine-tuned using question and character-level labels. BERT adjusts its internal weights based on the training data to make the model better suited to the head entity detection task. Before BERT training begins, the text needs to be converted into a format that the model can handle.

[0057] Taking the question "When was Hankou River Street in Wuhan built?" as an example, first, the question text is tokenized. For Chinese, it is usually split by characters. After tokenization, special tokens are added to the sequence. The [CLS] classification token is added at the beginning of the sequence, and the [SEP] separator token is added at the end to distinguish sentence boundaries. The input text is converted into a Token sequence: ["[CLS]", "请", "问", "武", "汉", "汉", "正", "街", "是", "什", "么", "时", "候", "修", "建", "的", "?", "[SEP]"]. Then, word embeddings, segment embeddings, and position embeddings are generated for each Token respectively, and the combined embeddings are input into the pre-trained BERT model.

[0058] The BERT model is composed of stacked Transformer encoding layers, and each layer can extract and integrate information learned from the previous layer. Each Transformer encoder unit contains two sub-layers: the multi-head attention mechanism and the feed-forward neural network. To avoid the problem of gradient disappearance or explosion during the training of deep networks, there are residual connections and layer normalization operations after each sub-layer. This deep and multi-head attention mechanism enables BERT to consider the context information of the entire sequence, allowing the model to comprehensively understand the semantics of Tokens in the text. Finally, each Token is converted into a high-dimensional feature vector, and these vectors contain rich context information learned from the entire input sequence.

[0059] For the CRF-based sequence labeling layer, CRF is a statistical modeling method commonly used for labeling and segmenting sequence data. In the BERT-CRF head entity detection model, the CRF layer receives the word feature vectors output from the BERT-based feature processing layer as input, calculates the prediction scores logits (unnormalized log probabilities) for each possible label, and is used for model optimization training and decoding the output sequence.

[0060] During the training process, the CRF-based sequence labeling layer calculates the loss based on the model's predicted logits and the correct labels labels. The loss function is the average of the negative log-likelihood. By minimizing the loss function, backpropagation is used to optimize the model parameters; at the same time, the CRF-based sequence labeling layer is also responsible for considering the transition probabilities between labels, optimizing the label prediction of the entire sequence, and using the Viterbi algorithm to find the optimal label sequence, thereby decoding and determining the head entity in the input question to be answered, and completing the head entity detection.

[0061] S202. Retrieve multiple candidate triples from the knowledge graph based on the head entity.

[0062] Specifically, the head entity in the detected question to be answered is used to retrieve multiple candidate triples in the knowledge graph.

[0063] In some embodiments, S103 specifically includes: using the head entity as a query keyword to perform head entity matching through the Match query of Elasticsearch, and retrieving multiple candidate triples.

[0064] Specifically, Elasticsearch is an open-source, distributed search and analytics engine that supports real-time operations. We use Elasticsearch to manage and query knowledge graph triple data. Knowledge graph triple data is first imported into the Elasticsearch index in batches, laying the foundation for subsequent search and analysis tasks.

[0065] After the head entity detection model identifies the head entity in the question to be answered, it is used as the query keyword through the Elasticsearch Match query to match the head entity to retrieve a set of candidate triples.

[0066] Elasticsearch's Match query process involves multiple key components, including text analysis, indexing, query execution, and relevance scoring.

[0067] The text analysis process involves word segmentation and text conversion. The IK Analyzer word segmentation tool is used to process continuous text, splitting it into meaningful terms and performing preprocessing operations such as removing useless symbols and lowercasing English. In addition to word segmentation, it may also include conversion operations such as stemming and lemmatization to normalize different forms of terms into the same word form.

[0068] During the indexing process, these terms are used to construct an inverted index. An inverted index is a data structure in which each term is associated with a list of documents, showing all the locations where the term appears in the dataset, such as term frequency and position offset. The inverted index allows Elasticsearch to quickly find documents containing the header entity.

[0069] When executing a Match query, the query string is subjected to the same word segmentation and standardization processing to determine the fields and query terms to be searched; with the help of the inverted index, the system can quickly locate documents containing the query terms.

[0070] After the search is completed, the BM25 algorithm will be used to calculate and sort the relevance scores, effectively returning candidate triplets related to the head entity. The scoring formula is as follows:

[0071]

[0072] Among them, IDF stands for inverse document frequency, q i represents the i-th word in the query, f(q i ,D) represents word q i The frequency of the word in document D, |D| represents the length of document D, avgdl represents the average length of all documents, k1 and b represent the free parameters of the algorithm, which are used to adjust the impact of word frequency and document length on the score.

[0073] S203. Input multiple candidate triples into the SBERT candidate triple ranking model for semantic similarity sorting, and output the target triples corresponding to the question to be answered in the knowledge graph.

[0074] Specifically, after retrieving multiple candidate triples corresponding to the question header entity in the knowledge graph, the multiple candidate triples are input into the SBERT candidate triple ranking model for semantic similarity sorting, so as to determine the target triple corresponding to the question to be answered.

[0075] The Twin BERT (SBERT) model consists of two BERTs with identical network structures and shared weights. This allows two input sentences to be encoded into a unified vector space for semantic similarity analysis. The two input sentences are fed into two BERTs, where features are extracted to obtain a vector representation for each token. After passing through a pooling layer, these are combined to form sentence embeddings u and v. The similarity between the two feature vectors is calculated using the cosine similarity function (similarity = cos(u, v)). During feature vector similarity training, cosine similarity is converted into a loss function (CosineSimilarityLoss = 1-cos(u, v)) to improve the similarity between the model output vector and the label vector.

[0076] The twin BERT architecture is based on a comparison strategy for paired samples, which allows the representation of the same text to be reused in multiple comparison tasks, thereby reducing repeated calculations. By sharing weights to process paired inputs, the text pairs are effectively encoded based on the BERT model. This weight sharing mechanism not only reduces the computational burden, but also ensures the consistency and reliability of the model in multiple text comparison scenarios. Therefore, the twin BERT model is used in the embodiment of this application to implement the sorting of candidate triples.

[0077] In some embodiments, the SBERT candidate triple ranking model in S104 is trained based on the following steps: setting sample types, including: positive samples, negative samples of entities with the same head, negative samples of ambiguous entities, and negative samples of irrelevant entities; setting the similarity scores and sample quantities of different sample types in a segmented manner, and randomly extracting triplets through Elasticsearch to construct a training data set; based on the constructed training data set, training the SBERT candidate triple ranking sub-model.

[0078] Specifically, when training the SBERT candidate triple ranking model, in addition to preparing <user question-answer triples> that accurately correspond to user questions as positive samples, a certain number of negative samples must also be prepared to enhance the model's generalization ability. A good negative sample generation mechanism is crucial for strengthening the model's generalization ability.

[0079] The core idea of ​​the difficult sample mining strategy is to prioritize negative examples that are difficult for the current model to correctly classify or recognize as training examples during training. In practice, negative examples (i.e., non-target examples) may far outnumber positive examples (target examples). This helps identify the most challenging examples from a large number of negative examples, allowing the training process to focus more on these potentially misclassified negative examples.

[0080] In the embodiment of the present application, difficult samples are mined based on the characteristics of triples, and a segmented sampling strategy is proposed to optimize the selection and utilization efficiency of negative samples. Table 1 is a segmented sampling table of the training data set provided in the embodiment of the present application.

[0081] Table 1

[0082]

[0083] With reference to Table 1, the segmented sampling strategy in the embodiment of the present application specifically includes:

[0084] Set the sample type, including: positive sample, negative sample of the same head entity, negative sample of ambiguous entity, negative sample of irrelevant entity. Q represents the question, (h, r, t) represents the correct triple, which is a positive sample; (h, r ′ ,t ‘ ) represents a triplet with the same head entity but different predicates, which is a negative sample of the same head entity; (h ′ ,r ′ ,t ‘ ) represents the triplet of ambiguous entities, which is the negative sample of ambiguous entities; (h0, r0, t0) represents the triplet of irrelevant entities, which is the negative sample of irrelevant entities.

[0085] We set similarity scores and sample sizes for different sample types in a segmented manner, and randomly extract triplets through Elasticsearch to construct a training dataset. Positive and negative samples are randomly extracted from triplets that meet the criteria through Elasticsearch. Different levels of similarity score labels and sample sizes are assigned based on the sample's characteristics to enhance the model's ability to recognize different entity features and accurately distinguish similar entities.

[0086] Based on the constructed training dataset, train the SBERT candidate triple ranking sub-model.

[0087] The embodiment of the present application uses a segmented sampling strategy to help the SBERT candidate triple ranking model more effectively extract and utilize important entity features, thereby accurately distinguishing similar entities and entity triplets.

[0088] In some embodiments, the training process of the SBERT candidate triple ranking sub-model adopts an attention pooling operation.

[0089] Specifically, the traditional SBERT model uses three pooling strategies to output fixed-size sentence embeddings, namely: Token embedding using special tags (such as CLS), average pooling, and maximum pooling.

[0090] In order to more accurately capture and emphasize the important semantic components of a sentence, an attention pooling mechanism is introduced in the embodiments of the present application. Attention pooling is a pooling method used to process sequence data or image data in deep learning. It optimizes the pooling process by introducing an attention mechanism. Attention pooling calculates the attention weight of each position in the input sequence and performs a weighted average of the input sequence based on these weights to extract key information, which can focus more on important parts of the data and improve the performance of the model.

[0091] This mechanism dynamically assigns importance weights to each word embedding in a sentence, enabling the model to extract key information more carefully, thereby optimizing the sentence similarity evaluation process. The calculation formula is as follows:

[0092]

[0093] Among them, X represents the word embedding matrix, w represents the trainable global attention weight, and x j represents the embedding vector of the jth word in the sequence, x i represents the embedding vector of the i-th word in the sequence, a i Represents the weight of each word, and the final v represents the comprehensive embedding of the sentence.

[0094] In the embodiments of the present application, the SBERT problem mapping model improves the accuracy of calculating the semantic similarity between questions and triples through a segmented sampling strategy and an improved sentence embedding method, and effectively solves the problems of entity ambiguity and head entity detection errors.

[0095] Figure 3 This is the second flowchart of the retrieval based on the SBERT question mapping model provided in the embodiment of the present application, such as Figure 3 As shown in the figure, the SBERT question mapping model proposed above is used to realize the retrieval of knowledge graphs. In order to train the question mapping model belonging to the computer field, 2000 computer-related questions are collected and the triple correspondence between them and the knowledge graph is completed. All triples of the constructed knowledge graph are stored in the Elasticsearch index, and then head entity detection, candidate triple retrieval and candidate triple sorting are performed in sequence to select the target triple with a similarity greater than 0.9 with the question.

[0096] In the embodiment of the present application, a retrieval method based on the SBERT question mapping model is introduced to effectively map students' questions to triples in the knowledge graph. This process improves the system's ability to accurately handle questions through steps such as head entity detection, candidate triple retrieval, and triple sorting.

[0097] ③ Retrieval based on vector similarity

[0098] A direct similarity search strategy based on the vector space model is introduced. By mapping student input and information in the knowledge graph into points in the vector space, the similarity index between them is calculated. By introducing vector similarity search, the system's fuzzy query processing capabilities are significantly improved, providing students with a more flexible and intelligent question-answering service experience.

[0099] During implementation, it is important to ensure that both entity and question vector representations are generated by the same embedding model to ensure consistency in the embedding space. We use OpenAIEmbeddings technology to convert entity data into fixed-dimensional numerical vectors that capture the core characteristics and deep semantic information of the entity. Cosine similarity is used as a metric to quantify the similarity between two vectors.

[0100] After completing the vector representation of the knowledge graph entities, a vector index was constructed in Neo4j, focusing on identifying conceptual entities with semantic content similar to the student's query. While the query process for this vector index is similar in nature to the Twin BERT search, it differs in that it only uses vectors to retrieve relevant entities, without distinguishing between ambiguous entities or determining specific content, providing more context for LLM analysis.

[0101] When instructing an agent to perform a vector-based search, the first step is to convert the query into its corresponding vector representation. The next step is to use this representation vector as a parameter, along with a specially constructed vector index, to retrieve relevant information from the database. The information retrieved is then returned to the agent in the form of relation / attribute triples for further processing and application.

[0102] In the embodiment of the present application, three retrieval methods are adopted, namely, retrieval based on Cypher query statements, retrieval based on the SBERT problem mapping model, and retrieval based on vector similarity, which respectively provide multi-angle retrieval capabilities for structured data retrieval, natural language query adaptability, and fuzzy query processing capabilities; the combined use of these methods improves the quality and speed of the system's response to student queries.

[0103] Among them, Neo4j search based on the Cypher graph query language can provide highly accurate results, but at the same time places high demands on the accuracy of student input. In contrast, search based on the twin BERT provides a more forgiving query environment, suitable for students with a less in-depth understanding of knowledge graph structures. Vector similarity search technology builds on this foundation, further reducing the reliance on input standardization. The combined application of these methods gives the question-answering system a high degree of flexibility, enabling it to effectively respond to a variety of query requirements. While ensuring the accuracy of query results, it also enhances the system's user-friendliness and adaptability.

[0104] In some embodiments, the knowledge graph is pre-built based on the following steps:

[0105] Use the pre-trained large language model fine-tuned based on LoRA to extract information from the textbooks corresponding to the question-answering subject area to obtain the knowledge ontology set, relationship set, and attribute set;

[0106] Based on the knowledge ontology set and relationship set, the chapters and sections in the textbook are mapped to the Chapter nodes in the first layer and the Section nodes in the second layer of the knowledge graph to build a structural knowledge graph; the concepts in the textbook are mapped to the Concept nodes in the third layer of the knowledge graph, and the concept-related knowledge is mapped to other nodes mapped to the fourth layer of the knowledge graph to build a conceptual knowledge graph;

[0107] Based on the structure-level knowledge graph and the concept-level knowledge graph and their relationships, combined with the attribute set, an instance-level knowledge graph is constructed and stored and visualized using the Neo4j graph database.

[0108] Specifically, information extraction for building knowledge graphs for computer science courses presents unique challenges. Course texts, often sourced from textbooks or web pages, are poorly structured and highly coupled. Conceptual relationships are often deeply embedded within the textbooks, making some relationships difficult to extract directly. However, the texts contain rich descriptions of conceptual attributes, such as the definition, functions, and social impact of computers. This information constitutes the properties of computer-related concepts. Therefore, extracting conceptual attributes and relationships from textbook texts is a key task in building knowledge graphs. Given the advantages of large language models in text understanding and generation, we consider using them to directly generate structured information.

[0109] By pre-training a large language model using a different dataset or pre-training multiple large language models using different datasets, we can perform a variety of information extraction tasks, such as entity recognition, relationship extraction, attribute extraction, and event extraction, based on the knowledge source corresponding to the question-answering topic. The knowledge source can be a textbook or a webpage; the examples in this application focus primarily on textbooks.

[0110] The LoRA fine-tuning method is a low-resource parameter fine-tuning technique for large language models. While traditional fine-tuning methods involve updating all parameters of a pre-trained model to adapt to the downstream task, LoRA proposes a more efficient strategy that aims to reduce the number of parameters that need to be updated during fine-tuning, thereby lowering computational and storage costs. Its core approach is to introduce additional learnable weights from a low-rank decomposition into the feedforward network of the pre-trained model, reducing the number of parameters and freezing the original weights, thereby achieving efficient model fine-tuning.

[0111] The LoRA fine-tuned pre-trained large language model is used to extract information from the textbooks corresponding to the question-answering subject area, and obtain the knowledge ontology set, relationship set and attribute set as the basis for knowledge graph generation.

[0112] Figure 4 This is a schematic diagram of the process of creating a knowledge graph provided by the embodiment of this application, such as Figure 4 As shown, in the embodiment of the present application, the knowledge graph is constructed using a top-down approach, and the construction of the knowledge graph is divided into two levels: the structural level and the conceptual level.

[0113] The structural level nodes are the top-level nodes of the knowledge graph, representing the order of course learning, determined directly from the textbook chapters. The conceptual level reflects more specific concept entity attributes and the relationships between concepts, offering greater flexibility. A fine-tuned large model is used to extract information from long texts to determine the attributes and relationships of concept nodes. At the same time, the existing knowledge graph is combined for knowledge fusion and alignment, and the Neo4j graph database is used for storage and visualization of the knowledge graph. The following is the knowledge graph construction process:

[0114] Regarding the construction of the ontology knowledge graph at the structural level: The setting at the structural level helps to understand the students’ poor grasp of a certain part, so that teachers can provide targeted guidance and personalized teaching. The construction method of the ontology knowledge graph at the structural level is as follows:

[0115] Considering the universality of the construction process, based on the knowledge ontology set and relationship set, the chapters and sections of the textbook are directly mapped to the Chapter node in the first layer of the knowledge graph and the Section node in the second layer. The Concept node is preliminarily determined in the third layer, which belongs to the concept level construction. The classes and their meanings are as follows:

[0116] Chapter corresponds to the chapter in the textbook, is the node with the largest granularity, and belongs to the structural level;

[0117] Section corresponds to the section in the textbook, which is the node under Chapter and belongs to the structural level;

[0118] Concept corresponds to the knowledge points in the textbook, is the node of the next level under Section, and belongs to the conceptual level.

[0119] Furthermore, in the course knowledge framework, knowledge points never appear in isolation, and knowledge points should be connected to form a network of knowledge structures. In this way, learners can extend the overall structure through a knowledge point, which helps learners to master knowledge flexibly. Therefore, the embodiment of the present application starts from the overall level and designs the following semantic relationships from the structural level to connect all nodes. It is assumed that the instance nodes of Chapter are {C1, C2, ..., Cn}, the instance nodes of Section are {S1, S2, ..., Sn}, and the instance nodes of Concept are {Con1, Con 2, ..., Conn}. The semantic relationships between nodes will be shown in the form of triples (A, relationship, B), all of which are unidirectional relationships from A to B, as follows:

[0120] Include relationship: (C1, includes, S1), (S1, includes, Con 1) is used to describe the relationship between chapters and sections, sections and concepts, and is usually used to represent a hierarchical structure.

[0121] Successor relationships: (C1, next, C2), (S1, next, S2), (Con 1, next, Con2) are used to describe the order between chapters, and are usually relationships between nodes at the same level.

[0122] Furthermore, relationships are designed to represent the connection between knowledge points and external entities, while attributes are designed to describe the inherent characteristics of knowledge points, allowing for further detailed attribute decomposition of specific knowledge points. Attributes for Chapter and Section nodes include "name" and "description."

[0123] Furthermore, instance creation is to find the specific case under each class, fill in the case's attributes and relationships under the corresponding class. Taking the C++ course as an example, according to the chapter structure of the textbook, directly construct instances of the Chapter node and Section node.

[0124] Regarding the construction of the ontology knowledge graph at the conceptual level: Unlike the ontology at the structural level, which has a relatively clear relationship hierarchy and attributes, the basic node at the conceptual level is the Concept node, and the Concept node cannot be completely determined by the textbook subheadings and long text. Because textbooks often describe the transitions and connections between knowledge points in detail to facilitate students' understanding, the construction of the knowledge graph should simplify this part and increase the structural relationships. The method for constructing the ontology knowledge graph at the conceptual level is as follows:

[0125] Based on the knowledge ontology and relationship sets, concepts in the textbook are mapped to the Concept node in the third layer of the knowledge graph, and concept-related knowledge is mapped to other nodes in the fourth layer of the knowledge graph. Considering the complexity of course concepts and to increase the structural and semantic relationship between concepts, in addition to the Concept node as the foundation, Feature node, Reason node, Method node, Direction node, Standard node, and Example node are designed to expand the Concept node. The classes and their meanings are as follows:

[0126] Concept: basic concept nodes, such as "computer" and "algorithm";

[0127] Feature: used to represent the characteristics of a concept, such as the characteristic "effectiveness" of "algorithm";

[0128] Reason: used to express the reason for a concept, for example, the reason why computers use “binary code” is “simple calculation”;

[0129] Method: The method used to represent the concept, such as the two methods of "numerical representation": "fixed-point number" and "floating-point number";

[0130] Example: used to represent a case of a concept, such as "algorithm design case";

[0131] Direction: used to indicate the development direction of a concept, for example, the development direction of "computer" is "miniaturization";

[0132] Standard: A standard used to represent a concept, such as the two indicators for measuring the performance of an algorithm: time complexity and space complexity.

[0133] Furthermore, to connect the Concept node with other nodes, seven semantic relationships are designed, which are reflected in the textbook level. Assuming that the instances of Concept are Con1 and Con2, the instance of Feature is F1, the instance of Reason is R1, the instance of Method is M1, the instance of Example is Exam1, the instance of Direction is Dir1, and the instance of Standard is St1, the relationships are as follows:

[0134] ① Between the Concept node and other nodes:

[0135] Feature relationship: (Con1, Feature, F1) is used to connect Concept and Feature nodes;

[0136] Cause relationship: (Con1, Reason, R1) is used to connect Concept and Reason nodes;

[0137] Representation method relationship: (Con1, representation method, M1) is used to connect Concept and Method nodes;

[0138] Case relationship: (Con1, Case, Exam1) is used to connect the Concept and Example nodes;

[0139] Development direction relationship: (Con1, development to, Dir1) is used to connect Concept and Direction nodes;

[0140] Performance indicator relationship: (Con1, performance indicator, St1) is used to connect Concept and Standard nodes;

[0141] ②The inter-concept relationships between Concept nodes: (Con1, related to, Con2), (Con1, belongs to, Con2), (Con1, includes, Con2), (Con1, classified, Con2), (Con1, stored in, Con2), (Con1, processed, Con2), (Con1, applied, Con2), (Con1, type, Con2), (Con1, composed of, Con2), (Con1, has, Con2), (Con1, involves, Con2), (Con1, field, Con2), (Con1, condition, Con2) and (Con1, adopted, Con2) are used to connect Concept and Concept nodes.

[0142] Furthermore, for several nodes other than the Concept node, two attributes, "name" and "description", are designed; and for the Concept node, the text content of the C++ textbook is counted and 26 attributes are designed as shown below: name, content, composition, representation method, description, time, format, application scope, function, source, value, application case, significance, unit, classification, core technology, characteristics, quantity, including, development trend, advantages, elements, principles, disadvantages, operation, and attention.

[0143] Furthermore, conceptual instances are created, and information is extracted and the conceptual structure is manually refined using the large language model fine-tuned by LoRA. The concepts are then processed into the form of <entity, relationship, entity> relationship triples or <entity, attribute, content> attribute triples.

[0144] Finally, a total of 697 relationship triples and 1,118 attribute triples were extracted. After manual inspection, 561 instances and 629 relationships of the knowledge graph were created using the Python library and neo4j.

[0145] In the embodiment of the present application, LoRA technology is used to perform instruction-level fine-tuning on the large language model to effectively extract key information from long texts. This not only improves the accuracy of information extraction, but also better integrates the extracted information into the knowledge graph, enriches the content of the knowledge graph, and accurately displays the connection between knowledge concepts; at the structural level of the knowledge graph, the logical relationship and learning sequence between course chapters are clearly displayed, providing learners with a clear learning path to help them better master the learning content; further through the construction of the conceptual level, the intrinsic connection between different knowledge points is revealed, which promotes the understanding of the overall course structure and deep knowledge system.

[0146] In some embodiments, the LLM-based knowledge graph generation-assisted teaching question-answering method also includes: if relevant knowledge of the question to be answered is not retrieved in the knowledge graph, relevant knowledge of the question to be answered is retrieved based on the local knowledge base.

[0147] Specifically, if the relevant knowledge of the question to be answered is not retrieved in the knowledge graph, text segmentation and similarity retrieval are performed in the local knowledge base to obtain the relevant knowledge of the question to be answered, which is then input into the large language model to combine the answers and return them to the user.

[0148] While knowledge graphs, with their clear structural characteristics, have been widely used in intelligent question-answering systems, providing an efficient framework for knowledge management and querying, the limitations of independent knowledge graphs are becoming increasingly apparent when addressing the diversity and complexity of student needs. In particular, for queries that rely on extensive background knowledge or unstructured data, the structural limitations of knowledge graphs can lead to insufficient information support. Furthermore, building and maintaining a fully updated knowledge graph requires significant human, material, and time resources, and places stringent demands on data integrity, real-time availability, and accuracy.

[0149] By introducing a local knowledge base, we can overcome these challenges and enhance the system's problem-solving capabilities. This system aggregates resources such as teaching syllabi, e-textbooks, and authoritative websites as information sources for the local knowledge base. This local knowledge base constrains the scope of the large language model's responses, ensuring the rigor and reliability of its output while also increasing the interpretability of the knowledge source. Compared to strategies based on fine-tuning on domain-specific data, this allows for the deployment of large, specialized domain models without requiring retraining.

[0150] On the other hand, local knowledge bases aggregate a wide range of unstructured data, encompassing a rich set of information to support responses to a wider variety of questions. Unlike structured knowledge graphs, local knowledge bases do not focus on the structured representation of data. This makes them particularly advantageous when answering open-ended, explanatory queries, or complex queries that require the integration of multiple information sources. By adding specialized question-and-answer modules based on local knowledge bases, we can expand the existing knowledge graph's answering capabilities and effectively broaden the intelligent question-and-answer system's response capabilities.

[0151] The technical solution provided in the embodiment of the present application is further illustrated below through a specific example.

[0152] Figure 5 The embodiment of the present application provides a second flow chart of the knowledge graph generation-assisted teaching question-answering method based on LLM provided in the embodiment of the present application, such as Figure 5 As shown, the method includes the following steps:

[0153] Step a: The user enters a question.

[0154] Step b: Use the BERT and Softmax-based text classification model to filter questions and determine whether the input question is related to computer topics. Computer-related questions will proceed to the next step, otherwise they will be rejected.

[0155] Step c: Enter LangChain and search the knowledge graph to obtain relevant knowledge for the input question. Search methods include: Cypher query-based search, SBERT question mapping model-based search, and vector similarity-based search. The general process of SBERT question mapping model-based search includes: head entity detection, candidate triple search, and candidate triple ranking.

[0156] Step d: If the relevant knowledge of the input question is not retrieved in the knowledge graph, the relevant knowledge of the input question is retrieved based on the local knowledge base.

[0157] Step e: Input the retrieved relevant knowledge into the large language model, generate the answer and return it to the user.

[0158] Figure 6 This is a schematic diagram of the architecture of the knowledge graph generation-assisted teaching question-answering system based on LLM provided in the embodiment of the present application. Figure 6 As shown, the system includes:

[0159] The question filtering module is used to filter the questions to be answered by the user and determine whether the questions to be answered are in line with the scientific and reasonable question answering subject area;

[0160] A professional question-answering module based on a knowledge graph is used to search for questions that match the Q&A topic domain in a pre-built knowledge graph, determine relevant knowledge for the questions, input relevant knowledge into a large language model, and generate answers to the questions. Questions that match the Q&A topic domain are searched using at least one of the following methods: Cypher query; search based on the SBERT question mapping model; or search based on vector similarity.

[0161] The professional question-and-answer module based on the local knowledge base is used to retrieve relevant knowledge of the question to be answered based on the local knowledge base if the relevant knowledge of the question to be answered is not retrieved in the knowledge graph; the relevant knowledge is input into the large language model to generate the answer to the question to be answered.

[0162] In some embodiments, the question filtering module is specifically used to: input the question to be answered into the text classification model based on the improved BERT for question filtering, and determine whether the question to be answered is consistent with the question answering topic area; the text classification model based on the improved BERT includes an input layer, a Transformer encoder layer, a fully connected layer, a Softmax layer and an output layer.

[0163] In some embodiments, the retrieval based on the SBERT question mapping model in the professional question-answering module based on the knowledge graph includes: inputting the question to be answered that meets the question-answering subject area into the BERT-CRF head entity detection model for head entity detection, and outputting the head entity in the question to be answered; retrieving multiple candidate triples in the knowledge graph based on the head entity; inputting the multiple candidate triples into the SBERT candidate triple ranking model for semantic similarity sorting, and outputting the target triples corresponding to the question to be answered in the knowledge graph.

[0164] In some embodiments, the BERT-CRF head entity detection submodel includes a data processing layer, a BERT-based feature processing layer and a CRF-based sequence labeling layer; outputting the head entity in the question to be answered includes: the data processing layer performs word segmentation and BIO labeling on the question to be answered, and outputs a BIO sequence; the BERT-based feature processing layer extracts the contextual features of each word based on the BIO sequence, and generates a corresponding word feature vector; the CRF-based sequence labeling layer performs label prediction based on the word feature vector, obtains the globally optimal label sequence based on the transition probability between labels, and decodes to determine the head entity in the question to be answered.

[0165] In some embodiments, multiple candidate triples are retrieved from the knowledge graph based on the head entity, including: using the head entity as a query keyword to perform head entity matching through Elasticsearch's Match query to retrieve multiple candidate triples.

[0166] In some embodiments, the SBERT candidate triple ranking submodel includes two BERTs with the same network structure and shared weights. The SBERT candidate triple ranking submodel is trained based on the following steps: setting sample types, including: positive samples, negative samples of entities with the same head, negative samples of ambiguous entities, and negative samples of irrelevant entities; setting the similarity scores and sample quantities of different sample types in a segmented manner, and randomly extracting triplets through Elasticsearch to construct a training data set; based on the constructed training data set, training the SBERT candidate triple ranking submodel.

[0167] In some embodiments, the training process of the SBERT candidate triple ranking sub-model adopts an attention pooling operation.

[0168] In some embodiments, the knowledge graph is pre-built based on the following steps:

[0169] Use the pre-trained large language model fine-tuned based on LoRA to extract information from the textbooks corresponding to the question-answering subject area to obtain the knowledge ontology set, relationship set, and attribute set;

[0170] Based on the knowledge ontology set and relationship set, the chapters and sections in the textbook are mapped to the Chapter nodes in the first layer and the Section nodes in the second layer of the knowledge graph to build a structural knowledge graph; the concepts in the textbook are mapped to the Concept nodes in the third layer of the knowledge graph, and the concept-related knowledge is mapped to other nodes mapped to the fourth layer of the knowledge graph to build a conceptual knowledge graph;

[0171] Based on the structure-level knowledge graph and the concept-level knowledge graph and their relationships, combined with the attribute set, an instance-level knowledge graph is constructed and stored and visualized using the Neo4j graph database.

[0172] In some embodiments, the relationship set includes the successor relationship between nodes of the same layer at the structural level, the inclusion relationship between nodes of different layers at the structural level and between the structural level and the conceptual level, the characteristic relationship, cause relationship, representation method relationship, case relationship, development direction relationship and performance indicator relationship between the Concept node and other nodes at the conceptual level, and the inter-concept relationship between different Concept nodes at the conceptual level; the attribute set includes name attributes and description attributes.

[0173] It is understood that the detailed functional implementation of each of the above-mentioned units / modules can be found in the description of the aforementioned method embodiment and will not be described in detail here. It should be understood that the above-mentioned device is used to execute the method in the above-mentioned embodiment, and the corresponding program modules in the device have similar implementation principles and technical effects as those described in the above-mentioned method. The working process of the device can refer to the corresponding process in the above-mentioned method and will not be described in detail here.

[0174] Based on the methods described in the above embodiments, embodiments of the present application provide an electronic device. The device may include: at least one memory for storing programs and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is configured to execute the methods described in the above embodiments.

[0175] Figure 7 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 7As shown, the electronic device may include: a processor (Processor) 701, a communication interface (Communications Interface) 702, a memory (Memory) 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other via the communication bus 704. The processor 701 can call the software instructions in the memory 703 to execute the method described in the above embodiment.

[0176] In addition, the logic instructions in the above-mentioned memory 703 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the relevant technology, or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.

[0177] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0178] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0179] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0180] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0181] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0182] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

[0183] It is easy for those skilled in the art to understand that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A knowledge graph generation-assisted teaching question answering method based on LLM, characterized by: include: Filter the questions to be answered by the user to determine whether the questions to be answered are in line with the scientific and reasonable question-answering subject areas; In the pre-constructed knowledge graph corresponding to the question-answering subject area, the LangChain agent is used to search for the questions to be answered that match the question-answering subject area, and determine the relevant knowledge of the questions to be answered; Inputting the relevant knowledge into a large language model to generate an answer to the question to be answered; The retrieval of the questions to be answered that are in the question-answering subject area adopts at least one of the following retrieval methods: retrieval based on Cypher query statements; retrieval based on the SBERT question mapping model; retrieval based on vector similarity; The retrieval based on the SBERT question mapping model includes: Input the question to be answered that meets the question answering topic area into the BERT-CRF head entity detection sub-model for head entity detection, and output the head entity in the question to be answered; Retrieving multiple candidate triples in the knowledge graph based on the head entity; Input the multiple candidate triples into the SBERT candidate triple ranking sub-model for semantic similarity sorting, and output the target triples corresponding to the question to be answered in the knowledge graph; The BERT-CRF head entity detection sub-model includes a data processing layer, a BERT-based feature processing layer, and a CRF-based sequence annotation layer; the output of the head entity in the question to be answered includes: The data processing layer performs word segmentation and BIO annotation on the question to be answered, and outputs a BIO sequence; The BERT-based feature processing layer extracts context features of each word based on the BIO sequence and generates a corresponding word feature vector; The CRF-based sequence annotation layer performs label prediction based on the word feature vector, obtains the globally optimal label sequence based on the transition probability between labels, and decodes to determine the head entity in the question to be answered; The retrieving multiple candidate triples in the knowledge graph based on the head entity includes: Using the head entity as a query keyword through the Match query of Elasticsearch to perform head entity matching, and retrieve the multiple candidate triples; The SBERT candidate triple ranking sub-model includes two BERTs with the same network structure and shared weights. The SBERT candidate triple ranking sub-model is trained based on the following steps: Set the sample type, including: positive sample, negative sample of the same entity, negative sample of ambiguous entity, and negative sample of irrelevant entity; The similarity scores and sample numbers for different sample types are set in a segmented manner, and triples are randomly extracted through Elasticsearch to construct a training dataset; Based on the constructed training dataset, the SBERT candidate triple ranking sub-model is trained.

2. The knowledge graph generation-assisted teaching question answering method according to claim 1 is characterized in that: The training process of the SBERT candidate triple ranking sub-model adopts the attention pooling operation.

3. The knowledge graph generation-assisted teaching question answering method according to claim 1 is characterized in that: The knowledge graph is pre-built based on the following steps: Use the pre-trained large language model fine-tuned based on LoRA to extract information from the textbooks corresponding to the question-answering subject area to obtain the knowledge ontology set, relationship set and attribute set; Based on the knowledge ontology set and the relationship set, the chapters and sections in the textbook are mapped to the Chapter nodes of the first layer and the Section nodes of the second layer of the knowledge graph to construct a structural level knowledge graph; the concepts in the textbook are mapped to the Concept nodes of the third layer of the knowledge graph, and the concept-related knowledge is mapped to other nodes of the fourth layer of the knowledge graph to construct a conceptual level knowledge graph; Based on the structure-level knowledge graph and the concept-level knowledge graph and the relationship between them, combined with the attribute set, an instance-level knowledge graph is constructed and stored and visualized using the Neo4j graph database.

4. The knowledge graph generation-assisted teaching question answering method according to claim 3 is characterized in that: The relationship set includes the successor relationship between nodes of the same layer at the structural level, the inclusion relationship between nodes of different layers at the structural level and between the structural level and the concept level, the characteristic relationship, cause relationship, representation method relationship, case relationship, development direction relationship and performance indicator relationship between the Concept node and other nodes at the concept level, and the inter-concept relationship between different Concept nodes at the concept level; The attribute set includes a name attribute and a description attribute.

5. The knowledge graph generation-assisted teaching question answering method according to claim 1 is characterized in that: The method further comprises: If relevant knowledge of the question to be answered is not retrieved in the knowledge graph, relevant knowledge of the question to be answered is retrieved based on the local knowledge base.

6. A knowledge graph generation-assisted teaching question answering system based on LLM, characterized by: include: A question filtering module is used to filter the questions to be answered by the user to determine whether the questions to be answered are in line with the scientific and reasonable question answering subject area; A professional question-answering module based on a knowledge graph is used to search for questions that match the question-answering topic domain in a pre-constructed knowledge graph corresponding to the question-answering topic domain, determine relevant knowledge about the questions to be answered, input the relevant knowledge into a large language model, and generate answers to the questions to be answered. The search for questions that match the question-answering topic domain adopts at least one of the following methods: Cypher query; search based on the SBERT question mapping model; search based on vector similarity; A professional question-answering module based on a local knowledge base is used to retrieve relevant knowledge of the question to be answered based on the local knowledge base if no relevant knowledge of the question to be answered is retrieved in the knowledge graph; input the relevant knowledge into the large language model to generate an answer to the question to be answered; The retrieval based on the SBERT question mapping model includes: Input the question to be answered that meets the question answering topic area into the BERT-CRF head entity detection sub-model for head entity detection, and output the head entity in the question to be answered; Retrieving multiple candidate triples in the knowledge graph based on the head entity; Input the multiple candidate triples into the SBERT candidate triple ranking sub-model for semantic similarity sorting, and output the target triples corresponding to the question to be answered in the knowledge graph; The BERT-CRF head entity detection sub-model includes a data processing layer, a BERT-based feature processing layer, and a CRF-based sequence annotation layer; the output of the head entity in the question to be answered includes: The data processing layer performs word segmentation and BIO annotation on the question to be answered, and outputs a BIO sequence; The BERT-based feature processing layer extracts context features of each word based on the BIO sequence and generates a corresponding word feature vector; The CRF-based sequence annotation layer performs label prediction based on the word feature vector, obtains the globally optimal label sequence based on the transition probability between labels, and decodes to determine the head entity in the question to be answered; The retrieving multiple candidate triples in the knowledge graph based on the head entity includes: Using the head entity as a query keyword through the Match query of Elasticsearch to perform head entity matching, and retrieve the multiple candidate triples; The SBERT candidate triple ranking sub-model includes two BERTs with the same network structure and shared weights. The SBERT candidate triple ranking sub-model is trained based on the following steps: Set the sample type, including: positive sample, negative sample of the same entity, negative sample of ambiguous entity, and negative sample of irrelevant entity; The similarity scores and sample numbers for different sample types are set in a segmented manner, and triples are randomly extracted through Elasticsearch to construct a training dataset; Based on the constructed training dataset, the SBERT candidate triple ranking sub-model is trained.

Images