Knowledge manual question-answering system and question-answering method based on large language model

Abstract

The invention relates to the technical field of model training, and discloses a knowledge manual question-answering system and question-answering method based on a large language model, and the system comprises a knowledge slicing module which is used for dividing a knowledge manual into a plurality of independent knowledge units according to a preset rule; the text vectorization and recall module is used for converting the knowledge units into text vectors through an embedded model and storing the text vectors, and is also used for retrieving the knowledge unit with the highest similarity in the knowledge base based on the question input by the user; the large language model initialization module is used for configuring a large language model special for the field and setting a cue word template; the workflow arrangement module is used for deploying knowledge retrieval nodes and big language model nodes through a workflow platform, inputting retrieved knowledge units as contexts into a big language model, generating final answers and feeding back the final answers to the user; the problems of low query efficiency and poor accuracy caused by insufficient training data in an existing query method based on a large language model are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of model training, and in particular, to a knowledge manual question-answering system and a question-answering method based on a large language model. Background Art

[0002] With the rapid development of artificial intelligence technology, generative artificial intelligence (Generative AI) has gradually become the focus of attention in various industries. Generative AI can generate new content or patterns based on existing data and is widely used in fields such as natural language processing, image generation, and music creation, thus providing a feasible solution to the above problems - using generative AI to generate problem solutions. However, developing generative AI applications faces many challenges, such as complex model selection, high deployment difficulty, data privacy and security issues, etc.

[0003] A large language model (Large Language Model, abbreviated as LLM) is an artificial intelligence technology based on deep learning, mainly used for natural language processing (NLP) tasks. These models learn the complex rules of language by training on large-scale datasets, so as to be able to generate natural language text or understand the meaning of language text. LLM usually adopts the Transformer architecture, which allows the model to process the entire sequence in parallel, thereby improving the processing efficiency. However, LLM is highly dependent on training data, and the size and quantity of training data will directly affect the performance of AI. If the training data is insufficient, it will lead to problems such as low query efficiency and poor query accuracy. It can be seen that the existing query methods based on large language models have problems of low query efficiency and poor accuracy due to insufficient training data. Summary of the Invention

[0004] The present invention provides a knowledge manual question-answering system and a question-answering method based on a large language model to solve the problems of low query efficiency and poor accuracy existing in the existing query methods based on large language models due to insufficient training data.

[0005] To achieve the above object, the present invention is realized through the following technical solutions: In the first aspect, the present invention provides a knowledge manual question-answering system based on a large language model, including: A knowledge slicing module, configured to divide a knowledge manual into multiple independent knowledge units according to a preset rule, and each knowledge unit includes a question and a solution corresponding to the question included in the knowledge unit; A text vectorization and recall module, configured to convert the knowledge unit into a text vector through an embedding model and store it, and also configured to retrieve the knowledge unit with the highest similarity in the knowledge base based on a user input question; A large language model initialization module for configuring a domain-specific large language model and setting a prompt template, where the prompt template includes a role definition, processing steps, and context integration rules; A workflow orchestration module for deploying knowledge retrieval nodes and large language model nodes through a workflow platform, inputting the retrieved knowledge units as context into the large language model, and generating a final answer by the large language model and feeding it back to the user.

[0006] Optionally, dividing the knowledge manual into multiple independent knowledge units according to a preset rule, including: Segmenting the knowledge manual by a custom delimiter, where each segmented paragraph corresponds to an independent combination of questions and solutions, and inserting and matching the delimiter through manual and / or automated tools.

[0007] Optionally, converting the knowledge units into text vectors through an embedding model, including: Using a pre-trained language representation model to tokenize, encode, and generate embedding vectors for the text, and representing the text as a semantic vector with a fixed dimension through average pooling.

[0008] Optionally, the recall method of the text vectorization and recall module is hybrid recall, including: Semantic recall based on cosine similarity and keyword recall based on an inverted index, where the proportion of semantic recall is 70% and the proportion of keyword recall is 30%, and the recall results are re-ranked and the top two most relevant knowledge units are returned.

[0009] Optionally, the configured large language model is a 72B-Chat model debugged for a specific domain; The prompt template is designed using the chain of thought and few-shot example methods to guide the model to generate answers that meet the domain requirements.

[0010] Optionally, the workflow platform is the Dify platform; The output content of the knowledge retrieval node includes: the retrieved knowledge units and the document identifiers of the source documents of the knowledge units; The large language model node generates a structured answer by integrating context and prompts.

[0011] Optionally, the system further includes: a feedback optimization module; The feedback optimization module is used to receive the user's evaluation of the answer and continuously optimize the knowledge base retrieval strategy and the large language model generation logic according to the evaluation content.

[0012] Second aspect, an embodiment of the present application provides a method for answering questions in a knowledge manual based on a large language model, which is applied to the knowledge manual question answering system based on a large language model as described in the first aspect. The method includes: Obtain the question input by the user, and divide the knowledge manual into multiple independent knowledge units according to a preset rule through a knowledge slicing module. Each knowledge unit contains a question and a solution corresponding to the question in the knowledge unit; Use the text vectorization and recall module to convert the knowledge unit into a text vector through an embedding model and store it, and retrieve the knowledge unit with the highest similarity in the knowledge base based on the question vector input by the user; Configure a domain-specific large language model through the large language model initialization module and set a prompt template, where the prompt template includes role definition, processing steps, and context integration rules; Use the workflow orchestration module to deploy a knowledge retrieval node and a large language model node on the workflow platform, input the retrieved knowledge unit as context into the large language model, and generate a final answer by the large language model and feedback it to the user.

[0013] Optionally, the similarity calculation in the vector retrieval uses the CoSENT algorithm, and the formula of the algorithm satisfies the following relationship: ; In the formula, CS represents similarity, represents the dot product of vector and vector , represents the product of the modulus lengths of vector and vector .

[0014] Beneficial effects: The knowledge manual question answering system based on a large language model provided by the present invention first establishes a knowledge base based on the existing knowledge manual. These knowledge bases divide the knowledge manual into multiple question-and-answer paragraphs, and these knowledge paragraphs contain existing questions and solutions in the past, which can be used for subsequent retrieval according to the input question and return paragraphs containing the retrieved questions and relevant answers; by using the retrieved paragraphs as context prompts (prompts) and inputting them into the LLM, the LLM then generates a final answer output according to the predefined Prompt and context content; through the workflow mechanism and the existing knowledge manual, reliable knowledge context is generated and handed over to the LLM, and then sorted and generated reliable answers according to the predefined Prompt, avoiding problems such as low query efficiency, poor accuracy due to insufficient training data, and time-consuming and laborious manual search for solutions. Description of the drawings

[0015] Figure 1Schematic diagram of the knowledge manual Q&A system based on the large language model according to the preferred embodiment of the present invention; Figure 2 Flow chart of the text vector embedding step according to the preferred embodiment of the present invention; Figure 3 Usage flow chart of the knowledge manual Q&A system based on the large language model according to the preferred embodiment of the present invention. Detailed implementation manners

[0016] The technical solutions of the present invention will be described clearly and completely below. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0017] Unless otherwise defined, the technical terms or scientific terms used in the present invention shall have the ordinary meanings understood by those of ordinary skill in the art to which the present invention belongs. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, the terms such as "a" or "one" do not denote a quantity limitation, but mean that there is at least one. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "up", "down", "left" and "right" are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationships will also change accordingly.

[0018] Please refer to Figure 1 , an embodiment of the present application provides a knowledge manual Q&A system based on a large language model, including: A knowledge slicing module, configured to divide a knowledge manual into multiple independent knowledge units according to a preset rule, and each knowledge unit includes a question and its corresponding solution; A text vectorization and recall module, configured to convert the knowledge units into text vectors through an embedding model and store them, and also configured to retrieve the knowledge unit with the highest similarity in the knowledge base based on the user input question; A large language model initialization module, configured to configure a domain-specific large language model and set a prompt template, where the prompt template includes a role definition, a processing step, and a context integration rule; A workflow orchestration module, configured to deploy a knowledge retrieval node and a large language model node through a workflow platform, input the retrieved knowledge unit as context into the large language model, generate a final answer and feedback it to the user.

[0019] In this embodiment, the specific implementation steps of the present invention are as follows: (1)Data preprocessing: Data cleaning: Remove duplicate and incomplete data in each knowledge manual and remove the serial numbers of the Q&A pairs in the knowledge manual. The serial numbers are used for convenient manual query of the knowledge manual and are meaningless in the Q&A system, so they need to be cleaned up.

[0020] Knowledge slicing: Segment the knowledge manual into paragraphs according to questions and solutions. Each slice contains a question and its corresponding solution. The slicing rule can use the default line break or a custom symbol as the slicing identifier. According to this rule, the continuous knowledge manual text will be split into several smaller blocks, and each block can be regarded as an independent knowledge unit or processing unit.

[0021] (2)Text vectorization and recall: Text preprocessing: Use the processing unit after obtaining the knowledge slices as the input text, and use a pre-trained language representation model to perform word segmentation, encoding, and embedded vector generation on the text. The pre-trained language representation model used here is the BERT model (Bidirectional Encoder Representation from Transformers, BERT). First, the input text is segmented into words or sub-word units by the BERT tokenizer, and special tokens [CLS] and [SEP] are added before and after the tokenization result, representing the start and end of the tokenization sequence respectively. After tokenization, use BertTokenizer to encode the tokenized text and convert it into the corresponding Token ID, which is a sequence of numbers that the model can understand.

[0022] Vector embedding: The steps of its text vector embedding are as Figure 2 shown. After the input text is converted into Token ID, these IDs are sent to the embedding layer of the model to generate the corresponding unique embedding vector. In order to capture the relationship between words in the text, the embedded vector will be sent to the Transformer encoder, which contains 12 layers of Transformer, and each layer processes the input embedded vector to generate the context representation of the text.

[0023] Vector pooling: What is output by the encoder is a sequence of embedded vectors. In order to represent the entire text as a vector, average pooling is used to represent the text as a vector. The formula for average pooling is: (1) Among them, represents the text vector, is the embedded vector of the text, is the masking technology used. Through the above operations, the finally obtained It is a 768-dimensional vector representing the semantic information of the entire input text.

[0024] Text recall: The purpose of text vectorization is to perform knowledge retrieval and matching in the knowledge base based on the user's input text later. Therefore, the recall method is crucial for the accuracy of retrieval and matching. To ensure the recall rate, a hybrid recall mode combining semantic recall and keyword recall is adopted, and the proportion of semantic recall and keyword recall is designed to be 70% to 30%. Among them, for semantic recall, the CoSENT algorithm is adopted to calculate the cosine similarity between the text vector converted from the user's input text and the text vectors in the knowledge base, and then measure their semantic similarity. The formula is: (2) Among them, represents the dot product of vector and vector , represents the product of the norms of vector and vector . The closer the similarity is to 1, the more similar the sentences are. For keyword recall, it is based on the inverted index, and relevant knowledge slices of the input text are quickly located through keyword matching. Finally, the recalled knowledge slices are re-ranked according to the weight ratio of the recall method, and the top 2 knowledge slices are returned.

[0025] (3)Model initialization: In this step, relying on the Dify platform, select the fine-tuned LLM on the Dify platform and set the corresponding prompt to guide the model to understand the specific requirements in the telecommunications field.

[0026] Domain large model: To better implement the Q&A of the knowledge manual, the 72B-Chat large model that has been debugged in the focused domain is selected as the basic LLM. Focused domain debugging means that when the model is applied in a certain field, the model is pre-trained using the pre-collected training data in that field to ensure that the model can adapt to its application in a certain field faster.

[0027] For example, if the 72B-Chat model is applied in the telecommunications field, the pre-collected training data related to telecommunications service processing is input into the model for pre-training. The trained 72B-Chat model is the model after focused domain debugging for the telecommunications field.

[0028] The above telecommunications field is only used as an example in this embodiment and is not limited. In different requirements, the model can be used in different fields.

[0029] Model Prompt: According to the card blocking knowledge manual, set the LLM to act as a telecommunications business processing expert, design the key steps to solve problems, and use methods such as CoT and few show to design the prompt to ensure the accuracy and practicality of the LLM. In the prompt, slice the recalled knowledge as context content to provide hints for the LLM.

[0030] (4) Workflow Orchestration: Through the workflow mechanism of the Dify platform, deploy the knowledge retrieval node and the LLM node, set the parameter transmission between the nodes, and implement the entire process from the user question input to the model giving an answer.

[0031] Knowledge Retrieval: Knowledge retrieval is responsible for taking the user's input text query as input, taking the recalled knowledge slices as the content field in the output object result, and adding a field knowledge in the result to store the knowledge manual to which the knowledge slices belong, for prompting the user about the document referenced by the knowledge fragment.

[0032] (5) Result Feedback: The user receives the answer generated by the system and can evaluate the result or provide feedback for the continuous optimization of the system.

[0033] Figure 3 This is the overall flowchart of the knowledge manual Q&A process based on the Dify platform of the present invention. After the user inputs a question, the system goes through steps such as vector transformation, knowledge retrieval, and LLM processing, and finally feeds back the result to the user.

[0034] The embodiments of the present application also provide a knowledge manual Q&A method based on a large language model, which is applied to a knowledge manual Q&A system based on a large language model. The method includes: Obtain the question input by the user, and divide the knowledge manual into multiple independent knowledge units according to preset rules through the knowledge slicing module, where each knowledge unit contains a question and the solution corresponding to the question in the knowledge unit; Use the text vectorization and recall module to convert the knowledge units into text vectors through an embedding model and store them, and retrieve the knowledge unit with the highest similarity in the knowledge base based on the question vector input by the user; Configure a domain-specific large language model through the large language model initialization module and set the prompt template, where the prompt template includes role definition, processing steps, and context integration rules; Use the workflow orchestration module to deploy the knowledge retrieval node and the large language model node on the workflow platform, take the retrieved knowledge units as context input into the large language model, and generate the final answer by the large language model and feed it back to the user.

[0035] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art shall fall within the protection scope determined by the claims.

Claims

1. A knowledge manual question-answering system based on a large language model, characterized in that: include: The knowledge slicing module is used to divide the knowledge manual into multiple independent knowledge units according to preset rules, and each knowledge unit contains a problem and the solution corresponding to the problem; A text vectorization and recall module, used to convert the knowledge unit into a text vector through an embedding model and store it, and also used to retrieve the knowledge unit with the highest similarity in the knowledge base based on the user input question; A large language model initialization module is used to configure a domain-specific large language model and set a prompt word template, wherein the prompt word template includes role definition, processing steps, and context integration rules; The workflow orchestration module is used to deploy knowledge retrieval nodes and large language model nodes through the workflow platform, input the retrieved knowledge units into the large language model as context, and the large language model generates the final answer and feeds it back to the user.

2. The knowledge manual question-answering system based on a large language model according to claim 1, characterized in that: Divide the knowledge manual into multiple independent knowledge units according to preset rules, including: The knowledge manual is segmented into paragraphs using custom delimiters. Each segmented paragraph corresponds to an independent combination of problems and solutions. Delimiter insertion and matching are completed through manual and / or automated tools.

3. The knowledge manual question-answering system based on a large language model according to claim 1, characterized in that: The knowledge unit is converted into a text vector through an embedding model, including: Use the pre-trained language representation model to segment, encode and generate embedding vectors for the text, and represent the text as a semantic vector of fixed dimension through average pooling.

4. The knowledge manual question-answering system based on a large language model according to claim 1, characterized in that: The recall mode of the text vectorization and recall module is a mixed recall mode, including: Semantic recall based on cosine similarity and keyword recall based on inverted index, where semantic recall accounts for 70% and keyword recall accounts for 30%. The recall results are re-sorted and the top two most relevant knowledge units are returned.

5. The knowledge manual question-answering system based on a large language model according to claim 1, characterized in that: The configured large language model is the 72B-Chat model that has been debugged in a focused field; The prompt word template is designed using chain thinking and a few-sample example method to guide the model to generate answers that meet domain requirements.

6. The knowledge manual question-answering system based on a large language model according to claim 1, characterized in that: The workflow platform is the Dify platform; The output content of the knowledge retrieval node includes: retrieved knowledge units and knowledge unit source document identifiers; The large language model node generates a structured answer by integrating context and prompt words.

7. The knowledge manual question-answering system based on a large language model according to claim 1, characterized in that: The system further comprises: a feedback optimization module; The feedback optimization module is used to receive the user's evaluation of the answer, and continuously optimize the knowledge base retrieval strategy and the large language model generation logic according to the evaluation content.

8. A knowledge manual question-answering method based on a large language model, applied to a knowledge manual question-answering system based on a large language model as claimed in any one of claims 1 to 7, characterized in that: The method comprises: Obtain the question input by the user, and divide the knowledge manual into multiple independent knowledge units according to preset rules through the knowledge slicing module, where each knowledge unit contains a question and the solution corresponding to the question; The knowledge unit is converted into a text vector by using the text vectorization and recall module through the embedding model and stored, and the knowledge unit with the highest similarity in the knowledge base is retrieved based on the question vector input by the user; Configure a domain-specific large language model and set a prompt word template through a large language model initialization module, wherein the prompt word template includes role definition, processing steps, and context integration rules; The workflow orchestration module is used to deploy knowledge retrieval nodes and large language model nodes on the workflow platform. The retrieved knowledge units are input into the large language model as context. The large language model generates the final answer and feeds it back to the user.

9. The knowledge manual question-answering method based on a large language model according to claim 8, characterized in that: The similarity calculation in the vector retrieval adopts the CoSENT algorithm, and the formula of the algorithm satisfies the following relationship: ； In the formula, CS represents similarity, Representation vector and vector The number product of Representation vector and vector The modulus-length product of .

Images