Question answering method and system based on langchain and large language model

Abstract

This application relates to a question-answering method and system based on LangChain and a large language model. The method includes: loading an LLM based on an accelerated inference framework; building a text vector knowledge base; vectorizing user questions to generate question vectors; performing similarity matching between the question vectors and the text vector knowledge base using two different vector retrieval libraries, returning two sets of similar text paragraphs; performing pairwise paragraph similarity matching on the two sets of similar text paragraphs to obtain the two similar text paragraphs with the highest paragraph similarity scores; when the highest paragraph similarity score is higher than or equal to a set threshold, performing content fusion on the two similar text paragraphs to generate a final knowledge paragraph, and generating a contextual hint template based on the final knowledge paragraph; when the highest paragraph similarity score is lower than the set threshold, using the user question as the contextual hint template; inputting the contextual hint template into the LLM, performing parallel inference, and streaming the answer.

Description

Technical Field

[0001] This application generally relates to the field of data processing, and more specifically to a question-answering method and system based on the LangChain framework and a large language model. Background Technology

[0002] Intelligent question answering systems (QA) are a high-level form of information retrieval systems. Their purpose is to accurately answer user questions using concise natural language. Compared to search engines, QA systems can better understand the true intent of user questions, thus more effectively meeting user needs. This has led to their increasing attention and promising future in the field of Natural Language Processing (NLP).

[0003] The development of question-answering systems can be summarized into three stages: structured data-based question-answering systems, free text-based question-answering systems, and question-answer pair-based question-answering systems. Structured data-based question-answering systems are primarily domain-specific, processing simple and highly structured data. They typically transform input questions into database queries and retrieve answers from the database. Free text-based question-answering systems mainly involve question analysis, document retrieval and paragraph segmentation, candidate answer extraction, answer ranking, and answer verification. Question-answer pair-based systems primarily involve two types: community question answering (CQA) and frequently asked questions (FAQ).

[0004] A knowledge base-based question-answering method and system have been proposed in the prior art. This method includes: acquiring question information and corresponding solution information, and inputting the question information and solution information into a preset knowledge base; acquiring question input data; matching the question input data with the preset knowledge base to generate a matching result; if the matching result is successful, generating answer feedback information corresponding to the question input data; if the matching result is unsuccessful, reporting the question input data to a list of questions to be improved. This method requires inputting question information and its corresponding solution information into a preset knowledge base, which consumes a significant amount of manpower and time, is cumbersome, and has high development costs.

[0005] On the other hand, since the advent of ChatGPT, Large Language Models (LLMs) have demonstrated astonishing capabilities and have been applied explosively to various fields related to NLP. Large language models refer to deep learning models trained using massive amounts of text data, typically possessing a large number of parameters, such as hundreds of millions or even trillions. Existing LLMs primarily employ the Transformer model architecture and pre-trained language as the target for modeling. The main difference between LLMs and small language models is that LLMs significantly expand the model size, pre-training data, and total computational cost (by a factor of magnitude). LLMs can better understand natural language and generate high-quality text based on given context (e.g., prompts).

[0006] Therefore, question-answering methods based on large language models and the LangChain framework have been proposed in related fields. LLM (Language Model) is already a model trained on massive datasets, thus it can provide relatively good answers to common domain questions, such as common sense, code, and mathematical calculation questions. However, for vertical domains, users need to fine-tune the LLM according to their needs, but fine-tuning the model is limited by data volume and GPU resources, often resulting in poor performance. The LangChain framework is a very powerful framework designed to help developers build end-to-end applications using language models. Through its provided tools, components, and interfaces, it can combine large language models with external data and allow the language model to interact with the runtime environment. Therefore, using LangChain in conjunction with a knowledge base for vertical domain question answering has become a popular and interesting solution for building intelligent question-answering systems. However, most current solutions only propose basic application processes and do not delve into performance, accuracy, and industry applications. Further exploration is needed to better apply large language models and the LangChain framework to the required domains. Summary of the Invention

[0007] In view of the above-mentioned problems, this application provides a local knowledge base question answering method and system based on the LangChain framework and a large language model, and applies it to a robot question answering dialogue system. The application also improves the question answering method and system to better enhance system performance under limited graphics card resources, so as to achieve more accurate and efficient intelligent question answering, while also saving costs.

[0008] According to one aspect of this application, a question-answering method based on the LangChain framework and a Large Language Model (LLM) is provided, comprising: loading the LLM based on an LLM-accelerated inference framework; building a text vector knowledge base in the LangChain framework; vectorizing the user question to generate a question vector; performing vector similarity matching between the question vector and the text vector knowledge base using two different vector retrieval libraries to return two sets of similar text paragraphs; performing pairwise paragraph text similarity matching on the two sets of similar text paragraphs to obtain two similar text paragraphs with the highest paragraph similarity score; when the highest paragraph similarity score is higher than or equal to a paragraph similarity threshold, performing content fusion on the two similar text paragraphs to generate a final knowledge paragraph most relevant to the question vector, and generating a contextual hint template based on the final knowledge paragraph; when the highest paragraph similarity score is lower than the paragraph similarity threshold, using the user question as the contextual hint template; and inputting the contextual hint template into the LLM for parallel inference and streaming output of the answer.

[0009] According to another aspect of this application, a question-answering system based on the LangChain framework and a Large Language Model (LLM) is provided, comprising: a processor, and an interface unit and a storage unit coupled to the processor. The interface unit is configured to receive user questions, and the storage unit is configured to store the LLM and computer program instructions. The processor is configured to execute the computer program instructions to: load the LLM based on an LLM-accelerated inference framework; build a text vector knowledge base in the LangChain framework; vectorize the user questions to generate question vectors; and perform vector similarity analysis between the question vectors and the text vector knowledge base using two different vector retrieval libraries. The system performs a matching process to return two sets of similar text paragraphs. For each of the two sets of similar text paragraphs, it performs pairwise paragraph text similarity matching to obtain two similar text paragraphs with the highest paragraph similarity score. When the highest paragraph similarity score is higher than or equal to the paragraph similarity threshold, it performs content fusion on the two similar text paragraphs to generate the final knowledge paragraph most relevant to the question vector, and generates a context hint template based on the final knowledge paragraph. When the highest paragraph similarity score is lower than the paragraph similarity threshold, it uses the user question as the context hint template. The context hint template is then input into the LLM for parallel reasoning and the answer is streamed through the interface unit.

[0010] According to another aspect of this application, a machine-readable storage medium storing program instructions is provided, wherein, when executed by a processor, the program instructions cause the processor to perform the question-answering method based on the LangChain framework and the Large Language Model (LLM) as described above. Attached Figure Description

[0011] This application can be better understood from the following description of specific embodiments in conjunction with the accompanying drawings, wherein:

[0012] Figure 1 A flowchart illustrating a question-answering method based on the LangChain framework and the Large Language Model (LLM) according to an example embodiment of this application is shown.

[0013] Figure 2 This illustrates the process of building a text vector knowledge base in the LangChain framework according to an example embodiment of this application;

[0014] Figure 3 A flowchart of a question-answering method based on the LangChain framework and Large Language Model (LLM) according to another example embodiment of this application is shown; and

[0015] Figure 4 A schematic block diagram of a question-answering system based on the LangChain framework and the Large Language Model (LLM) according to an example embodiment of this application is shown. Detailed Implementation

[0016] The features and exemplary embodiments of various aspects of this application will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a comprehensive understanding of this application. However, it will be apparent to those skilled in the art that this application can be implemented without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of this application by illustrating examples. This application is by no means limited to any specific configuration presented below, but covers any modifications, substitutions, and improvements to elements, components, and algorithms without departing from the spirit of this application. Well-known structures and techniques are not shown in the accompanying drawings and the following description in order to avoid unnecessary obfuscation of this application.

[0017] In view of the shortcomings of the existing solutions mentioned in the background, this application proposes a new local knowledge base question answering method and system based on the LangChain framework and a large language model, and applies it to a robot question answering dialogue system. The application also improves the question answering method and system to better enhance system performance under limited graphics card resources, so as to achieve more accurate and efficient intelligent question answering, while also saving costs.

[0018] First, this application utilizes the LangChain framework to apply advanced large language modeling technology to a question-and-answer dialogue system, improving performance. According to embodiments of this application, based on LLM and LangChain technology, the tedious task of inputting questions and corresponding answers in traditional question-and-answer systems can be eliminated, reducing the complexity of knowledge base construction and significantly saving manpower and system development time. Furthermore, the knowledge base document format is no longer limited to text, but also supports images and portable document formats (PDF), increasing the diversity of the knowledge base. Additionally, the proposed question-and-answer system can be integrated into a robot; by changing the knowledge base in different fields, robots with different functions can be created. For example, building a legal knowledge base can create a legal service robot. Moreover, using LLM, casual conversation between humans and robots can be achieved, enhancing the fun of human-computer interaction.

[0019] Furthermore, existing question-answering systems based on the LLM and LangChain frameworks lack proposals for performance optimization. To better build intelligent question-answering systems based on the LLM and LangChain frameworks under limited graphics card resources, some embodiments of this application have made one or more improvements in the following aspects:

[0020] 1. To prevent errors from a single vector similarity match that could lead to incorrect answers, two different vector retrieval libraries (e.g., Faiss and Chroma) are used simultaneously for vector similarity matching, each returning two sets of similar text paragraphs. Then, for each set of similar text paragraphs, pairwise text similarity matching is performed. Paragraphs with matching scores greater than a threshold are merged and returned as the final knowledge paragraph, thus improving the accuracy of knowledge reference.

[0021] 2. Since LLMs typically consume a significant amount of GPU memory, most researchers and institutions, with the exception of a few, are limited by GPU resources. How to maximize the number of question-answers within limited GPU memory is a current research hotspot. According to some embodiments of this application, after matching and returning similar text paragraphs based on the LangChain framework, the returned paragraphs are again matched with the question for similarity, returning the most similar sentence. This reduces the number of words in the prompt template, thereby reducing the GPU memory size of the LLM during inference and increasing the number of parallel inferences within limited GPU memory resources. Simultaneously, it avoids irrelevant sentences in the paragraphs from misleading the LLM model, leading to incorrect final answers.

[0022] 3. Use an LLM accelerated reasoning framework for LLM transformation and reasoning. This framework can increase LLM response speed by tens of times, significantly reducing user waiting time and improving user experience.

[0023] 4. Leveraging the advantages of the accelerated inference framework for efficient parallel inference, improvements have been made for multi-user parallel inference. The parallel inference logic is designed to minimize user waiting time without increasing graphics card costs.

[0024] The following will be combined with the appendix Figures 1 to 4 This paper details a question-answering method and system based on the LangChain framework and the Large Language Model (LLM) according to example embodiments of this application.

[0025] Figure 1 A flowchart illustrating a question-answering method based on the LangChain framework and Large Language Model (LLM) according to an example embodiment of this application is shown. Figure 1 As shown, the question-and-answer method may include steps 110 to 190. These steps will be described in detail below.

[0026] In step 110, the LLM is loaded based on the LLM accelerated inference framework.

[0027] According to the example embodiments of this application, LLM models, such as FastLLM and vLLM, can be loaded based on an accelerated inference framework. This accelerated inference framework utilizes innovative memory management principles and a highly efficient inference mechanism, achieving inference speeds tens of times faster than the original Transformer framework. For example, using an Nvidia TITANRTX graphics card with 24GB of VRAM for testing, inferring 10 questions (not input simultaneously) using the original Transformer framework takes nearly 90 seconds, while using the accelerated inference framework takes only 35 seconds. If the batch operation in the accelerated inference framework is used to input 10 questions simultaneously for inference, it takes only 16 seconds.

[0028] In step 120, the text vector knowledge base in the LangChain framework is built.

[0029] Figure 2 The process of building a text vector knowledge base in the LangChain framework according to an example embodiment of this application is illustrated. The process may include: loading a local knowledge document and reading the document as text; splitting the text into text blocks; vectorizing the text blocks to generate text block vectors; and storing the text block vectors in a repository to generate a text vector knowledge base.

[0030] Specifically, it is necessary to prepare local knowledge documents for the domain required by the question-answering system. These documents can be in various formats, such as text, images, and portable document formats (e.g., PDF, Markdown, TXT, JPG, etc.). Then, the documents are loaded according to their format and read as text. Next, the text is split into text blocks. For example, the text can be split according to characters, length, or semantics using the text segmentation methods provided in the LangChain framework. In some exemplary embodiments of this application, the text can be split according to characters plus length. Then, the text blocks need to be vectorized. The main purpose of text vectorization is to represent the text as a series of vectors that can express the semantics of the text, that is, to use numerical vectors to represent the semantics of the text. In some embodiments of this application, embedding models such as text2vec and m3e can be used to encode the split text blocks to obtain vector representations of the text blocks. Finally, all text block vectors are stored in the vector_store repository to complete the construction of the local text vector knowledge base.

[0031] In step 130, the user's question is vectorized to generate a question vector. For example, the user's input question is encoded using language embedding models such as text2vec or m3e to generate a question vector `query_vector`. It is important to note that question vectorization and text block vectorization should use the same quantization method to ensure comparison within the same space.

[0032] In step 140, two different vector retrieval libraries are used to perform vector similarity matching between the question vector and the text vector knowledge base to return two sets of similar text paragraphs. Specifically, the question vector is matched with the text block vectors in the repository for similarity, and the k similar text paragraphs with the highest k similarity scores are returned in sorted order. These paragraphs can be considered the most useful knowledge for answering the question, and the number of words in the paragraph context can be controlled by chunk_size. The similarity matching operation can be performed using vector retrieval libraries such as milvus, chroma, and faiss.

[0033] According to the example embodiment of this application, two different vector retrieval libraries (such as Faiss and Chroma) are used to perform vector similarity matching operations simultaneously, and two sets of k similar text paragraphs are returned respectively (for example, k is 5 and chunk_size is 250 characters).

[0034] Then, in step 150, for the two groups of k similar text paragraphs, a text similarity algorithm, such as cosine distance, Levenshtein distance, etc., is used to perform pairwise paragraph text similarity matching on the two groups of similar text paragraphs to find the two similar text paragraphs with the highest paragraph similarity score.

[0035] When the highest paragraph similarity score is higher than or equal to the paragraph similarity threshold, in step 160, the two similar text paragraphs are fused (intersection) to generate the final knowledge paragraph most relevant to the question vector, and a contextual prompt template is generated based on the final knowledge paragraph.

[0036] When the highest paragraph similarity score is lower than the paragraph similarity threshold, it means that none of the results obtained using the two vector retrieval libraries are similar, which is equivalent to the user question not matching any relevant text paragraphs in the text vector knowledge base. In this case, the user question can be directly used as a contextual prompt template in step 170.

[0037] According to some example embodiments of this application, the prompt template may, for example, have the following form: [Known information: {context}, answer the user's question based on the above known information, the question is: {question}]. Sentences from the final knowledge paragraph obtained in step 160 can be used as known information to replace the {context} part of the template, while the {question} part is replaced with the user-inputted question. If the user-inputted question does not match a relevant text paragraph in the text vector knowledge base, the above template is no longer used; instead, the user-inputted question is directly used as the prompt template. The content in the prompt template can be adjusted according to requirements, but it should be noted that the more content in the prompt, the more GPU memory is consumed during inference.

[0038] Then, in step 180, the contextual hint template is input into the LLM for parallel reasoning, and in step 190, the answer is streamed.

[0039] According to exemplary embodiments of this application, the LLM can generate and output answers based on the aforementioned contextual prompt template. For example, LLM options include Galactica, LLaMA, ChatGLM, the GPT series, and Vicuna. The default output method for the accelerated reasoning framework is holistic output, meaning all answers are generated and then returned all at once. If the answers are long, the output return time will also be long, resulting in a poor user experience. In embodiments of this application, the output method can be adjusted to streamed output of answers. For accelerated reasoning frameworks that do not provide a streaming generation scheme, this can be implemented independently.

[0040] Furthermore, this application proposes a multi-user parallel answering scheme. Common parallel answering schemes in the prior art involve inputting a user question into the LLM to generate an answer as soon as it arrives. While it appears that multiple questions are answered simultaneously, the LLM internally operates in a serial manner. In the embodiments of this application, multi-user parallel inference logic is formulated based on the efficient parallel inference features of the accelerated inference framework. For example, two queues can be set up, named the inference queue and the waiting queue. Ten positions are allocated in each queue for the graphics card used. When a user request is detected, a timer starts, collecting all user requests within 3 seconds, and then sending them together into the inference queue for inference. This effectively utilizes the batch inference feature of the accelerated framework, resulting in a shorter answering time when using the same graphics memory resources. If there are at least two empty slots in the inference queue, user questions will still be collected and added to the queue in 3-second intervals. If all slots in the inference queue are full, the remaining user requests will enter the waiting queue and a timer will start. If at least two slots in the inference queue are released within 6 seconds (the release time varies, with shorter answers resulting in faster release), then the corresponding number of threads in the waiting queue will be sent to the inference queue for parallel inference. If the waiting time for questions in the waiting queue exceeds 6 seconds, a message such as "The system is busy, please ask your question later" can be returned.

[0041] Generally, an inference queue and a waiting queue can be set up for an LLM. When a user query request is detected, all user questions received within a preset time are collected and sent to the inference queue for parallel inference. When all positions in the inference queue are full, the remaining user questions received within the preset time are sent to the waiting queue. When a position is released in the inference queue, the corresponding user questions in the waiting queue are sent to the inference queue for parallel inference. When the waiting time for user questions in the waiting queue exceeds the waiting time threshold, a system busy message is returned.

[0042] Therefore, according to the embodiments of this application, the advantages of efficient parallel inference of the accelerated inference framework can be used to realize multi-user parallel inference, which can shorten the user waiting time as much as possible without increasing the cost of the graphics card.

[0043] Furthermore, in order to maximize the number of questions and answers with limited GPU memory resources, some exemplary embodiments of this application propose that after matching and returning similar text paragraphs based on the LangChain framework, the returned paragraphs are again matched with the questions to return the most similar sentences, thereby reducing the number of words in the prompt template and thus reducing the GPU memory resource consumption of LLM during inference.

[0044] Figure 3A flowchart is shown for a question-answering method based on the LangChain framework and the Large Language Model (LLM) according to another example embodiment of this application. Figure 3 The question-and-answer method shown is... Figure 1 This is an improvement on the question-and-answer method shown. For the sake of brevity and clarity, Figure 3 The part with is omitted in the middle. Figure 1 The same steps 110 to 130 are repeated. Figure 3 The flowchart of the question-and-answer method shown is as follows: Figure 1 The difference in the question-answering scheme process shown is that after generating the final knowledge segment most relevant to the question vector, the returned final knowledge segment is again matched with the user's question for similarity.

[0045] In other words, Figure 1 Step 160 in the process can be Figure 3 Steps 1610 to 1650 are replaced in the previous steps. Specifically, in step 1610, content fusion is performed on two similar text paragraphs to generate the final knowledge paragraph most relevant to the question vector; in step 1620, the final knowledge paragraph is split into individual sentences, and each individual sentence is vectorized to generate a single-sentence vector; in step 1630, for each single-sentence vector, the single-sentence vector is matched with the question vector to obtain the corresponding single-sentence similarity score; in step 1640, the final knowledge paragraph is updated by deleting single sentences whose single-sentence similarity scores are lower than the single-sentence similarity threshold; and in step 1650, a contextual hint template is generated based on the updated final knowledge paragraph.

[0046] In an exemplary embodiment of this application, the final knowledge segment most relevant to the question vector obtained in step 1610 can be segmented into sentences according to characters. Characters are those that can end sentences, including periods, question marks, exclamation marks, etc. Then, the similarity between the question vector and each sentence can be calculated. For example, an embedding model can be used to vectorize each sentence to generate a sentence vector, and then the cosine distance similarity algorithm described above can be used sequentially to calculate the similarity between the question vector and each sentence vector to obtain the corresponding sentence similarity score. Next, sentences with sentence similarity scores lower than the sentence similarity threshold are deleted from the final knowledge segment; that is, sentences with sentence similarity scores not lower than the sentence similarity threshold are retained, resulting in an updated final knowledge segment. In special cases, such as if a sentence ends with a question mark (indicating that the sentence may be a question in QA) and the similarity score of the next sentence is lower than the similarity threshold, the sentence and its next sentence are retained to prevent the loss of useful information.

[0047] Based on such Figure 3The question-answering method shown can reduce the number of words in the contextual hint template, thereby reducing the memory resource consumption of the LLM during inference and increasing the number of parallel inferences with limited memory resources. At the same time, it can also avoid irrelevant sentences in paragraphs causing errors in the LLM model, leading to incorrect final answers.

[0048] The above combination Figures 1 to 3 This paper describes the specific process and corresponding technical effects of a question-answering method based on the LangChain framework and Large Language Model (LLM) according to embodiments of this application. This question-answering method can be used in scenarios such as... Figure 4 This is implemented using the question-and-answer system shown. Figure 4 A schematic block diagram of a question-answering system 400 based on the LangChain framework and a Large Language Model (LLM) according to an example embodiment of this application is shown. The question-answering system 400 may include a processor 420, an interface unit 410 coupled to the processor, and a storage unit 430. The interface unit 410 is configured to receive user questions, and the storage unit 430 is configured to store the LLM and computer program instructions. In this question-answering system, the processor 410 is configured to execute the computer program instructions to implement the above-referenced appendix. Figures 1 to 3 The question-answering method described, based on the LangChain framework and a large language model, also possesses the technical advantages mentioned above.

[0049] The foregoing references "exemplary embodiments," "implementations," and "some embodiments." It should be understood that features mentioned in various embodiments are not necessarily applicable only to that embodiment, but may be used in other embodiments. A feature in one embodiment may be applied to another embodiment, or may be included in another embodiment.

[0050] Furthermore, various operations or steps are described as multiple discrete operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations or steps must depend on the order. In particular, these operations or steps do not need to be performed in the order presented.

[0051] This application may be implemented in other specific forms without departing from its spirit and essential characteristics. For example, features described in a particular embodiment may be modified without departing from the basic spirit of this application. Therefore, the present embodiments are to be regarded as exemplary rather than limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description, and all changes falling within the meaning and scope of the claims and their equivalents are thus included within the scope of this application.

Claims

1. A question-answering method based on the LangChain framework and the Large Language Model (LLM), comprising: The LLM is loaded based on an LLM-accelerated inference framework. Build the text vector knowledge base in the LangChain framework; Vectorize user questions to generate question vectors; Using a first vector retrieval library and a second vector retrieval library different from the first vector retrieval library, vector similarity matching is performed between the question vector and the text vector knowledge base to return a first group of similar text paragraphs and a second group of similar text paragraphs; For text paragraph pairs consisting of text paragraphs selected from the first group of similar text paragraphs and text paragraphs selected from the second group of similar text paragraphs, paragraph text similarity matching is performed to obtain a pair of target similar text paragraphs with the highest paragraph similarity score; When the highest paragraph similarity score is higher than or equal to the paragraph similarity threshold, the content of the pair of target similar text paragraphs is fused to generate the final knowledge paragraph most relevant to the question vector, and a contextual prompt template is generated based on the final knowledge paragraph. When the highest paragraph similarity score is lower than the paragraph similarity threshold, the user question will be used as the context prompt template. and The contextual hint template is input into the LLM for parallel reasoning and streaming output of the answer.

2. The question-and-answer method according to claim 1, wherein, Generating contextual prompt templates based on the final knowledge paragraphs includes: The final knowledge paragraph is split into individual sentences, and each individual sentence is vectorized to generate a single sentence vector; For each sentence vector, the sentence vector is matched with the question vector to obtain the corresponding sentence similarity score; The final knowledge paragraph is updated by deleting sentences whose sentence similarity scores are lower than a sentence similarity threshold from the final knowledge paragraph; and The contextual hint template is generated based on the updated final knowledge paragraph.

3. The question-and-answer method according to claim 1 or 2, wherein, Loading the LLM based on the LLM-accelerated inference framework includes: Set up an inference queue and a waiting queue for the LLM; When a user query request is detected, all user questions received within a preset time are collected and sent to the inference queue for parallel inference. When all positions in the inference queue are full, the remaining user questions received within the preset time period are sent to the waiting queue. Furthermore, when a position is released in the inference queue, the corresponding user question from the waiting queue is sent to the inference queue for parallel inference. When the waiting time for a user's question in the waiting queue exceeds the waiting time threshold, a "system busy" message is returned.

4. The question-and-answer method according to claim 1 or 2, wherein, Building the text vector knowledge base in the LangChain framework includes: Load the local knowledge document and read it as text; The text is split into text blocks; The text block is vectorized to generate a text block vector; and The text block vectors are stored in a repository to generate the text vector knowledge base.

5. The question-and-answer method according to claim 4, wherein, The quantization method used to vectorize the user question is the same as the quantization method used to vectorize the text block.

6. The question-and-answer method according to claim 1 or 2, wherein, Each group of similar text paragraphs includes k similar text paragraphs with the highest k similarity scores obtained from the corresponding vector retrieval library.

7. The question-and-answer method according to claim 2, wherein, Breaking the final knowledge paragraph into individual sentences includes breaking the final knowledge paragraph into individual sentences according to character format.

8. The question-and-answer method according to claim 4, wherein, Splitting the text into text blocks includes splitting the text into text blocks according to the number of characters plus the length.

9. The question-and-answer method according to claim 4, wherein, The local knowledge document is in one or more of the following formats: text, image, portable document format.

10. The question-and-answer method according to claim 4, wherein, The local knowledge documents are selected and loaded based on question-and-answer related domains.

11. A question-answering system based on the LangChain framework and Large Language Model (LLM), comprising: A processor, and an interface unit and a storage unit coupled to the processor, the interface unit being configured to receive user questions, and the storage unit being configured to store the LLM and computer program instructions, wherein the processor is configured to execute the computer program instructions to: The LLM is loaded based on an LLM-accelerated inference framework. Build the text vector knowledge base in the LangChain framework; The user questions are vectorized to generate question vectors; Using a first vector retrieval library and a second vector retrieval library different from the first vector retrieval library, vector similarity matching is performed between the question vector and the text vector knowledge base to return a first group of similar text paragraphs and a second group of similar text paragraphs; For text paragraph pairs consisting of text paragraphs selected from the first group of similar text paragraphs and text paragraphs selected from the second group of similar text paragraphs, paragraph text similarity matching is performed to obtain a pair of target similar text paragraphs with the highest paragraph similarity score; When the highest paragraph similarity score is higher than or equal to the paragraph similarity threshold, the content of the pair of target similar text paragraphs is fused to generate the final knowledge paragraph most relevant to the question vector, and a contextual prompt template is generated based on the final knowledge paragraph. When the highest paragraph similarity score is lower than the paragraph similarity threshold, the user question will be used as the contextual prompt template; and The contextual hint template is input into the LLM for parallel reasoning and the answer is streamed through the interface unit.

12. The question-answering system according to claim 11, wherein, Generating contextual prompt templates based on the final knowledge paragraphs includes: The final knowledge paragraph is split into individual sentences, and each individual sentence is vectorized to generate a single sentence vector; For each sentence vector, a similarity match is performed between the sentence vector and the question vector to obtain the corresponding sentence similarity score; and The final knowledge paragraph is updated by deleting sentences whose sentence similarity scores are lower than the sentence similarity threshold, so that the contextual prompt template can be generated based on the updated final knowledge paragraph.

13. The question-answering system according to claim 11 or 12, wherein, Loading the LLM based on the LLM-accelerated inference framework includes: Set up an inference queue and a waiting queue for the LLM; When a user query request is detected, all user questions received within a preset time are collected and sent to the inference queue for parallel inference. When all positions in the inference queue are full, the remaining user questions received within the preset time period are sent to the waiting queue. Furthermore, when a position is released in the inference queue, the corresponding user question from the waiting queue is sent to the inference queue for parallel inference. When the waiting time for a user's question in the waiting queue exceeds the waiting time threshold, a "system busy" message is returned.

14. The question-answering system according to claim 11 or 12, wherein, Building the text vector knowledge base in the LangChain framework includes: Load the local knowledge document and read it as text; The text is split into text blocks; The text block is vectorized to generate a text block vector; and The text block vectors are stored in a repository to generate the text vector knowledge base.

15. The question-answering system according to claim 14, wherein, The quantization method used to vectorize the user question is the same as the quantization method used to vectorize the text block.

16. The question-answering system according to claim 11 or 12, wherein, Each group of similar text paragraphs includes k similar text paragraphs with the highest k similarity scores obtained from the corresponding vector retrieval library.

17. The question-answering system according to claim 12, wherein, Breaking the final knowledge paragraph into individual sentences includes breaking the final knowledge paragraph into individual sentences according to character format.

18. The question-answering system according to claim 14, wherein, Splitting the text into text blocks includes splitting the text into text blocks according to the number of characters plus the length.

19. The question-answering system according to claim 14, wherein, The local knowledge document is in one or more of the following formats: text, image, portable document format.

20. The question-answering system according to claim 14, wherein, The local knowledge documents are selected and loaded based on question-and-answer related domains.

21. A machine-readable storage medium storing program instructions, wherein, When the program instructions are executed by the processor, the processor performs the question-answering method based on the LangChain framework and the Large Language Model (LLM) as described in any one of claims 1 to 10.

Images