Question and answer large model implementation method and system based on fine tuning technology

Abstract

The invention relates to the technical field of consulting questions and answers, and discloses a question and answer large model implementation method and system based on a fine tuning technology. According to the method, firstly, a question and answer knowledge base is constructed, a question and answer model of a large language model capable of retrieving and using the knowledge from the question and answer knowledge base is established, the model adopts a fine tuning technical model to fuse domain knowledge, and output control is carried out on the fine-tuned domain model through instruction prompt; the problem that a large language model question-answering system is unstable in a specific field is avoided, the reliability of a large model is improved, the application range of the large model is widened, and the requirements of consultation question-answering and other intelligent question-answering are met.

Description

Technical Field

[0001] The present invention relates to the technical field of question and answer, and specifically to a method and system for implementing a large question and answer model based on fine-tuning technology. Background Art

[0002] Intelligent question and answer systems, as an important branch of artificial intelligence applications, aim to efficiently complete tasks specified by users through dialogue interactions. Therefore, to conduct research on question and answer systems, high-quality question and answer data is required as training and test corpora. With the rapid development of artificial intelligence technology, large language models (LLMs) such as ChatGPT, GPT-4, and LLaMA3 have gradually become the leaders in the field of natural language processing. These large language models not only possess powerful text generation capabilities but also demonstrate amazing understanding and reasoning abilities in various tasks. However, when it comes to professional knowledge questions in specific fields, the large language models may be unstable in answering questions or even fabricate answers. Therefore, how to build a highly secure and intelligent question and answer system in vertical fields has become a research hotspot.

[0003] The hallucination problem of large language models refers to the deviation or error that occurs when the model processes input tasks, maintains the coherence of the output context, and conforms to the facts of the real world. The hallucination problem of large models can be attributed to three main manifestations:

[0004] 1) The answer does not match the input task, that is, the answer generated by the model is not relevant to the task input by the user;

[0005] 2) The context of the answer content is inconsistent, that is, the answer generated by the model is self-contradictory;

[0006] 3) The answer contradicts established facts, that is, the answer generated by the model conflicts with verifiable facts in the real world.

[0007] These hallucination problems affect the reliability of large models and limit their application scope. Hallucination problems may pose threats to human life, health, and property safety. Regarding the hallucination problems of large language models, current researchers are more focused on the quantitative evaluation of large model hallucinations and the research on the sources and mitigation techniques of hallucinations. For the quantitative evaluation of large model hallucinations, some studies have designed queries that can guide the model to generate hallucinatory answers and observed the ability of large models to generate factual statements. The root causes of hallucinations in large language models are diverse. During the training phase, errors in the training dataset may cause large language models to memorize incorrect knowledge; biased alignment data may also cause large language models to tend to agree with user opinions and ignore facts, thus affecting the accuracy of the generated content. The most direct way to mitigate hallucination problems during the training phase is to clean and carefully design the training data and alignment data to reduce unreliable data, but this also requires a large amount of manual operation. Therefore, a method and system for implementing a large question-answering model based on fine-tuning technology are proposed. Summary of the Invention

[0008] In view of the deficiencies of the prior art, the present invention provides a method and system for implementing a large question-answering model based on fine-tuning technology to solve the problems in the background technology.

[0009] In a first aspect, to achieve the above object, the present invention provides the following technical solution: A method for implementing a large question-answering model based on fine-tuning technology, the method comprising the following steps:

[0010] Construction of a knowledge question-answering library:

[0011] Data collection: In combination with the consultation service situation of a certain unit and user needs analysis, negotiate with the big data department of the unit to obtain data related to question-answering consultations of the unit;

[0012] Data cleaning: Remove irrelevant content information such as expressions and symbols from the data;

[0013] Data marking: Adopt the method of manual review to extract the question-answer pair data related to consultations, and mark the extracted question-answer pair data;

[0014] Data storage: Store the marked dataset in a database to form a knowledge question-answering library.

[0015] Model construction:

[0016] Construction of a domain fine-tuning dataset: Use the marked common service consultation dataset to construct prompts, and each prompt consists of a question and an answer;

[0017] Model fine-tuning: Use the ChatGLM-6B model as the base model, and use the obtained prompts to perform domain fine-tuning on the base model ChatGLM-6B to obtain a model;

[0018] Control output: The domain model for fine-tuning is instructed through input prompts. Each piece of input information is modified into a text string called a prompt. Based on the prompt, the model generates corresponding answers through a knowledge Q&A library.

[0019] Model performance evaluation: Three evaluation metrics, Rouge_1, Rouge_2, and Rouge_L, are used to evaluate the answer generation ability of the above model.

[0020] Preferably, in the data storage step, a relational database is used to store data, facilitating the large language model to quickly and accurately retrieve and use this knowledge during the Q&A process.

[0021] Preferably, in the data marking step, the marked content also needs to be submitted to the big data department of the unit for review to ensure the correctness of data marking.

[0022] Preferably, in the model fine-tuning step, the specific fine-tuning technique used is the P-tuningv2 technique, which adds Prompts tokens as input to each layer.

[0023] Second aspect: A question-answering large model implementation system based on fine-tuning technology, which includes five layers: a data layer, a model layer, an algorithm layer, a service layer, and an application layer, for implementing the question-answering large model implementation method described in the first aspect.

[0024] The data layer is the foundation of the architecture, mainly responsible for the work of data extraction, processing, and storage in this system.

[0025] The model layer is jointly driven by a deep learning framework and the large language model ChatGLM-6b.

[0026] The algorithm layer includes, but is not limited to, a knowledge Q&A module, a knowledge extraction module, and a dialogue management module.

[0027] The service layer provides different types of knowledge Q&A, including but not limited to household registration and transportation.

[0028] The application layer mainly includes two parts, namely a user client and a background management.

[0029] Preferably, the deep learning framework uses PyTorch, which provides flexible tools and libraries for building deep learning models, is dedicated to natural language processing tasks, and encapsulates the basic functions of large model fine-tuning and invocation.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] The present invention first constructs a question-and-answer knowledge base and establishes a question-and-answer model of a large language model that can retrieve and use this knowledge from the question-and-answer knowledge base. This model uses fine-tuning technology to fuse domain knowledge into the model and controls the output of the fine-tuned domain model through instruction prompts, avoiding the instability problem of the large language model question-and-answer system in specific domains, improving the reliability of the large model, and expanding the application scope of the large model to meet the needs of consultation question-and-answer and other intelligent question-and-answer scenarios.

[0032] Other features and advantages of the present invention will be described in the following specification, and will, in part, be obvious from the specification, or can be learned by implementing the present invention. The objectives and other advantages of the present invention can be realized and achieved by the structures pointed out in the specification, claims, and drawings. Description of the Drawings

[0033] Figure 1 is the question-and-answer model of the present invention based on a large language model;

[0034] Figure 2 is the system architecture diagram of the question-and-answer large model implementation of the present invention based on fine-tuning technology;

[0035] Figure 3 is the flowchart for constructing the knowledge question-and-answer library of the present invention;

[0036] Figure 4 is the schematic diagram of the intelligent question-and-answer algorithm of the large language model of the present invention;

[0037] Figure 5 is the schematic diagram of the system and debugging of the present invention;

[0038] Figure 6 is the diagram of the question-and-answer data marking types of the present invention. Detailed Embodiments

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art in the technical field of the present invention without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0040] Embodiment 1:

[0041] Please refer to Figures 1 - 6 , this embodiment provides a method for implementing a question-and-answer large model based on fine-tuning technology, mainly including two parts: the construction of a question-and-answer knowledge base and the construction of a question-and-answer model of a large language model.

[0042] Before building the Q&A knowledge base, it is necessary to clarify the scope and type of domain knowledge based on the user's consultation needs and business characteristics; obtain the support of the big data department within the unit through coordination within the unit, obtain Q&A data within the past two months, and perform tasks such as data cleaning and data tagging on the obtained data; finally, store the data in the form of a relational database to facilitate the large language model to quickly and accurately retrieve and use this knowledge during the Q&A process;

[0043] The Q&A model of the large language model conducts a comparative analysis of Q&A systems in various fields, and summarizes the advantages and disadvantages of model fine-tuning methods. Based on this, a Q&A model of PLLM (Police Large Language) integrating domain knowledge is proposed. The large model is injected with domain knowledge through the P-tuning v2 fine-tuning technology, and the output of the fine-tuned domain model is controlled through instruction prompts.

[0044] This method includes the following steps:

[0045] Knowledge Q&A library construction, as Figure 3 shown:

[0046] Data collection: Combining the consultation service situation of a certain unit and user needs analysis, negotiating with the big data department of the unit to obtain data related to Q&A consultations of the unit (using Scrapy crawler technology to obtain domain knowledge-related documents including online public knowledge websites and public materials of a certain unit), specifically 20,000 pieces of data;

[0047] Data cleaning: Remove irrelevant content information such as emojis and symbols in the data;

[0048] Data tagging: Adopt the method of manual review to extract Q&A pair data related to consultations, and perform data tagging on the extracted Q&A pair data. As Figure 6 shown, and then submit the marked content to the big data department of the unit for re-review to ensure the correctness of data tagging;

[0049] Data storage: Store the re-reviewed marked data set in the database. The database stores data in the form of a relational database to facilitate the large language model to quickly and accurately retrieve and use this knowledge during the Q&A process, constituting a knowledge Q&A library.

[0050] Model construction, as Figure 4 shown:

[0051] This part designs a Q&A model of PLLM (Police Large Language Model) based on the large language model to improve the level and accuracy of consultation services. The PLLM framework diagram is as Figure 4As shown in the figure; the base large model of PLLM selects the open-source large language model released by Tsinghua University - ChatGLM3-6B, with a parameter count of 6 billion.

[0052] Since the large model designed this time is a lightweight model and the sample dataset is 20,000, the fine-tuning technique adopted is P-tuning v2. It only needs to optimize a small number of prompt parameters, reducing the training burden. The soft prompt can be flexibly inserted into any position of the model to adapt to different types of tasks. Compared with the P-tuning v2 fine-tuning technique, supervised learning SFT (Supervised Learning Fine-tuning) requires a large amount of labeled data, and once the data is mislabeled, it will affect the performance of the model. The Lora efficient parameter fine-tuning technique optimizes parameters by adding adapters between model layers, which may lead to a decline in model performance. The reinforcement learning fine-tuning method RLHF (Reinforcement Learning) requires relying on a large amount of computing resources and time to train the reinforcement learning algorithm.

[0053] The construction process of the PLLM model mainly includes the following three parts, namely domain fine-tuning dataset construction, model fine-tuning, and controlling model output. The construction process of the PLLM model is as follows:

[0054] Step 1, domain fine-tuning dataset construction: Use the labeled common service consultation dataset to construct prompts, and each prompt consists of a question and an answer. For example: {"content": "How to endorse a residence permit", "summary": "For a residence permit holder who continuously resides in the place of residence, within 1 month before the expiration of each full year of residence, log in to the 'Internet + Platform of a certain unit 1', select the residence permit endorsement item, upload the corresponding supporting materials, and have them reviewed by a certain unit 2; after the data review is completed, the residence permit holder shall go to a certain unit 3 at the pre-selected place of residence with the residence permit for on-site endorsement."};

[0055] Step 2, model fine-tuning: Adopt the ChatGLM-6B model as the base model, and use the obtained prompts to perform domain fine-tuning on the base model ChatGLM-6B. The fine-tuning technique used is the P-tuning v2 technique to obtain the model; the P-tuning v2 technique adds Prompts tokens as input to each layer, rather than just adding them to the input layer, which brings two benefits:

[0056] (1) More learnable parameters (increased from 0.01% of P-tuning and Prompt Tuning to 0.1% - 3%), and at the same time, it is also parameter-efficient enough;

[0057] (2) The Prompt added to the deeper structure can have a more direct impact on the model prediction.

[0058] Step 3, controlling the output: The fine-tuned domain model is instruction-guided through the input prompt words. The template of its specific instruction is set as T ′ , where the information of each input is modified into a text string called a prompt and set as X ′ , such as Figure 1 shown as T ′ :(x,D)→X ′ , and based on the prompt model, corresponding answers are generated through the knowledge Q&A library.

[0059] Model performance evaluation: Three evaluation indicators, Rouge_1, Rouge_2, and Rouge_L, are used to evaluate the answer generation ability of the above model. Rouge is a set of evaluation indicators for NLP (Natural Language Processing) tasks (Recall-Oriented Understudy for Gisting Evaluation). Its basic idea is that multiple experts make manual markings to form a benchmark dataset, compare the fused generated information with the manually marked benchmark dataset, and evaluate the quality of the fused generated information by calculating the number of overlapping basic units (n-grams, word order, and word pairs). By comparing with the marked benchmark dataset, the stability and robustness of the evaluation system are improved.

[0060] The ROUGE criterion consists of a series of evaluation methods, including ROUGE-N and ROUGE-L, etc. The Rouge_1, Rouge_2, and Rouge_L indicators used in the invention. In ROUGE-N, the value of n is determined according to the data, and n takes an integer value. In most cases, ROUGE is equivalent to recall, and ROUGE-N is expressed as:

[0061]

[0062] where n represents the length of the n-gram, {ReferenceSummaries} represents the short-sighted description information marked manually, match Count(gramn) represents the number of times gram appears simultaneously in the short-sighted description information marked and the short video description information recognized by the model, and Count(gramn) represents the number of n-grams that appear in the short-sighted description information marked manually.

[0063] OUGE-L is the initial letter in LCS, and Rouge_L is an indicator evaluated through the longest common subsequence. The calculation method of Rouge_L is as follows:

[0064]

[0065]

[0066] where LCS(X, Y) is the length of the longest common subsequence of X and Y, m and n represent the lengths of the manually marked data and the fused generated information respectively, R lcs , P lcs represent the recall rate and the precision rate respectively. In most cases, β is set to a very large number, and F lcs is equivalent to ROUGE-L.

[0067] Example Two:

[0068] This example provides a system for implementing a question-answering large model based on the fine-tuning technology. As Figure 2 shown, the system includes five layers: a data layer, a model layer, an algorithm layer, a service layer, and an application layer, which are used to implement the method for implementing a question-answering large model based on the fine-tuning technology in Example One;

[0069] The data layer is the foundation of the architecture and is mainly responsible for the work of data extraction, processing, and storage in this system;

[0070] The model layer is jointly driven by a deep learning framework and a large language model ChatGLM-6b. The deep learning framework uses PyTorch, which provides flexible tools and libraries for building deep learning models, is dedicated to natural language processing tasks, and encapsulates the basic functions of fine-tuning and calling large models;

[0071] The algorithm layer includes, but is not limited to, a knowledge question-answering module, a knowledge extraction module, and a dialogue management module;

[0072] The service layer provides different types of knowledge question-answering, including but not limited to household registration and transportation;

[0073] The application layer mainly includes two parts, namely a user client and a background management.

[0074] Example Three:

[0075] Test the system constructed in Example Two. The test uses the black-box testing method, aiming to verify whether the functions of the system meet the expectations from the user's perspective and not caring about the internal implementation details of the system. In black-box testing, various functions and user operations of the system will be tested without the need to understand the internal structure and code implementation of the system. By simulating the actual operation behaviors of users, the various inputs and outputs of the system will be tested to verify whether the responses and behaviors of the system in various situations meet the expectations. As Figure 5Specifically, black box testing includes functional testing, interface testing, performance testing, etc., to comprehensively evaluate the quality and reliability of the system. Through black box testing, we can find possible functional defects and user experience problems in the system, so as to repair and improve them in time, and improve the stability of the system and user satisfaction.

[0076] 1) Functional testing

[0077] Login interface test: The goal of this test is to comprehensively evaluate the security, stability and user experience of the login module. A series of rigorous security protocol tests, user interface evaluations and stability tests were conducted to ensure that the login module can provide users with a smooth and intuitive operating experience while maintaining user information security.

[0078] Data acquisition and processing module testing: This module verifies the functionality of the system in receiving and preprocessing user data. The core goal of the test is to ensure the accuracy and efficiency of the system in the data acquisition and preprocessing process, so as to ensure that the data can meet the predetermined quality standards and requirements before entering the subsequent analysis stage. Through this test, we hope to fully evaluate the performance of the system in this key link.

[0079] Algorithm interface module test: This part of the test focuses on evaluating the accuracy and response speed of the consultation question and answer interface. The test aims to ensure that the interface can demonstrate excellent stability and accuracy when processing diverse question texts.

[0080] 2) Non-functional performance testing

[0081] System performance indicators: The network backbone bandwidth is required to be guaranteed at 1000Mbps, the data transmission between the switch and the user is required to be guaranteed at 100Mbps, and the actual network bandwidth average utilization is required to be no more than 20%. The average server CPU utilization is no more than 30%. The average response time of the application system to the user is no more than 10 seconds. The average comprehensive response time of the model is no more than 15 seconds.

[0082] System reliability: During the use of the system, it has strong fault tolerance. When a local functional failure occurs, it will not affect the normal use of other functions; when the user makes an operation error, the system should give appropriate prompt information.

Claims

1. A method for implementing a large question-answering model based on fine-tuning technology, characterized in that The method includes the following steps: Knowledge Q&A library construction: Data collection: In combination with the consultation service situation of a certain unit and user requirement analysis, negotiate with the big data department of the unit to obtain data related to Q&A consultations of the unit; Data cleaning: Remove irrelevant content information such as emojis and symbols from the data; Data tagging: Adopt the method of manual review to extract Q&A pair data related to consultations, and perform data tagging on the extracted Q&A pair data; Data storage: Store the labeled data set in a database to form a knowledge Q&A library. Model construction: Construction of domain fine-tuning data set: Use the labeled common service consultation data set to construct prompts, and each prompt consists of a question and an answer; Model fine-tuning: Use the ChatGLM-6B model as the base model, and use the obtained prompts to perform domain fine-tuning on the base model ChatGLM-6B to obtain a model; Controlling the output: Guide the fine-tuned domain model through input prompt words, where each input information is modified into a text string called a prompt, and the corresponding answer is generated based on the prompt model through the knowledge Q&A library; Model performance evaluation: Use three evaluation metrics, Rouge_1, Rouge_2, and Rouge_L, to evaluate the answer generation ability of the above model.

2. A method for implementing a large question-answering model based on fine-tuning technology according to claim 1, wherein, In the data storage step, the data is stored in the way of a relational database to facilitate the large language model to quickly and accurately retrieve and use this knowledge during the Q&A process.

3. A method for implementing a large question-answering model based on fine-tuning technology according to claim 1, characterized in that, In the data tagging step, the labeled content also needs to be submitted to the big data department of the unit for re-review to ensure the correctness of data tagging.

4. A method for implementing a large question-answering model based on fine-tuning technology according to claim 1, characterized in that, In the model fine-tuning step, the specific fine-tuning technology used is the P-tuningv2 technology, and Prompts tokens are added as inputs at each layer in the P-tuningv2 technology.

5. A question-answering large model implementation system based on fine-tuning technology, characterized in that, The system includes five layers: a data layer, a model layer, an algorithm layer, a service layer, and an application layer, and is used to implement the Q&A large model implementation method based on fine-tuning technology described in claims 1-4; The data layer is the foundation of the architecture and is mainly responsible for the work of data extraction, processing, and storage of the system; The model layer is jointly driven by a deep learning framework and the large language model ChatGLM-6b; The algorithm layer includes, but is not limited to, a knowledge Q&A module, a knowledge extraction module, and a dialogue management module; The service layer provides different types of knowledge Q&A, including but not limited to household registration and transportation; The application layer mainly includes two parts, namely a user client and a background management.

6. The implementation system of a question-answering large model based on the fine-tuning technology according to claim 5, wherein, Among them, the deep learning framework uses PyTorch. PyTorch provides flexible tools and libraries for building deep learning models, is dedicated to natural language processing tasks, and encapsulates the basic functions of fine-tuning and calling large models.

Images