Domain Q&A large model training and Q&A methods, related devices and program products

Abstract

The present application discloses a method for training and answering questions of a domain question-answering large model, related devices and program products, which relates to the field of natural language processing technology. The present application introduces a referee model, which can score the preference of the input answer and evaluate the quality of the answer. Domain question-answering data is extracted based on a domain knowledge base. When iteratively training an initial large model, the referee large model scores the first answer corresponding to the domain question generated by the initial large model and the second answer in the domain question-answering data respectively. Based on the scoring results, the answers that meet the preference requirements are selected and combined with the domain questions to form target training data, and the initial large model is trained using the target training data to obtain a final domain question-answering large model, which is trained using high-quality domain question-answering training data. The present application can obtain high-quality target training data without manually proofreading the extracted domain question-answering data, saving labor costs and improving training efficiency.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology. More specifically, it relates to a method for training and answering questions of a domain question-answering large model, related devices, and program products. Background Art

[0002] General large models are usually trained based on extensive public literature and network data, lacking the accumulation of professional knowledge and industry data. Therefore, they have deficiencies in terms of industry pertinence and accuracy. However, users have relatively high requirements for the professional services of industry large models and low tolerance for errors. Once an industry large model provides incorrect information to the public, serious consequences may occur. Retraining and fine-tuning general large models with industry data to build highly available question-answering systems can solve the above problems to a certain extent.

[0003] Technical standards and domain text data in industry data can only be used for the pre-training of models. What is needed for building a question-answering system for subsequent question answering is instruction data, that is, question-and-answer pair data in the industry. Existing technologies generally use some artificial intelligence methods to generate some instruction data. The generated instruction data may have defects such as factual errors. To ensure the quality of the instruction data, a large amount of manual proofreading is still required before it can be applied to model training. Therefore, the current training process of domain question-answering large models still requires a large amount of manual work to proofread instruction data. Summary of the Invention

[0004] In view of the above problems, this application is proposed to provide a method for training and answering questions of a domain question-answering large model, related devices, and program products, so as to reduce the manual proofreading work of instruction data in the training process of the domain question-answering large model, reduce costs, and improve training efficiency. The specific solutions are as follows:

[0005] In a first aspect, a method for training a domain question-answering large model is provided, including:

[0006] Obtaining an initial large model with question-answering capabilities and a referee model trained with question-answer data marked with preference rankings;

[0007] Obtaining a domain knowledge base, and extracting domain question-answer data based on the domain knowledge base, where the domain knowledge base includes domain knowledge information;

[0008] Iteratively train the initial large model using the domain Q&A data. In each round of training, the referee model scores the first answer corresponding to the domain question generated by the initial large model and the second answer corresponding to the domain question in the domain Q&A data respectively. Based on the scoring results, select the answers that meet the preference requirements and form target training data with the domain questions, and use the target training data to train the initial large model. After the iterative training, a domain Q&A large model is obtained.

[0009] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, during the iterative training of the initial large model, the domain Q&A data used in different training rounds is different.

[0010] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the process of using the domain Q&A data to perform any round of training on the initial large model includes:

[0011] Send the domain questions in the domain Q&A data into the initial large model to obtain the first answer generated by the initial large model;

[0012] Score the first answer and the second answer respectively through the referee model, and select the answers that meet the preference requirements based on the scoring results and form target training data with the domain questions;

[0013] Use the target training data to continue training the initial large model obtained from the previous round of training to obtain the initial large model after this round of training.

[0014] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the process of selecting the answers that meet the preference requirements based on the scoring results and forming target training data with the domain questions includes:

[0015] Sort the answers according to the scores in the scoring results, and select the top N answers in the sorting and form target training data with the domain questions, where N is an integer greater than or equal to 1.

[0016] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the domain knowledge base is an extensible domain knowledge base, and the method further includes:

[0017] When it is detected that new knowledge information is added to the extensible domain knowledge base, extract new domain Q&A data based on the new knowledge information, and use the new domain Q&A data to update and train the domain Q&A large model to obtain an updated domain Q&A large model.

[0018] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the referee model is a referee large model obtained by supervising and training a general large model with question-and-answer data marked with preference rankings.

[0019] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of extracting domain question-and-answer data based on the domain knowledge base includes:

[0020] For each modality of domain knowledge information in the domain knowledge base, perform vectorization processing respectively, and store the vector representation corresponding to the domain knowledge information in a vector database;

[0021] Vectorize the configured query question, and retrieve the top K candidate domain knowledge information with the highest similarity to the query question based on vector similarity, where K is a set positive integer;

[0022] Add the top K candidate domain knowledge information and the query question to the first prompt instruction prompt, and submit it to the general large model to instruct the general large model to generate the answer most matching the query question, and then form domain question-and-answer data with the query question.

[0023] In a possible design, in another implementation of the first aspect of the embodiments of the present application, the process of extracting domain question-and-answer data based on the domain knowledge base includes:

[0024] Call the general large model to instruct the general large model to extract several questions and corresponding answers from the provided domain knowledge information;

[0025] The questions and corresponding answers extracted by the general large model form domain question-and-answer data.

[0026] In a second aspect, a question-and-answer method is provided, including:

[0027] Obtain a domain question;

[0028] Send the domain question into a configured domain question-and-answer large model to obtain the answer output by the model;

[0029] Wherein, the domain question-and-answer large model is trained by using the domain question-and-answer large model training method in any implementation of the first aspect of the embodiments of the present application.

[0030] In a third aspect, a domain question-and-answer large model training device is provided, including:

[0031] A model acquisition unit, configured to acquire an initial large model with question-and-answer capabilities and a referee model trained with question-and-answer data marked with preference rankings;

[0032] A knowledge base processing unit, configured to obtain a domain knowledge base and extract domain Q&A data based on the domain knowledge base, where the domain knowledge base includes domain knowledge information;

[0033] A model training unit, configured to iteratively train the initial large model by using the domain Q&A data. In each round of training, the referee model scores the first answer corresponding to the domain question generated by the initial large model and the second answer in the domain Q&A data respectively, selects the answer that meets the preference requirements based on the scoring results, and forms target training data with the domain question, and uses the target training data to train the initial large model until the domain Q&A large model is obtained after the last round of training.

[0034] In a fourth aspect, an electronic device is provided, including: a memory and a processor;

[0035] The memory is configured to store a program;

[0036] The processor is configured to execute the program to implement each step of the domain Q&A large model training method described in any implementation manner of the first aspect of the embodiments of the present application, or implement each step of the Q&A method described in the second aspect.

[0037] In a fifth aspect, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, each step of the domain Q&A large model training method described in any implementation manner of the first aspect of the embodiments of the present application is implemented, or each step of the Q&A method described in the second aspect is implemented.

[0038] In a sixth aspect, a computer program product is provided, including a computer program. When the computer program is executed by a processor, each step of the domain Q&A large model training method described in any implementation manner of the first aspect of the embodiments of the present application is implemented, or each step of the Q&A method described in the second aspect is implemented.

[0039] With the above technical solution, the present application uses an initial large model with question-and-answer capabilities as the base of the domain question-and-answer large model, and at the same time introduces a referee model, which is trained using question-and-answer data marked with preference rankings and can perform preference scoring on different input answers to evaluate the quality of different answers. The present application configures a domain knowledge base, which can extract domain question-and-answer data based on the domain knowledge information in the domain knowledge base. When iteratively training the initial large model, the present application does not directly use the extracted domain question-and-answer data for training, but the referee large model scores the first answer corresponding to the domain question generated by the initial large model and the second answer corresponding to the domain question in the domain question-and-answer data respectively, and selects the answer that meets the preference requirements based on the scoring results to form target training data with the domain question. Obviously, by scoring the first answer and the second answer by the referee large model, higher-quality answers can be selected to form high-quality target training data with the domain question, and then the initial large model can be trained using the target training data. After one or more iterative trainings in this way, the final domain question-and-answer large model can be obtained, which is trained using high-quality domain question-and-answer training data, better understands the semantics and specifications of the domain, and provides stronger question-and-answer capabilities in terms of domain professionalism and practicality. Moreover, the solution of the present application can obtain high-quality target training data without manual proofreading of the extracted domain question-and-answer data, saving labor costs and improving training efficiency. Description of the Drawings

[0040] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of illustrating the preferred embodiments and are not considered to be a limitation of the present application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0041] Figure 1 It is a schematic diagram of an implementation system architecture for the large model training method and question-and-answer method provided by an embodiment of the present application;

[0042] Figure 2 It is a schematic diagram of the flow of a domain question-and-answer large model training method provided by an embodiment of the present application;

[0043] Figure 3 It exemplifies a schematic diagram of the acquisition method of the annotation data required for training a referee model;

[0044] Figure 4 It exemplifies a schematic diagram of the processing flow of extracting domain question-and-answer data from a domain knowledge base;

[0045] Figure 5 It exemplifies another schematic diagram of the flow of a domain question-and-answer large model training method;

[0046] Figure 6Schematic structural diagram of a training device for a domain question-answering large model provided by an embodiment of the present application;

[0047] Figure 7 Schematic structural diagram of an electronic device provided by an embodiment of the present application. Specific embodiments

[0048] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0049] The domain large model can also be called an industry large model, which is generally defined as retraining and fine-tuning a general large model with domain data to solve domain problems. There are two ways of using domain data. The first is to use domain data to continue training, fine-tune and other methods that change the model weights; the second is to use the in context learning ability to inject domain knowledge through prompts or use an external database without changing the weights of the general large model. The former can be called training / fine-tuning a domain large model because it changes the model weights, while the latter can basically only be regarded as an application of the general large model.

[0050] The technical standards and domain text data in domain data can only be used for the pre-training of the model. To build a question-answering system for subsequent question answering, instruction data is needed. Existing technologies generally use some artificial intelligence methods to generate some instruction data (domain question-answering data), but in order to ensure factuality, a large amount of manual proofreading is still required. High-quality industry instruction data is the bottleneck for model fine-tuning. The present application introduces a referee model to score the extracted domain question-answering data and the feedback data of the initial large model respectively, and selects high-quality answers and domain questions to form target training data based on the scoring results, so as to optimize the initial large model with the target training data to obtain the final domain question-answering large model.

[0051] The present application provides a training method for a domain question-answering large model and a question-answering method based on the domain question-answering large model. The method of the present application can be applied to a variety of fields, such as the intelligent vehicle field, the medical field, the education field, etc.

[0052] The method provided by the present application can be divided into a training stage and an inference stage. The training stage is the stage of training the domain question-answering large model, and the inference stage is the process of using the trained domain question-answering large model to perform domain question-answering tasks. Among them, the training stage and the inference stage can be deployed in the same device or in different devices.

[0053] Combined Figure 1 With the system structure shown, the system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 Taking the inclusion of one server as an example for illustration).

[0054] For example, the training phase may be deployed in the server 200, and the inference phase may be deployed in the terminal 100, such as in a mobile phone, a tablet, an intelligent vehicle, a robot, or a wearable device, etc.

[0055] For ease of understanding, this application separately introduces the processes of the training phase and the inference phase.

[0056] I. Training Phase

[0057] Referring to Figure 2 , a flowchart of a method for training a domain question-answering large model provided by an embodiment of this application may specifically include the following steps:

[0058] Step S100: Obtain an initial large model with question-answering capabilities and a referee model trained using question-answering data labeled with preference rankings.

[0059] The initial large model serves as the foundation of the domain question-answering large model to be trained in this application. To improve training efficiency, the initial large model obtained in this embodiment is a large model with question-answering capabilities, that is, the initial large model can answer user instructions.

[0060] For currently existing general large models, some general large models already have question-answering capabilities themselves, so in this embodiment, such general large models can be directly used as the initial large model. For some other general large models without question-answering capabilities, in this embodiment, question-answering data can also be constructed using the collected public datasets, and the general large model can be supervised and trained using the question-answering data to obtain an initial large model with question-answering capabilities.

[0061] For public datasets, they can be obtained from various sources such as the Internet or professional databases. Taking the maternal and infant industry as an example, the Maternal and Infant (MATINF) Dataset is a large-scale jointly labeled dataset that can be used for classification, question-answering, and summarization in the field of Chinese maternal and infant care. This dataset collected nearly 2 million pairs of question-answer pairs from large maternal and infant care Q&A websites and was constructed through automatic and manual data cleaning. In this embodiment, the public dataset can be used to supervise and train (Supervised Fine-Tuning, SFT) the general large model foundation to obtain a model with basic question-answering capabilities, which is called the initial large model, or sft-model.

[0062] Furthermore, in this embodiment, a referee model is also obtained. The referee model is trained using question-and-answer data marked with preference rankings. It can score the answers corresponding to the input questions, and the scoring results can evaluate the quality of the answers. For example, the higher the score, the higher the quality of the answer, that is, it is more in line with the user's preference for the answer.

[0063] The referee model in this embodiment can also be called a reward model. It is trained using question-and-answer data marked with preference rankings. Different answers in the marked data can be obtained through an initial large model, a general large model, a domain knowledge base, etc. Further, artificial experts perform preference ranking annotations on different answers to obtain the final question-and-answer data marked with preference rankings.

[0064] Refer to Figure 3 for an example of the acquisition method of the marked data required for training a referee model.

[0065] Taking the question "What is hypersensitivity?" as an example, answers corresponding to the question can be obtained through various channels. Figure 3 shows different answers obtained through a large model and multiple different knowledge bases, which are defined as Answer A, Answer B, and Answer C from left to right.

[0066] Then, experts can perform preference ranking annotations on Answers A, B, and C. For example, the ranking result is B > C > A. Then, the marked question-and-answer data can be used to train the referee model.

[0067] This application focuses on professional question-and-answer systems based on vertical domain-specific knowledge / data and aims to solve the requirements of specific scenarios in vertical domains. Therefore, it is required that experts focus on professional verification and authenticity verification during data annotation and give the partial order relationship of different answers based on these two dimensions. Among them, professional verification requires the model to search, integrate, and reason relevant professional knowledge according to the questions raised by users to provide accurate, comprehensive, and in-depth answers; authenticity verification determines whether the answers output by the model contain fictional, incorrect, or statements that do not conform to reality, or whether the model can correctly distinguish when faced with untrue information.

[0068] After experts annotate the answers according to the above professional verification and authenticity verification standards to obtain the marked question-and-answer data, training the referee model using this question-and-answer data can make the answer scores given by the referee model meet the professional verification and authenticity verification standards, that is, it can give more accurate quality scores.

[0069] The referee model in this embodiment can adopt a variety of neural network structures, for example, various types of regression models. In one possible implementation, in order to improve the performance of the referee model and reduce the training time of the referee model, the referee model can be trained on the basis of a general large model to obtain a trained large referee model. The capabilities of the general large model base can be fully utilized to improve the performance of the large referee model. At the same time, there is no need to train a completely new model architecture from scratch, which can improve the training efficiency of the referee model.

[0070] When the referee model is a large referee model, it is essentially to use the large model to perform a regression task, that is, a multi-layer perceptron MLP is added at the [CLS] position of the output layer of the model, and the output value is the score given by the large referee model.

[0071] In one possible implementation, the process of training the referee model on the basis of the general large model can adopt LORA or other possible training strategies.

[0072] Step S110: Obtain a domain knowledge base, and extract domain Q&A data based on the domain knowledge base. The domain knowledge base includes domain knowledge information.

[0073] In order to enable the domain Q&A large model to better understand the semantics, norms, and knowledge of the domain and provide stronger Q&A capabilities in terms of domain professionalism and practicality, a domain knowledge base is configured in this embodiment. The domain knowledge base contains knowledge information of the target domain. The target domain is the domain to which the Q&A method is applied, such as the intelligent vehicle domain, the medical domain, the education domain, etc.

[0074] The domain knowledge information in the domain knowledge base can cover a variety of different information sources, including but not limited to: community Q&A (Community QA) data obtained from online forums, document Q&A (DocumentQA) data extracted from domain professional documents, knowledge graph Q&A (Knowledge-based QA) data extracted from knowledge graphs, etc. In practical applications, document Q&A is mainly used, and users are supported to upload documents independently to generate personal knowledge bases.

[0075] In one possible implementation, the domain knowledge base of the present application may include one or more modalities of domain knowledge information, including but not limited to: text modality, image modality, audio, video modality, etc.

[0076] The domain knowledge base in this embodiment can also support expansion, that is, an expandable domain knowledge base. Users can dynamically add new knowledge information to the domain knowledge base.

[0077] In this embodiment, based on the knowledge information in the domain knowledge base, domain Q&A data can be extracted. When extracting the domain Q&A data, a rule-based extraction method can be adopted, or an extraction method based on artificial intelligence algorithms can be used, so as to obtain a large number of paired domain Q&A data, which are applied to the subsequent model training process.

[0078] Step S120: Use the domain Q&A data to iteratively train the initial large model. In each round of training, the referee model scores the answers generated by the initial large model and the answers in the domain Q&A data respectively, and guides the training of the initial large model based on the scoring results to obtain the finally trained domain Q&A large model.

[0079] Among them, the domain Q&A data includes domain questions and corresponding answers (defined as the second answers).

[0080] This embodiment adopts a training strategy similar to reinforcement learning. In the process of iteratively training the initial large model, in each round of training:

[0081] The referee model scores the answers corresponding to the domain questions generated by the initial large model (defined as the first answers) and the second answers in the domain Q&A data respectively. Based on the scoring results, the answers that meet the preference requirements are selected and combined with the domain questions to form the target training data, and the initial large model is trained using the target training data. After iterative training, the domain Q&A large model is obtained.

[0082] In an optional implementation manner, the process of using the domain Q&A data to perform any round of training on the initial large model includes:

[0083] S1: Send the domain questions in the domain Q&A data into the initial large model to obtain the first answers generated by the initial large model.

[0084] S2: The referee model scores the first answers and the second answers respectively, and based on the scoring results, selects the answers that meet the preference requirements and combines them with the domain questions to form the target training data.

[0085] S3: Use the target training data to continue training the initial large model obtained in the previous round of training to obtain the initial large model after this round of training.

[0086] Among them, the initial large model after each round of training serves as the initial large model for the next round, that is, the initial large model is continuously updated as the training progresses through iterative rounds. After multiple rounds of iterative training, the trained initial large model is used as the domain Q&A large model.

[0087] Since in each round of the training process, the referee model scores the first answer output by the initial large model and the second answer in the domain Q&A data respectively, it can measure the quality of the two answers, and select the answer that meets the preference requirements from the first answer and the second answer, and form the target training data with the domain question for training the initial large model. The initial large model can continuously improve its domain Q&A ability based on the high-quality target training data. In this way, in the next round of training, the initial large model can generate a higher-quality first answer, and after being scored and evaluated by the referee model, a higher-quality answer can be selected, thus continuously promoting the training effect of the initial large model.

[0088] In a possible implementation, the process of selecting the answer that meets the preference requirements based on the scoring result and forming the target training data with the domain question includes:

[0089] Sort the scores of each answer in the scoring result, and select the top N answers in the sorting to form the target training data with the domain question, where N is an integer greater than or equal to 1. When the set preference requirement is to select the answer with the highest score, N is equal to 1, and when the set preference requirement is to select several answers with the top scores, N is greater than 1.

[0090] Since the second answer is the answer corresponding to the domain question in the domain Q&A data, and the domain Q&A data is extracted from the domain knowledge base, it is possible that one or more answers are extracted for one domain question, that is, the second answer corresponding to one domain question may be one or more.

[0091] When N = 1, that is, select the answer with the highest score from the first answer and the second answer to form the target training data with the domain question.

[0092] When N > 1, two or more answers can be selected from the first answer and the second answer, and respectively form the target training data with the domain question.

[0093] It can be understood that the domain knowledge base can contain a large amount of domain knowledge information, so a large number of domain Q&A data can be extracted for different domain knowledge information. When using the extracted domain Q&A data to iteratively train the initial large model, the domain Q&A data used in different training rounds can be the same or different.

[0094] As an example, in each round of the training process, M pairs of domain Q&A data are used to train the initial large model. After the end of this round of training, new M pairs of domain Q&A data are selected to be used as the next round of training data.

[0095] Another example is to use domain knowledge information from different entries in the domain knowledge base as units, extract domain question and answer data for any domain knowledge information entry, and use the extracted domain question and answer data in one round of training. In the next round of training, the domain question and answer data extracted from the domain knowledge information of another entry in the domain knowledge base is used.

[0096] The process of iterative training of the initial large model can have various conditions for ending training, such as: all domain question and answer data have been used up, or the number of iterations has reached a set number, or the training time has reached a set time, or the performance of the trained initial large model meets the set requirements, etc.

[0097] The domain question and answer big model training method provided in the embodiment of the present application uses the initial big model with question and answer capabilities as the base of the domain question and answer big model, and introduces a referee model, which is trained using question and answer data marked with preference rankings, and can perform preference scoring on different input answers and evaluate the quality of different answers. The present application is configured with a domain knowledge base, and domain question and answer data can be extracted based on the domain knowledge information in the domain knowledge base. When iteratively training the initial big model, the present application does not directly use the extracted domain question and answer data for training, but the referee big model scores the first answer corresponding to the domain question generated by the initial big model and the second answer in the domain question and answer data respectively. Based on the scoring results, the answer and domain question that meet the preference requirements are selected to form the target training data. Obviously, after the referee big model scores the first answer and the second answer, higher quality answers and domain questions can be selected to form high-quality target training data, and then the target training data can be used to train the initial big model. After one or more iterative trainings, the final domain question and answer big model can be obtained. It uses high-quality domain question and answer training data for training, better understands the semantics and specifications of the domain, and provides more professional and practical question and answer capabilities in the domain. Furthermore, the present application solution can obtain high-quality target training data without the need for manual proofreading of the extracted domain question-and-answer data, thereby saving labor costs and improving training efficiency.

[0098] In real-world scenarios, the process of generating domain data is inherently fluid. In terms of data timeliness, the training cycle of large models inherently makes it difficult to achieve timeliness. Therefore, supplementing large models with highly time-sensitive domain data is a prerequisite for large domain models. Taking the financial sector as an example, data timeliness is one of the challenges in implementing large models. How to input real-time data such as emergencies and financial information into large models is directly related to whether large financial models can accurately perform analytical decisions. To this end, this embodiment provides an extensible domain knowledge base.

[0099] After the training of the domain Q&A large model is completed based on the extensible domain knowledge base, the domain Q&A large model can be deployed in the application scenario. Subsequently, users can also periodically or in real time add new knowledge information to the extensible domain knowledge base.

[0100] In a possible implementation, when the large model training method of the present application detects new knowledge information in the extensible domain knowledge base, it can further extract new domain Q&A data based on the new knowledge information, and use the new domain Q&A data to update and train the domain Q&A large model to obtain an updated domain Q&A large model.

[0101] In this embodiment, the process of using the new domain Q&A data to update and train the domain Q&A large model is the same as the process of iteratively training the initial large model with the domain Q&A data in the foregoing step S120. Only the domain Q&A large model to be updated needs to be used as the initial large model in step S120, and then the new domain Q&A data can be used to iteratively update and train the initial large model. In each round of training, the referee model scores the answers generated by the initial large model and the answers in the domain Q&A data respectively, and guides the training of the initial large model based on the scoring results. After iterative training, an updated domain Q&A large model is obtained.

[0102] Among them, the timing of updating and training the domain Q&A large model can be set by the user, including but not limited to any one or more combinations of the following:

[0103] The number of new knowledge information in the extensible domain knowledge base reaches a set number threshold, the importance of the new knowledge information reaches a set importance level, and a set large model update period is reached.

[0104] In this embodiment, by configuring the extensible domain knowledge base, knowledge can be updated quickly and the timeliness of the answers of the domain Q&A large model can be improved. At the same time, the existence of the extensible domain knowledge base can largely eliminate the hallucinations of the large model and improve the accuracy of the model's answers.

[0105] In some embodiments of the present application, the process of extracting domain Q&A data based on the domain knowledge base in the foregoing step S110 is described.

[0106] The domain knowledge base may include domain knowledge information in multiple modalities, such as knowledge information in multiple formats under multiple modalities such as text, image, audio, video, and library tables. In a possible implementation, the process of extracting domain Q&A data based on the domain knowledge base may include the following steps:

[0107] S1. For each modality of domain knowledge information in the domain knowledge base, perform vectorization processing respectively, and store the vector representation corresponding to the domain knowledge information in the vector database.

[0108] Specifically, multi-modal alignment technology can be adopted to align the vectorized representations of domain knowledge information in different modalities into the same vector space, obtain the vector representations corresponding to each piece of domain knowledge information, and store them in the vector database.

[0109] S2. Vectorize the configured query problem, and retrieve the top K (topK) candidate domain knowledge information with the highest similarity to the query problem based on vector similarity.

[0110] Among them, the query problem can be pre-set or extracted from the domain knowledge information. The query problem can be a domain problem, so as to ensure that matching domain knowledge information can be retrieved in the domain knowledge base. Vectorize the query problem, then calculate the similarity with each vector in the vector database, and select the candidate domain knowledge information corresponding to the top K vectors with the highest similarity as the candidate answers corresponding to the query problem.

[0111] S3. Add the top K candidate domain knowledge information and the query problem to the first prompt, and submit it to the general large model to instruct the general large model to generate the answer most matching the query problem, and then form domain Q&A data with the query problem.

[0112] The above top K candidate answers retrieved through vector similarity may not be accurate enough. Therefore, in this embodiment, the ability of the large model can be invoked to generate the answer most matching the query problem through the large model.

[0113] In a possible implementation, obtain the format template of the first prompt, where the prompt format template includes a task instruction, a question slot, and a candidate answer slot, and the task instruction is used to instruct the large model to determine the answer most matching the query problem from the candidate answers provided in the candidate answer slot according to the query problem in the question slot.

[0114] Fill the query problem into the question slot, fill the top K candidate domain knowledge information as candidate answers into the candidate answer slot, obtain the first prompt, input the first prompt into the general large model, obtain the final answer output by the large model, and form domain Q&A data with the query problem.

[0115] Combined with Figure 4 as shown, it exemplifies a processing flow for extracting domain Q&A data from the domain knowledge base.

[0116] For various types of domain knowledge information included in the domain knowledge base, including but not limited to: text data, image data, audio data, table data, etc. These domain knowledge information are vectorized, aligned to a unified vector space, to obtain the vector representation of each knowledge data, and stored in the vector database.

[0117] For the configured query question Q, it can be a question set based on a template or a question extracted from the domain knowledge information. The query question is vectorized to obtain a query question vector. Using this query question vector, vector similarity retrieval is performed in the vector database to obtain the top K knowledge data with the highest similarity.

[0118] To improve the accuracy of the finally extracted domain Q&A data, in this embodiment, the top K knowledge data and the query question can be filled into the prompt template to form a prompt instruction prompt. This prompt is used to instruct the large model to combine the query question and the top K knowledge data to determine the answer most matching the query question, and obtain the final answer A corresponding to the query question output by the large model. A pair of domain Q&A data is composed of the query question Q and the final answer A.

[0119] The domain Q&A data extraction solution in the above example combines vector retrieval technology and large model technology, and can improve the quality of the obtained domain Q&A data.

[0120] In another implementation, another possible implementation solution for extracting domain Q&A data based on the domain knowledge base is introduced.

[0121] In this embodiment, the large model can be directly called to perform the extraction work of domain Q&A data. Exemplarily, a general large model is called to instruct the general large model to extract several questions and corresponding answers from the provided domain knowledge information. The questions and corresponding answers extracted by the general large model form domain Q&A data.

[0122] The following exemplifies a possible prompt instruction prompt:

[0123] Extract several questions and corresponding answers from the following information, requiring them to be logical and factual, and give the results in the form of "Question:" and "Answer:". [Domain knowledge information].

[0124] Domain knowledge information can be extracted from the domain knowledge base, filled into the above prompt, and then the large model is called to perform the extraction work of questions and answers.

[0125] It can be understood that considering the input length limit of the large model, the domain knowledge information in the domain knowledge base can be segmented, and a part of the domain knowledge information is sent each time for the large model to perform the extraction work of questions and answers.

[0126] In this embodiment, by injecting domain knowledge information into the prompt, the large model is instructed to extract questions and corresponding answers from the domain knowledge information, and the extraction of domain Q&A data can be achieved by leveraging the capabilities of the large model, with higher efficiency.

[0127] Referring to Figure 5 , a training process for a domain Q&A large model is provided.

[0128] Based on a general large model, by using a publicly available dataset to extract Q&A data pairs, the general large model is fine-tuned through SFT training to obtain an SFT large model, which has preliminary Q&A capabilities. The SFT large model can be used as the initial large model to be trained.

[0129] Furthermore, for multiple-source different answers to the same Q&A, Q&A data marked with preference rankings can be obtained through expert annotation. Then, the supervised training RM of the general large model can be carried out using the annotated Q&A data, and the trained referee large model (which can also be called the reward model) can be obtained. The referee large model can score the answers corresponding to the input domain questions, and the score can measure the quality of the answers.

[0130] This embodiment can also configure an extensible domain knowledge base, which stores domain knowledge information and supports dynamic updates. Domain Q&A data QA can be extracted from the domain knowledge base.

[0131] During the iterative training process of the initial large model, in each round of training, the domain question Q in the domain Q&A data is sent into the initial large model to obtain the answer A' generated by the initial large model. The domain question Q, the answer A, and the answer A' are sent into the referee large model together. The referee large model gives preference scores to the answers A and A', and selects the answer with the highest score, which is combined with the domain question Q to form the target training data. The initial large model is trained using the target training data.

[0132] After multiple rounds of iterative training, the initial large model after the last round of training is used as the final domain Q&A large model.

[0133] The domain Q&A large model trained in this embodiment supports private deployment, reducing the risks of data leakage and security vulnerabilities, and providing higher credibility and protection for tasks involving sensitive information. The private deployment of the domain Q&A large model enables enterprises to better manage and control data governance.

[0134] The existence of an extensible domain knowledge base can largely eliminate model hallucinations and improve the accuracy of the answers given by the domain question-answering large model. At the same time, the extensible domain knowledge base can quickly update knowledge and improve the timeliness of the answers given by the domain question-answering large model. The introduction of the referee large model enriches the source of high-quality domain question-answering data, maximizes the role of professional knowledge and industry data accumulation, and at the same time eliminates the need for manual proofreading of domain question-answering data, saving labor costs.

[0135] II. Inference stage

[0136] Based on the domain question-answering large model obtained after training according to the foregoing embodiments, this embodiment further provides a question-answering method, which specifically includes:

[0137] First, obtain a domain question.

[0138] Further, send the domain question into the configured domain question-answering large model to obtain the answer output by the model.

[0139] Specifically, in the domain question-answering scenario, the user can interact with the domain question-answering large model, send the user's interaction content as a domain question into the domain question-answering large model, and the large model generates the corresponding answer and outputs it to the user.

[0140] Based on the domain question-answering large model training method introduced in the foregoing embodiments, it can be seen that the model training process uses extensible domain knowledge information, enabling the model to better understand the semantics, norms, and knowledge of the domain, and thus being able to provide stronger question-answering capabilities in terms of domain professionalism and practicality.

[0141] Next, a description will be given of the domain question-answering large model training device provided by the embodiments of the present application. The domain question-answering large model training device described below can be correspondingly referred to the domain question-answering large model training method described above.

[0142] See Figure 6 , Figure 6 which is a schematic structural diagram of a domain question-answering large model training device disclosed in the embodiments of the present application.

[0143] As Figure 6 shown, the device may include:

[0144] A model acquisition unit 11, configured to acquire an initial large model with question-answering capabilities and a referee model trained using question-answering data marked with preference rankings;

[0145] A knowledge base processing unit 12, configured to acquire a domain knowledge base and extract domain question-answering data based on the domain knowledge base, where the domain knowledge base includes domain knowledge information;

[0146] A model training unit 13 is configured to iteratively train the initial large model by using the domain Q&A data. In each round of training, the referee model scores the first answer corresponding to the domain question generated by the initial large model and the second answer corresponding to the domain question in the domain Q&A data respectively. Based on the scoring results, the answers that meet the preference requirements are selected and combined with the domain questions to form target training data, and the initial large model is trained by using the target training data. After the iterative training, a domain Q&A large model is obtained.

[0147] In a possible implementation, during the process of the model training unit iteratively training the initial large model, the domain Q&A data used in different training rounds is different.

[0148] In a possible implementation, the process of the model training unit using the domain Q&A data to perform any round of training on the initial large model includes:

[0149] Sending the domain questions in the domain Q&A data into the initial large model to obtain the first answer generated by the initial large model;

[0150] Scoring the first answer and the second answer respectively through the referee model, and selecting the answers that meet the preference requirements based on the scoring results and combining them with the domain questions to form target training data;

[0151] Using the target training data to continue training the initial large model obtained in the previous round of training to obtain the initial large model after this round of training.

[0152] In a possible implementation, the process of the model training unit selecting the answers that meet the preference requirements based on the scoring results and combining them with the domain questions to form target training data includes:

[0153] Sorting the answers according to the scores in the scoring results, and selecting the top N answers in the sorting and combining them with the domain questions to form target training data, where N is an integer greater than or equal to 1.

[0154] In a possible implementation, if the domain knowledge base is an extensible domain knowledge base, the device of the present application may further include:

[0155] A model update training unit is configured to, when detecting new knowledge information in the extensible domain knowledge base, extract new domain Q&A data based on the new knowledge information, and use the new domain Q&A data to perform update training on the domain Q&A large model to obtain an updated domain Q&A large model.

[0156] In a possible implementation, the referee model obtained by the model acquisition unit is a referee large model obtained by supervising and training a general large model by using Q&A data marked with preference rankings.

[0157] In a possible implementation, the process by which the knowledge base processing unit extracts domain Q&A data based on the domain knowledge base includes:

[0158] For each modality of domain knowledge information in the domain knowledge base, perform vectorization processing respectively, and store the vector representation corresponding to the domain knowledge information in the vector database;

[0159] Vectorize the configured query question, and retrieve the top K candidate domain knowledge information that matches the query question based on vector similarity;

[0160] Add the top K candidate domain knowledge information and the query question to the first prompt instruction prompt, and submit it to the general large model to instruct the general large model to generate the answer that best matches the query question, and then form domain Q&A data with the query question.

[0161] In another possible implementation, the process by which the knowledge base processing unit extracts domain Q&A data based on the domain knowledge base includes:

[0162] Call the general large model to instruct the general large model to extract a number of questions and corresponding answers from the provided domain knowledge information;

[0163] The questions and corresponding answers extracted by the general large model form domain Q&A data.

[0164] In the embodiments of the present application, an electronic device is further provided. Refer to Figure 7 As shown, it shows a schematic structural diagram of an electronic device suitable for implementing the electronic device in the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, tablet computers, teaching large screens, wearable devices, and the like. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present application.

[0165] Such as Figure 7As shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603, so as to implement the domain question-and-answer large model training method of the foregoing embodiments of the present application, or the question-and-answer method of the foregoing embodiments. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0166] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 7 an electronic device with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0167] In an embodiment of the present application, there is also provided a computer program product including computer-readable instructions. When the computer-readable instructions run on an electronic device, the electronic device is enabled to implement any one of the domain question-and-answer large model training methods provided in the embodiments of the present application, or the question-and-answer method of the foregoing embodiments.

[0168] In an embodiment of the present application, there is also provided a computer-readable storage medium carrying one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can be enabled to implement any one of the domain question-and-answer large model training methods provided in the embodiments of the present application, or the question-and-answer method of the foregoing embodiments.

[0169] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in the present application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0170] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions accomplished by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, in more cases, software program implementation is a better embodiment for the present application. Based on such understanding, the technical solution of the present application, in essence or the part that makes contributions to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disc of a computer, etc., and includes several instructions for causing a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0171] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0172] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device, or data center to another website, computer, training device, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0173] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A method for training a large domain question-answering model, characterized in that, Including: Obtaining an initial large model with question - answering capabilities and a referee model trained using question - answering data marked with preference rankings; Obtaining a domain knowledge base and extracting domain question - answering data based on the domain knowledge base, where the domain knowledge base includes domain knowledge information; Iteratively training the initial large model using the domain question - answering data. In each round of training, the referee model scores the first answer corresponding to the domain question generated by the initial large model and the second answer corresponding to the domain question in the domain question - answering data respectively. Based on the scoring results, select the answers that meet the preference requirements and form target training data with the domain questions, and use the target training data to train the initial large model. After the iterative training, a domain question - answering large model is obtained; The process of using the domain question - answering data to perform any round of training on the initial large model includes: Sending the domain questions in the domain question - answering data into the initial large model to obtain the first answer generated by the initial large model; Scoring the first answer and the second answer respectively through the referee model, and based on the scoring results, select the answers that meet the preference requirements and form target training data with the domain questions; Using the target training data to continue training the initial large model obtained in the previous round of training to obtain the initial large model after this round of training.

2. The method according to claim 1, characterized in that During the process of iteratively training the initial large model, the domain question - answering data used in different training rounds is different.

3. The method according to claim 1, characterized in that, The process of selecting the answers that meet the preference requirements and forming target training data with the domain questions based on the scoring results includes: Sorting the scores of each answer in the scoring results, and selecting the top N answers in the sorting and forming target training data with the domain questions, where N is an integer greater than or equal to 1.

4. The method according to claim 1, characterized in that, The domain knowledge base is an extensible domain knowledge base, and the method further includes: When detecting new knowledge information in the extensible domain knowledge base, extracting new domain question - answering data based on the new knowledge information, and using the new domain question - answering data to update and train the domain question - answering large model to obtain an updated domain question - answering large model.

5. The method according to claim 1, wherein The referee model is a referee large model obtained by supervising and training a general large model using question - answering data marked with preference rankings.

6. The method according to any one of claims 1 to 5, characterized in that The process of extracting domain question - answering data based on the domain knowledge base includes: For each modality of domain knowledge information in the domain knowledge base, perform vectorization processing respectively, and store the vector representation corresponding to the domain knowledge information in a vector database; Vectorize the configured query question, and retrieve the top K candidate domain knowledge information with the highest similarity to the query question based on vector similarity, where K is a set positive integer; Add the top K candidate domain knowledge information and the query question to the first prompt instruction prompt and submit it to the general large model to instruct the general large model to generate the answer most matching the query question, and then form domain question - answering data with the query question.

7. The method according to any one of claims 1 to 5, characterized in that The process of extracting domain question - answering data based on the domain knowledge base includes: Invoke a general large model to instruct the general large model to extract a number of questions and corresponding answers from the provided domain knowledge information; The questions and corresponding answers extracted by the general large model form domain Q&A data.

8. A question-and-answer method, characterized in that, It includes: Obtain domain questions; Send the domain questions into the configured domain Q&A large model to obtain the answers output by the model; Among them, the domain Q&A large model is trained by using the domain Q&A large model training method described in any one of claims 1 to 7.

9. An apparatus for training a large model for domain question answering, characterized in that, It includes: A model acquisition unit for acquiring an initial large model with Q&A capabilities and a referee model trained by using Q&A data marked with preference rankings; A knowledge base processing unit for acquiring a domain knowledge base and extracting domain Q&A data based on the domain knowledge base, where the domain knowledge base includes domain knowledge information; A model training unit for iteratively training the initial large model by using the domain Q&A data. In each round of training process, the referee model scores the first answer corresponding to the domain question generated by the initial large model and the second answer in the domain Q&A data respectively, selects the answer that meets the preference requirements based on the scoring results, forms target training data with the domain question, and uses the target training data to train the initial large model until the domain Q&A large model is obtained after the last round of training; Among them, the process of using the domain Q&A data to perform any round of training on the initial large model includes: Send the domain questions in the domain Q&A data into the initial large model to obtain the first answer generated by the initial large model; Score the first answer and the second answer respectively through the referee model, and select the answer that meets the preference requirements based on the scoring results to form target training data; Use the target training data to continue training the initial large model obtained in the previous round of training to obtain the initial large model after this round of training.

10. An electronic device, characterized in that, It includes: A memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the domain Q&A large model training method described in any one of claims 1 to 7, or to implement each step of the Q&A method described in claim 8.

11. A readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, it implements each step of the domain Q&A large model training method described in any one of claims 1 to 7, or implements each step of the Q&A method described in claim 8.

12. A computer program product, comprising a computer program, characterized in that, When this computer program is executed by the processor, it implements each step of the domain Q&A large model training method described in any one of claims 1 to 7, or implements each step of the Q&A method described in claim 8.

Images