A method and device for joint parameter adjustment of classification and question-answering large models

Abstract

The present invention provides a method and device for joint parameter adjustment of classification and question-answering models, belonging to the technical field of model parameter adjustment. The method obtains professional question-answer pairs and general question-answering data sets respectively, and mixes the two according to a preset ratio to construct a data set required for model parameter adjustment; the model includes a classification model and a question-answering model; the classification model is used to determine the question category of the input question, and the question category and question text are spliced ​​into prompt words and then input into the question-answering model for processing; the classification model parameters are frozen, and the question-answering model parameters are adjusted; the question-answering model parameters are frozen, and the classification model parameters are adjusted; the above parameter adjustment process is repeated until the predetermined model standard is reached. The present invention realizes the joint parameter adjustment of classification and question-answering models, while improving the professional knowledge of the question-answering model, retaining the general knowledge generalization ability of the model as much as possible, improving the accuracy, reliability and efficiency of question responses, and improving the question-answering effect.

Description

Technical Field

[0001] The present invention relates to the technical field of model parameter adjustment, and in particular to a method and device for joint parameter adjustment of a classification and question-answering large model. Background Art

[0002] With the development of big model technology, big question-answering models are becoming increasingly widely used. By pre-training on large-scale, multi-domain datasets, big question-answering models can learn a wealth of general and domain knowledge, providing users with accurate and professional answers. As a result, big question-answering models have been gradually applied in various fields, including healthcare, law, finance, and education.

[0003] In order to improve the large-scale question-answering model's understanding of professional knowledge, it is necessary to use professional knowledge to adjust the parameters of the question-answering model, thereby improving the model's performance in professional fields. Although this process can significantly enhance the accuracy and depth of the model in answering professional questions, it will inevitably bring an inherent challenge to the model: that is, to a certain extent, it will affect its original generalization ability. Due to the complex interactivity of the model's internal parameters, focusing on the optimization of specific domain knowledge often causes the model to exhibit "forgetting" when dealing with unrelated fields or general problems. That is, while the model remembers new information, it may forget or weaken the previously learned extensive knowledge and general reasoning ability, thereby affecting the overall question-answering effect and the breadth of the user experience.

[0004] To further improve the accuracy of the Q&A model in handling professional questions while retaining its general knowledge and generalization capabilities as much as possible, a classification step can be added before the questions are input into the model. Specifically, by introducing a classification model, the input questions can be accurately divided into two categories: professional questions and non-professional questions. The Q&A model then makes intelligent judgments based on this classification result: for questions classified as professional, the Q&A model will use its optimized professional knowledge to provide in-depth and professional answers; for non-professional questions, the Q&A model will use its existing general knowledge and generalization capabilities to provide comprehensive and accurate responses.

[0005] Considering that existing open source classification models lack understanding of professional knowledge and cannot accurately classify questions, it is necessary to jointly adjust the parameters of the classification model and the question-answering model. While improving the professional ability of the model, the generalization ability should be retained as much as possible. While effectively improving the quality of the model's answers in professional fields, its flexibility and accuracy in dealing with general problems should be retained to the maximum extent. In this way, the professionalism and practicality of the question-answering model can be significantly enhanced without sacrificing general ability. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a method and device for joint parameter adjustment of a classification and question-answering large model, so as to solve the problem that the question classification accuracy of the existing large model is not high and the accuracy of answering professional questions is not accurate enough.

[0007] The technical solution adopted in the present invention is:

[0008] In a first aspect, the present invention provides a method for joint parameter adjustment of a classification and question-answering model, comprising:

[0009] Obtain the professional question-answer pairs and general question-answer datasets required for model training respectively, and mix the professional question-answer pairs and question-answer datasets in a preset ratio to construct a dataset ; The model includes a classification model and a question-answering model;

[0010] The classification model is used to determine the question category of the user's input question, and the question category and question text are combined into a prompt word and then input into the question-answering model;

[0011] Freeze the classification model parameters according to the dataset Calculate the first loss function of the question-answering model , adjust the parameters of the question-answering model;

[0012] After the question-answering model parameter adjustment is completed, the question-answering model parameters are frozen, and the classification model parameters are adjusted using the generated results of the question-answering model as supervision data. The classification score of the classification model is set, and the second loss function of the classification model is calculated based on the classification score. ;

[0013] Freeze the classification model parameters and question-answering model parameters respectively, and iteratively adjust the parameters of the question-answering model and classification model until the question-answering model and classification model reach the predetermined model standards.

[0014] Furthermore, the professional question-answer pairs and general question-answer datasets required for model training are obtained respectively, and the professional question-answer pairs and the question-answer datasets are mixed according to a preset ratio to construct a dataset. ,include:

[0015] Collect information related to professional questions and store it in the object database. Based on the information in the object database, use the big model to generate corresponding professional question and answer pairs.

[0016] The open source question-answering dataset is used as a general question-answering dataset, and the professional question-answering pairs and the question-answering dataset are mixed according to the preset ratio to construct a dataset. .

[0017] Furthermore, the method of generating corresponding professional question-answer pairs based on the information in the object database using the large model includes:

[0018] The data in the object database are divided into sections of 200 words each, which are used as reference materials for generating question-answer pairs. For paragraphs with less than 200 words, the paragraph is directly used as a reference material.

[0019] For each reference material, a large model is used to generate 2-5 question-answer pairs that are closely related to the content of the reference material.

[0020] Furthermore, the method of determining the question category of the question input by the user through the classification model, splicing the question category and the question text into prompt words and then inputting them into the question-answering model includes:

[0021] The BERT model is used as the classification model. After the user enters a question, the question text is input into the BERT model for question classification. A fully connected layer is used to transform the output of the BERT model to obtain a score for each question category.

[0022] The scores of question categories are converted into probability distributions through the softmax function to determine the probability that the question text belongs to each question category. , and the category with the highest score is considered as the predicted question category of the question text;

[0023] The question category and question text are concatenated as prompt words and input into the question-answering model to answer questions and obtain the probability of the question-answering model output. and use Weighted to get the final output probability .

[0024] Furthermore, the freezing of the classification model parameters is based on the data set Calculate the first loss function of the question-answering model , adjust the question answering model parameters, including:

[0025] Freeze the classification model parameters, and for each question in the question-answer pair, obtain the corresponding question classification probability and question category based on the classification model. Then, concatenate the question category and the question-answer pair as prompt words and input them into the question-answer model. Then, calculate the first loss function based on the actual answer in the question-answer pair and the output of the question-answer model. , through the first loss function Adjust the parameters of the question-answering model;

[0026] The loss function Using the logarithmic loss function, the goal is to minimize the loss of each token in the output sequence:

[0027] ;

[0028] in, is the probability of the final output of the question-answering model.

[0029] Furthermore, the parameters of the question-answering model are frozen, and the generation results of the question-answering model are used as supervision data to adjust the parameters of the classification model, and the classification score of the classification model is set, and the second loss function of the classification model is calculated based on the classification score. ,include:

[0030] After completing the parameter adjustment of the question-answering model, freeze the parameters of the question-answering model and use the generated results of the question-answering model as supervision data to adjust the parameters of the classification model;

[0031] Setting the classification score for the classification model for:

[0032] ;

[0033] in, Indicates the category of the problem; represents a set of problem categories; Represents the output probability of the question-answering model;

[0034] The second loss function is calculated by minimizing the KL divergence , through the second loss function Adjust the classification model parameters to make the problem classification probability and classification scores Close to the same; the second loss function for:

[0035] ;

[0036] in, represents the KL divergence, represents the probability of question classification, Represents the classification score of the classification model.

[0037] In a second aspect, the present invention provides a device for joint parameter adjustment of a classification and question-answering large model, comprising:

[0038] The dataset construction module is used to obtain the professional question-answer pairs and general question-answer datasets required for model training, and mix the professional question-answer pairs and question-answer datasets in a preset ratio to construct a dataset. ; The model includes a classification model and a question-answering model;

[0039] The knowledge question answering module is used to determine the question category of the user's input question through the classification model, and then combine the question category and question text into prompt words and input them into the question answering model;

[0040] The first parameter adjustment module is used to freeze the classification model parameters according to the data set Calculate the first loss function of the question-answering model , adjust the parameters of the question-answering model;

[0041] The second parameter adjustment module is used to freeze the question-answering model parameters after the question-answering model parameters are adjusted, adjust the classification model parameters using the generated results of the question-answering model as supervision data, set the classification score of the classification model, and calculate the second loss function of the classification model based on the classification score. ;

[0042] The joint adjustment module is used to freeze the classification model parameters and the question-answering model parameters in turn, and iteratively adjust the parameters of the question-answering model and the classification model until the question-answering model and the classification model meet the predetermined model standards.

[0043] In summary, the beneficial effects of the present invention are as follows:

[0044] A method for joint parameter adjustment of a classification and question-answering model provided by the present invention, the method respectively obtains professional question-answer pairs and general question-answering data sets, and mixes the two according to a preset ratio to construct a data set required for model parameter adjustment; the model includes a classification model and a question-answering model; the classification model is used to determine the question category of the input question, and the question category and question text are spliced ​​into prompt words and then input into the question-answering model for processing; the classification model parameters are frozen, and the question-answering model parameters are adjusted; the question-answering model parameters are frozen, and the classification model parameters are adjusted; the above parameter adjustment process is repeated until the predetermined model standard is reached. The present invention realizes the joint parameter adjustment of the classification and question-answering model, while improving the professional knowledge of the question-answering model, retains the general knowledge generalization ability of the model as much as possible, improves the accuracy, reliability and efficiency of the question response, and improves the question-answering effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work, and these are all within the scope of protection of the present invention.

[0046] Figure 1 This is a flow chart of a method for joint parameter adjustment of a classification and question-answering large model of the present invention;

[0047] Figure 2 This is a functional module diagram of a joint parameter adjustment device for a classification and question-answering large model of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. If there is no conflict, the various features of the present invention and the embodiments can be combined with each other and are all within the scope of protection of the present invention.

[0049] The overall concept of the joint parameter adjustment method for the classification and question-answering models proposed in this invention is as follows: professional data is collected through various channels, question-answer data is generated based on the collected data, and subsequent model parameter adjustment is supported; a question-answering system is constructed based on the classification model and the question-answering model; the classification model parameters are frozen and the question-answering model parameters are adjusted; the question-answering model parameters are frozen and the classification model parameters are adjusted; and this parameter adjustment process is repeated until the predetermined standard is met. The detailed implementation process of this invention is shown in the following examples.

[0050] Example 1: Reference Figure 1 As shown, Figure 1 This is a flow chart of a joint parameter adjustment method for a classification and question-answering model of the present invention. Figure 1 As shown, the joint parameter adjustment method of the classification and question-answering large model of the present invention includes:

[0051] Obtain the professional question-answer pairs and general question-answer datasets required for model training respectively, and mix the professional question-answer pairs and question-answer datasets in a preset ratio to construct a dataset ; The model includes a classification model and a question-answering model;

[0052] The classification model is used to determine the question category of the user's input question, and the question category and question text are combined into a prompt word and then input into the question-answering model;

[0053] Freeze the classification model parameters according to the dataset Calculate the first loss function of the question-answering model , adjust the parameters of the question-answering model;

[0054] After the question-answering model parameter adjustment is completed, the question-answering model parameters are frozen, and the classification model parameters are adjusted using the generated results of the question-answering model as supervision data. The classification score of the classification model is set, and the second loss function of the classification model is calculated based on the classification score. ;

[0055] Freeze the classification model parameters and question-answering model parameters respectively, and iteratively adjust the parameters of the question-answering model and classification model until the question-answering model and classification model reach the predetermined model standards.

[0056] Furthermore, in an embodiment of the present invention, the professional question-answer pairs and the general question-answer dataset required for model training are obtained respectively, and the professional question-answer pairs and the question-answer dataset are mixed according to a preset ratio to construct a dataset. ,include:

[0057] Collect information related to professional questions and store it in the object database. Based on the information in the object database, use the big model to generate corresponding professional question and answer pairs.

[0058] The open source question-answering dataset is used as a general question-answering dataset, and the professional question-answering pairs and the question-answering dataset are mixed according to the preset ratio to construct a dataset. .

[0059] Specifically, in this embodiment of the present invention, specialized materials such as books, reports, and reviews are collected from libraries, archives, and the internet, and stored in an object database to support subsequent training dataset generation and model parameter adjustment. Based on this collected specialized material, a large model is used to automatically generate high-quality specialized question-and-answer pairs, enhancing the model's expertise.

[0060] Then, we use general open source question-answering datasets such as OMGEval as question-answering datasets to ensure the general knowledge of the model. We mix professional knowledge (professional question-answer pairs) and general knowledge (question-answering datasets) in a ratio of 1:3 to construct the dataset. , which is used to adjust the parameters of subsequent classification and question-answering models, improving the model's understanding and answering capabilities of professional knowledge and ensuring the accuracy of generated responses. In addition, other ratios can be selected to construct datasets based on actual needs.

[0061] Among them, an object database is a database used to store and manage object data. Compared with traditional relational databases, object databases are more suitable for storing complex data structures and object-oriented data models, such as various common unstructured data (such as pictures, videos, log files, etc.).

[0062] Furthermore, in an embodiment of the present invention, based on the information in the object database, a large model is used to generate corresponding professional question-answer pairs, specifically including:

[0063] The data in the object database are divided into sections of 200 words each and used as reference materials for generating question-answer pairs. For paragraphs with less than 200 words, the paragraph is directly used as a reference material.

[0064] For each reference material, use large-scale models (such as Qwen and llama) to generate 2-5 question-answer pairs closely related to the reference material. The generated content should strictly comply with laws and regulations. The generated questions should cover key information in the reference material, ensuring diversity and coverage. The generated answers should accurately reflect the information in the reference material to avoid misleading or erroneous information.

[0065] Furthermore, in an embodiment of the present invention, professional knowledge question and answer includes two parts: a classification model and a question and answer model. The main process is: after the user inputs a question, the classification model is first used to determine the question category, and then the question category and the question are spliced ​​as a prompt and input into the question and answer model.

[0066] Specifically, the classification model determines the question category of the user's input question, concatenates the question category and question text into prompt words, and then inputs them into the question-answering model. This includes the following process:

[0067] In order to improve the model's ability to understand and generate professional knowledge, the question-answer data Adjust the parameters of the question-answering model. Use the BERT model as the classification model. After the user enters a question, the question text is fed into the BERT model for question classification. A fully connected layer is used to transform the BERT model's output to obtain a score for each question category.

[0068] The scores of question categories are converted into probability distributions through the softmax function to determine the probability that the question text belongs to each question category. , and the category with the highest score is considered as the predicted question category of the question text;

[0069] The question category and question text are concatenated as prompt words and input into the question-answering model to answer questions and obtain the probability of the question-answering model output. , and use Weighted to get the final output probability .

[0070] Question-answering models can utilize open-source models, including large open-source models such as qwen, llama, and baichuan. Question-answering models use natural language to interact, analyzing user questions and providing answers based on pre-learned knowledge. Question-answering models typically operate in several steps: question understanding, which converts user questions into a form that computers can understand and process, identifying key words, grammatical structures, and semantic information; and answer generation, which generates the final answer based on the question and pre-learned knowledge.

[0071] The probability of the final output of the question-answering model The probability of question classification is Weighted output probability:

[0072] .

[0073] The BERT model is a pre-trained model based on the Transformer architecture. It aims to address the limitations of traditional language models in understanding text semantics by introducing bidirectional contextual information. The BERT model's structure primarily consists of an embedding layer and the Transformer encoder. The embedding layer consists of token embeddings, segment embeddings, and position embeddings. The Transformer encoder is responsible for capturing contextual information in the input sequence.

[0074] Furthermore, in one embodiment of the present invention, in order to improve the model's ability to understand and generate professional knowledge, the question-answer data is used Adjust the parameters of the question-answering model. The answer is expressed as express, .

[0075] For each question, we first obtain the classification probability and result based on the classification model. Then, we concatenate the question category and the question-answer pair as prompt words and input them into the question-answering model. We calculate the loss function based on the actual answer and the output of the question-answering model, and adjust the model parameters.

[0076] In this process, the classification model The classification result is For each result , the data input to the question answering model is , indicating that the classification result is , the problem is , the answer is .

[0077] When adjusting the parameters of the question-answering model, the logarithmic loss function is used, with the goal of minimizing the loss of each token in the output sequence:

[0078] ;

[0079] in, Representing a dataset The pieces of data; Indicates the classification category, Indicates non-professional questions, Indicates professional issues.

[0080] Embodiments of the present invention adjust question-answering model parameters, taking both question classification probability and answer generation probability into account when adjusting model parameters. This allows the question-answering model to better generate answers based on question classification. Even the most advanced classifiers can make mistakes and return inaccurate classification results. By adjusting model parameters, the question-answering model can generate correct answers when given incorrect classification results, thereby ignoring misleading classification results and enhancing its ability to identify and answer questions.

[0081] Furthermore, in an embodiment of the present invention, after the question-answering model parameters are adjusted, the question-answering model parameters are frozen and the classification model parameters are adjusted. Considering that the classification results are closely related to the generation of the question-answering model, the generation results of the question-answering model are used as supervision data to adjust the classification model parameters.

[0082] Based on the above classification model parameter adjustment process, for data , the classification score of the classification model for:

[0083] ;

[0084] in, Indicates a category of problem; All categories in the set representing the problem category; represents a set of problem categories; Indicates the problem , when the problem is classified as When , the question answering model output is probability.

[0085] Category Rating The higher the value, the more effective the corresponding classification result is in improving the output accuracy of the question-answering model. The goal of adjusting the classification model parameters is to assign a higher score to the classification results that can better improve the accuracy of the question-answering model. , that is, the probability of problem classification and classification scores Be as consistent as possible.

[0086] The above goal is achieved by minimizing the KL divergence, and the loss function for adjusting the classification model parameters is:

[0087] ;

[0088] Furthermore, KL divergence is an asymmetric measure of the difference between two probability distributions, and is generally used to measure the "distance" between two probability distribution functions.

[0089] For two probability distributions P and Q, the KL divergence can be expressed as:

[0090] ;

[0091] in represents a random variable, and Respectively expressed in When P=Q, the KL divergence is 0, indicating that the two distributions are exactly the same; the larger the KL divergence, the greater the difference between the two distributions.

[0092] Finally, the embodiment of the present invention freezes the classification model and the question-answering large model respectively, and iteratively adjusts the parameters of the question-answering large model and the classification model until both the question-answering large model and the classification model meet the predetermined model standards.

[0093] Furthermore, the predetermined model standards include three aspects:

[0094] (1) Stable loss function value: During the parameter adjustment process, the loss function value decreases as the number of iterations increases. When the loss function value in the dataset tends to be stable or the decrease rate becomes smaller, it can be considered that the model has approached convergence.

[0095] (2) Use appropriate evaluation metrics (such as accuracy for classification models) to measure model performance. When the evaluation metrics of the model on the validation set reach the expected level or tend to be stable, the model parameters can be considered to have been adjusted.

[0096] (3) Iterate to a certain number of steps: A certain number of iteration steps can be set. When the iteration reaches a certain number of steps, the parameter adjustment is considered completed and the parameter adjustment process is terminated.

[0097] The method of the embodiment of the present invention has the following technical effects:

[0098] 1. Use the classification model to classify input questions into professional and non-professional questions, thereby improving the judgment and response capabilities of the question-answering model and enhancing the professionalism of the answers;

[0099] Realize joint parameter adjustment of classification and question-answering models, while improving the professional knowledge of question-answering models, retain the generalization ability of the models as much as possible, improve the accuracy, reliability and efficiency of question responses, and improve the question-answering effect.

[0100] Example 2: Reference Figure 2 As shown, an embodiment of the present invention further provides a device for joint parameter adjustment of a classification and question-answering large model, which includes:

[0101] The dataset construction module is used to obtain the professional question-answer pairs and general question-answer datasets required for model training, and mix the professional question-answer pairs and question-answer datasets in a preset ratio to construct a dataset. ; The model includes a classification model and a question-answering model;

[0102] The knowledge question answering module is used to determine the question category of the user's input question through the classification model, and then combine the question category and question text into prompt words and input them into the question answering model;

[0103] The first parameter adjustment module is used to freeze the classification model parameters according to the data set Calculate the first loss function of the question-answering model , adjust the parameters of the question-answering model;

[0104] The second parameter adjustment module is used to freeze the question-answering model parameters after the question-answering model parameters are adjusted, adjust the classification model parameters using the generated results of the question-answering model as supervision data, set the classification score of the classification model, and calculate the second loss function of the classification model based on the classification score. ;

[0105] The joint adjustment module is used to freeze the classification model parameters and the question-answering model parameters in turn, and iteratively adjust the parameters of the question-answering model and the classification model until the question-answering model and the classification model meet the predetermined model standards.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A joint parameter adjustment method for classification and question-answering large models, characterized by: include: Obtain the professional question-answer pairs and general question-answer datasets required for model training respectively, and mix the professional question-answer pairs and question-answer datasets in a preset ratio to construct a dataset ; The model includes a classification model and a question-answering model; The classification model is used to determine the question category of the user's input question, and the question category and question text are combined into a prompt word and then input into the question-answering model; Freeze the classification model parameters according to the dataset Calculate the first loss function of the question-answering model , adjust the parameters of the question-answering model, including: freezing the classification model parameters, obtaining the corresponding question classification probability and question category for each question in the question-answer pair based on the classification model, then concatenating the question category and the question-answer pair as prompt words and inputting them into the question-answering model, and then calculating the first loss function based on the actual answer in the question-answer pair and the output of the question-answering model , through the first loss function Adjust the parameters of the question-answering model; the first loss function Using the logarithmic loss function, the goal is to minimize the loss of each token in the output sequence: ； in, is the probability of the final output of the question-answering model; After the question-answering model parameter adjustment is completed, the question-answering model parameters are frozen, and the classification model parameters are adjusted using the generated results of the question-answering model as supervision data. The classification score of the classification model is set, and the second loss function of the classification model is calculated based on the classification score. ,include: After completing the parameter adjustment of the question-answering model, freeze the parameters of the question-answering model and use the generated results of the question-answering model as supervision data to adjust the parameters of the classification model; Setting the classification score for the classification model for: ； in, Indicates the category of the problem; represents a set of problem categories; Represents the probability of the final output of the question-answering model; Indicates that the problem category is The final output probability of the question answering model under the condition of; The second loss function is calculated by minimizing the KL divergence , through the second loss function Adjust the classification model parameters to make the problem classification probability and classification scores Close to the same; the second loss function for: ； in, represents the KL divergence, represents the probability of question classification, represents the classification score of the classification model; Freeze the classification model parameters and question-answering model parameters respectively, and iteratively adjust the parameters of the question-answering model and classification model until the question-answering model and classification model reach the predetermined model standards.

2. The joint parameter adjustment method of the classification and question-answering model according to claim 1 is characterized in that: The professional question-answer pairs and general question-answer datasets required for model training are obtained separately, and the professional question-answer pairs and the question-answer datasets are mixed according to a preset ratio to construct a dataset. ,include: Collect information related to professional issues and store it in the object database. Based on the information in the object database, use the big model to generate corresponding professional question and answer pairs. The open source question-answering dataset is used as a general question-answering dataset, and the professional question-answering pairs and the question-answering dataset are mixed according to the preset ratio to construct a dataset. .

3. The joint parameter adjustment method of the classification and question-answering model according to claim 2 is characterized in that: The method of generating corresponding professional question-answer pairs based on the information in the object database using the large model includes: The data in the object database are divided into sections of 200 words each, which are used as reference materials for generating question-answer pairs. For paragraphs with less than 200 words, the paragraph is directly used as a reference material. For each reference material, a large model is used to generate 2-5 question-answer pairs that are closely related to the content of the reference material.

4. The joint parameter adjustment method for the classification and question-answering model according to claim 1 is characterized in that: The method of determining the question category of the question input by the user through the classification model, combining the question category and the question text into a prompt word and then inputting it into the question-answering model includes: The BERT model is used as the classification model. After the user enters a question, the question text is input into the BERT model for question classification. A fully connected layer is used to transform the output of the BERT model to obtain a score for each question category. The scores of question categories are converted into probability distributions through the softmax function to determine the probability that the question text belongs to each question category. , and the category with the highest score is considered as the predicted question category of the question text; The question category and question text are concatenated as prompt words and input into the question-answering model to answer questions and obtain the probability of the question-answering model output. and use Weighted to get the final output probability .

5. A joint parameter adjustment device for a classification and question-answering large model, characterized in that: include: The dataset construction module is used to obtain the professional question-answer pairs and general question-answer datasets required for model training, and mix the professional question-answer pairs and question-answer datasets in a preset ratio to construct a dataset. ; The model includes a classification model and a question-answering model; The knowledge question answering module is used to determine the question category of the user's input question through the classification model, and then combine the question category and question text into prompt words and input them into the question answering model; The first parameter adjustment module is used to freeze the classification model parameters according to the data set Calculate the first loss function of the question-answering model , adjust the parameters of the question-answering model, including: freezing the classification model parameters, obtaining the corresponding question classification probability and question category for each question in the question-answer pair based on the classification model, then concatenating the question category and the question-answer pair as prompt words and inputting them into the question-answering model, and then calculating the first loss function based on the actual answer in the question-answer pair and the output of the question-answering model , through the first loss function Adjust the parameters of the question-answering model; the first loss function Using the logarithmic loss function, the goal is to minimize the loss of each token in the output sequence: ； in, is the probability of the final output of the question-answering model; The second parameter adjustment module is used to freeze the question-answering model parameters after the question-answering model parameters are adjusted, adjust the classification model parameters using the generated results of the question-answering model as supervision data, set the classification score of the classification model, and calculate the second loss function of the classification model based on the classification score. ,include: After completing the parameter adjustment of the question-answering model, freeze the parameters of the question-answering model and use the generated results of the question-answering model as supervision data to adjust the parameters of the classification model; Setting the classification score for the classification model for: ； in, Indicates the category of the problem; represents a set of problem categories; Represents the probability of the final output of the question-answering model; Indicates that the problem category is The final output probability of the question answering model under the condition of; The second loss function is calculated by minimizing the KL divergence , through the second loss function Adjust the classification model parameters to make the problem classification probability and classification scores Close to the same; the second loss function for: ； in, represents the KL divergence, represents the probability of question classification, represents the classification score of the classification model; The joint adjustment module is used to freeze the classification model parameters and the question-answering model parameters in turn, and iteratively adjust the parameters of the question-answering model and the classification model until the question-answering model and the classification model meet the predetermined model standards.

Images