Model training method and device based on large model, question and answer method and device based on large model and electronic equipment

Abstract

The invention provides a model training method and device based on a large model, a question and answer method and device and electronic equipment, and relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing, deep learning, reinforcement learning and large models. The method comprises the following steps: taking a first large model as a teacher model of a language model, and carrying out distillation learning on the language model based on the first large model; inputting a first prompt text into the language model, and outputting a plurality of first response texts of the first prompt text by the language model; determining a reference response text of the first prompt text from the plurality of first response texts; and training the language model based on the reference response text of the first prompt text.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of natural language processing, deep learning, reinforcement learning, and large model technology, and in particular to a model training method, question-answering method, device, electronic device, storage medium, and computer program product based on a large model. Background Art

[0002] At present, language models have the advantages of high accuracy and high processing efficiency, and have been widely used in the fields of dialogue, question answering, translation, etc. However, in order to train language models, most of the related technologies need to rely on large models and manual labeling of a large number of training samples. The labeling time of language model training samples is relatively long, resulting in low training efficiency of language models. Summary of the invention

[0003] The present disclosure proposes a model training method, question-answering method, device, electronic device, storage medium and computer program product based on a large model.

[0004] According to a first aspect of the present disclosure, a large-model-based model training method is proposed, comprising: using a first large model as a teacher model of a language model, and performing distillation learning on the language model based on the first large model; inputting a first prompt text into the language model, and having the language model output a plurality of first response texts of the first prompt text; determining a reference response text of the first prompt text from the plurality of first response texts; and training the language model based on the reference response text of the first prompt text.

[0005] According to the second aspect of the present disclosure, a question-answering method is proposed, comprising: obtaining a question text; obtaining a target prompt text based on the question text; inputting the target prompt text into a question-answering model, and having the question-answering model output an answer text of the question text, wherein the question-answering model is obtained by adopting the large-model-based model training method proposed in the first aspect above.

[0006] According to a third aspect of the present disclosure, a model training device based on a large model is proposed, including: a first training module, used to use the first large model as a teacher model of a language model, and perform distillation learning on the language model based on the first large model; a processing module, used to input a first prompt text into the language model, and the language model outputs multiple first response texts of the first prompt text; a determination module, used to determine a reference response text of the first prompt text from multiple first response texts; a second training module, used to train the language model based on the reference response text of the first prompt text.

[0007] According to a fourth aspect of the present disclosure, a question-and-answer device is proposed, comprising: a first acquisition module for acquiring a question text; a second acquisition module for obtaining a target prompt text based on the question text; a processing module for inputting the target prompt text into a question-and-answer model, and the question-and-answer model outputs an answer text of the question text, wherein the question-and-answer model is obtained by adopting the large-model-based model training method proposed in the first aspect above.

[0008] According to the fifth aspect of the present disclosure, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the large model-based model training method proposed in the first aspect above, or execute the question-answering method proposed in the second aspect above.

[0009] According to the sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is proposed, wherein the computer instructions are used to enable the computer to execute the large model-based model training method proposed in the first aspect, or to execute the question-answering method proposed in the second aspect.

[0010] According to the seventh aspect of the present disclosure, a computer program product is proposed, including a computer program, which, when executed by a processor, implements the large model-based model training method proposed in the first aspect, or implements the question-answering method proposed in the second aspect.

[0011] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0013] Figure 1 A flowchart of a model training method based on a large model according to an embodiment of the present disclosure;

[0014] Figure 2 A flowchart of a model training method based on a large model according to another embodiment of the present disclosure;

[0015] Figure 3 A flowchart of a model training method based on a large model according to another embodiment of the present disclosure;

[0016] Figure 4A flowchart of a model training method based on a large model according to another embodiment of the present disclosure;

[0017] Figure 5 A flowchart of a training method for a reward model according to an embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of a model training method based on a large model according to an embodiment of the present disclosure;

[0019] Figure 7 A flowchart of a question-and-answer method according to an embodiment of the present disclosure;

[0020] Figure 8 It is a structural schematic diagram of a model training device based on a large model according to an embodiment of the present disclosure;

[0021] Fig. 9 It is a structural schematic diagram of a question-answering device according to an embodiment of the present disclosure;

[0022] Fig.10 A schematic block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0024] AI (Artificial Intelligence) is a technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. At present, AI technology has the advantages of high automation, high precision and low cost, and has been widely used.

[0025] NLU (Natural Language Processing) is a science that studies computer systems that can effectively implement natural language communication, especially the software systems therein. It is an important direction in the fields of computer science and artificial intelligence.

[0026] DL (Deep Learning) is a new research direction in the field of ML (Machine Learning). It is a science that learns the inherent laws and representation levels of sample data, enabling machines to have analytical learning capabilities like humans and to recognize data such as text, images, and sounds. It is widely used in speech and image recognition.

[0027] RL (Reinforcement Learning), also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent can maximize its rewards or achieve specific goals through learning strategies during its interaction with the environment.

[0028] A large model refers to a machine learning model with a large parameter scale and complexity. It requires a lot of computing resources and storage space to train and store, and often requires distributed computing and special hardware acceleration technology. Large models have stronger generalization and expression capabilities. Large models include LLM (Large Language Model). A large language model refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question and answer, dialogue, etc., and are an important path to artificial intelligence.

[0029] Figure 1 FIG. 1 is a flow chart of a model training method based on a large model according to an embodiment of the present disclosure. Figure 1 As shown, the method includes:

[0030] S101, using the first large model as the teacher model of the language model, and performing distillation learning on the language model based on the first large model.

[0031] It should be noted that the execution subject of the model training method based on the large model of the embodiment of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a workstation, a server, a computer, a user terminal and other intelligent devices. Among them, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a car terminal, etc.

[0032] It should be noted that the language model in the embodiments of the present disclosure may be any language model in the relevant technology, without too much limitation here, for example, it may include a dialogue model, a question-and-answer model, a translation model, a text classification model, etc.

[0033] It should be noted that the first large model can be implemented by any large model in the relevant technology, and no excessive restrictions are made here. For example, it can be a Transformer model, a large language model, etc. It should be noted that the Transformer model is a neural network model based on the self-attention mechanism. Based on the first large model, the language model can be distilled and learned by any knowledge distillation method in the relevant technology, and no excessive restrictions are made here.

[0034] S102: input a first prompt text into a language model, and the language model outputs a plurality of first response texts of the first prompt text.

[0035] For example, the first prompt text is generated based on the input text of the client.

[0036] For example, when the language model is a dialogue model, the first prompt text may carry the first dialogue text, and the first response text may include the second dialogue text for the first dialogue text. For example, the first dialogue text is "What day is today?", and the first prompt text "You are a chat robot, please have a dialogue with the user, user: What day is today?" can be input into the dialogue model, and the dialogue model outputs the first response text such as "Today is Tuesday," "Tuesday," and "Tuesday."

[0037] For example, when the language model is a question-answering model, the first prompt text may carry the question text, and the first response text may include the answer text for the question text. For example, if the question text is "What is a big model?", the first prompt text "You are a question-answering robot, please answer the user's question, the user's question: What is a big model?" can be input into the question-answering model, and the question-answering model outputs the first response text such as "A big model refers to a machine learning model with a large parameter scale and complexity", "A big model has stronger generalization and expression capabilities", and "A big model includes a big language model".

[0038] For example, when the language model is a translation model, the first prompt text may carry the text to be translated, and the first response text may include the text after the text to be translated is translated.

[0039] For example, when the language model is a text classification model, the first prompt text may carry the text to be classified, and the first response text may include the category of the text to be classified, and / or the probability of the text to be classified in each category.

[0040] It should be noted that there is no excessive limitation on the number of first response texts to a first prompt text, for example, it can be 5.

[0041] S103: Determine a reference response text of the first prompt text from a plurality of first response texts.

[0042] It should be noted that the multiple first response texts include reference response texts, the number of reference response texts for a first prompt text is at least one, and the reference response texts of different first prompt texts may be different.

[0043] It is understandable that the first response text generated by the language model is not necessarily correct and may contain content errors, grammatical errors, spelling errors, semantic errors, poor readability, etc. In the training scenario after the language model has undergone multiple rounds of iterations, the accuracy of multiple first response texts generated by the language model is high, so the reference response text of the determined first prompt text is closer to the manual annotation result and has higher accuracy.

[0044] In one embodiment, determining a reference response text of a first prompt text from a plurality of first response texts includes obtaining a correct probability of the first response text, and determining a reference response text of the first prompt text from a plurality of first response texts based on the correct probability. Thus, the correct probability can be taken into consideration to determine a reference response text of the first prompt text from a plurality of first response texts, which helps to obtain a reference response text of the first prompt text with a higher correct probability, thereby improving the accuracy of the reference response text of the first prompt text.

[0045] In some examples, obtaining the correct probability of the first response text includes inputting the first response text into a correct probability prediction model, and the correct probability prediction model outputs the correct probability of the first response text. It should be noted that there are no excessive restrictions on the correct probability prediction model, for example, it may include a deep learning model, a mechanism model, etc.

[0046] In some examples, based on the correctness probability, a reference response text for the first prompt text is determined from multiple first response texts, including using the first response text corresponding to the greatest correctness probability as the reference response text for the first prompt text.

[0047] In some examples, based on the probability of correctness, a reference response text for the first prompt text is determined from multiple first response texts, including using the first response text corresponding to the probability of correctness greater than a set threshold as the reference response text for the first prompt text, and / or sorting the multiple first response texts in descending order according to the probability of correctness, and using the first N first response texts in the sort as the reference response text for the first prompt text, where N is a positive integer.

[0048] In one embodiment, determining a reference response text of a first prompt text from a plurality of first response texts includes performing problem detection on the first response text to obtain a problem detection result of the first response text, and determining a reference response text of the first prompt text from a plurality of first response texts based on the problem detection result. Thus, the problem detection result can be taken into consideration to determine a reference response text of the first prompt text from a plurality of first response texts, which helps to obtain a reference response text of the first prompt text without a problem, thereby improving the accuracy of the reference response text of the first prompt text.

[0049] It should be noted that the problem detection for the first response text can be implemented by using any text problem detection method in the relevant technology, and no excessive restrictions are made here.

[0050] In some examples, problem detection is performed on the first response text to obtain a problem detection result of the first response text, including inputting the first response text into a problem detection model, and the problem detection model outputs the problem detection result of the first response text.

[0051] In some examples, problem detection is performed on the first response text to obtain a problem detection result for the first response text, including identifying whether the first response text complies with rules in a set rule base, and in response to the first response text not complying with at least one rule in the set rule base, generating a problem detection result indicating that there is a problem with the first response text.

[0052] In some examples, based on the problem detection result, a reference response text for the first prompt text is determined from multiple first response texts, including using the first response text for which the problem detection result indicates that there is no problem as the reference response text for the first prompt text.

[0053] S104: Training a language model based on a reference response text of the first prompt text.

[0054] It should be noted that training the language model based on the reference response text of the first prompt text means training the language model using the reference response text of the first prompt text as a label. Training the language model based on the reference response text of the first prompt text can be implemented by any model training method in the relevant technology, and no further limitation is made here.

[0055] In one embodiment, training a language model based on a reference response text of a first prompt text includes inputting the first prompt text into the language model, outputting a fourth response text of the first prompt text by the language model, and training the language model based on the fourth response text and the reference response text of the first prompt text. Thus, the first prompt text can be re-input into the language model, the language model continues to generate the fourth response text of the first prompt text, and the language model is trained in consideration of the fourth response text and the reference response text of the first prompt text.

[0056] In some examples, the language model is trained based on the fourth response text and the reference response text of the first prompt text, including obtaining a loss function of the language model based on the fourth response text and the reference response text of the first prompt text, and training the language model based on the loss function of the language model. It should be noted that there are no excessive restrictions on the loss function, for example, it can include CE (Cross Entropy), MSE (Mean-Square Error), KL (Kullback-Leibler) divergence, contrast loss function, etc.

[0057] For example, taking the language model as a dialogue model, the first prompt text 1 "You are a chat robot, please have a dialogue with the user, user: What day is today" can be input into the dialogue model, and the dialogue model outputs first response texts such as "Today is Tuesday", "Tuesday", and "Tuesday", and determines the reference response text of the first prompt text 1 from the above multiple first response texts. For example, the reference response text of the first prompt text 1 is "Today is Tuesday", and the dialogue model is trained based on the reference response text of the first prompt text 1.

[0058] For example, continuing to take the language model as the dialogue model, the first prompt text 2 "You are a chat robot, please have a dialogue with the user, user: How is the air quality today?" can be input into the dialogue model, and the dialogue model outputs first response texts such as "Today's air quality is excellent", "Excellent", "Today's air quality is excellent, the air is very good, you can move normally, go breathe fresh air", and determine the reference response text of the first prompt text 2 from the above multiple first response texts. For example, the reference response text of the first prompt text 2 is "Today's air quality is excellent, the air is very good, you can move normally, go breathe fresh air", and train the dialogue model based on the reference response text of the first prompt text 2.

[0059] For example, if the language model is a question-and-answer model, the first prompt text 3 "You are a question-and-answer robot, please answer the user's question. The user's question is: What kind of model is a big model?" can be input into the question-and-answer model, and the question-and-answer model can output first response texts such as "A big model refers to a machine learning model with a large parameter scale and complexity", "A big model has stronger generalization and expression capabilities", and "A big model includes a large language model". The reference response text of the first prompt text 3 is determined from the above multiple first response texts. For example, the reference response text of the first prompt text 3 is "A big model refers to a machine learning model with a large parameter scale and complexity". The question-and-answer model is trained based on the reference response text of the first prompt text 3.

[0060] It should be noted that the training content of other types of language models can refer to the training content of dialogue models and question-answering models, and will not be repeated here.

[0061] The large-model-based model training method proposed in the present invention uses the first large model as the teacher model of the language model, and performs distillation learning on the language model based on the first large model. The first prompt text is input into the language model, and the language model outputs multiple first response texts of the first prompt text, determines the reference response text of the first prompt text from the multiple first response texts, and trains the language model based on the reference response text of the first prompt text. Therefore, the first large model can be used as the teacher model of the language model to perform distillation learning on the language model, that is, the knowledge of the first large model can be transferred to the language model. The language model has the advantages of small size, high precision and fast reasoning speed. In addition, the reference response text of the first prompt text can be determined from the multiple first response texts generated by the language model to train the language model, which can realize the automatic acquisition of the reference response text of the language model and the self-evolution of the language model. There is no need to rely on a large model or manual labeling, which can save the labeling time of the training samples of the language model and improve the training efficiency of the language model. It is especially suitable for training scenarios after multiple rounds of iterations of the language model. In addition, in this solution, the first prompt text and the reference response text of the first prompt text can be obtained in the real business scenario of the language model, so that the training samples of the language model are closer to the real business scenario, which improves the authenticity and diversity of the training samples of the language model, and helps to improve the generalization and applicability of the language model in real business scenarios.

[0062] In the above embodiment, in step S103, the reference response text for determining the first prompt text from the plurality of first response texts may be combined with Figure 2 Further understanding, Figure 2 FIG. 1 is a flow chart of a model training method based on a large model according to another embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0063] S201, using the first large model as the teacher model of the language model, and performing distillation learning on the language model based on the first large model.

[0064] S202: Input a first prompt text into a language model, and the language model outputs a plurality of first response texts of the first prompt text.

[0065] The relevant contents of steps S201 - S202 can be found in the above embodiment and will not be described again here.

[0066] S203: Obtain a score for the first response text.

[0067] It is understandable that different first response texts may be scored differently.

[0068] In one implementation, obtaining the score of the first response text includes inputting the first response text into a text scoring model, and having the text scoring model output the score of the first response text.

[0069] In one embodiment, obtaining the score of the first response text includes inputting the first prompt text and the first response text into a reward model, outputting the reward of the first response text by the reward model, and using the reward of the first response text as the score of the first response text. Thus, the reward model can be used to obtain the reward of the first response text to obtain the score of the first response text, that is, the reward model can be constructed based on the reinforcement learning technology to achieve the acquisition of the score of the first response text, thereby improving the accuracy of the score of the first response text.

[0070] It should be noted that there are not too many restrictions on the reward model, for example, it may include deep learning models, mechanism models, etc.

[0071] In one embodiment, obtaining the score of the first response text includes inputting the first prompt text and the first response text into the second largest model, and the second largest model outputting the score of the first response text. Thus, the score of the first response text can be obtained using the second largest model, that is, the score of the first response text can be obtained based on the deep learning technology, which improves the accuracy of the score of the first response text. The second largest model has good generalization and is suitable for scoring scenarios of multi-category first response texts.

[0072] It should be noted that the relevant contents of the second largest model can refer to the relevant contents of the first largest model, and will not be repeated here. For example, the second largest model is consistent with the first largest model.

[0073] In some examples, the method further includes obtaining a second response text to the first prompt text based on the first prompt text through the second large model, and obtaining a score for the first response text based on the first response text and the second response text through the second large model. Thus, the second response text to the first prompt text can be generated through the second large model, and the score for the first response text can be obtained by comprehensively considering the first response text and the second response text.

[0074] For example, the score of the first response text is obtained based on the first response text and the second response text through the second largest model, including obtaining the similarity between the first response text and the second response text through the second largest model as the score of the first response text.

[0075] For example, the score of the first response text is obtained based on the first response text and the second response text through the second largest model, including performing feature extraction on the first response text through the second largest model to obtain first text features, and performing feature extraction on the second response text to obtain second text features, and obtaining the similarity between the first text features and the second text features through the second largest model as the score of the first response text.

[0076] S204: Determine a reference response text for the first prompt text from the plurality of first response texts based on the score.

[0077] In one embodiment, based on the score, a reference response text for the first prompt text is determined from multiple first response texts, including taking the first response text with a score greater than a set threshold as the reference response text for the first prompt text, and / or sorting the multiple first response texts in descending order according to the score, and taking the first N first response texts in the sort as the reference response text for the first prompt text, where N is a positive integer.

[0078] S205: Training a language model based on a reference response text of the first prompt text.

[0079] The relevant contents of step S205 can be found in the above embodiment and will not be described again here.

[0080] The model training method based on the large model proposed in the present disclosure obtains the score of the first response text, and based on the score, determines the reference response text of the first prompt text from multiple first response texts. Therefore, the reference response text of the first prompt text can be determined from multiple first response texts taking into account the score, which helps to obtain the reference response text of the first prompt text with a higher score, thereby improving the accuracy of the reference response text of the first prompt text.

[0081] In the above embodiment, regarding the reference response text based on the first prompt text in step S104, the language model is trained, which can be combined with Figure 3 Further understanding, Figure 3 FIG. 1 is a flow chart of a model training method based on a large model according to another embodiment of the present disclosure. Figure 3 As shown, the method includes:

[0082] S301, using the first largest model as a teacher model of the language model, and performing distillation learning on the language model based on the first largest model.

[0083] S302: input a first prompt text into a language model, and the language model outputs a plurality of first response texts of the first prompt text.

[0084] S303: Determine a reference response text of the first prompt text from multiple first response texts.

[0085] The relevant contents of steps S301 - S303 can be found in the above embodiment and will not be described again here.

[0086] S304: Associating the first prompt text with a reference response text of the first prompt text to obtain a first training sample of a language model.

[0087] It should be noted that the association between the first prompt text and the reference response text of the first prompt text can be achieved by using any data association method in the relevant technology, and no further limitation is given here. For example, a mapping relationship, a corresponding relationship, etc. can be established between the first prompt text and the reference response text of the first prompt text. For example, the reference response text of the first prompt text can be used as a label to associate the first prompt text with the reference response text of the first prompt text to obtain the first training sample of the language model.

[0088] For instance, continuing with the first prompt text 1 in the above embodiment as an example, the first prompt text 1 "You are a chat robot, please have a conversation with the user, user: What day is today?" and the reference response text of the first prompt text 1 "Today is Tuesday" can be associated to obtain the first training sample A of the dialogue model.

[0089] For instance, continuing with the first prompt text 2 in the above embodiment as an example, the first prompt text 2 "You are a chat robot, please have a conversation with the user, user: How is the air quality today?" and the reference response text of the first prompt text 2 "Today's air quality is excellent, the air is very good, you can move normally, go breathe some fresh air" can be associated to obtain the first training sample B of the dialogue model.

[0090] For instance, continuing to take the first prompt text 3 in the above embodiment as an example, the first prompt text 3 "You are a question-and-answer robot, please answer the user's question, the user's question: What kind of model is the big model" and the reference response text of the first prompt text 3 "The big model refers to a machine learning model with a huge parameter scale and complexity" can be associated to obtain the first training sample C of the question-and-answer model.

[0091] S305: Train the language model based on the first training sample.

[0092] It should be noted that, based on the first training sample, the language model can be trained by any model training method in the related art, and no further limitation is made here. For example, the relevant content of step S305 can refer to the relevant content of step S104, and no further description is given here.

[0093] For example, the dialogue model may be trained based on the first training samples A and B of the dialogue model.

[0094] For example, the question-answering model may be trained based on the first training sample C of the question-answering model.

[0095] The model training method based on the large model proposed in the present disclosure associates the first prompt text and the reference response text of the first prompt text to obtain the first training sample of the language model, and trains the language model based on the first training sample. Thus, the first prompt text and its reference response text can be associated to obtain the first training sample to train the language model. During the training process, the language model can learn the relationship between the first prompt text and its reference response text, so that the trained language model can generate the response text of the prompt text based on the prompt text.

[0096] In the above embodiment, regarding the reference response text based on the first prompt text in step S104, the language model is trained, which can be combined with Figure 4 Further understanding, Figure 4 FIG. 1 is a flow chart of a model training method based on a large model according to another embodiment of the present disclosure. Figure 4 As shown, the method includes:

[0097] S401, using the first large model as a teacher model of the language model, and performing distillation learning on the language model based on the first large model.

[0098] S402: Input a first prompt text into a language model, and the language model outputs a plurality of first response texts of the first prompt text.

[0099] S403: Determine a reference response text of the first prompt text from a plurality of first response texts.

[0100] The relevant contents of steps S401 - S403 can be found in the above embodiment and will not be described again here.

[0101] S404: Use the remaining first response text except the reference response text of the first prompt text as the third response text.

[0102] It should be noted that the plurality of first response texts include the third response text, and the number of the third response texts of a first prompt text is at least one. For example, the number of the third response texts of the first prompt text is M-1, where M is the number of the first response texts of the first prompt text, and M is a positive integer greater than 1.

[0103] For instance, continuing to take the first prompt text 1 in the above embodiment as an example, the first prompt text 1 is "You are a chat robot, please have a conversation with the user, user: what day is today", and the multiple first response texts of the first prompt text 1 include "Today is Tuesday", "Tuesday", "Tuesday", etc. The reference response text of the first prompt text 1 is "Today is Tuesday", and "Tuesday" and "Tuesday" can be used as the third response text of the first prompt text 1.

[0104] For example, continuing to take the first prompt text 2 in the above embodiment as an example, the first prompt text 2 is "You are a chat robot, please have a conversation with the user, user: How is the air quality today?", and the multiple first response texts of the first prompt text 2 include "Today's air quality is excellent", "Excellent", "Today's air quality is excellent, the air is very good, you can move normally, go breathe fresh air", etc., and the reference response text of the first prompt text 2 is "Today's air quality is excellent, the air is very good, you can move normally, go breathe fresh air", and "Today's air quality is excellent" and "Excellent" can be used as the third response text of the first prompt text 2.

[0105] For instance, continuing to take the first prompt text 3 in the above embodiment as an example, the first prompt text 3 is "You are a question-and-answer robot, please answer the user's question. The user's question is: What kind of model is the big model?" The multiple first response texts of the first prompt text 3 include "A big model refers to a machine learning model with a large parameter scale and complexity", "A big model has stronger generalization and expression capabilities", "A big model includes a large language model", etc. The reference response text of the first prompt text 3 is "A big model refers to a machine learning model with a large parameter scale and complexity". "A big model has stronger generalization and expression capabilities" and "A big model includes a large language model" can be used as the third response text of the first prompt text 3.

[0106] S405: Training a language model based on the third response text and the reference response text of the first prompt text.

[0107] It should be noted that the language model is trained based on the third response text and the reference response text of the first prompt text, and any model training method in the relevant technology can be used to achieve it, and no further limitation is made here. For example, the relevant content of step S405 can refer to the relevant content of step S104, and no further description is given here.

[0108] In one embodiment, a language model is trained based on a third response text and a reference response text of the first prompt text, including obtaining a loss function of the language model based on the third response text and the reference response text of the first prompt text, and training the language model based on the loss function of the language model.

[0109] For instance, continuing with the first prompt text 1 in the above embodiment as an example, the dialogue model can be trained based on the third response text "Tuesday", "Tuesday" of the first prompt text 1, and the reference response text "Today is Tuesday" of the first prompt text 1.

[0110] For instance, continuing with the first prompt text 2 in the above embodiment as an example, the dialogue model can be trained based on the third response text of the first prompt text 2 "Today's air quality is excellent", "Excellent", and the reference response text of the first prompt text 2 "Today's air quality is excellent, the air is very good, you can carry out normal activities, go and breathe fresh air".

[0111] For instance, continuing with the first prompt text 3 in the above embodiment as an example, the question-answering model can be trained based on the third response text of the first prompt text 3 "The large model has stronger generalization and expression capabilities", "The large model includes a large language model", and the reference response text of the first prompt text 3 "The large model refers to a machine learning model with a large parameter scale and complexity".

[0112] The model training method based on the large model proposed in the present disclosure uses the remaining first response texts other than the reference response text of the first prompt text as the third response text, and trains the language model based on the third response text and the reference response text of the first prompt text. Thus, the language model can be trained directly based on the remaining first response texts other than the reference response text of the first prompt text and the reference response text of the first prompt text, without the need to continue to obtain the remaining response texts other than the first response text to train the language model, which can save the time for obtaining training samples of the language model and improve the training efficiency of the language model.

[0113] Based on any of the above embodiments, the language model is distilled and learned based on the first large model, including inputting the second prompt text into the first large model, outputting the reference response text of the second prompt text by the first large model, associating the second prompt text with the reference response text of the second prompt text, obtaining the second training sample of the language model, and training the language model based on the second training sample. Thus, the reference response text of the second prompt text can be obtained by using the first large model, and the second prompt text and its reference response text can be associated to obtain the second training sample to train the language model. During the training process, the language model can learn the relationship between the second prompt text and its reference response text, so that the trained language model can generate the response text of the prompt text based on the prompt text.

[0114] It should be noted that, the relevant content of the second prompt text can refer to the relevant content of the first prompt text, and the relevant content of the reference response text of the second prompt text can refer to the relevant content of the reference response text of the first prompt text. The second prompt text and the reference response text of the second prompt text are associated to obtain the relevant content of the second training sample of the language model, which can refer to the relevant content of step S304. The relevant content of training the language model based on the second training sample can refer to the relevant content of step S305, which will not be repeated here.

[0115] In the above embodiment, the training content of the reward model can be combined with Figure 5 Further understanding, Figure 5 FIG. 1 is a flow chart of a method for training a reward model according to an embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0116] S501, obtaining a sample prompt text, and a positive response text and a negative response text of the sample prompt text.

[0117] It should be noted that the accuracy of the positive response text is higher than that of the negative response text, that is, the positive response text refers to the sample response text with higher accuracy of the sample prompt text, and the negative response text refers to the sample response text with lower accuracy of the sample prompt text. For example, the positive response text refers to the correct sample response text of the sample prompt text, and the negative response text refers to the incorrect sample response text of the sample prompt text.

[0118] In one embodiment, the method further includes obtaining a reference response text of the sample prompt text sent by the client of the annotated user as the positive response text. Thus, the positive response text can be obtained by relying on manual annotation.

[0119] In one embodiment, the method further includes obtaining a plurality of sample response texts of the sample prompt text, and annotating the sample response texts as positive response texts or negative response texts based on the feedback data of the user group for the sample response texts. Thus, the sample response texts can be annotated as positive response texts or negative response texts in consideration of the feedback data of the user group for the sample response texts, and automatic annotation of positive response texts and negative response texts can be achieved without relying on manual annotation, which can save the annotation time of the training samples of the reward model and improve the training efficiency of the reward model.

[0120] It is understandable that the feedback data of the user group on the sample response text may include the feedback data of the user group on the sample response text when the sample response text is displayed on the client corresponding to the user group. For example, the above feedback data may include the number of likes, number of views, viewing time, number of complaints, etc. of the user group on the sample response text.

[0121] In some examples, based on the feedback data of the user group on the sample response text, the sample response text is labeled as positive response text or negative response text, including obtaining a score of the sample response text based on the feedback data of the user group on the sample response text, and labeling the sample response text corresponding to a score greater than a set threshold as positive response text, and labeling the sample response text corresponding to a score less than or equal to the set threshold as negative response text.

[0122] It should be noted that the relevant content of the sample prompt text can refer to the relevant content of the first prompt text, and the relevant content of the sample response text can refer to the relevant content of the first response text, which will not be repeated here.

[0123] S502, inputting the sample prompt text, the positive response text and the negative response text into the reward model.

[0124] S503, obtaining a predicted reward for the positive response text based on the sample prompt text and the positive response text through a reward model.

[0125] S504, obtaining a predicted reward for the negative response text based on the sample prompt text and the negative response text through a reward model.

[0126] It is understandable that the predicted reward for a positive response text and the predicted reward for a negative response text may be different.

[0127] S505 , training a reward model based on the predicted reward of the positive response text and the predicted reward of the negative response text.

[0128] It should be noted that the reward model is trained based on the predicted reward of the positive response text and the predicted reward of the negative response text. Any model training method in the relevant technology can be used to achieve this, and no further restrictions are made here.

[0129] It is understandable that the accuracy of positive response text is higher than that of negative response text. In theory, the predicted reward of positive response text should be greater than that of negative response text. However, the predicted reward generated by the reward model during training is not necessarily correct. There may be a situation where the predicted reward of positive response text is less than or equal to the predicted reward of negative response text.

[0130] In one embodiment, the reward model is trained based on the predicted reward of the positive response text and the predicted reward of the negative response text, including obtaining the loss function of the reward model based on the difference parameter between the predicted reward of the positive response text and the predicted reward of the negative response text, and training the reward model based on the loss function of the reward model. Thus, the difference parameter between the predicted reward of the positive response text and the predicted reward of the negative response text can be taken into account to obtain the loss function of the reward model to train the reward model, without the need to mark the specific value of the reward, which can save the marking time of the training samples of the reward model and improve the training efficiency of the reward model.

[0131] It should be noted that there are no excessive restrictions on the loss function of the reward model, for example, it may include a contrast loss function, etc.

[0132] In one embodiment, the reward model is trained based on the predicted reward of the positive response text and the predicted reward of the negative response text, including obtaining a first loss function of the reward model based on the predicted reward of the positive response text and the reference reward of the positive response text, obtaining a second loss function of the reward model based on the predicted reward of the negative response text and the reference reward of the negative response text, and training the reward model based on the first loss function and the second loss function. For example, the first loss function and the second loss function are summed to obtain a total loss function of the reward model, and the reward model is trained based on the total loss function of the reward model.

[0133] The training method of the reward model proposed in the present disclosure obtains a sample prompt text, and a positive response text and a negative response text of the sample prompt text, inputs the sample prompt text, the positive response text and the negative response text into the reward model, obtains the predicted reward of the positive response text based on the sample prompt text and the positive response text through the reward model, obtains the predicted reward of the negative response text based on the sample prompt text and the negative response text through the reward model, and trains the reward model based on the predicted reward of the positive response text and the predicted reward of the negative response text. Thus, the sample prompt text, the positive response text and the negative response text can be input into the reward model, and the predicted rewards of the positive response text and the negative response text can be obtained respectively through the reward model, and the reward model is trained taking into account the predicted rewards of the positive response text and the negative response text, thereby improving the training accuracy of the reward model. During the training process, the reward model can learn the relationship between the reward of the positive response text and the reward of the negative response text, so that the trained reward model can obtain the reward of the response text based on the prompt text and the response text.

[0134] Based on any of the above embodiments, Figure 6 As shown, the first large model is used as the teacher model of the language model, that is, the language model is now the student model, and the first large model is used to obtain the second training sample of the language model, and the language model is iterated multiple times based on the second training sample.

[0135] After multiple rounds of iterations of the language model based on the second training sample, the first prompt text is input into the language model, and the language model outputs multiple first response texts of the first prompt text. The first prompt text and the first response text are input into the reward model, and the reward model outputs the reward of the first response text to determine a reference response text of the first prompt text from the multiple first response texts, and based on the first prompt text and the reference response text of the first prompt text, a first training sample of the language model is obtained, and the language model is trained based on the first training sample.

[0136] Figure 7 FIG. 1 is a flow chart of a question-answering method according to an embodiment of the present disclosure. Figure 7 As shown, the method includes:

[0137] S701, obtain question text.

[0138] It should be noted that the execution subject of the question-answering method of the embodiment of the present disclosure may be a hardware device with data information processing capabilities and / or the necessary software required to drive the hardware device to work. Optionally, the execution subject may include a workstation, a server, a computer, a user terminal and other intelligent devices. Among them, the user terminal includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle-mounted terminal, etc.

[0139] It should be noted that there are no excessive restrictions on the question text, for example, it can be composed of at least one language such as Chinese, English, etc. The question text can be obtained by using any method for obtaining the question text in any question-answering method in the related art, and there are no excessive restrictions here.

[0140] For example, it can receive question text sent by the client.

[0141] For example, a question-and-answer request sent by a client may be received, and the question text may be extracted from the question-and-answer request.

[0142] S702, obtaining a target prompt text based on the question text.

[0143] It should be noted that obtaining the target prompt text based on the question text can be achieved by using any method for obtaining the prompt text of a language model in the relevant technology, and no further limitations are made here.

[0144] For example, a template of a prompt text may be obtained, and the text to be replaced in the template of the prompt text may be replaced with the question text to obtain the target prompt text.

[0145] For example, the question text can be used as the target prompt text.

[0146] S703: Input the target prompt text into the question-answering model, and the question-answering model outputs the answer text of the question text.

[0147] It should be noted that in the embodiments of the present disclosure, the question-answering model is obtained by using the large-model-based model training method proposed in the present disclosure.

[0148] For example, if the question text is "What is a big model?", based on the above question text, the target prompt text "You are a question-answering robot, please answer the user's question, the user's question: What is a big model?" is obtained, and the target prompt text is input into the question-answering model, and the question-answering model outputs the answer text of the above question text. For example, the answer text of the above question text is "A big model refers to a machine learning model with a large parameter scale and complexity."

[0149] The question-answering method proposed in the present disclosure obtains a question text, obtains a target prompt text based on the question text, inputs the target prompt text into a question-answering model, and the question-answering model outputs an answer text of the question text, wherein the question-answering model is obtained by adopting the model training method based on a large model proposed in the present disclosure, and the question-answering model has the advantages of small size, high precision, and fast reasoning speed, which improves the generation efficiency of the answer text, that is, improves the question-answering efficiency, thereby improving the user experience in the question-answering scenario, and in addition, the question-answering model has good generalization and applicability, improves the accuracy of the answer text, and is applicable to a variety of question-answering scenarios.

[0150] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0151] According to an embodiment of the present disclosure, the present disclosure also provides a large-model-based model training device for implementing the above-mentioned large-model-based model training method.

[0152] Figure 8 It is a block diagram of a model training device based on a large model according to an embodiment of the present disclosure.

[0153] like Figure 8 As shown, the model training device 800 based on the large model includes: a first training module 801, a processing module 802, a determination module 803 and a second training module 804.

[0154] A first training module 801 is used to use the first large model as a teacher model of the language model, and perform distillation learning on the language model based on the first large model;

[0155] A processing module 802 is used to input a first prompt text into the language model, and the language model outputs a plurality of first response texts of the first prompt text;

[0156] A determination module 803, configured to determine a reference response text of the first prompt text from a plurality of the first response texts;

[0157] The second training module 804 is used to train the language model based on the reference response text of the first prompt text.

[0158] In one embodiment of the present disclosure, the determination module 803 is further used to: obtain a score of the first response text; and determine a reference response text of the first prompt text from a plurality of first response texts based on the score.

[0159] In one embodiment of the present disclosure, the determination module 803 is further used to: input the first prompt text and the first response text into a reward model, and the reward model outputs a reward for the first response text; and use the reward for the first response text as a score for the first response text.

[0160] In one embodiment of the present disclosure, the determination module 803 is further used to: input the first prompt text and the first response text into a second large model, and the second large model outputs a score of the first response text.

[0161] In one embodiment of the present disclosure, the determination module 803 is further used to: obtain a second response text to the first prompt text based on the first prompt text through the second large model; and obtain a score of the first response text based on the first response text and the second response text through the second large model.

[0162] In one embodiment of the present disclosure, the second training module 804 is further used to: associate the first prompt text with a reference response text of the first prompt text to obtain a first training sample of the language model; and train the language model based on the first training sample.

[0163] In one embodiment of the present disclosure, the second training module 804 is further used to: use the remaining first response text other than the reference response text of the first prompt text as the third response text; and train the language model based on the third response text and the reference response text of the first prompt text.

[0164] In one embodiment of the present disclosure, the first training module 801 is further used to: input a second prompt text into the first large model, and have the first large model output a reference response text of the second prompt text; associate the second prompt text with the reference response text of the second prompt text to obtain a second training sample of the language model; and train the language model based on the second training sample.

[0165] In one embodiment of the present disclosure, the large model-based model training device 800 also includes: a third training module, which is used to: obtain sample prompt text, and positive response text and negative response text of the sample prompt text; input the sample prompt text, the positive response text and the negative response text into the reward model; obtain the predicted reward of the positive response text based on the sample prompt text and the positive response text through the reward model; obtain the predicted reward of the negative response text based on the sample prompt text and the negative response text through the reward model; and train the reward model based on the predicted reward of the positive response text and the predicted reward of the negative response text.

[0166] In one embodiment of the present disclosure, the third training module is further used to: obtain the loss function of the reward model based on the difference parameter between the predicted reward of the positive response text and the predicted reward of the negative response text; and train the reward model based on the loss function of the reward model.

[0167] In one embodiment of the present disclosure, the third training module is further used to: obtain multiple sample response texts of the sample prompt text; and label the sample response text as the positive response text or the negative response text based on feedback data of the user group for the sample response text.

[0168] The large-model-based model training device proposed in the present disclosure uses the first large model as the teacher model of the language model, and performs distillation learning on the language model based on the first large model, inputs a first prompt text into the language model, and the language model outputs multiple first response texts of the first prompt text, determines a reference response text of the first prompt text from the multiple first response texts, and trains the language model based on the reference response text of the first prompt text. Therefore, the first large model can be used as the teacher model of the language model to perform distillation learning on the language model, that is, the knowledge of the first large model can be transferred to the language model. The language model has the advantages of small size, high precision and fast reasoning speed. In addition, the reference response text of the first prompt text can be determined from the multiple first response texts generated by the language model to train the language model, which can realize the automatic acquisition of the reference response text of the language model and the self-evolution of the language model. There is no need to rely on a large model or manual labeling, which can save the labeling time of the training samples of the language model and improve the training efficiency of the language model. It is especially suitable for training scenarios after multiple rounds of iterations of the language model. In addition, in this solution, the first prompt text and the reference response text of the first prompt text can be obtained in the real business scenario of the language model, so that the training samples of the language model are closer to the real business scenario, which improves the authenticity and diversity of the training samples of the language model, and helps to improve the generalization and applicability of the language model in real business scenarios.

[0169] According to an embodiment of the present disclosure, the present disclosure also provides a question-answering device for implementing the above-mentioned question-answering method.

[0170] Fig. 9 is a block diagram of a question-and-answer device according to an embodiment of the present disclosure.

[0171] like Fig. 9 As shown, the question-answering device 900 includes: a first acquisition module 901 , a second acquisition module 902 and a processing module 903 .

[0172] The first acquisition module 901 is used to acquire the question text;

[0173] A second acquisition module 902 is used to obtain a target prompt text based on the question text;

[0174] Processing module 903 is used to input the target prompt text into the question-answering model, and the question-answering model outputs the answer text of the question text, wherein the question-answering model is obtained by adopting the large model-based model training method proposed in the present disclosure.

[0175] The question-answering device proposed in the present disclosure obtains a question text, obtains a target prompt text based on the question text, inputs the target prompt text into a question-answering model, and the question-answering model outputs an answer text of the question text, wherein the question-answering model is obtained by adopting the model training method based on a large model proposed in the present disclosure, and the question-answering model has the advantages of small size, high precision, and fast reasoning speed, which improves the generation efficiency of the answer text, that is, improves the question-answering efficiency, thereby improving the user experience in the question-answering scenario, and in addition, the question-answering model has good generalization and applicability, improves the accuracy of the answer text, and is applicable to a variety of question-answering scenarios.

[0176] According to an embodiment of the present disclosure, the present disclosure also proposes an electronic device, a readable storage medium, and a computer program product.

[0177] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0178] like Fig.10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0179] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1006, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0180] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as a model training method based on a large model and a question-and-answer method. For example, in some embodiments, a model training method based on a large model and a question-and-answer method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the model training method based on the large model and the question-and-answer method described above may be executed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute a large model-based model training method or a question-answering method in any other appropriate manner (e.g., by means of firmware).

[0181] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0182] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be presented to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0183] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0184] To propose interactions with a user account, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user account; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user account can propose input to the computer. Other types of devices may also be used to propose interactions with the user account; for example, feedback proposed to the user account may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user account may be received in any form (including acoustic input, voice input, or tactile input).

[0185] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user account computer with a graphical user account interface or a web browser through which a user account can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0186] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0187] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the steps of the model training method and the question-answering method based on the large model described in the above embodiments of the present disclosure are implemented.

[0188] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0189] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A model training method based on a large model, comprising: Using the first large model as a teacher model of the language model, and performing distillation learning on the language model based on the first large model; Inputting a first prompt text into the language model, and having the language model output a plurality of first response texts of the first prompt text; Determine a reference response text of the first prompt text from a plurality of the first response texts; The language model is trained based on a reference response text of the first prompt text.

2. The method according to claim 1, wherein: The step of determining a reference response text of the first prompt text from a plurality of the first response texts includes: Obtaining a score for the first response text; Based on the score, a reference response text for the first prompt text is determined from a plurality of the first response texts.

3. The method according to claim 2, wherein: The obtaining the score of the first response text includes: Inputting the first prompt text and the first response text into a reward model, and having the reward model output a reward for the first response text; The reward for the first response text is used as the score of the first response text.

4. The method according to claim 2, wherein: The obtaining the score of the first response text includes: The first prompt text and the first response text are input into a second large model, and the second large model outputs a score of the first response text.

5. The method according to claim 4, wherein: The method further comprises: Obtaining a second response text to the first prompt text based on the first prompt text by using the second large model; A score of the first response text is obtained by using the second large model based on the first response text and the second response text.

6. The method according to claim 1, wherein: The training of the language model based on the reference response text of the first prompt text includes: Associating the first prompt text with a reference response text of the first prompt text to obtain a first training sample of the language model; The language model is trained based on the first training sample.

7. The method according to claim 1, wherein: The training of the language model based on the reference response text of the first prompt text includes: The remaining first response texts other than the reference response text of the first prompt text are used as third response texts; The language model is trained based on the third response text and a reference response text of the first prompt text.

8. The method according to claim 1, wherein: The performing distillation learning on the language model based on the first large model includes: Inputting a second prompt text into the first large model, and having the first large model output a reference response text of the second prompt text; Associating the second prompt text with a reference response text of the second prompt text to obtain a second training sample of the language model; The language model is trained based on the second training sample.

9. The method for question-answering according to claim 3, wherein: The method further comprises: Obtaining a sample prompt text, and a positive response text and a negative response text of the sample prompt text; inputting the sample prompt text, the positive response text, and the negative response text into the reward model; Obtaining a predicted reward for the positive response text based on the sample prompt text and the positive response text by the reward model; Obtaining a predicted reward for the negative response text based on the sample prompt text and the negative response text by the reward model; The reward model is trained based on the predicted reward of the positive response text and the predicted reward of the negative response text.

10. The method according to claim 9, wherein: The step of training the reward model based on the predicted reward of the positive response text and the predicted reward of the negative response text comprises: Obtaining a loss function of the reward model based on a difference parameter between the predicted reward of the positive response text and the predicted reward of the negative response text; The reward model is trained based on the loss function of the reward model.

11. The method according to claim 9, wherein: The method further comprises: Obtaining multiple sample response texts of the sample prompt text; Based on feedback data of a user group on the sample response text, the sample response text is marked as the positive response text or the negative response text.

12. A question-answering method, comprising: Get the question text; Based on the question text, a target prompt text is obtained; The target prompt text is input into a question-answering model, and the question-answering model outputs an answer text to the question text, wherein the question-answering model is obtained by adopting the large-model-based model training method described in any one of claims 1-11.

13. A model training device based on a large model, comprising: A first training module, used for using the first large model as a teacher model of the language model, and performing distillation learning on the language model based on the first large model; a processing module, configured to input a first prompt text into the language model, and output a plurality of first response texts of the first prompt text by the language model; A determination module, used for determining a reference response text of the first prompt text from a plurality of the first response texts; The second training module is used to train the language model based on a reference response text of the first prompt text.

14. The device according to claim 13, wherein: The determining module is further used for: Obtaining a score for the first response text; Based on the score, a reference response text for the first prompt text is determined from a plurality of the first response texts.

15. The device according to claim 14, wherein: The determining module is further used for: Inputting the first prompt text and the first response text into a reward model, and having the reward model output a reward for the first response text; The reward for the first response text is used as the score of the first response text.

16. The device according to claim 14, wherein: The determining module is further used for: The first prompt text and the first response text are input into a second large model, and the second large model outputs a score of the first response text.

17. The device according to claim 16, wherein: The determining module is further used for: Obtaining a second response text to the first prompt text based on the first prompt text by using the second large model; A score of the first response text is obtained by using the second large model based on the first response text and the second response text.

18. The device according to claim 13, wherein: The second training module is further used for: Associating the first prompt text with a reference response text of the first prompt text to obtain a first training sample of the language model; The language model is trained based on the first training sample.

19. The device according to claim 13, wherein: The second training module is further used for: The remaining first response texts other than the reference response text of the first prompt text are used as third response texts; The language model is trained based on the third response text and a reference response text of the first prompt text.

20. The device according to claim 13, wherein: The first training module is further used for: Inputting a second prompt text into the first large model, and having the first large model output a reference response text of the second prompt text; Associating the second prompt text with a reference response text of the second prompt text to obtain a second training sample of the language model; The language model is trained based on the second training sample.

21. The device according to claim 15, wherein: The device further comprises: a third training module, configured to: Obtaining a sample prompt text, and a positive response text and a negative response text of the sample prompt text; inputting the sample prompt text, the positive response text, and the negative response text into the reward model; Obtaining a predicted reward for the positive response text based on the sample prompt text and the positive response text by the reward model; Obtaining a predicted reward for the negative response text based on the sample prompt text and the negative response text by the reward model; The reward model is trained based on the predicted reward of the positive response text and the predicted reward of the negative response text.

22. The device according to claim 21, wherein The third training module is further used for: Obtaining a loss function of the reward model based on a difference parameter between the predicted reward of the positive response text and the predicted reward of the negative response text; The reward model is trained based on the loss function of the reward model.

23. The device according to claim 21, wherein The third training module is further used for: Obtaining multiple sample response texts of the sample prompt text; Based on feedback data of a user group on the sample response text, the sample response text is marked as the positive response text or the negative response text.

24. A question-answering device, comprising: The first acquisition module is used to acquire the question text; A second acquisition module is used to obtain a target prompt text based on the question text; A processing module, used for inputting the target prompt text into a question-answering model, and having the question-answering model output an answer text of the question text, wherein the question-answering model is obtained by adopting the large model-based model training method described in any one of claims 1-11.

25. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-12.

27. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images