Model training method, device, equipment, medium and product

Abstract

The invention provides a model training method and device, equipment, a medium and a product, and the method comprises the steps: obtaining sample data related to a target knowledge domain from a knowledge base; determining at least one question text related to the target knowledge field from the sample data; for each question text in the at least one question text, executing the following steps: determining reference data related to the question text from a knowledge base; according to the question text and reference data, determining a reply text corresponding to the question text; and training the target language model by taking each question text and the reply text corresponding to the question text as training data. In the method, each question and answer pair in the knowledge field is used as the training data to train the language model, and the reasoning performance of the language model in the specific knowledge field is improved on the basis that data labeling or incremental pre-training does not need to be carried out.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of computer technology, language models have emerged. Language models usually have natural language processing capabilities and can handle different types of natural language tasks. In order to improve the reasoning performance of language models, the industry usually uses retrieval-augmented generation (RAG) technology to assist language models in task processing.

[0003] Specifically, RAG technology is divided into two stages: data retrieval stage and reply generation stage. The data retrieval stage is used to retrieve knowledge data related to the input text from the knowledge base, and the reply generation stage is used to combine the knowledge data to generate reply text.

[0004] After applying RAG technology, in general fields that are not related to professional technology, the language model can refer to the retrieved data and generate more reasonable and accurate response text. However, in knowledge fields related to professional technology, it is difficult to effectively improve the reasoning performance of the language model. Summary of the invention

[0005] The present application provides a model training method. The method can improve the reasoning performance of a language model in a specific knowledge domain without data annotation or incremental pre-training. The present application also provides a device, an electronic device, a computer-readable storage medium, and a computer program product corresponding to the above method.

[0006] In a first aspect, the present application provides a model training method, the method comprising:

[0007] From the knowledge base, obtain sample data related to the target knowledge domain;

[0008] determining at least one question text related to the target knowledge domain from the sample data;

[0009] For each question text in the at least one question text, the following steps are performed: determining reference data related to the question text from the knowledge base; determining a reply text corresponding to the question text based on the question text and the reference data;

[0010] Each of the question texts and the reply text corresponding to the question text is used as training data to train the target language model.

[0011] In some possible implementations, determining a reply text corresponding to the question text according to the question text and the reference data includes:

[0012] Generate a first prompt word; wherein the first prompt word includes the question text, the reference data and information for indicating the generation of a reply text for the question text;

[0013] The first prompt word is sent to a general language model, and a reply text corresponding to the question text and returned by the general language model is received.

[0014] In some possible implementations, determining a reply text corresponding to the question text according to the question text and the reference data includes:

[0015] Generate multiple candidate response texts based on the question text and the reference data;

[0016] A reply text corresponding to the question text is determined based on the multiple candidate reply texts.

[0017] In some possible implementations, generating a plurality of candidate reply texts according to the question text and the reference data includes:

[0018] Generate a second prompt word; send the second prompt word to multiple universal language models respectively, and receive multiple candidate reply texts returned by the multiple universal language models; wherein the second prompt word includes the question text, the reference data and information indicating the reply text generated for the question text; or,

[0019] Generate multiple second prompt words; send the multiple second prompt words to the general language model in sequence, and receive multiple candidate reply texts returned in sequence by the general language model; wherein the multiple second prompt words all include the question text, the reference data and information indicating the reply text generated for the question text, and the arrangement order of the reference data in the multiple second prompt words is different.

[0020] In some possible implementations, determining a reply text corresponding to the question text according to the multiple candidate reply texts includes:

[0021] Generate a third prompt word; wherein the third prompt word includes the multiple candidate reply texts and information for indicating the evaluation of the multiple candidate reply texts;

[0022] Sending the third prompt word to a universal language model, and receiving a target reply text returned by the universal language model;

[0023] According to the target reply text, a reply text corresponding to the question text is determined.

[0024] In some possible implementations, the target reply text is generated based on the second prompt word, and determining the reply text corresponding to the question text according to the target reply text includes:

[0025] Generate a fourth prompt word; wherein the fourth prompt word includes the target reply text, the second prompt word and information for instructing to polish the target reply text;

[0026] The fourth prompt word is sent to the general language model, and a reply text returned by the general language model is received.

[0027] In some possible implementations, determining at least one question text related to the target knowledge domain from the sample data includes:

[0028] Generate a fifth prompt word; wherein the fifth prompt word includes the sample data, the question type related to the target knowledge field and information for instructing to generate a question text;

[0029] The fifth prompt word is sent to a general language model, and at least one question text related to the target knowledge domain returned by the general language model is received.

[0030] In some possible implementations, determining reference data related to the question text from the knowledge base includes:

[0031] Determine the similarity between the question text and the candidate data in the knowledge base;

[0032] The candidate data whose similarity meets the set conditions is determined as reference data.

[0033] In a second aspect, the present application provides a model training device, the device comprising:

[0034] An acquisition module is used to acquire sample data related to the target knowledge domain from the knowledge base;

[0035] A first determination module, configured to determine at least one question text related to the target knowledge domain from the sample data;

[0036] The second determination module is used to perform the following steps for each question text in the at least one question text: determining reference data related to the question text from the knowledge base; determining a reply text corresponding to the question text based on the question text and the reference data;

[0037] The training module is used to train the target language model by taking each of the question texts and the reply texts corresponding to the question texts as training data.

[0038] In some possible implementations, the second determining module is specifically configured to:

[0039] Generate a first prompt word; wherein the first prompt word includes the question text, the reference data and information for indicating the generation of a reply text for the question text;

[0040] The first prompt word is sent to a general language model, and a reply text corresponding to the question text and returned by the general language model is received.

[0041] In some possible implementations, the second determining module is specifically configured to:

[0042] Generate multiple candidate response texts based on the question text and the reference data;

[0043] A reply text corresponding to the question text is determined based on the multiple candidate reply texts.

[0044] In some possible implementations, the second determining module is specifically configured to:

[0045] Generate a second prompt word; send the second prompt word to multiple universal language models respectively, and receive multiple candidate reply texts returned by the multiple universal language models; wherein the second prompt word includes the question text, the reference data and information indicating the reply text generated for the question text; or,

[0046] Generate multiple second prompt words; send the multiple second prompt words to the general language model in sequence, and receive multiple candidate reply texts returned in sequence by the general language model; wherein the multiple second prompt words all include the question text, the reference data and information indicating the reply text generated for the question text, and the arrangement order of the reference data in the multiple second prompt words is different.

[0047] In some possible implementations, the second determining module is specifically configured to:

[0048] Generate a third prompt word; wherein the third prompt word includes the multiple candidate reply texts and information for indicating the evaluation of the multiple candidate reply texts;

[0049] Sending the third prompt word to a universal language model, and receiving a target reply text returned by the universal language model;

[0050] According to the target reply text, a reply text corresponding to the question text is determined.

[0051] In some possible implementations, the target reply text is generated based on the second prompt word, and the second determination module is specifically configured to:

[0052] Generate a fourth prompt word; wherein the fourth prompt word includes the target reply text, the second prompt word and information for instructing to polish the target reply text;

[0053] The fourth prompt word is sent to the general language model, and a reply text returned by the general language model is received.

[0054] In some possible implementations, the first determining module is specifically configured to:

[0055] Generate a fifth prompt word; wherein the fifth prompt word includes the sample data, the question type related to the target knowledge field and information for instructing to generate a question text;

[0056] The fifth prompt word is sent to a general language model, and at least one question text related to the target knowledge domain returned by the general language model is received.

[0057] In some possible implementations, the second determining module is specifically configured to:

[0058] Determine the similarity between the question text and the candidate data in the knowledge base;

[0059] The candidate data whose similarity meets the set conditions is determined as reference data.

[0060] In a third aspect, the present application provides an electronic device, comprising a processor and a memory. The processor and the memory communicate with each other. The processor is used to execute instructions stored in the memory so that the electronic device performs the model training method in the first aspect or any implementation of the first aspect.

[0061] In a fourth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and the instructions instruct an electronic device to execute the model training method described in the above-mentioned first aspect or any one of the implementations of the first aspect.

[0062] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, enables the electronic device to execute the model training method described in the first aspect or any one of the implementations of the first aspect.

[0063] Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations.

[0064] It can be seen from the above technical solutions that this application has the following advantages:

[0065] The present application provides a model training method, which first obtains sample data related to the target knowledge field from a knowledge base, and determines at least one question text related to the target knowledge field from the sample data. Then, for each question text in the at least one question text, the following steps are performed: from the knowledge base, reference data related to the question text is determined, and based on the question text and the reference data, a reply text corresponding to the question text is determined. Each question text and the reply text corresponding to the question text are used as training data to train the target language model.

[0066] In this method, sample data related to the knowledge domain is obtained, and question texts related to the knowledge domain are extracted from the sample data to determine the questions that users may ask in the knowledge domain. For each question text, the corresponding reply text is determined in combination with reference data to form a question-answer pair of "question text-reply text". Each question-answer pair in the knowledge domain is used as training data to train the language model, thereby improving the reasoning performance of the language model in a specific knowledge domain without the need for data annotation or incremental pre-training. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required for use in the embodiments are briefly introduced below.

[0068] Figure 1 A flowchart of a model training method provided in an embodiment of the present application;

[0069] Figure 2 A schematic diagram of the structure of a model training device provided in an embodiment of the present application;

[0070] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The terms "first" and "second" in the embodiments of the present application are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features.

[0072] First, some technical terms and application scenarios involved in the embodiments of this application are introduced.

[0073] With the rapid development of computer technology, language models have emerged. Language models have powerful expression and learning capabilities and can handle many different types of natural language tasks, such as dialogue tasks, translation tasks, speech recognition tasks, text generation tasks, semantic analysis tasks, etc.

[0074] Generally, the language model can perform natural language processing on the user's input text and generate corresponding reply text, wherein the input text may indicate a question, an instruction, etc.

[0075] In order to improve the reasoning performance of the language model, the industry usually uses retrieval-augmented generation (RAG) technology to assist the language model in task processing. Specifically, RAG technology is divided into two stages: data retrieval stage and reply generation stage. In the data retrieval stage, for the user's input text, by calling the recall model, the knowledge data related to the input text is retrieved from the knowledge base, for example, the knowledge data similar to the input text is retrieved from the knowledge base. In the reply generation stage, the knowledge data obtained in the data retrieval stage is sent to the language model together with the input text, so that the language model can analyze the input text in combination with the knowledge data, and then generate a reply text.

[0076] After applying RAG technology, in general fields unrelated to professional technology, the retrieved knowledge data related to the input text can assist the language model in generating replies, so the language model can generate more reasonable and accurate reply texts. However, in knowledge fields related to professional technology, due to the high professionalism of the knowledge data related to the input text, it is difficult for the language model to quickly and accurately understand the knowledge data, and it is difficult to effectively improve the rationality and accuracy of the reply text, and it is difficult to effectively improve the reasoning performance of the language model.

[0077] For example, when the knowledge field related to the professional and technical field is mathematics, the input text is "What is the limit", and the knowledge data related to the input text can be data in the mathematics field such as advanced mathematics textbooks. Due to the high professionalism of the mathematics field, for the input text "What is the limit", the language model is difficult to give a highly accurate response text after analyzing data in the mathematics field such as advanced mathematics textbooks.

[0078] To address the above issues, the industry usually uses the following two methods to improve the reasoning performance of language models in knowledge domains: incremental pretraining (CT) and data annotation. Incremental pretraining refers to the method of introducing knowledge data from a specific knowledge domain to further train the model based on the existing pretrained language model, and using knowledge data from a specific knowledge domain to enhance the ability and performance of the language model in that specific knowledge domain.

[0079] The data labeling method refers to the manual labeling of knowledge data in a specific knowledge field by researchers in that knowledge field. Since researchers have professional knowledge in that knowledge field, they can obtain question and answer data in that knowledge field through manual labeling, and then train the language model based on the data labeling results to enhance the ability and performance of the language model in the specific knowledge field.

[0080] However, the incremental pre-training method relies on a large amount of knowledge data and consumes a lot of computing resources for training. The data annotation method relies on manual data annotation, which makes it difficult to obtain high-quality data annotation results and results in poor training results.

[0081] In view of this, the present application provides a model training method, which first obtains sample data related to the target knowledge field from the knowledge base, and determines at least one question text related to the target knowledge field from the sample data. Then, for each question text in the at least one question text, the following steps are performed: from the knowledge base, reference data related to the question text is determined, and based on the question text and the reference data, a reply text corresponding to the question text is determined. Each question text and the reply text corresponding to the question text are used as training data to train the target language model.

[0082] In this method, sample data related to the knowledge domain is obtained, and question texts related to the knowledge domain are extracted from the sample data to determine the questions that users may ask in the knowledge domain. For each question text, the corresponding reply text is determined in combination with reference data to form a question-answer pair of "question text-reply text". Each question-answer pair in the knowledge domain is used as training data to train the language model, thereby improving the reasoning performance of the language model in a specific knowledge domain without the need for data annotation or incremental pre-training.

[0083] To facilitate understanding of the technical solution provided by the embodiments of the present application, it will be described below with reference to the accompanying drawings.

[0084] See also Figure 1 The flowchart of a model training method provided in the embodiment of the present application shown in FIG. 1 specifically includes:

[0085] S101: Obtain sample data related to the target knowledge domain from the knowledge base.

[0086] The process of obtaining sample data related to the target knowledge domain can be understood as the process of collecting data from a knowledge base. Among them, a knowledge base generally refers to a data set related to knowledge, for example, a data set related to knowledge in different knowledge domains. In an embodiment of the present application, the knowledge base may be related to the target knowledge domain, and the knowledge base may store data related to the target knowledge domain.

[0087] In specific implementation, the knowledge base can be a vector knowledge base. Data related to the target knowledge field is collected from different source channels, the data related to the target knowledge field is processed in blocks, and the data after block processing is vectorized to generate vectorized representations corresponding to the data related to the target knowledge field, and the vectorized representations corresponding to the data related to the target knowledge field are stored to build a knowledge base. In this way, unified storage of different types of data is achieved, which is convenient for subsequent data retrieval.

[0088] In the embodiment of the present application, any amount and type of data can be selected from the knowledge base related to the target knowledge field as sample data related to the target knowledge field. For example, when the target knowledge field is the product sales knowledge field, the sample data related to the target knowledge field can include product introduction data, customer case data of multiple companies, introduction data of multiple companies, etc.

[0089] Among them, the knowledge domain can be understood as a concept corresponding to the general domain. The general domain is usually unrelated to professional technology. For example, the general domain can include the common sense of life and language communication. The knowledge domain is usually related to professional technology. For example, the knowledge domain can include the mathematics and physics fields.

[0090] In other words, for general domains, users do not need specific expertise to understand the knowledge in the general domain. For knowledge domains, users need specific expertise to understand the knowledge in the knowledge domain. For example, users need expertise in mathematics to understand the knowledge in the mathematics domain, and users need expertise in physics to understand the knowledge in the physics domain.

[0091] In actual business scenarios, knowledge domains can also include domains related to specific businesses. For example, in a product sales scenario, knowledge related to product sales (such as knowledge related to product introduction, knowledge related to product sales cases, knowledge related to potential customers, etc.) can constitute the product sales knowledge domain. For another example, in a store management scenario, knowledge related to store management (such as knowledge related to store information, knowledge related to business conditions, etc.) can constitute the store management knowledge domain.

[0092] In the embodiment of the present application, the target knowledge field can be understood as any knowledge field related to professional technology, and the sample data related to the target knowledge field can be understood as the knowledge related to the professional technology of the target knowledge field. For example, when the target knowledge field is the field of mathematics, the sample data related to the target knowledge field can be mathematical knowledge, and when the target knowledge field is the field of product sales knowledge, the sample data related to the target knowledge field can be knowledge related to product sales. In the embodiment of the present application, the knowledge related to the professional technology of the target knowledge field can be used as a sample for extracting question text, and therefore, is called sample data.

[0093] The sample data related to the target knowledge domain may include data of various types. For example, the sample data related to the target knowledge domain may be data of a document type, data of a web page type, and the like.

[0094] S102: Determine at least one question text related to the target knowledge domain from the sample data.

[0095] In the embodiment of the present application, for each sample data, at least one question text related to the target knowledge field is determined. The at least one question text is associated with the sample data, and the sample data can be used to reply to the at least one question text. In other words, the at least one question text can be understood as a question text extracted based on the content described by the sample data.

[0096] For example, if the target knowledge domain is the product sales knowledge domain, sample data A is the customer case data of enterprise A, and the content described in the sample data is "After enterprise A uses product A, it improves work efficiency in many aspects." In this case, the question text can be "How does product A improve the efficiency of the enterprise in recruitment?", "How does product A help the enterprise improve online collaboration efficiency?", etc. Sample data A can respond to the above question text. For another example, if the target knowledge domain is the product sales knowledge domain, sample data B is the introduction data of product A, and the content described in the sample data is "Multiple functions of product A." In this case, the question text can be "How does product A optimize the recruitment process?", "How does product A connect with functional module A to facilitate recruitment?", "What is the working method of functional module B of product A?", etc. Sample data B can respond to the above question text.

[0097] Since the sample data is related to the target knowledge domain, the question text extracted based on the content described by the sample data can also be a question text related to the target knowledge domain. In other words, at least one question text is a question related to the target knowledge domain, which can be understood as a question that the user may have or ask in the target knowledge domain.

[0098] In some possible implementations, the automatic determination of the question text can be achieved with the help of a language model. The language model may have natural language processing capabilities, be able to understand the meaning of natural language, and process different types of natural language tasks. For example, the language model may be a deep learning model trained using text data.

[0099] The language model can determine the target knowledge domain based on prompt learning. Among them, prompt words can be used to guide the language model to perform specific outputs in generative tasks (such as text generation tasks, question-answering tasks, and dialogue tasks). By configuring prompt words, the language model can understand the background and requirements of the task, and the language model can handle different types of natural language processing tasks without retraining the language model, increasing the scalability and flexibility of the language model.

[0100] In a specific implementation, a fifth prompt word is generated, the fifth prompt word is sent to a general language model, and at least one question text related to the target knowledge field returned by the general language model is received. The fifth prompt word includes sample data, a question type related to the target knowledge field, and information for indicating the generation of the question text.

[0101] In the embodiments of the present application, the general language model can be understood as a language model that does not utilize data from knowledge fields to enhance model performance. In other words, the general language model has similar performance in model reasoning in various knowledge fields and does not have outstanding reasoning performance for a particular knowledge field.

[0102] Since the fifth prompt word includes sample data, question types related to the target knowledge domain, and information for instructing to generate question texts, the universal language model can extract question texts from the sample data based on the prompt capability of the fifth prompt word by referring to question types related to the target knowledge domain, and then output at least one question text related to the target knowledge domain. In this way, a large number of question texts related to the target knowledge domain are collected.

[0103] S103: For each question text in at least one question text, perform the following steps: determine reference data related to the question text from a knowledge base; and determine a reply text corresponding to the question text based on the question text and the reference data.

[0104] After extracting the question text from the sample data, the corresponding reply text can be determined for each question text, that is, multiple "question text-reply text" question-answer pairs in the target knowledge field are generated. Considering that it is difficult to obtain accurate and comprehensive reply texts with only a single sample data for generating question texts, in the embodiment of the present application, the reply texts corresponding to each question text are generated by recalling reference data and combining the rich reference data.

[0105] The reference data can be understood as data related to a question text and used to assist in generating a reply text corresponding to the question text. The reference data can be data in a knowledge base. In specific implementation, the similarity between the question text and the candidate data in the knowledge base is determined, and the candidate data whose similarity meets the set conditions is determined as the reference data.

[0106] The candidate data can be understood as data stored in the knowledge base related to the target knowledge domain. In other words, the process of determining the reference data can be understood as the process of retrieving data from the knowledge base. Considering that the candidate data in the knowledge base can be stored in the form of vectorized representation, the question text can be vectorized to obtain the vectorized representation corresponding to the question text, and then the similarity between the vectorized representation corresponding to the question text and the vectorized representation corresponding to the candidate data in the knowledge base can be determined, thereby determining the reference data.

[0107] In the embodiment of the present application, there is no limitation on the method of determining the similarity. For example, the similarity between the question text and the candidate data in the knowledge base can be determined using measurement methods such as cosine similarity and Euclidean distance.

[0108] The setting condition can be understood as the condition for determining the candidate data as the reference data. For example, the setting condition can be that the similarity is greater than the similarity threshold, that is, the candidate data with similarity greater than the similarity threshold is determined as the reference data. For another example, the setting condition can be the top K (top-K) of the similarity sorting result, that is, the candidate data is sorted from high to low according to the similarity, and the candidate data with the top K in the sorting result is determined as the reference data, where K is an integer greater than 0.

[0109] After determining the reference data related to the question text, the question text can be analyzed in combination with the reference data to determine the reply text corresponding to the question text. In some possible implementations, the reply text is directly output with the help of a language model. Specifically, a first prompt word is generated, the first prompt word is sent to a universal language model, and the reply text corresponding to the question text returned by the universal language model is received.

[0110] The first prompt word includes the question text, reference data and information for indicating the generation of a reply text for the question text.

[0111] Since the first prompt word includes the question text, reference data, and information indicating the response text for generating the question text, the universal language model can analyze the question text based on the prompt capability of the first prompt word and the content described by the reference data, and then output the response text corresponding to the question text. In this way, based on the natural language processing capability of the language model, a question-answer pair of "question text-response text" is formed.

[0112] In some other possible implementations, the reply text is not generated directly. Specifically, a plurality of candidate reply texts are generated according to the question text and reference data, and then, a reply text corresponding to the question text is determined according to the plurality of candidate reply texts.

[0113] That is to say, multiple candidate reply texts are generated first, and then the multiple candidate reply texts are integrated to determine the reply text. In this way, the directly generated reply text is prevented from being inaccurate and the quality of the reply text is improved.

[0114] The embodiments of the present application support generating multiple candidate reply texts in different ways. In some embodiments, a second prompt word is generated, and the second prompt word is sent to multiple universal language models respectively, and multiple candidate reply texts returned by the multiple universal language models are received. The second prompt word includes the question text, reference data, and information for indicating the reply text generated for the question text.

[0115] That is to say, multiple candidate reply texts can be generated by multiple different general language models. Among them, multiple general language models can be different versions of the same language model, or different variants of the language model. Since the second prompt word includes the question text, reference data and information for indicating the reply text generated for the question text, multiple general language models can analyze the question text based on the prompt ability of the second prompt word, combined with the content described by the reference data, and output candidate reply texts respectively, to obtain multiple candidate reply texts.

[0116] In other embodiments, multiple second prompt words are generated, the multiple second prompt words are sent to the general language model in sequence, and multiple reply texts returned in sequence by the general language model are received. The multiple second prompt words all include the question text, reference data, and information indicating the reply text for generating the question text, and the reference data in the multiple second prompt words are arranged in different orders.

[0117] That is to say, multiple reply texts can be generated by a general language model based on different prompt words. Since the reference data can include multiple data, multiple second prompt words can be generated by adjusting the arrangement order of the reference data. For example, the reference data includes data A, data B and data C, the arrangement order of the reference data in the second prompt word A can be data A, data B and data C, the arrangement order of the reference data in the second prompt word B can be data A, data C and data B, the arrangement order of the reference data in the second prompt word C can be data B, data A and data C, the arrangement order of the reference data in the second prompt word D can be data B, data C and data A, the arrangement order of the reference data in the second prompt word E can be data C, data A and data B, and the arrangement order of the reference data in the second prompt word F can be data C, data B and data A.

[0118] Since the reference data arrangement orders are different in multiple second prompt words, after sending the multiple second prompt words to the universal language model, the universal language model can combine the reference data in different arrangement orders to sequentially generate candidate reply texts with certain differences, and obtain multiple candidate reply texts.

[0119] In the embodiment of the present application, the second prompt word may also include information indicating the format of the reply text, information indicating the content to be included in the reply text, and information indicating that the content included in the reply text needs to be derived from reference data. In this way, the quality of multiple candidate reply texts is improved.

[0120] The embodiment of the present application does not limit the method for adjusting the arrangement order of the reference data in the second prompt word. For example, the arrangement order of the reference data in the second prompt word can be adjusted by setting a random seed.

[0121] After obtaining multiple candidate reply texts, the multiple candidate reply texts are integrated to determine the reply text. In specific implementation, a third prompt word is generated, the third prompt word is sent to the general language model, the target reply text returned by the general language model is received, and the reply text corresponding to the question text is determined according to the target reply text.

[0122] The third prompt word includes a plurality of candidate reply texts and information for indicating evaluation of the plurality of candidate reply texts.

[0123] That is to say, in the embodiment of the present application, the quality of multiple candidate reply texts is evaluated with the help of the natural language processing capability of the language model, and then the target reply text with the best quality is determined from the multiple candidate reply texts, and then the reply text is determined based on the target reply text.

[0124] In some possible implementations, the information in the third prompt word used to indicate the evaluation of multiple candidate reply texts may be information used to indicate the evaluation of multiple candidate reply texts based on the best of N samples (BoN). In this case, the universal language model can determine the target reply text by comparing multiple candidate reply texts in pairs. That is, the universal language model can first compare candidate reply text A with candidate reply text B to obtain Bo2, then compare candidate reply text C with Bo2 to obtain Bo3, and so on, to finally obtain BoN, and use BoN as the target reply text.

[0125] Furthermore, pairwise comparison of candidate reply texts can be achieved through the good-same-bad (GSB) evaluation metric, that is, the general language model first compares the GSB of candidate reply text A and candidate reply text B to obtain Bo2, then compares the GSB of candidate reply text C and Bo2 to obtain Bo3, and so on, finally obtaining BoN.

[0126] Next, the reply text corresponding to the question text can be determined based on the target reply text. In some embodiments, the target reply text can be directly determined as the reply text corresponding to the question text. In other words, the candidate reply text with the best quality among multiple candidate reply texts is determined as the reply text corresponding to the question text.

[0127] In other embodiments, the target reply text may be polished, and the polished target reply text may be determined as the reply text corresponding to the question text. In specific implementation, the target reply text is generated based on the second prompt word, a fourth prompt word is generated, the fourth prompt word is sent to the general language model, and the reply text returned by the general language model is received.

[0128] The fourth prompt word includes the target reply text, the second prompt word and information for instructing to polish the target reply text.

[0129] Since the target reply text is generated based on the second prompt word, by configuring the target reply text, the second prompt word and the information for instructing to polish the target reply text in the fourth prompt word, the universal language model can judge whether the target reply text meets the question response requirements represented by the second prompt word based on the prompt ability of the fourth prompt word, and then polish the target reply text, further improve the reply quality, and output the reply text.

[0130] It should be noted that the embodiment of the present application does not limit the number of times the candidate reply text is polished. For example, after the target reply text is polished once, the polished target reply text may be polished again. In this case, the prompt word input into the universal language model may include the polished target reply text, the second prompt word, and information for instructing to polish the polished target reply text.

[0131] Thus, by executing the above steps, the reply text corresponding to the question text is determined. After executing the above steps for each text in at least one question text, multiple question-answer pairs of "question text-reply text" can be obtained, and question-answer scenarios that may be involved in the target knowledge field can be constructed.

[0132] S104: Use each question text and the reply text corresponding to the question text as training data to train the target language model.

[0133] The target language model can be understood as a language model to be trained. In some possible implementations, the target language model is trained using low-rank adaptation (LoRA) technology. The LoRA technology injects a trainable low-rank decomposition matrix into each layer of the transformer architecture of the target language model to fine-tune some parameters in the target language model, thereby reducing the number of adjustment parameters while maintaining the initial performance of the target language model and achieving efficient model training.

[0134] In this way, by using multiple “question text-reply text” question-answer pairs in the target knowledge domain as training data, the reasoning ability of the target language model in the target knowledge domain is improved.

[0135] In this method, sample data related to the knowledge domain is obtained, and question texts related to the knowledge domain are extracted from the sample data to determine the questions that users may ask in the knowledge domain. For each question text, the corresponding reply text is determined in combination with reference data to form a question-answer pair of "question text-reply text". Each question-answer pair in the knowledge domain is used as training data to train the language model, thereby improving the reasoning performance of the language model in a specific knowledge domain without the need for data annotation or incremental pre-training.

[0136] Combination of the above Figure 1 The model training method provided in the embodiment of the present application is introduced in detail. The following will introduce the devices and equipment provided in the embodiment of the present application in conjunction with the accompanying drawings.

[0137] See also Figure 2 The structural diagram of the model training device shown in FIG. 20 includes:

[0138] The acquisition module 201 is used to acquire sample data related to the target knowledge domain from the knowledge base;

[0139] A first determination module 202 is used to determine at least one question text related to the target knowledge field from the sample data;

[0140] The second determination module 203 is used to perform the following steps for each question text in the at least one question text: determine reference data related to the question text from the knowledge base; determine a reply text corresponding to the question text based on the question text and the reference data;

[0141] The training module 204 is used to train the target language model by using each of the question texts and the reply texts corresponding to the question texts as training data.

[0142] In some possible implementations, the second determining module 203 is specifically configured to:

[0143] Generate a first prompt word; wherein the first prompt word includes the question text, the reference data and information for indicating the generation of a reply text for the question text;

[0144] The first prompt word is sent to a general language model, and a reply text corresponding to the question text and returned by the general language model is received.

[0145] In some possible implementations, the second determining module 203 is specifically configured to:

[0146] Generate multiple candidate response texts based on the question text and the reference data;

[0147] A reply text corresponding to the question text is determined based on the multiple candidate reply texts.

[0148] In some possible implementations, the second determining module 203 is specifically configured to:

[0149] Generate a second prompt word; send the second prompt word to multiple universal language models respectively, and receive multiple candidate reply texts returned by the multiple universal language models; wherein the second prompt word includes the question text, the reference data and information indicating the reply text generated for the question text; or,

[0150] Generate multiple second prompt words; send the multiple second prompt words to the general language model in sequence, and receive multiple candidate reply texts returned in sequence by the general language model; wherein the multiple second prompt words all include the question text, the reference data and information indicating the reply text generated for the question text, and the arrangement order of the reference data in the multiple second prompt words is different.

[0151] In some possible implementations, the second determining module 203 is specifically configured to:

[0152] Generate a third prompt word; wherein the third prompt word includes the multiple candidate reply texts and information for indicating the evaluation of the multiple candidate reply texts;

[0153] Sending the third prompt word to a universal language model, and receiving a target reply text returned by the universal language model;

[0154] According to the target reply text, a reply text corresponding to the question text is determined.

[0155] In some possible implementations, the target reply text is generated based on the second prompt word, and the second determination module 203 is specifically configured to:

[0156] Generate a fourth prompt word; wherein the fourth prompt word includes the target reply text, the second prompt word and information for instructing to polish the target reply text;

[0157] The fourth prompt word is sent to the general language model, and a reply text returned by the general language model is received.

[0158] In some possible implementations, the first determining module 202 is specifically configured to:

[0159] Generate a fifth prompt word; wherein the fifth prompt word includes the sample data, the question type related to the target knowledge field and information for instructing to generate a question text;

[0160] The fifth prompt word is sent to a general language model, and at least one question text related to the target knowledge domain returned by the general language model is received.

[0161] In some possible implementations, the second determining module 203 is specifically configured to:

[0162] Determine the similarity between the question text and the candidate data in the knowledge base;

[0163] The candidate data whose similarity meets the set conditions is determined as reference data.

[0164] The model training device 20 according to the embodiment of the present application may correspond to the method described in the embodiment of the present application, and the above and other operations and / or functions of each module / unit of the model training device 20 are respectively to implement Figure 1 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.

[0165] The present application also provides an electronic device. The electronic device is specifically used to implement Figure 2 The functions of the model training device 20 in the illustrated embodiment.

[0166] Figure 3 A schematic diagram of the structure of an electronic device 300 is provided. Figure 3 As shown, the electronic device 300 includes a bus 301, a processor 302, a communication interface 303 and a memory 304. The processor 302, the memory 304 and the communication interface 303 communicate with each other via the bus 301.

[0167] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0168] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0169] The communication interface 303 is used for communicating with the outside. For example, the communication interface 303 can be used for communicating with a terminal.

[0170] The memory 304 may include a volatile memory, such as a random access memory (RAM). The memory 304 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0171] The memory 304 stores executable code, and the processor 302 executes the executable code to perform the aforementioned model training method.

[0172] Specifically, in implementing Figure 2 In the case of the embodiment shown, and Figure 2 When each module or unit of the model training device 20 described in the embodiment is implemented by software, the execution Figure 2 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 304. The processor 302 executes the program code corresponding to each unit stored in the memory 304 to perform the aforementioned model training method.

[0173] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned model training method applied to the model training device 20.

[0174] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.

[0175] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0176] When the computer program product is executed by a computer, the computer executes any of the aforementioned model training methods. The computer program product may be a software installation package, and when any of the aforementioned model training methods is needed, the computer program product may be downloaded and executed on a computer.

[0177] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to each embodiment of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0178] The units involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the unit / module does not, in some cases, constitute a limitation on the unit itself.

[0179] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0180] In the context of the present application embodiment, machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0181] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0182] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0183] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0184] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0185] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model training method, characterized in that: The method comprises: From the knowledge base, obtain sample data related to the target knowledge domain; determining at least one question text related to the target knowledge domain from the sample data; For each question text in the at least one question text, the following steps are performed: determining reference data related to the question text from the knowledge base; determining a reply text corresponding to the question text based on the question text and the reference data; Each of the question texts and the reply text corresponding to the question text is used as training data to train the target language model.

2. The method according to claim 1, characterized in that The step of determining a reply text corresponding to the question text according to the question text and the reference data includes: Generate a first prompt word; wherein the first prompt word includes the question text, the reference data and information for indicating the generation of a reply text for the question text; The first prompt word is sent to a general language model, and a reply text corresponding to the question text and returned by the general language model is received.

3. The method according to claim 1, characterized in that The step of determining a reply text corresponding to the question text according to the question text and the reference data includes: Generate multiple candidate response texts based on the question text and the reference data; A reply text corresponding to the question text is determined based on the multiple candidate reply texts.

4. The method according to claim 3, characterized in that: The step of generating a plurality of candidate reply texts according to the question text and the reference data includes: Generate a second prompt word; send the second prompt word to multiple universal language models respectively, and receive multiple candidate reply texts returned by the multiple universal language models; wherein the second prompt word includes the question text, the reference data and information indicating the reply text generated for the question text; or, Generate multiple second prompt words; send the multiple second prompt words to the general language model in sequence, and receive multiple candidate reply texts returned in sequence by the general language model; wherein the multiple second prompt words all include the question text, the reference data and information indicating the reply text generated for the question text, and the arrangement order of the reference data in the multiple second prompt words is different.

5. The method according to claim 3, characterized in that: The step of determining a response text corresponding to the question text according to the plurality of candidate response texts includes: Generate a third prompt word; wherein the third prompt word includes the multiple candidate reply texts and information for indicating the evaluation of the multiple candidate reply texts; Sending the third prompt word to a universal language model, and receiving a target reply text returned by the universal language model; According to the target reply text, a reply text corresponding to the question text is determined.

6. The method according to claim 5, characterized in that The target reply text is generated based on the second prompt word, and determining the reply text corresponding to the question text according to the target reply text includes: Generate a fourth prompt word; wherein the fourth prompt word includes the target reply text, the second prompt word and information for instructing to polish the target reply text; The fourth prompt word is sent to the general language model, and a reply text returned by the general language model is received.

7. The method according to claim 1, characterized in that The step of determining at least one question text related to the target knowledge domain from the sample data comprises: Generate a fifth prompt word; wherein the fifth prompt word includes the sample data, the question type related to the target knowledge field and information for instructing to generate a question text; The fifth prompt word is sent to a general language model, and at least one question text related to the target knowledge domain returned by the general language model is received.

8. The method according to any one of claims 1 to 7, characterized in that: Determining reference data related to the question text from the knowledge base includes: Determine the similarity between the question text and the candidate data in the knowledge base; The candidate data whose similarity meets the set conditions is determined as reference data.

9. A model training device, characterized in that: The device comprises: An acquisition module is used to acquire sample data related to the target knowledge domain from the knowledge base; A first determination module, configured to determine at least one question text related to the target knowledge domain from the sample data; The second determination module is used to perform the following steps for each question text in the at least one question text: determining reference data related to the question text from the knowledge base; determining a reply text corresponding to the question text based on the question text and the reference data; The training module is used to use each of the question texts and the reply texts corresponding to the question texts as training data to train the target language model.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory; The processor is configured to execute instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The method comprises instructions, wherein the instructions instruct an electronic device to execute the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises computer readable instructions for implementing the method according to any one of claims 1 to 8.

Images