Large language model training method and device, computer equipment and storage medium

Abstract

The invention relates to a big language model training method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring original answer text data; removing the first answer text data in the original answer text data to obtain second answer text data; the first answer text data is an answer text comprising a target language unit; the target language unit is a language unit used for expressing answer refusing semantics; removing third answer text data in the second answer text data to obtain fourth answer text data; the third answer text data is an answer text which is recognized through a pre-trained first large language model and contains an answer refusal intention; and training the to-be-trained second large language model by adopting the training sample data containing the fourth answer text data to obtain a target large language model. By adopting the method, the problem that an answer text output by a large language model when facing a question of a user has limitation can be solved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method, apparatus, computer device, computer-readable storage medium and computer program product for training a large language model. Background Art

[0002] With the development of artificial intelligence technology, natural language processing (NLP) technology based on large language models (LLM) has been widely used in dialogue generation, text creation, intelligent question and answer and other fields. Large language models are a natural language processing model based on deep learning. They are usually built through a neural network architecture and trained using a large-scale corpus. They have strong language understanding and generation capabilities. However, existing large language models are usually set with specific values ​​by their developers. When faced with some questions input by users, the output answer text is often limited due to different values. Summary of the invention

[0003] Based on this, it is necessary to provide a large language model training method, apparatus, computer equipment, computer-readable storage medium and computer program product that can solve the problem that the answer text output by the large language model when facing user questions has limitations, in response to the above technical problems.

[0004] In a first aspect, the present application provides a method for training a large language model, comprising:

[0005] Get the original answer text data;

[0006] Removing the first answer text data from the original answer text data to obtain the second answer text data; the first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer;

[0007] Removing the third answer text data from the second answer text data to obtain fourth answer text data; the third answer text data is an answer text containing an intention to refuse to answer identified by the pre-trained first language model;

[0008] The second large language model to be trained is trained using the training sample data including the fourth answer text data to obtain a target large language model; the target large language model is used to output an answer text that does not reject the user's question.

[0009] In one embodiment, before removing the third answer text data from the second answer text data to obtain the fourth answer text data, the method further includes:

[0010] Constructing a text screening instruction; the text screening instruction includes the second answer text data and a text screening prompt word; the text screening prompt word is used to instruct the first language model to recognize the answer text containing the intention of refusing to answer in the second answer text data;

[0011] The text screening instruction is input into the first language model to obtain the third answer text data.

[0012] In one embodiment, the step of training the second large language model to be trained using the training sample data including the fourth answer text data to obtain the target large language model comprises:

[0013] Using the training sample data including the fourth answer text data to train the second language model to be trained, to obtain a pre-trained third language model;

[0014] The pre-trained third language model is trained using dialogue data generated based on the data generation model to obtain the target large language model; the model parameter scale of the data generation model is greater than the model parameter scale of the pre-trained third language model.

[0015] In one embodiment, the data generation model includes a first data generation model and a second data generation model, and the first data generation model and the second data generation model are the same data generation model; before training the pre-trained third language model using the conversation data generated based on the data generation model, the method further includes:

[0016] At least one set of dialogue data is generated based on the mutual dialogue content between the first data generation model and the second data generation model.

[0017] In one embodiment, the step of training the pre-trained third language model using the conversation data generated based on the data generation model includes:

[0018] Inputting the dialogue data generated based on the data generation model into the evaluation model, and scoring each group of the dialogue data based on at least one text quality evaluation dimension by the evaluation model to obtain a scoring result;

[0019] Determining, in each group of the conversation data, the conversation data whose scoring result is higher than the scoring threshold as target conversation data;

[0020] The pre-trained large language model is trained using the target dialogue data.

[0021] In one embodiment, before the second language model to be trained is trained using the training sample data including the fourth answer text data, the method further includes:

[0022] The training sample data is generated according to the preset text data, the business scenario data and the fourth answer text data.

[0023] In a second aspect, the present application also provides a large language model training device, comprising:

[0024] An acquisition module, used to acquire original answer text data;

[0025] A first removal module is used to remove the first answer text data in the original answer text data to obtain the second answer text data; the first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer;

[0026] A second removal module is used to remove the third answer text data from the second answer text data to obtain fourth answer text data; the third answer text data is the answer text containing the intention of refusing to answer identified by the pre-trained first language model;

[0027] The training module is used to train the second language model to be trained using the training sample data containing the fourth answer text data to obtain a target language model; the target language model is used to output an answer text that does not reject the user's question.

[0028] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0030] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0031] The training method, apparatus, computer device, computer-readable storage medium and computer program product of the above-mentioned large language model obtain original answer text data; remove the first answer text data in the original answer text data to obtain second answer text data, wherein the first answer text data is an answer text including a target language unit, wherein the target language unit is a language unit used to express the semantics of refusing to answer; remove the third answer text data in the second answer text data to obtain fourth answer text data, wherein the third answer text data is an answer text containing the intention of refusing to answer identified by the pre-trained first large language model; train the second large language model to be trained using training sample data containing the fourth answer text data to obtain a target large language model, wherein the target large language model is used to output an answer text that does not reject the user's question. By screening the original answer text data in stages, answer text data containing specific semantic expressions and specific intentions are eliminated, so that the data retained in the training data set focuses more on answer diversity and problem-solving capabilities. This enables the trained target large language model to avoid outputting answer texts that reject user questions when faced with user questions, thereby improving the coverage, diversity and flexibility of the large language model's answer content and solving the problem that the answer text output by the large language model when faced with user questions is limited. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 A diagram of an application environment of a large language model training method in one embodiment;

[0034] Figure 2 A schematic diagram of a flow chart of a method for training a large language model in an embodiment;

[0035] Figure 3 is a flowchart of a method for training a large language model in another embodiment;

[0036] Figure 4 is a structural block diagram of a large language model training device in one embodiment;

[0037] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] The training method of the large language model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains the original answer text data; the terminal 102 removes the first answer text data in the original answer text data to obtain the second answer text data; the first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer; the terminal 102 removes the third answer text data in the second answer text data to obtain the fourth answer text data; the third answer text data is an answer text containing the intention of refusing to answer identified by the pre-trained first large language model; the terminal 102 uses the training sample data containing the fourth answer text data to train the second large language model to be trained to obtain the target large language model; the target large language model is used to output the answer text that does not reject the user's question. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. Head-mounted devices may be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides cloud computing services.

[0040] In an exemplary embodiment, Figure 2 As shown, a method for training a large language model is provided, and the method is applied to Figure 1 The terminal 102 in FIG. 1 is used as an example for explanation, including:

[0041] Step S202, obtaining original answer text data.

[0042] Among them, the original answer text data may be composed of a large amount of answer data from multiple fields and scenarios, and may include various forms of answers after users ask questions, covering a wide range of semantic content and expressions.

[0043] Exemplarily, the original answer text data can be obtained from multiple sources, such as dialogue system logs, open corpora, social media comments, forum questions and answers, knowledge base content, etc.

[0044] Step S204, removing the first answer text data from the original answer text data to obtain the second answer text data.

[0045] The first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer.

[0046] In a specific implementation, the target language unit may be a language element in the text that directly conveys the semantics of refusing to answer. The target language unit includes, but is not limited to, specific words (such as "sorry", "can't", "can't", etc.), phrases (such as "I'm sorry, I can't answer"), and sentence patterns (such as "I'm not sure about the answer to this question"). Optionally, the target language unit may be explicitly defined by rules or templates for easy identification and matching.

[0047] In a specific implementation, the first answer text data is an answer text including the target language unit, which can be expressed as an explicit or identifiable rejection semantic expression. For example, the first answer text data can include an explicit negative expression, such as "I can't help you"; the first answer text data can also include rejection modal particles, such as "I'm sorry" and "I'm afraid".

[0048] In a specific implementation, the first answer text data in the original answer text data may be firstly screened out, and then the first answer text data in the original answer text data may be removed to obtain the second answer text data.

[0049] Exemplarily, the method of filtering out the first answer text data in the original answer text data may include filtering based on rule-based judgment, for example, detecting the target language unit using keyword matching, phrase templates or regular expressions.

[0050] The second answer text data is the remaining answer text after removing the first answer text data from the original answer text data.

[0051] This step is the first round of screening of the original answer text data, removing the answer text data containing specific target language units. These target language units are language content that conveys the semantics of refusing answers, and their existence will affect the quality of subsequent training sample data and the generation ability of the target large language model. By removing the first answer text data from the original answer text data and obtaining the second answer text data, the foundation is laid for the subsequent screening steps and model training, ensuring that the final target large language model can focus more on generating high-quality answer texts without the semantics of refusing answers.

[0052] Step S206, removing the third answer text data from the second answer text data to obtain fourth answer text data.

[0053] Among them, the third answer text data is the answer text containing the intention of refusing to answer, which is recognized by the pre-trained first language model.

[0054] It should be noted that although the first answer text data containing the target language unit (explicit rejection semantics) has been removed from the second answer text data, there may still be some texts that do not explicitly express rejection semantics but have the intention of rejecting the answer. These texts often have implicit negation and evasive expressions, which are not easy to be screened and identified by simple rules.

[0055] Among them, the first language model has a strong natural language understanding ability, and can identify whether the text contains the intention of refusing to answer through context and semantic analysis. The first language model can identify the third answer text data containing the intention of refusing to answer from the second answer text data.

[0056] The third answer text data is the part of the second answer text data that is identified as having the intention of refusing to answer after being analyzed by the first language model. The third answer text data may express an implicit or indirect intention of rejection, for example, "This may be beyond my ability", although no explicit rejection words are used, but the overall semantics convey the intention of avoiding or denying the answer.

[0057] The fourth answer text data is the answer text remaining after removing the third answer text data from the second answer text data. The fourth answer text data contains neither explicit rejection expression nor implicit rejection intention, and is more suitable for the training requirements of the target large language model.

[0058] This step uses the pre-trained first large language model to perform deep semantic analysis on the second answer text data, eliminates the third answer text data that contains the intention of rejecting the answer, further improves the accuracy and pertinence of data screening, and generates the fourth answer text data that meets the target training requirements, so that the training sample data is more focused on the ability to generate positive and constructive answers, avoids learning negative or evasive patterns during the training process, and ultimately ensures that the target large language model can generate high-quality answer texts without rejection intentions.

[0059] Step S208: training the second large language model to be trained using the training sample data including the fourth answer text data to obtain a target large language model.

[0060] Among them, the target large language model is used to output the answer text that does not reject the user's question.

[0061] Among them, the second largest language model is the initial, untrained large language model. The second largest language model can be trained through self-supervised learning, using a large amount of unlabeled text data to learn the grammar, semantics, and contextual relationships of the language by predicting the next word or filling in missing words.

[0062] In a specific implementation, the fourth answer text data of the training sample data can be segmented to obtain the first half of the training data and the second half of the training data; the second half of the training data is used as the true value, and the second half of the predicted data based on the first half of the training data is used as the predicted value; and according to the difference between the true value and the predicted value, the target large language model is trained.

[0063] In the above-mentioned large language model training method, original answer text data is obtained; the first answer text data in the original answer text data is removed to obtain the second answer text data, wherein the first answer text data is an answer text including a target language unit, wherein the target language unit is a language unit used to express the semantics of refusing to answer; the third answer text data in the second answer text data is removed to obtain the fourth answer text data, wherein the third answer text data is an answer text containing the intention of refusing to answer identified by the pre-trained first large language model; the second large language model to be trained is trained using the training sample data containing the fourth answer text data to obtain the target large language model, wherein the target large language model is used to output the answer text that does not reject the user's question. By screening the original answer text data in stages, the answer text data containing specific semantic expressions and specific intentions are eliminated, so that the data retained in the training data set pays more attention to the ability to diversify answers and solve problems, so that the trained target large language model can avoid outputting the answer text that rejects the user's question when facing the user's question, thereby improving the coverage, diversity and flexibility of the large language model's answer content, and solving the problem that the answer text output by the large language model when facing the user's question has limitations.

[0064] In another embodiment, before removing the third answer text data from the second answer text data to obtain the fourth answer text data, it also includes: constructing a text filtering instruction; the text filtering instruction includes the second answer text data and a text filtering prompt word; the text filtering prompt word is used to instruct the first large language model to recognize the answer text containing the intention of refusing to answer in the second answer text data; the text filtering instruction is input into the first large language model to obtain the third answer text data.

[0065] Among them, the text screening prompt words are used to provide clear task instructions for the first language model, clearly convey the requirements of the screening task, and help the first language model understand the screening task that needs to be completed. The first language model can analyze and judge the semantics in the input text through its semantic understanding ability, and identify which answer texts contain the intention of refusing to answer. Exemplarily, the text screening prompt words can be "Please identify all texts containing the intention of refusing to answer", "Please identify all texts containing evasive expressions", etc.

[0066] In the specific implementation, the constructed text filtering instruction is input into the first large language model. The text filtering instruction includes the second answer text data and text filtering prompt words. The first large language model extracts the answer text containing the intention of refusing to answer from the second answer text data according to the guidance of the text filtering prompt words as the third answer text data.

[0067] The technical solution of this embodiment refines the screening target of the first language model through clear language expressions using text screening prompt words, so that the model can accurately locate texts containing rejection intentions; constructs text screening instructions to structure the screening tasks, reduces the ambiguity of the model in the screening tasks, and improves the efficiency of the screening process; by adjusting the text screening prompt words, the screening criteria can be flexibly modified according to the needs of different scenarios, thereby enhancing the adaptability of the method; utilizes the semantic understanding ability of the first language model, combined with clear screening prompts, eliminates possible texts with ambiguous expressions or implicit rejection intentions, and improves the accuracy and pertinence of data screening.

[0068] In another embodiment, the second largest language model to be trained is trained with training sample data including the fourth answer text data to obtain a target large language model, including: training the second largest language model to be trained with training sample data including the fourth answer text data to obtain a pre-trained third largest language model; training the pre-trained third largest language model with dialogue data generated based on the data generation model to obtain the target large language model; the model parameter scale of the data generation model is greater than the model parameter scale of the pre-trained third largest language model.

[0069] Among them, the pre-trained third language model is an intermediate model trained using training sample data containing the fourth answer text data, and has strong language generation capabilities and the ability to accept user questions.

[0070] Among them, the data generation model can be a large language model, but the parameter scale of the data generation model is larger than the parameter scale of the pre-trained third largest language model. It should be noted that due to the large parameter scale of the data generation model, it is not suitable for use in business lines, but it can be used to provide rich, high-quality dialogue data materials to train the target large language model. It can be seen that during the training process, a large language model with a large parameter scale can be used to provide training data to train a large language model with a smaller parameter scale, thereby improving the flexibility of the training process.

[0071] Among them, the dialogue data generated by the data generation model covers a variety of scenarios, contexts and expressions. The dialogue data generated by the data generation model is used to further fine-tune the pre-trained third language model to obtain the target language model, which can further improve the diversified expression ability and dialogue adaptation ability of the target language model. Optionally, the dialogue data generated by the data generation model can also be customized according to the target scenario to make it more suitable for the actual application needs of the target language model.

[0072] The technical solution of this embodiment is divided into a two-stage training process. First, the fourth answer text data is used to pre-train the second largest language model to be trained to obtain a pre-trained third largest language model, which can avoid outputting answer texts that reject user questions when facing user questions; then, high-quality dialogue data is generated through a large-parameter-scale data generation model, and the pre-trained third largest language model is further fine-tuned. This can improve the adaptability of the target large language model to complex contexts and the coherence and semantic relevance in multiple rounds of dialogues, and can handle more complex and diverse user inputs. This staged training method makes full use of high-quality screening data and generation data to obtain a target large language model that is efficient, has excellent performance, and is suitable for multiple scenarios.

[0073] In another embodiment, the data generation model includes a first data generation model and a second data generation model, and the first data generation model and the second data generation model are the same data generation model; before using the conversation data generated based on the data generation model to train the pre-trained third language model, it also includes: generating at least one set of conversation data based on the mutual conversation content between the first data generation model and the second data generation model.

[0074] The first data generation model and the second data generation model are data generation models with the same architecture, parameter scale and language generation capability. The dialogue between the two data generation models generates logically coherent and diverse dialogue data by simulating real multi-round dialogue scenarios.

[0075] In the specific implementation, in the process of mutual dialogue, the first data generation model and the second data generation model take turns as the active party and the responding party of the dialogue. The dialogue between the models may start from a predefined starting question or scenario, and then gradually generate complex dialogue content through multiple rounds of interaction. Since both data generation models have strong semantic understanding capabilities, the generated dialogue is highly coherent in logic and semantics. The language generation capabilities of the two data generation models make the dialogue content rich and diverse in language style and semantic expression. The dialogue scene can be designed as needed, so that the generated dialogue data is more in line with the actual application needs. The multiple rounds of interaction between the two data generation models will generate several groups of dialogue data, each group of dialogue data can contain the context and reply content of multiple rounds of dialogue, simulating the real interaction between the user and the virtual character. For example, let the first data generation model play the role of the user and the second data generation model play the role of the virtual character. The first data generation model can be requested to output the conversation content of the user role, and then the conversation content of the user role is input into the second data generation model, and the virtual character's reply is output through the second data generation model. Then the above process continues until the set dialogue round is reached.

[0076] Based on the generated dialogue data, further training the pre-trained third language model can enhance its learning ability of dialogue logic and language expression.

[0077] In the specific implementation, the conversation data is used to train the pre-trained third language model. It can be a supervised machine learning task. Each training data is a group of conversation data. The second to last content of each group of conversation data is the user's input, and the penultimate content is the reply of the virtual character. In the training process, the first to the second to last content in the group of conversation data can be used as input, and the penultimate content can be used as a label. First, a predicted content is obtained based on the input content, and then the generated predicted content is compared with the label. The difference between the two is minimized to adjust the model parameters. For example, the difference between the two can be measured by the cross entropy loss function. Finally, the target large language model is obtained.

[0078] Through rich conversation data, the model can learn more complex semantic associations and context adaptation capabilities, improve the logic and coherence of the model in multi-round conversations, and optimize its adaptability to complex scenarios.

[0079] The technical solution of this embodiment generates at least one set of multi-round dialogue data through the dialogue between two identical data generation models. These data are rich in content, semantically coherent and diverse, and are used to further train the pre-trained third language model, significantly improving the language generation ability of the target large language model and its adaptability to complex dialogue scenarios.

[0080] In another embodiment, a pre-trained third language model is trained using conversation data generated based on a data generation model, including: inputting the conversation data generated based on the data generation model into an evaluation model, scoring each group of conversation data based on at least one text quality evaluation dimension through the evaluation model to obtain a scoring result; determining, in each group of conversation data, conversation data with a scoring result higher than a scoring threshold as target conversation data; and training the pre-trained large language model using the target conversation data.

[0081] The evaluation model can be an artificial intelligence model used to evaluate the quality of text data, which can be designed based on a large language model or a machine learning model in a specific field. The evaluation model is used to quantitatively score the various characteristics of the input text and assist in screening out high-quality training data.

[0082] Optionally, the text quality evaluation dimensions may include five dimensions: role compliance, logical coherence, initiative, repetition, and format compliance. Among them, "role compliance" can be used to evaluate whether the text meets the identity, tone, and behavior requirements of each role in the dialogue; "logical coherence" can be used to evaluate whether the text is logical, especially whether the context of multiple rounds of dialogue is coherent and reasonable; "initiative" can be used to evaluate whether the text actively promotes the development of the dialogue, rather than passively or mechanically answering questions; "repetition" can be used to evaluate whether there is repeated content in the text, especially whether the text generated in multiple rounds of dialogue avoids language repetition and content redundancy; "format compliance" can be used to evaluate whether the text meets the format requirements in a specific task or scenario, such as whether the paragraph structure of the dialogue and the use of punctuation are standardized.

[0083] The evaluation model can score each group of dialogue data based on these text quality evaluation dimensions, and determine the dialogue data with a score higher than the scoring threshold as the target dialogue data, and then use the target dialogue data to train the pre-trained large language model.

[0084] The technical solution of this embodiment uses an evaluation model to perform quality scoring on the generated dialogue data, and only retains the target dialogue data with a scoring result higher than a threshold, thereby screening out high-quality data with semantic coherence, clear logic, and fluent language, and avoiding interference of low-quality data on model training; and, based on specific text quality evaluation dimensions, the data is screened to ensure that the training data is more in line with the target scenario requirements, and ultimately the model is more suitable for specific task applications; using the target dialogue data screened by the evaluation model to train the pre-trained third language model can improve the model's semantic understanding ability and context adaptation ability, so that it exhibits higher logic, coherence, and task relevance in dialogue generation.

[0085] In other embodiments, before using the training sample data including the fourth answer text data to train the second language model to be trained, it also includes: generating training sample data according to preset text data, business scenario data and the fourth answer text data.

[0086] Among them, the preset text data is sample data used to improve the creative ability of the target large language model. For example, the preset text data can be pre-written novel data. The pre-written novel data can contain rich language expressions, diverse scenes, complex semantic relationships, and multi-character dialogues, which helps the model learn more diverse language styles and semantic structures.

[0087] Among them, business scenario data is used to improve the business-specific capabilities of the target large language model. Business scenario data can include corpus data designed specifically for specific business needs, such as customer service conversations, intelligent question and answer logs, or question and answer data in specific fields (such as legal, medical and other business scenarios), which helps to enhance the model's adaptability in specific business scenarios and make the text it generates more in line with specific scenario needs.

[0088] Among them, the fourth answer text data is used to improve the non-rejection ability of the target large language model. The fourth answer text data is high-quality answer text data retained after multiple rounds of screening. It does not contain rejection semantics or rejection intent. It is the core quality standard to ensure the training data set and is used to guide the model to generate positive answer texts without rejection intent.

[0089] Then, the data from the above three data sources are integrated to generate the final training sample data.

[0090] The technical solution of this embodiment generates diversified and high-quality training sample data by integrating preset text data, business scenario data and fourth answer text data, which not only covers a wide range of language expressions, but also can meet the task requirements of specific scenarios, ultimately improving the model's language generation ability, task adaptability and actual application effect, while ensuring that the generated answer text meets the core goal of no rejection intention.

[0091] In another embodiment, Figure 3 As shown, a method for training a large language model is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0092] S302, constructing a text filtering instruction.

[0093] The text screening instruction includes the second answer text data and a text screening prompt word; the text screening prompt word is used to instruct the first language model to recognize the answer text containing the intention of refusing to answer in the second answer text data.

[0094] S304: Input the text screening instruction into the first language model to obtain third answer text data.

[0095] S306, removing the first answer text data from the original answer text data to obtain second answer text data.

[0096] The first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer.

[0097] S308, removing the third answer text data from the second answer text data to obtain fourth answer text data.

[0098] The third answer text data is the answer text containing the intention of refusing to answer, which is recognized by the pre-trained first language model.

[0099] S310, generating the training sample data according to the preset text data, the business scenario data and the fourth answer text data.

[0100] S312, using the training sample data including the fourth answer text data to train the second language model to be trained, to obtain a pre-trained third language model.

[0101] S314: Generate at least one set of dialogue data based on the dialogue content between the first data generation model and the second data generation model.

[0102] The model parameter scale of the data generation model is greater than the model parameter scale of the pre-trained third language model. The data generation model includes a first data generation model and a second data generation model, and the first data generation model and the second data generation model are the same data generation model.

[0103] S316: Input the dialogue data generated based on the data generation model into the evaluation model, and score each group of dialogue data based on at least one text quality evaluation dimension through the evaluation model to obtain a scoring result.

[0104] S318: Determine, in each group of conversation data, conversation data with a score result higher than a score threshold as target conversation data.

[0105] S320, using the target conversation data to train the pre-trained large language model to obtain a target large language model.

[0106] The target large language model is used to output answer text that does not reject the user's question.

[0107] It should be noted that the specific limitations of the above steps can refer to the specific limitations of the training method for a large language model above.

[0108] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiment of the present application also provides a large language model training device for implementing the large language model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more large language model training devices provided below can refer to the limitations of the large language model training method above, and will not be repeated here.

[0110] In an exemplary embodiment, Figure 4 As shown, a training device for a large language model is provided, comprising:

[0111] The acquisition module 410 is used to acquire original answer text data.

[0112] The first removal module 420 is used to remove the first answer text data in the original answer text data to obtain the second answer text data; the first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer.

[0113] The second removal module 430 is used to remove the third answer text data from the second answer text data to obtain fourth answer text data; the third answer text data is the answer text containing the intention of refusing to answer identified by the pre-trained first language model.

[0114] The training module 440 is used to train the second large language model to be trained using the training sample data containing the fourth answer text data to obtain a target large language model; the target large language model is used to output an answer text that does not reject the user's question.

[0115] In one embodiment, the second removal module 430 is specifically used to construct a text filtering instruction; the text filtering instruction includes second answer text data and a text filtering prompt word; the text filtering prompt word is used to instruct the first large language model to recognize the answer text containing the intention of refusing to answer in the second answer text data; the text filtering instruction is input into the first large language model to obtain the third answer text data.

[0116] In one embodiment, the training module 440 is specifically used to train the second largest language model to be trained using the training sample data containing the fourth answer text data to obtain a pre-trained third largest language model; use the dialogue data generated based on the data generation model to train the pre-trained third largest language model to obtain the target large language model; the model parameter scale of the data generation model is greater than the model parameter scale of the pre-trained third largest language model.

[0117] In one of the embodiments, the data generation model includes a first data generation model and a second data generation model, and the first data generation model and the second data generation model are the same data generation model;

[0118] In one embodiment, the training module 440 is specifically configured to generate at least one set of dialogue data based on the dialogue content between the first data generation model and the second data generation model.

[0119] In one embodiment, the training module 440 is specifically used to input the conversation data generated based on the data generation model into the evaluation model, and score each group of the conversation data based on at least one text quality evaluation dimension through the evaluation model to obtain a scoring result; in each group of the conversation data, the conversation data with the scoring result higher than the scoring threshold is determined as the target conversation data; and the pre-trained large language model is trained using the target conversation data.

[0120] In one embodiment, the training module 440 is specifically configured to generate the training sample data according to preset text data, business scenario data and the fourth answer text data.

[0121] Each module in the training device of the large language model can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0122] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a training method for a large language model is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0123] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0126] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0128] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0130] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for training a large language model, characterized in that: The method comprises: Get the original answer text data; Removing the first answer text data from the original answer text data to obtain the second answer text data; the first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer; Removing the third answer text data from the second answer text data to obtain fourth answer text data; the third answer text data is an answer text containing an intention to refuse to answer identified by the pre-trained first language model; The second large language model to be trained is trained using the training sample data including the fourth answer text data to obtain a target large language model; the target large language model is used to output an answer text that does not reject the user's question.

2. The method according to claim 1, characterized in that Before removing the third answer text data from the second answer text data to obtain the fourth answer text data, the method further includes: Constructing a text screening instruction; the text screening instruction includes the second answer text data and a text screening prompt word; the text screening prompt word is used to instruct the first language model to recognize the answer text containing the intention of refusing to answer in the second answer text data; The text screening instruction is input into the first language model to obtain the third answer text data.

3. The method according to claim 1, characterized in that The training of the second language model to be trained using the training sample data including the fourth answer text data to obtain the target language model comprises: Using the training sample data including the fourth answer text data to train the second language model to be trained, to obtain a pre-trained third language model; The pre-trained third language model is trained using dialogue data generated based on the data generation model to obtain the target large language model; the model parameter scale of the data generation model is greater than the model parameter scale of the pre-trained third language model.

4. The method according to claim 3, characterized in that The data generation model includes a first data generation model and a second data generation model, and the first data generation model and the second data generation model are the same data generation model; Before training the pre-trained third language model using the dialogue data generated based on the data generation model, the method further includes: At least one set of dialogue data is generated based on the mutual dialogue content between the first data generation model and the second data generation model.

5. The method according to claim 3, characterized in that: The method of training the pre-trained third language model using the dialogue data generated based on the data generation model includes: Inputting the dialogue data generated based on the data generation model into the evaluation model, and scoring each group of the dialogue data based on at least one text quality evaluation dimension by the evaluation model to obtain a scoring result; Determining, in each group of the conversation data, the conversation data whose scoring result is higher than the scoring threshold as target conversation data; The pre-trained large language model is trained using the target dialogue data.

6. The method according to claim 1, characterized in that Before the second language model to be trained is trained using the training sample data including the fourth answer text data, the method further includes: The training sample data is generated according to the preset text data, the business scenario data and the fourth answer text data.

7. A large language model training device, characterized in that: The device comprises: An acquisition module, used to acquire original answer text data; A first removal module is used to remove the first answer text data in the original answer text data to obtain the second answer text data; the first answer text data is an answer text including a target language unit; the target language unit is a language unit used to express the semantics of refusing to answer; A second removal module is used to remove the third answer text data from the second answer text data to obtain fourth answer text data; the third answer text data is the answer text containing the intention of refusing to answer identified by the pre-trained first language model; The training module is used to train the second language model to be trained using the training sample data containing the fourth answer text data to obtain a target language model; the target language model is used to output an answer text that does not reject the user's question.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images