Language model training method, natural language processing method and device

Abstract

The invention provides a language model training method and a natural language processing method and device. The language model training method comprises the steps that for each target question in an original data set, multiple times of reasoning are conducted on the target question through a language model, multiple pieces of answer data are obtained, and the temperature values of the language model are different during reasoning each time; screening out first answer data of which the first repetition degree is greater than a first threshold value and second answer data of which the second repetition degree is smaller than a second threshold value from the multiple pieces of answer data; taking the target question, the first answer data and the second answer data as preference data, and adding the preference data to a preference data set; and training a language model by using the preference data set.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, and in particular to a language model training method, a natural language processing method and a device. Background Art

[0002] AIGC (Artificial Intelligence Generated Content) refers to the technology based on artificial intelligence technical methods such as generative adversarial networks and large pre-trained models, which generates relevant content with appropriate generalization capabilities through learning and recognition of existing data. AIGC has been very popular in recent years, and various large models for AIGC have emerged.

[0003] Existing large models have high output repetition rates, low model performance, and poor user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, this specification provides a language model training method, a natural language processing method and a device.

[0005] According to a first aspect of an embodiment of the present specification, a method for training a language model is provided, the method comprising: for each target question in an original data set, using a language model to perform multiple inferences on the target question to obtain multiple answer data, wherein the temperature value of the language model is different in each inference; screening out first answer data having a first repetition greater than a first threshold and second answer data having a second repetition less than a second threshold from the multiple answer data; adding the target question, the first answer data and the second answer data as preference data to a preference data set; and training the language model using the preference data set.

[0006] According to a second aspect of an embodiment of this specification, a natural language processing method is provided, comprising: inputting a question text into a language model to obtain answer data, wherein the language model is trained according to the language model training method of an embodiment of this specification.

[0007] According to a third aspect of an embodiment of the present specification, a language model training device is provided, comprising: an inference module, for performing multiple inferences on each target question in an original data set using a language model to obtain multiple answer data, wherein the temperature value of the language model is different in each inference; a screening module, for screening out first answer data having a first degree of repetition greater than a first threshold and second answer data having a second degree of repetition less than a second threshold from the multiple answer data; an adding module, for adding the target question, the first answer data and the second answer data as preference data to a preference data set; and a training module, for training the language model using the preference data set.

[0008] According to the fourth aspect of the embodiments of this specification, a natural language processing device is provided, including: an input module for inputting a question text into a language model to obtain answer data, wherein the language model is trained according to the language model training method of the embodiments of this specification.

[0009] According to a fifth aspect of the embodiments of this specification, a language model training device is provided, including:

[0010] processor;

[0011] a memory for storing processor-executable instructions;

[0012] The processor is configured to execute the language model training method of the embodiment of this specification and / or the natural language processing method of the embodiment of this specification.

[0013] The technical solutions provided by the embodiments of this specification may have the following beneficial effects:

[0014] In the embodiments of this specification, the language model can be automatically trained to save labor costs and improve production efficiency. In addition, the training method can reduce the repetition rate of model answers, improve model performance, and thus enhance the user experience of the model.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0017] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.

[0018] Figure 2It is a flowchart of a method for training a language model according to an exemplary embodiment of this specification.

[0019] Figure 3 The figure is a flowchart of a method for training a language model according to another exemplary embodiment.

[0020] Figure 4A is a schematic diagram of a first duplication checking mechanism according to an exemplary embodiment.

[0021] Figure 4B is a schematic diagram of a second duplication checking mechanism according to an exemplary embodiment.

[0022] Figure 5 It is a hardware structure diagram of the computer device where the language model training device of the embodiment of this specification is located.

[0023] Figure 6 It is a block diagram of a language model training device shown in this specification according to an exemplary embodiment.

[0024] Figure 7 It is a block diagram of a natural language processing device shown in this specification according to an exemplary embodiment. DETAILED DESCRIPTION

[0025] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Instead, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0026] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. The singular forms "a", "the" and "the" used in this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0028] Next, the embodiments of this specification are described in detail.

[0029] The following combination Figure 1 The system architecture of the training method, natural language processing method and device to which the language model can be applied in the embodiments of this specification is described. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of this specification can be applied, to help those skilled in the art understand the technical content of this specification, but it does not mean that the embodiments of this specification cannot be used in other devices, systems, environments or scenarios.

[0030] Figure 1 It is a schematic diagram of a system architecture shown in this specification according to an exemplary embodiment.

[0031] like Figure 1 As shown, the system architecture may include, for example, a terminal device, a network and a server. The network is used to provide a medium for a communication link between the terminal device and the server. The network may include various connection types, such as wired and / or wireless communication links, etc.

[0032] Users can use terminal devices to interact with servers through the network to receive or send messages, etc. Various communication client applications can be installed on the terminal devices, such as service access clients, web browser applications, search applications, instant messaging tools, email clients and / or social platform software.

[0034] The terminal device may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0035] The server may be a server that provides various services, such as a background management server that provides support for the content browsed by the user using the terminal device. The background management server may analyze and process the received data such as the user request, and feed back the processing results (such as the web page, information, or data obtained or generated according to the user request) to the terminal device.

[0036] The following is a detailed description of the language model training method provided in the embodiments of this specification. Figure 2 As shown, Figure 2 It is a flowchart of a language model training method shown in this specification according to an exemplary embodiment. The method can be applied to the server shown above. The language model training method provided in the embodiment of this specification may include the following steps.

[0037] In step 210, for each target question in the original data set, the language model is used to perform multiple inferences on the target question to obtain multiple answer data, and the temperature value of the language model is different during each inference.

[0038] According to the embodiments of this specification, different temperature values ​​can be set for the language model at each inference, and the temperature value is a hyperparameter that adjusts the output distribution of the model. By setting different temperature values, the language model can generate different answer information, thereby improving the diversity of the model answer information.

[0039] According to an embodiment of the present specification, the original data set may include one or more target questions. The original data set may be traversed, and each target question in the original data set may be input into a language model with different temperature values ​​to obtain multiple answer data corresponding to the target question.

[0040] In step 220, first answer data having a first degree of repetition greater than a first threshold and second answer data having a second degree of repetition less than a second threshold are screened out from the plurality of answer data.

[0041] According to an embodiment of the present specification, the repetition of answer data can be used to indicate the degree of repetition of the answer data in multiple answer data. The first repetition can be, for example, the repetition calculated using a first algorithm. The second repetition can be, for example, the repetition calculated using a second algorithm. The first threshold and the second threshold can be set according to actual needs.

[0042] In step 230 , the target question, the first answer data, and the second answer data are added to the preference data set as preference data.

[0043] According to an embodiment of the present specification, the preference data may include a prompt field, a chosen field, and a rejected field. The chosen field may be used to store answer data expected by the model. The rejected field is used to store answer data not expected by the model. Based on this, the target question may be used as the value of the prompt field, the first answer data may be used as the value of the rejected field, and the second answer data may be used as the value of the chosen field.

[0044] In step 240, the language model is trained using the preference dataset.

[0045] According to an embodiment of the present specification, for example, a loss value may be calculated based on each preference data in the preference data set and an objective function, and the parameters of the language model may be adjusted based on the loss value.

[0046] According to an embodiment of the present specification, after a language model is trained using a preference data set, the language model can be used for natural language processing. For example, a question text can be input into the language model to obtain answer data corresponding to the question text.

[0047] According to the language model training method of the embodiment of this specification, the language model can be automatically trained to save labor costs and improve production efficiency. In addition, based on the repetition of the preference data, the preference data is used to train the language model, which can alleviate the repetition rate of the model answer, improve the model performance, and thus enhance the user experience of the model.

[0048] Optionally, filtering out first answer data having a repetition degree greater than a first threshold from multiple answer data may, for example, include: performing word segmentation processing on each answer data in the multiple answer data to obtain a word sequence for each answer data; extracting at least one first text fragment from the word sequence of each answer data to obtain a first text fragment sequence, the first text fragment including N1 words that appear consecutively in the word sequence, N1 being a positive integer, and the value of N1 may be set according to actual needs; merging the first text fragment sequences of multiple answer data to obtain a first total set; for each answer data, determining a first repetition degree of the answer data according to the number of times each first text fragment in the first text fragment sequence of the answer data appears repeatedly in the first total set; and determining the answer data having a first repetition degree greater than the first threshold among the multiple answer data as the first answer data.

[0049] For example, N1 can be 10, and the first threshold can be 5. Exemplarily, in this embodiment, 10-grams can be extracted from the word sequence of each answer data to obtain a 10-gram sequence, that is, a first text fragment sequence, and each 10-gram in the 10-gram sequence is the first text fragment. The 10-gram sequences of multiple answer data are merged into a set, that is, the first total set. For each answer data, the number of times each 10-gram in the 10-gram sequence of the answer data is repeated in the first total set can be determined, and the maximum number of these repeated occurrences is used as the first repetition degree of the answer data. If the first repetition degree of the answer data is greater than 5 (that is, the maximum number of repetitions is greater than 5), the answer data is determined as the first answer data.

[0050] Optionally, filtering out second answer data whose second repetition degree is less than a second threshold from multiple answer data may, for example, include: performing word segmentation processing on each answer data in the multiple answer data to obtain a word sequence for each answer data; extracting at least one second text fragment from the word sequence of each answer data to obtain a second text fragment sequence, the second text fragment including N2 words that appear consecutively in the word sequence, N2 being a positive integer, and the value of N2 may be set according to actual needs; merging the second text fragment sequences of multiple answer data to obtain a second total set; for each answer data, calculating the number of times each second text fragment in the second text fragment sequence of the answer data appears repeatedly in the second total set as the second repetition degree of the answer data; determining the answer data whose second repetition degree is less than the second threshold among the multiple answer data as the second answer data.

[0051] For example, N2 can be 2, and the second threshold value can be 3. Exemplarily, in this embodiment, 2-grams can be extracted from the word sequence of each answer data to obtain a 2-gram sequence, that is, a second text fragment sequence, and each 2-gram in the 2-gram sequence is a second text fragment. The 2-gram sequences of multiple answer data are merged into a set, that is, the second total set. For each answer data, the number of times each 2-gram in the 2-gram sequence of the answer data appears repeatedly in the second total set can be determined, and the maximum number of these repeated occurrences is used as the second repetition degree of the answer data. If the second repetition degree of the answer data is less than 3 (that is, the maximum number of repetitions is greater than 3), the answer data is determined as the second answer data.

[0052] Optionally, after training the language model using the preference data set, the answer repetition rate of the language model can also be detected. If the answer repetition rate of the language model is greater than the repetition rate threshold, the operation of performing multiple reasoning on the target question using the language model for each target question in the original data set to obtain multiple answer data is returned, that is, the language model is trained again until the answer repetition rate of the language model is less than or equal to the repetition rate threshold. The repetition rate threshold can be set according to actual needs.

[0053] Alternatively, the loss value may be calculated, for example, according to the following formula:

[0054]

[0055] Among them, y ω is the expected answer in the preference data (corresponding to the selected field), y r is the undesired answer in the preference data (corresponding to the rejection field), π θ (y ω |x) is the current model generates y when the current given input is x ωThe probability of ref (y ω |x) is the reference model (language model with unchanged parameters) that generates y when the current input is x. ω The probability of θ (y r |x) is the current model generates y when the current given input is x r The probability of ref (y r |x) is the reference model that generates y when the current given input is x. r probability.

[0056] like Figure 3 FIG. 1 is a flowchart of a method for training a language model according to another exemplary embodiment. The method may include the following steps:

[0057] Step 31: Select the prompt of the training set, that is, the question used to input the language model.

[0058] For example, in this embodiment, a prompt can be manually selected, and the selected prompt can be a prompt used in training or a prompt extracted from an open source data set. The selected prompt can be used to construct a data set dataset, and the prompt constructed in this step can be recorded as {xi}.

[0059] Step 32: Create labels for the training set.

[0060] For example, in this embodiment, for the prompt constructed in step 1, each question in the data set can be traversed, and the language model can be used to infer each question to generate 5 answers. For example, a language model t is deployed to infer each question, and a different temperature value, tempeture, is set for model t each time during inference, so that model t generates different answers, thereby ensuring the diversity of model answers. The tempeture values ​​can be: 0.1, 0.3, 0.5, 0.7, 0.9. When tempeture = 0.1, prompt{x i} is input to model t, and the answer data of model t is recorded as And so on, use other templates to obtain the model's answer data. Finally, for each prompt, 5 answer data will be generated, namely and x i Assembled as

[0061] Step 33: Build a complete training dataset.

[0062] Exemplarily, in this embodiment, the first duplication check mechanism (duplicate detection 1) can be used to check and filter the generated in step 2. After checking, only the repeated data (i.e., the first answer data) and the non-repeated data (i.e., the second answer data) are retained to construct a preferred data set that meets the needs of model training. Answers with a high degree of repetition are regarded as answers that the model does not expect, and answers with a low degree of repetition are regarded as answers that the model prefers. The data set finally generated in this step is {xi, No repetition, Repeat}, this dataset is used as the dataset for model training.

[0063] According to the embodiments of this specification, the first duplicate check mechanism can be a strong check mechanism. Since the data items in the training data set are generally large, ranging from more than 10,000 to unlimited, if manual annotation of duplicate data is used, this will be a huge manpower cost. However, if the duplicate check mechanism is unreliable and the error rate of the judgment result is high, then as training data, it is likely to cause the model to learn bad data, thereby causing the model's ability to decline. Therefore, for massive data, the first duplicate check mechanism is a strong check mechanism, which will check out deterministic non-duplicate data and duplicate data, and suspected duplicate and suspected non-duplicate data will be discarded to prevent false detection. In this way, although the effective data will become less after screening through the inspection mechanism, the accuracy of the inspection result is higher.

[0064] The first duplicate check mechanism can be used to check the generated 5 in The values ​​are checked, and finally the deterministic non-duplicate data and duplicate data are detected to construct the preference data.

[0065] Figure 4A FIG. 1 is a schematic diagram of a first duplication check mechanism according to an exemplary embodiment. Figure 4A As shown, the first duplication check mechanism is as follows:

[0066] Step 41: For each Standardize the sentence segmentation. For example, you can use the punctuation marks in the text to Use unified symbols to separate sentences.

[0067] Step 42: Use N-gram algorithm to Perform 2-gram and 10-gram calculations. After 2-gram segmentation, if the maximum number of repetitions is within 3, it is judged as No repetition, marked as 0. After 10-gram segmentation, if the maximum number of repetitions is more than 5, it is judged as Repeat, mark as 1, and leave the others unmarked.

[0068] Step 43: Filter and assemble the labeled Marked as 0 As the chosen data of the preference data set, the ones marked as 1 As the rejected data in the preference data set, we finally produce Training data set. The format of the training data set can be as follows, where the first element in the output array output corresponds to No repetition, the second element corresponds to repeat:

[0069]

[0070]

[0071] Step 34: Train the model t. The data set uses the preference data set constructed in the above steps. In this embodiment, the model t can be trained based on DPO training. DPO is a RLHF algorithm that aims to improve the ability to follow instructions. In this embodiment, this ability to follow instructions can be used to increase the probability of non-repeated answers and reduce the probability of repeated answers, thereby improving the ability to follow instructions that humans do not expect the model to answer repeatedly. The core principle of DPO training to improve the model's ability to follow instructions is as follows:

[0072]

[0073] When the optimization goal during model training is to minimize L DPO At this time, the model y ω The probability of answering is maximized to be greater than y r The probability of answering, y ω The corresponding data set is No repeat, y r The corresponding data set is Repeat. That is to say, after DPO training, the probability of the model answering repeated data will be reduced, which can alleviate the problem of repeated answers in the model. The new model generated after training can be recorded as model t+1.

[0074] Step 35, perform repeatability evaluation on model t+1. According to the embodiments of this specification, an evaluation data set can be prepared, and then the evaluation data set and the second repeat detection mechanism can be used to perform repeatability evaluation on the model. On the one hand, the repeatability rate of the answer of the trained model can be measured through evaluation, and on the other hand, the evaluation result can be compared with the model before training to determine whether the model repeatability problem has been alleviated.

[0075] In the evaluation phase, the method of making an evaluation data set may include: constructing a 500-item repeated test set, with the goal of constructing prompts that easily allow the model to answer repeated questions. The data construction method is as follows: by asking and answering questions to the language model, screen out prompts that will answer obviously repeated questions, or manually collect user feedback to answer repeated questions.

[0076] During the evaluation phase, a second duplication detection mechanism can be used to check the duplication rate.

[0077] The second duplicate detection mechanism library is a second-strongest inspection mechanism that focuses on evaluation rather than training data set inspection. Since the evaluation test set generally has a small amount of data, and each evaluation data must have a corresponding evaluation result to achieve a reliable evaluation level, if a strong inspection mechanism such as the second duplicate detection mechanism is used, although manpower is saved, only deterministic complete duplications and complete non-duplications can be checked. Such an evaluation result set is likely to be a subset of the test set. Although manpower is saved, the evaluation result is unreliable. For data sets of the order of 500 and 100,000, the savings and gains brought by automated testing of 500 data compared to manual testing are not particularly large. If you want to take into account both accuracy and labor cost savings, you can make a balance by combining automated testing with manual screening. For the second-strongest inspection mechanism of the second duplicate detection mechanism, you can first establish a hypothesis, assuming that the model answer repetition rate is low, then you can assume that there are a large number of non-duplicate answers and a small number of duplicate answers in the 500 answers. A large number of non-duplicate answers can be detected automatically, and the remaining small number of duplicate answers can be screened manually, which can save labor costs and take into account evaluation accuracy.

[0078] The comparison between the first duplicate detection mechanism and the second duplicate detection mechanism can be shown in Table 1.

[0079]

[0080] Table 1

[0081] Figure 4B FIG. 1 is a schematic diagram of a second duplication check mechanism according to an exemplary embodiment. Figure 4B As shown, the process of performing duplicate detection and evaluation using the second duplicate detection mechanism may include, for example:

[0082] (1) The model answers based on the punctuation marks in the text Standardize sentence segmentation.

[0083] (2) Based on the classified sentences, the distance-based similarity judgment method is used to determine the duplication (the higher the similarity, the more similar the strings are). If the similarity is greater than 80%, it is determined that there is duplicate content. This part of the data will be marked as 1 (obviously repeated), if the similarity is less than 80%, this part It will go into step (3) to continue judging;

[0084] (3) Use the N-gram algorithm to perform 2-gram, 4-gram, and 6-gram word segmentation calculations on the standardized sentence. If the maximum number of repetitions after 2-gram segmentation is greater than 6, the maximum number of repetitions after 4-gram segmentation is greater than 4, and the maximum number of repetitions after 6-gram segmentation is greater than 2, then it is considered There is repetition in the sentence. It will be marked as 2 (suspected duplication), otherwise It will be marked as 0. The reason why it is marked as suspected repetition instead of complete repetition 1 is that in Chinese, the same sentence may have different meanings due to different contexts. Therefore, in order to ensure the accuracy of the evaluation, it is necessary to perform manual judgment in step (4) on the data marked as 1 and 2.

[0085] (4) After the above judgment and processing, each There are corresponding marks. In the next step, the final screening is done manually for the ones marked 1 and 2. The ones marked 0 do not need to be screened manually because they are the confirmed results. The reason for manual inspection is that there may be repeated sentences in Chinese, but due to different contexts, the semantics of the expressions are different. At this time, such repeated sentences cannot be judged as repeated.

[0086] (5) Calculate the model answer repetition rate: After the above steps, the answers to the 500 test questions are marked with corresponding 0 and 1. The model answer repetition rate = the number of questions marked as 1 / 500.

[0087] Corresponding to the embodiments of the aforementioned method, this specification also provides embodiments of a language model training device and a terminal to which it is applied.

[0088] The embodiment of the training device of the language model of this specification can be applied to a computer device, such as a server or a terminal device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 5 As shown, it is a hardware structure diagram of the computer device where the language model training device of the embodiment of this specification is located, except Figure 5 In addition to the processor 510, memory 530, network interface 520, and non-volatile memory 540 shown, the server or electronic device where the device 531 is located in the embodiment may also include other hardware according to the actual function of the computer device, which will not be described in detail.

[0089] like Figure 6 As shown, Figure 6 is a block diagram of a language model training device according to an exemplary embodiment of the present specification, the device comprising:

[0090] The reasoning module 610 is used to perform multiple reasoning on each target question in the original data set using the language model to obtain multiple answer data, and the temperature value of the language model is different during each reasoning;

[0091] A screening module 620, configured to screen out first answer data having a first repetition degree greater than a first threshold and second answer data having a second repetition degree less than a second threshold from the plurality of answer data;

[0092] An adding module 630, configured to add the target question, the first answer data and the second answer data as preference data to the preference data set;

[0093] The training module 640 is used to train the language model using the preference data set.

[0094] Optionally, the screening module may include:

[0095] A first word segmentation submodule is used to perform word segmentation processing on each answer data among the multiple answer data to obtain a word sequence of each answer data;

[0096] A first extraction submodule is used to extract at least one first text segment from the word sequence of each answer data to obtain a first text segment sequence, wherein the first text segment includes N1 words that appear consecutively in the word sequence, where N1 is a positive integer;

[0097] A first merging submodule, used for merging first text segment sequences of a plurality of answer data to obtain a first total set;

[0098] A first calculation submodule is used to calculate, for each answer data, the number of times each first text segment in the first text segment sequence of the answer data is repeatedly appeared in the first total set as the first repetition degree of the answer data;

[0099] The first determination submodule is used to determine answer data having a first degree of repetition greater than a first threshold among the plurality of answer data as first answer data.

[0100] Optionally, the screening module may include:

[0101] The second word segmentation submodule is used to perform word segmentation processing on each answer data among the multiple answer data to obtain a word sequence of each answer data;

[0102] A second extraction submodule is used to extract at least one second text segment from the word sequence of each answer data to obtain a second text segment sequence, wherein the second text segment includes N2 words that appear consecutively in the word sequence, where N2 is a positive integer;

[0103] A second merging submodule, used for merging the second text segment sequences of the plurality of answer data to obtain a second total set;

[0104] A second calculation submodule is used to determine, for each answer data, a second repetition degree of the answer data according to the number of times each second text segment in the second text segment sequence of the answer data appears repeatedly in the second total set;

[0105] The second determination submodule is used to determine answer data with a second degree of repetition less than a second threshold among the plurality of answer data as second answer data.

[0106] Optionally, the training module may include:

[0107] The loss calculation submodule is used to calculate the loss value according to each preference data and the objective function in the preference data set;

[0108] The adjustment submodule is used to adjust the parameters of the language model according to the loss value.

[0109] Optionally, the above device may further include:

[0110] A detection module, used for detecting the answer repetition rate of the language model after training the language model using the preference data set;

[0111] The return module is used to return the operation of performing multiple inferences on the target question using the language model to obtain multiple answer data for each target question in the original data set if the answer repetition rate of the language model is greater than the repetition rate threshold.

[0112] According to the language model training method of the embodiment of this specification, the language model can be automatically trained to save labor costs and improve production efficiency. In addition, based on the repetition of the preference data, the preference data is used to train the language model, which can alleviate the repetition rate of the model answer, improve the model performance, and thus enhance the user experience of the model.

[0113] like Figure 7 As shown, Figure 7is a block diagram of a natural language processing device according to an exemplary embodiment of the present specification, the device comprising:

[0114] The input module 710 is used to input the question text into the language model to obtain answer data. The language model is trained according to the language model training method shown in the embodiment of this specification.

[0115] Accordingly, the present specification also provides a language model training device, which includes a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute the language model training method of the embodiment of the present specification and / or the natural language processing method of the embodiment of the present specification.

[0116] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0117] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0118] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] Those skilled in the art will readily appreciate other embodiments of the specification after considering the specification and practicing the invention claimed herein. The specification is intended to cover any variations, uses or adaptations of the specification that follow the general principles of the specification and include common knowledge or customary techniques in the art that are not claimed in the specification. The specification and examples are to be considered exemplary only, and the true scope and spirit of the specification are indicated by the following claims.

[0120] It should be understood that the present description is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present description is limited only by the appended claims.

[0121] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. A method for training a language model, characterized in that: The method comprises: For each target question in the original data set, the language model is used to perform multiple inferences on the target question to obtain multiple answer data, and the temperature value of the language model is different during each inference; Filter out, from the plurality of answer data, first answer data with a first degree of repetition greater than a first threshold and second answer data with a second degree of repetition less than a second threshold; adding the target question, the first answer data and the second answer data as preference data to a preference data set; The language model is trained using the preference dataset.

2. The method according to claim 1, characterized in that Screening out first answer data with a first degree of repetition greater than a first threshold from the plurality of answer data comprises: Perform word segmentation on each of the plurality of answer data to obtain a word sequence of each answer data; Extract at least one first text segment from the word sequence of each answer data to obtain a first text segment sequence, wherein the first text segment includes N1 words that appear consecutively in the word sequence, where N1 is a positive integer; Merging the first text segment sequences of the plurality of answer data to obtain a first total set; For each answer data, calculating the number of times each first text segment in the first text segment sequence of the answer data appears repeatedly in the first total set as the first repetition degree of the answer data; Answer data having a first degree of repetition greater than a first threshold among the plurality of answer data is determined as the first answer data.

3. The method according to claim 1, characterized in that The step of selecting the second answer data having a second degree of repetition less than a second threshold from the plurality of answer data comprises: Perform word segmentation on each of the plurality of answer data to obtain a word sequence of each answer data; Extract at least one second text segment from the word sequence of each answer data to obtain a second text segment sequence, wherein the second text segment includes N2 words that appear consecutively in the word sequence, where N2 is a positive integer; Merging the second text segment sequences of the plurality of answer data to obtain a second total set; For each answer data, determining a second repetition degree of the answer data according to the number of times each second text segment in the second text segment sequence of the answer data repeatedly appears in the second total set; Answer data having a second degree of repetition less than a second threshold among the plurality of answer data are determined as the second answer data.

4. The method according to claim 1, characterized in that: The using the preference data set to train the language model comprises: Calculate a loss value according to each preference data in the preference data set and the objective function; According to the loss value, the parameters of the language model are adjusted.

5. The method according to claim 1, characterized in that After training the language model using the preference data set, the method further includes: Detecting the repetition rate of the answer of the language model; If the answer repetition rate of the language model is greater than the repetition rate threshold, then the operation of using the language model to perform multiple inferences on the target question for each target question in the original data set to obtain multiple answer data is returned.

6. A natural language processing method, characterized in that: The method comprises: The question text is input into a language model to obtain answer data, wherein the language model is trained according to the method according to any one of claims 1-5.

7. A language model training device, characterized in that: The device comprises: An inference module is used to perform multiple inferences on each target question in the original data set using a language model to obtain multiple answer data, wherein the temperature value of the language model is different during each inference; A screening module, used for screening out first answer data with a first repetition greater than a first threshold and second answer data with a second repetition less than a second threshold from the plurality of answer data; an adding module, configured to add the target question, the first answer data and the second answer data as preference data to a preference data set; A training module is used to train the language model using the preference data set.

8. The device according to claim 7, characterized in that The screening module comprises: A first word segmentation submodule is used to perform word segmentation processing on each answer data of the plurality of answer data to obtain a word sequence of each answer data; A first extraction submodule is used to extract at least one first text segment from the word sequence of each answer data to obtain a first text segment sequence, wherein the first text segment includes N1 words that appear consecutively in the word sequence, and N1 is a positive integer; A first merging submodule, configured to merge the first text segment sequences of the plurality of answer data to obtain a first total set; A first calculation submodule is used to calculate, for each answer data, the number of times each first text segment in the first text segment sequence of the answer data is repeated in the first total set as a first repetition degree of the answer data; The first determination submodule is used to determine answer data having a first degree of repetition greater than a first threshold among the plurality of answer data as the first answer data.

9. A natural language processing device, characterized in that: The device comprises: An input module is used to input the question text into a language model to obtain answer data, wherein the language model is trained according to the method according to any one of claims 1-5.

10. A language model training device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images