Optimization method and device of large language model and related equipment
By adjusting the inference degree of the large language model based on the annealing simulation algorithm, and updating the model with the target temperature value and loss value, the problem of poor output diversity of the large language model is solved, and the model's inference efficiency and generation ability are improved.
Patent Information
- Application Number
- CN202411147753.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-12-06
AI Technical Summary
The existing large language model has poor output diversity due to local optimal problems during text generation, and cannot fully explore better global solutions.
By obtaining preset large language models and training samples, adjust the inference degree of the model based on the annealing simulation algorithm, generate target temperature values, and update the model according to the target loss values to improve the diversity and stability of the output.
It improves the inference efficiency and output diversity of large language models, solves local optimal problems, and enhances the generation ability of the model.
Smart Images

Figure CN119090013A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular, to a method, apparatus, and related equipment for optimizing a large language model. Background Art
[0002] In the field of natural language processing, language models (such as GPT-4) are widely used in tasks such as text generation, machine translation, and dialogue systems. Existing text generation methods such as Greedy Search and Beam Search will lack diversity in the generated results due to local optimal problems during the generation process. The lack of diversity is generally improved by sampling or temperature adjustment, but its limitation is that it is easy to fall into the local optimal solution and cannot fully explore a better global solution, resulting in poor output diversity of large language models. Summary of the invention
[0003] The embodiments of the present application provide a method, apparatus and related equipment for optimizing a large language model to solve the problem of poor output diversity of large language models in the prior art.
[0004] To solve the above problems, this application is implemented as follows:
[0005] In a first aspect, an embodiment of the present application provides a method for optimizing a large language model, the method comprising:
[0006] Obtaining a preset large language model and a training sample, wherein the preset large language model is a text inference model, and the training sample includes a training text and label information corresponding to the training text;
[0007] Generating a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm;
[0008] Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value;
[0009] The preset large language model is updated according to a target loss value to obtain a target large language model, wherein the target loss value is a loss value obtained by calculating by inputting the label information and the prediction value into a target loss function.
[0010] Optionally, generating a target temperature value according to the training sample includes:
[0011] Determining the text type of the training sample, the text type including: writing type, extraction type or translation type;
[0012] When the text type of the training sample is a writing type, the first temperature value is determined as the target temperature value; when the text type of the training sample is a refinement type, the second temperature value is determined as the target temperature value; when the text type of the training sample is a translation type, the third temperature value is determined as the target temperature value;
[0013] The first temperature value is greater than the second temperature value, and the second temperature value is greater than the third temperature value.
[0014] Optionally, inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value includes:
[0015] Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating multiple candidate solutions;
[0016] Calculating acceptance probabilities corresponding to the plurality of candidate solutions respectively to obtain a plurality of acceptance probabilities, wherein the acceptance probabilities are used to represent acceptance probabilities of the corresponding candidate solutions becoming prediction values in the preset large language model;
[0017] Screening the multiple candidate solutions according to the multiple acceptance probabilities and a preset threshold to obtain multiple target candidate solutions, the multiple candidate solutions include the multiple target candidate solutions, and the number of the multiple target candidate solutions is less than the preset threshold;
[0018] Iterative reasoning is performed based on the multiple target candidate solutions to obtain the predicted value.
[0019] Optionally, calculating the acceptance probabilities corresponding to the multiple candidate solutions respectively to obtain multiple acceptance probabilities includes:
[0020] Calculating the output probability values corresponding to the multiple candidate solutions respectively to obtain multiple output probability values, where the output probability values are probability values output by a normalized exponential function;
[0021] Calculating the differences between the multiple output probability values and the target probability value respectively to obtain multiple difference values corresponding to the multiple candidate solutions respectively, wherein the target probability value is the largest probability value among the multiple probability values;
[0022] Ratios of the plurality of difference values to the target temperature value are calculated respectively to obtain the plurality of acceptance probabilities.
[0023] Optionally, the method further includes:
[0024] When the difference between any two of the acceptance probabilities among the multiple acceptance probabilities is greater than or equal to a preset probability, increasing the target temperature value;
[0025] When the difference between any two of the multiple acceptance probabilities is smaller than the preset probability, the target temperature value is reduced.
[0026] Optionally, performing iterative reasoning based on the multiple target candidate solutions to obtain the predicted value includes:
[0027] Perform iterative reasoning based on the multiple target candidate solutions to obtain a target prediction value;
[0028] Scoring the length of the target prediction value according to a first scoring rule to obtain a first score, and scoring the lexical diversity of the target prediction value according to a second scoring rule to obtain a second score;
[0029] Adjust the target temperature value according to the first score and the second score to obtain an updated temperature value;
[0030] The target predicted value is iteratively inferred according to the updated temperature value to obtain the predicted value.
[0031] In a second aspect, the embodiment of the present application further provides an optimization device for a large language model, including:
[0032] An acquisition module, used to acquire a preset large language model and training samples, wherein the preset large language model is a text inference model, and the training samples include training texts and label information corresponding to the training texts;
[0033] A generating module, configured to generate a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm;
[0034] An inference module, used for inputting the training sample into the preset large language model, performing inference based on the target temperature value, and generating a predicted value;
[0035] An updating module is used to update the preset large language model according to a target loss value to obtain a target large language model, wherein the target loss value is a loss value calculated by inputting the label information and the prediction value into a target loss function.
[0036] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor; the processor is used to read the program in the memory to implement the steps of the method described in the first aspect above.
[0037] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the method described in the first aspect above.
[0038] In a fifth aspect, an embodiment of the present application further provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the steps in the method described in the first aspect.
[0039] The present application provides an optimization method, device and related equipment for a large language model, the method comprising: obtaining a preset large language model and a training sample, the preset large language model being a text inference model, the training sample comprising a training text and label information corresponding to the training text; generating a target temperature value according to the training sample, the target temperature value being used to adjust the degree of inference of the preset large language model to the training sample based on an annealing simulation algorithm; inputting the training sample into the preset large language model, inferring based on the target temperature value, and generating a predicted value; updating the preset large language model according to a target loss value, and obtaining a target large language model, the target loss value being a loss value calculated by inputting the label information and the predicted value into a target loss function. The present application determines a target temperature value based on a training sample, infers the training sample through the target temperature value and the preset large language model, and obtains a predicted value, thereby comparing the predicted value with the label information of the training sample to determine the target loss value to update the preset large language model, thereby solving the problem of poor output diversity of the large language model and improving the inference efficiency of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0041] Figure 1 It is a flowchart of a method for optimizing a large language model provided in an embodiment of the present application;
[0042] Figure 2 It is a structural schematic diagram of an optimization device for a large language model provided in an embodiment of the present application;
[0043] Figure 3 It is a schematic diagram of the structure of an electronic device provided by the implementation of this application. DETAILED DESCRIPTION
[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0045] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing the inclusion of separate A, separate B, separate C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.
[0046] See also Figure 1 , Figure 1 is a flow chart of the optimization method of a large language model provided in an embodiment of the present application, such as Figure 1 As shown, the optimization method of the large language model may include the following steps:
[0047] Step 101: Obtain a preset large language model and training samples, wherein the preset large language model is a text inference model, and the training samples include training texts and label information corresponding to the training texts.
[0048] In this embodiment, the preset large language model may be a text reasoning model, which is a natural language processing (NLP) model that aims to understand and reason about implicit information in texts. It is usually used to judge the relationship between texts, such as whether one statement can be derived from another statement, or whether two texts are semantically contradictory.
[0049] Among them, the training samples include training text and label information corresponding to the training text. Specifically, for example, the training text can be "generate a historical story of the Three Kingdoms period", and the corresponding label information is the corresponding "historical story". In this embodiment, it can also be other training texts such as text summary, text generation, text translation, etc., which are not specifically limited in this embodiment.
[0050] It should be noted that the present application is applied to a large language model, and is mainly used to optimize the large language model to improve the diversity of its text reasoning.
[0051] Step 102: Generate a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm.
[0052] In this embodiment, the temperature is dynamically adjusted according to the characteristics of different training samples. Specifically, for example, for tasks that require more innovation, the target temperature value may be set higher, while for tasks that require less innovation, the target temperature value may be set lower.
[0053] Specifically, the target temperature value is used to adjust the reasoning degree of the preset large language model on the training sample based on the simulated annealing algorithm, wherein the simulated annealing algorithm (SA) is a random optimization algorithm inspired by the annealing process in physics. It is used to find the global optimal solution, especially when solving combinatorial optimization problems, and has a good effect.
[0054] The specific execution steps of the annealing simulation algorithm include: selecting an initial solution and setting the initial temperature. Randomly select a new solution in the neighborhood of the current solution. Calculate the objective function value (for example, cost, energy, etc.) of the new solution and the current solution. If the new solution is better, accept the new solution; if the new solution is not better, accept the new solution with a certain probability, and this probability is related to the difference between the current temperature and the solution. The Metropolis criterion is usually used to decide whether to accept the new solution. Gradually lower the temperature, usually following a certain cooling strategy (such as linear cooling, exponential cooling, etc.). When the temperature drops to a certain threshold or reaches the maximum number of iterations, terminate the algorithm. It should be noted that in this embodiment, the target temperature value is the initial temperature.
[0055] Step 103: input the training sample into the preset large language model, perform inference based on the target temperature value, and generate a predicted value.
[0056] In this embodiment, the training sample is input into the preset large language model, and reasoning is performed according to the initially set target temperature value. After multiple iterations and reasoning, the predicted value is finally output. It should be noted that the reasoning process through the target temperature value will have multiple iterations. For example, to generate text, it is necessary to gradually generate one word at a time. After each word is generated, the subsequent text needs to be inferred and generated, and the temperature value is adjusted at the same time until the reasoning is completed and the predicted value is finally generated.
[0057] Step 104: Update the preset large language model according to the target loss value to obtain a target large language model, wherein the target loss value is a loss value calculated by inputting the label information and the prediction value into a target loss function.
[0058] In this embodiment, the target loss value is calculated based on the target loss value, label information and target loss function. Specifically, the loss function is a function used in machine learning and deep learning to evaluate the difference between model prediction and actual results. It is a key component in the model training process and helps guide the optimization of the model so that it can better fit the data. The target loss value is finally obtained by inputting the label information and the predicted value into the target loss function to calculate the difference value, and the preset large language model is updated by the target loss value, and finally the updated target large model is obtained, thereby improving the diversity and stability of the large model output.
[0059] The present application provides an optimization method for a large language model, the method comprising: obtaining a preset large language model and a training sample, the preset large language model being a text inference model, the training sample comprising a training text and label information corresponding to the training text; generating a target temperature value according to the training sample, the target temperature value being used to adjust the degree of inference of the preset large language model on the training sample based on an annealing simulation algorithm; inputting the training sample into the preset large language model, inferring based on the target temperature value, and generating a predicted value; updating the preset large language model according to a target loss value, and obtaining a target large language model, the target loss value being a loss value calculated by inputting the label information and the predicted value into a target loss function. The present application determines a target temperature value based on a training sample, infers the training sample through the target temperature value and the preset large language model, and obtains a predicted value, thereby comparing the predicted value with the label information of the training sample to determine the target loss value to update the preset large language model, thereby solving the problem of poor output diversity of the large language model and improving the inference efficiency of the large language model.
[0060] In some feasible implementations, optionally, generating a target temperature value according to the training sample includes:
[0061] Determining the text type of the training sample, the text type including: writing type, extraction type or translation type;
[0062] When the text type of the training sample is a writing type, the first temperature value is determined as the target temperature value; when the text type of the training sample is a refinement type, the second temperature value is determined as the target temperature value; when the text type of the training sample is a translation type, the third temperature value is determined as the target temperature value;
[0063] The first temperature value is greater than the second temperature value, and the second temperature value is greater than the third temperature value.
[0064] In this embodiment, a target temperature value, namely, the initial temperature T_0, is set. Specifically, the domain knowledge dynamically sets the initial temperature. Specifically, the target temperature value is determined according to the text type of the training text, thereby dynamically adjusting the temperature according to the characteristics of different text generation tasks. For example, for tasks that require more innovation, a higher temperature can be maintained. Exemplarily, the text type includes: writing type, refining type or translation type, as shown in the following formula:
[0065]
[0066] It should be noted that the coefficient is set as a hyperparameter and can be adjusted according to the scenario. Specifically, the first temperature value corresponding to the writing type is greater than the second temperature value corresponding to the extraction type, which is greater than the third temperature value corresponding to the translation type.
[0067] It should be noted that during the reasoning process, the maximum number of acceptable solution sets at each step is top_k, the minimum value of acceptable probability is P_min, and the cooling coefficients are α and θ.
[0068] Optionally, inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value includes:
[0069] Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating multiple candidate solutions;
[0070] Calculating acceptance probabilities corresponding to the plurality of candidate solutions respectively to obtain a plurality of acceptance probabilities, wherein the acceptance probabilities are used to represent acceptance probabilities of the corresponding candidate solutions becoming prediction values in the preset large language model;
[0071] Screening the multiple candidate solutions according to the multiple acceptance probabilities and a preset threshold to obtain multiple target candidate solutions, the multiple candidate solutions include the multiple target candidate solutions, and the number of the multiple target candidate solutions is less than the preset threshold;
[0072] Iterative reasoning is performed based on the multiple target candidate solutions to obtain the predicted value.
[0073] In this embodiment, the reasoning step is taken as an example to explain. Specifically, the training sample is input into the preset large language model to generate multiple candidate solutions, which are recorded as S = {S0, ..., S NThe acceptance probability corresponding to each candidate solution is calculated to obtain multiple acceptance probabilities, and the multiple acceptance probabilities correspond to the multiple candidate solutions one by one. Specifically, the acceptance probability is used to represent the acceptance probability of the corresponding candidate solution becoming a prediction value in the preset large language model.
[0074] Thus, multiple candidate solutions are screened by multiple acceptance probabilities and preset thresholds to obtain multiple target candidate solutions, wherein all solutions in S are calculated, and then all solutions with P>P_min are screened, and the number is ensured to be less than top_k, where P is the acceptance probability and P_min is the preset threshold.
[0075] Multiple candidate solutions are screened by a preset threshold to obtain multiple target candidate solutions, and iterative reasoning is performed based on the multiple target candidate solutions to finally obtain a predicted value.
[0076] Optionally, calculating the acceptance probabilities corresponding to the multiple candidate solutions respectively to obtain multiple acceptance probabilities includes:
[0077] Calculating the output probability values corresponding to the multiple candidate solutions respectively to obtain multiple output probability values, where the output probability values are probability values output by a normalized exponential function;
[0078] Calculating the differences between the multiple output probability values and the target probability value respectively to obtain multiple difference values corresponding to the multiple candidate solutions respectively, wherein the target probability value is the largest probability value among the multiple probability values;
[0079] Ratios of the plurality of difference values to the target temperature value are calculated respectively to obtain the plurality of acceptance probabilities.
[0080] In this embodiment, the output probability values corresponding to the multiple candidate solutions are probability values output by the normalized exponential function. In this embodiment, the softmax function is taken as an example. The softmax function is a commonly used mathematical function, which is mainly used to convert a set of real numbers into a probability distribution. It is usually used in the output layer of multi-class classification problems, especially in neural networks.
[0081] Specifically, generate candidate solutions, denoted as S = {S0,...,S N}; Calculate the difference of candidate solutions, Δe = -(e ′ -e), where e ′ is the probability value of the softmax output of the current solution (i.e. the predicted token); e is the maximum probability value corresponding to all tokens in S; calculate the acceptance probability of the candidate solution:
[0082]
[0083] Where T is the temperature. In step 1, T = T_0. Calculate all solutions in S, then filter all solutions with P>P_min, and ensure that the number is less than top_k.
[0084] Optionally, the method further includes:
[0085] When the difference between any two of the acceptance probabilities among the multiple acceptance probabilities is greater than or equal to a preset probability, increasing the target temperature value;
[0086] When the difference between any two of the multiple acceptance probabilities is smaller than the preset probability, the target temperature value is reduced.
[0087] In this embodiment, during the reasoning process, the target temperature value can also be dynamically adjusted. For example, when the difference in acceptance probability is greater than or equal to the preset probability, the target temperature value is increased to lower the acceptance probability, or when the difference in acceptance probability is less than the preset probability, the target temperature value is reduced to increase the acceptance probability.
[0088] Specifically, it is dynamically adjusted according to the change of the current score. For example, when the probability gap is large, it means that the probability of the model predicting this token is far lower than the most certain answer, that is, the answer is more unique, so the temperature can be increased to make the acceptance probability lower; when the score gap is small, it means that the model is wavering in this reasoning result, and the temperature can be lowered to make it more likely to accept; the formula is as follows:
[0089]
[0090] In the formula, α<1 is to ensure that as the number of prediction steps increases, the diversity of the model gradually decreases, that is, the more pre-constraints are met, the fewer reliable answers are, which also increases the speed of later reasoning; θ is a small constant to prevent the denominator from being 0, which is set to 1e-5.
[0091] Optionally, performing iterative reasoning based on the multiple target candidate solutions to obtain the predicted value includes:
[0092] Perform iterative reasoning based on the multiple target candidate solutions to obtain a target prediction value;
[0093] Scoring the length of the target prediction value according to a first scoring rule to obtain a first score, and scoring the lexical diversity of the target prediction value according to a second scoring rule to obtain a second score;
[0094] Adjust the target temperature value according to the first score and the second score to obtain an updated temperature value;
[0095] The target predicted value is iteratively inferred according to the updated temperature value to obtain the predicted value.
[0096] In this embodiment, a feedback mechanism is introduced in the inference process to adjust the temperature in real time according to the length and diversity of the generated results. Specifically, the length of the predicted value and the vocabulary diversity score are set. Specifically,
[0097] That is, by calculating the length index and vocabulary diversity score of the generated text, the temperature of the next step is dynamically adjusted. The formula is as follows:
[0098]
[0099] Where L and V are the length and number of words of the current inference text respectively; L max is the hyperparameter, the maximum allowed inference length, V max For the hyperparameter, set the maximum vocabulary diversity. Integration, dynamic temperature value:
[0100]
[0101] Finally, based on the updated temperature value, the iteration is repeated to obtain the predicted value.
[0102] The present application determines a target temperature value based on a training sample, infers the training sample through the target temperature value and a preset large language model to obtain a predicted value, thereby comparing the predicted value with the label information of the training sample to determine a target loss value to update the preset large language model, thereby solving the problem of poor output diversity of the large language model and improving the reasoning efficiency of the large language model.
[0103] See also Figure 2 , Figure 2 is a structural diagram of the optimization device for a large language model provided in an embodiment of the present application. Figure 2 As shown, the optimization device 200 for a large language model includes:
[0104] An acquisition module 210 is used to acquire a preset large language model and a training sample, wherein the preset large language model is a text inference model, and the training sample includes a training text and label information corresponding to the training text;
[0105] A generating module 220, configured to generate a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm;
[0106] An inference module 230, configured to input the training sample into the preset large language model, perform inference based on the target temperature value, and generate a prediction value;
[0107] The updating module 240 is used to update the preset large language model according to the target loss value to obtain a target large language model. The target loss value is the loss value calculated by inputting the label information and the prediction value into a target loss function.
[0108] Optionally, the generating module 220 includes:
[0109] Generating a target temperature value according to the training sample comprises:
[0110] A first determination submodule is used to determine the text type of the training sample, where the text type includes: writing type, extraction type or translation type;
[0111] The second determination submodule is used to determine the first temperature value as the target temperature value when the text type of the training sample is a writing type; determine the second temperature value as the target temperature value when the text type of the training sample is a refinement type; and determine the third temperature value as the target temperature value when the text type of the training sample is a translation type;
[0112] The first temperature value is greater than the second temperature value, and the second temperature value is greater than the third temperature value.
[0113] Optionally, the reasoning module 230 includes:
[0114] A first reasoning submodule, configured to input the training sample into the preset large language model, perform reasoning based on the target temperature value, and generate multiple candidate solutions;
[0115] A calculation submodule, used to calculate the acceptance probabilities corresponding to the multiple candidate solutions respectively, to obtain multiple acceptance probabilities, wherein the acceptance probabilities are used to represent the acceptance probabilities of the corresponding candidate solutions becoming prediction values in the preset large language model;
[0116] a screening submodule, configured to screen the plurality of candidate solutions according to the plurality of acceptance probabilities and a preset threshold value to obtain a plurality of target candidate solutions, wherein the plurality of candidate solutions include the plurality of target candidate solutions, and the number of the plurality of target candidate solutions is less than the preset threshold value;
[0117] The second reasoning submodule is used to perform iterative reasoning based on the multiple target candidate solutions to obtain the predicted value.
[0118] Optionally, the computing submodule includes:
[0119] A first calculation unit is used to calculate the output probability values corresponding to the multiple candidate solutions respectively, to obtain multiple output probability values, wherein the output probability values are probability values output by a normalized exponential function;
[0120] A second calculation unit is used to calculate the difference between the multiple output probability values and the target probability value respectively, to obtain multiple difference values corresponding to the multiple candidate solutions respectively, wherein the target probability value is the largest probability value among the multiple probability values;
[0121] The third calculation unit is used to calculate the ratios of the multiple difference values and the target temperature value respectively to obtain the multiple acceptance probabilities.
[0122] Optionally, also include:
[0123] A first adjustment unit, configured to increase the target temperature value when a difference between any two of the acceptance probabilities among the multiple acceptance probabilities is greater than or equal to a preset probability;
[0124] The second adjustment unit is configured to reduce the target temperature value when a difference between any two of the acceptance probabilities among the multiple acceptance probabilities is less than the preset probability.
[0125] Optionally, the second reasoning submodule includes:
[0126] A first reasoning unit, configured to perform iterative reasoning based on the multiple target candidate solutions to obtain a target prediction value;
[0127] a scoring unit, configured to score the length of the target prediction value according to a first scoring rule to obtain a first score, and to score the lexical diversity of the target prediction value according to a second scoring rule to obtain a second score;
[0128] a third adjusting unit, configured to adjust the target temperature value according to the first score and the second score to obtain an updated temperature value;
[0129] The second reasoning unit is used to iteratively reason the target predicted value according to the updated temperature value to obtain the predicted value.
[0130] The present application determines a target temperature value based on a training sample, infers the training sample through the target temperature value and a preset large language model to obtain a predicted value, thereby comparing the predicted value with the label information of the training sample to determine a target loss value to update the preset large language model, thereby solving the problem of poor output diversity of the large language model and improving the reasoning efficiency of the large language model.
[0131] The present application also provides an electronic device. Figure 3 , the electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and executable on the processor 301.
[0132] When the program 3021 is executed by the processor 301, it can achieve Figure 1 Any step in the corresponding method embodiment:
[0133] Obtaining a preset large language model and a training sample, wherein the preset large language model is a text inference model, and the training sample includes a training text and label information corresponding to the training text;
[0134] Generating a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm;
[0135] Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value;
[0136] The preset large language model is updated according to a target loss value to obtain a target large language model, wherein the target loss value is a loss value obtained by calculating by inputting the label information and the prediction value into a target loss function.
[0137] Optionally, generating a target temperature value according to the training sample includes:
[0138] Determining the text type of the training sample, the text type including: writing type, extraction type or translation type;
[0139] When the text type of the training sample is a writing type, the first temperature value is determined as the target temperature value; when the text type of the training sample is a refinement type, the second temperature value is determined as the target temperature value; when the text type of the training sample is a translation type, the third temperature value is determined as the target temperature value;
[0140] The first temperature value is greater than the second temperature value, and the second temperature value is greater than the third temperature value.
[0141] Optionally, inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value includes:
[0142] Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating multiple candidate solutions;
[0143] Calculating acceptance probabilities corresponding to the plurality of candidate solutions respectively to obtain a plurality of acceptance probabilities, wherein the acceptance probabilities are used to represent acceptance probabilities of the corresponding candidate solutions becoming prediction values in the preset large language model;
[0144] Screening the multiple candidate solutions according to the multiple acceptance probabilities and a preset threshold to obtain multiple target candidate solutions, the multiple candidate solutions include the multiple target candidate solutions, and the number of the multiple target candidate solutions is less than the preset threshold;
[0145] Iterative reasoning is performed based on the multiple target candidate solutions to obtain the predicted value.
[0146] Optionally, calculating the acceptance probabilities corresponding to the multiple candidate solutions respectively to obtain multiple acceptance probabilities includes:
[0147] Calculating the output probability values corresponding to the multiple candidate solutions respectively to obtain multiple output probability values, where the output probability values are probability values output by a normalized exponential function;
[0148] Calculating the differences between the multiple output probability values and the target probability value respectively to obtain multiple difference values corresponding to the multiple candidate solutions respectively, wherein the target probability value is the largest probability value among the multiple probability values;
[0149] Ratios of the plurality of difference values to the target temperature value are calculated respectively to obtain the plurality of acceptance probabilities.
[0150] Optionally, the method further includes:
[0151] When the difference between any two of the acceptance probabilities among the multiple acceptance probabilities is greater than or equal to a preset probability, increasing the target temperature value;
[0152] When the difference between any two of the multiple acceptance probabilities is smaller than the preset probability, the target temperature value is reduced.
[0153] Optionally, performing iterative reasoning based on the multiple target candidate solutions to obtain the predicted value includes:
[0154] Perform iterative reasoning based on the multiple target candidate solutions to obtain a target prediction value;
[0155] Scoring the length of the target prediction value according to a first scoring rule to obtain a first score, and scoring the lexical diversity of the target prediction value according to a second scoring rule to obtain a second score;
[0156] Adjust the target temperature value according to the first score and the second score to obtain an updated temperature value;
[0157] The target predicted value is iteratively inferred according to the updated temperature value to obtain the predicted value.
[0158] The present application determines a target temperature value based on a training sample, infers the training sample through the target temperature value and a preset large language model to obtain a predicted value, thereby comparing the predicted value with the label information of the training sample to determine a target loss value to update the preset large language model, thereby solving the problem of poor output diversity of the large language model and improving the reasoning efficiency of the large language model.
[0159] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned large language model optimization method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the computer-readable storage medium is such as a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0160] The embodiment of the present application further provides a computer program product, which is stored in a storage medium. The computer program product is executed by at least one processor to implement the various processes of the above-mentioned large language model optimization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0161] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0163] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A method for optimizing large language models. It is characterized in that The method comprises: Obtaining a preset large language model and a training sample, wherein the preset large language model is a text inference model, and the training sample includes a training text and label information corresponding to the training text; Generating a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm; Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value; The preset large language model is updated according to a target loss value to obtain a target large language model, wherein the target loss value is a loss value obtained by calculating by inputting the label information and the prediction value into a target loss function.
2. The method according to claim 1, It is characterized in that Generating a target temperature value according to the training sample includes: Determining the text type of the training sample, the text type including: writing type, extraction type or translation type; When the text type of the training sample is a writing type, the first temperature value is determined as the target temperature value; when the text type of the training sample is a refinement type, the second temperature value is determined as the target temperature value; when the text type of the training sample is a translation type, the third temperature value is determined as the target temperature value; The first temperature value is greater than the second temperature value, and the second temperature value is greater than the third temperature value.
3. The method according to claim 1, It is characterized in that The step of inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating a predicted value includes: Inputting the training sample into the preset large language model, performing reasoning based on the target temperature value, and generating multiple candidate solutions; Calculating acceptance probabilities corresponding to the plurality of candidate solutions respectively to obtain a plurality of acceptance probabilities, wherein the acceptance probabilities are used to represent acceptance probabilities of the corresponding candidate solutions becoming prediction values in the preset large language model; Screening the multiple candidate solutions according to the multiple acceptance probabilities and a preset threshold to obtain multiple target candidate solutions, the multiple candidate solutions include the multiple target candidate solutions, and the number of the multiple target candidate solutions is less than the preset threshold; Iterative reasoning is performed based on the multiple target candidate solutions to obtain the predicted value.
4. The method according to claim 3, It is characterized in that The calculating the acceptance probabilities corresponding to the plurality of candidate solutions respectively to obtain a plurality of acceptance probabilities includes: Calculating the output probability values corresponding to the multiple candidate solutions respectively to obtain multiple output probability values, where the output probability values are probability values output by a normalized exponential function; Calculating the differences between the multiple output probability values and the target probability value respectively to obtain multiple difference values corresponding to the multiple candidate solutions respectively, wherein the target probability value is the largest probability value among the multiple probability values; Ratios of the plurality of difference values to the target temperature value are calculated respectively to obtain the plurality of acceptance probabilities.
5. The method according to claim 4, It is characterized in that The method further comprises: When the difference between any two of the acceptance probabilities among the multiple acceptance probabilities is greater than or equal to a preset probability, increasing the target temperature value; When the difference between any two of the multiple acceptance probabilities is smaller than the preset probability, the target temperature value is reduced.
6. The method according to claim 3, It is characterized in that The iterative reasoning based on the multiple target candidate solutions to obtain the predicted value includes: Perform iterative reasoning based on the multiple target candidate solutions to obtain a target prediction value; Scoring the length of the target prediction value according to a first scoring rule to obtain a first score, and scoring the lexical diversity of the target prediction value according to a second scoring rule to obtain a second score; Adjust the target temperature value according to the first score and the second score to obtain an updated temperature value; The target predicted value is iteratively inferred according to the updated temperature value to obtain the predicted value.
7. An optimization device for a large language model, It is characterized in that The device comprises: An acquisition module, used to acquire a preset large language model and training samples, wherein the preset large language model is a text inference model, and the training samples include training texts and label information corresponding to the training texts; A generating module, configured to generate a target temperature value according to the training sample, wherein the target temperature value is used to adjust the degree of reasoning of the preset large language model on the training sample based on an annealing simulation algorithm; An inference module, used for inputting the training sample into the preset large language model, performing inference based on the target temperature value, and generating a predicted value; An updating module is used to update the preset large language model according to a target loss value to obtain a target large language model, wherein the target loss value is a loss value calculated by inputting the label information and the prediction value into a target loss function.
8. An electronic device, include: A memory, a processor, and a program stored in the memory and executable on the processor; wherein the processor is used to read the program in the memory to implement the steps in the method for optimizing a large language model as described in any one of claims 1 to 6.
9. A readable storage medium for storing a program, It is characterized in that When the program is executed by a processor, the steps in the large language model optimization method according to any one of claims 1 to 6 are implemented.
10. A computer program product, It is characterized in that The computer program product is stored in a storage medium, and the computer program product is executed by at least one processor to implement the steps in the large language model optimization method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Text processing method and device based on large language model
CN118278411A
Cited By
Temperature sampling method and electronic equipment
CN121597834A
