Method and apparatus for optimizing iterative generation problem of large language model

Abstract

To provide a method and a device for optimizing a large language model repetition generation problem, a storage medium and a computer program.SOLUTION: The present invention introduces a neural network in the inference process of a large language model, and reduces the probability of an ID / text unit that is prone to repeated generation according to the occurrence probability of the repeated generation phenomenon, thereby improving the repeated generation problem of the large language model, improving the readability of the text generated by the large language model, and improving the user experience.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the technical fields of machine learning and natural language processing (NLP), and in particular to an optimization method and apparatus for solving a large-scale language model iterative generation problem, a storage medium, and a computer program. [Background technology]

[0002] A large language model (LLM) is a model based on deep learning technology, which generally includes a large number of parameters for processing and generating natural language text. By pre-training on large-scale text data, the large language model learns relationships, semantic information, and patterns between texts, thereby having powerful language understanding and generation capabilities.

[0003] For large-scale language models, the structure of the large-scale language model is generally a decoder structure in a transformer, or the structure of the large-scale language model includes at least a decoder. However, because the decoder structure generates the t+1th character from the previous t characters, this type of structure may cause the large-scale language model to be overly dependent on previous content, which may lead to the phenomenon of repetitive generation (also known as infinite repetitive generation). For example, when a user inputs "Q1", the response of the large-scale language model is generated repeatedly based on Q1 and the characters generated by each large-scale language model itself. If the response of the large-scale language model includes multiple consecutive repeated text blocks, this is considered to be the repetitive generation phenomenon, which will have a serious impact on user experience and semantic analysis.

[0004] To address the infinite repetition phenomenon, related technologies use a method of sampling the logtis of the model output in the inference stage, but instead of selecting the character with the highest probability each time, they select characters according to a predetermined rule to improve diversity and reduce the infinite repetition phenomenon. However, even if the method using diversity avoids the infinite repetition phenomenon as much as possible, it does not essentially solve the problem. Summary of the Invention [Problem to be solved by the invention]

[0005] At least one embodiment of the present invention provides a method and apparatus, a storage medium, and a computer program for optimizing an iterative generation problem of a large-scale language model, which can improve the iterative generation problem of a large-scale language model. [Means for solving the problem]

[0006] In one aspect of the present invention, an embodiment of the present invention comprises: obtaining a neural network by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output is a hidden layer representation obtained by the large-scale language model based on an input ID list, the output of the neural network is a probability that repetition occurs at a location of an ID corresponding to the first output, the first training data includes an ID list and a label, the ID list is generated by the large-scale language model based on a prompt text, and the label is used to indicate whether a location of each ID in the ID list is a location where repetition occurs; When performing inference using the large-scale language model, inputting the text to be inferred into the large-scale language model, obtaining a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, the i-th ID position including the location of each ID generated by the large-scale language model, adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID, and determining the ID at the i-th ID position and its corresponding text unit based on the third probability of each candidate ID; The present invention provides an optimization method for the iterative generation problem of large-scale language models including

[0007] Optionally, before obtaining the neural network by training using a first training set including a plurality of first training data, further generating a first training data set based on a first prompt text; obtaining a plurality of first training data generated based on a plurality of first prompt texts to obtain the first training set; Here, the step of generating one first training data based on one first prompt text includes: inputting the first prompt text into a large-scale language model to obtain a second ID list generated based on a first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text; concatenating the first ID list and the second ID list to obtain a third ID list; generating a label for each ID in the first list of IDs to be used to indicate that the corresponding ID should be ignored in the training process; generating a label for each ID in the second ID list to indicate whether the location of the corresponding ID is a location where recursive generation occurs; concatenating a label for each ID in the first ID list with a label for each ID in the second ID list to obtain a label for the third ID list; and generating a first training data set based on the third ID list and the labels of the third ID list.

[0008] Optionally, the step of obtaining a neural network by training using a first training set including a plurality of first training data includes: The method includes steps of freezing parameters of the large-scale language model, inputting an ID list in the first training data into the large-scale language model, calculating a loss value based on the output of the neural network and the labels of the ID list, and continuing to adjust the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining a trained neural network.

[0009] Optionally, a first probability of each candidate ID at a specific ID position is generated by the large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. A recurrence probability that a recurrence occurs at a specific ID position is generated by a neural network based on partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The fourth ID list is an ID list composed of IDs corresponding to each text unit in the inference target text.

[0010] Optionally, adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID includes: The method includes a step of reducing the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID, wherein the first candidate ID is an ID having the largest first probability, the degree of reduction in the first probability of the first candidate ID is positively correlated with the second probability, and the reduction rate of the first probability of other candidate IDs other than the first candidate ID is smaller than that of the first candidate ID.

[0011] In the step of selectively reducing the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID, specifically: Calculate the third probability of the first candidate ID using the formula max_score-a*logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents a weighting, which is a hyperparameter or a learnable parameter.

[0012] Optionally, before performing inference using the large-scale language model, the method further comprises: The method includes fine-tuning the large-scale language model and / or the neural network using a second training set and a third training set, where the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data, and the fine-tuning process includes the steps of: if the input training data is the second training data, using the second training data to train and fine-tune the large-scale language model and the neural network; and if the input training data is the first training data, using the first training data to train the neural network.

[0013] In a second aspect of the present invention, an embodiment of the present invention comprises: a first training module that obtains a neural network by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output is a hidden layer representation obtained by the large-scale language model based on an input ID list, the output of the neural network is a probability that repetition occurs at a location of an ID corresponding to the first output, the first training data includes an ID list and a label, the ID list is generated by the large-scale language model based on a prompt text, and the label is used to indicate whether a location of each ID in the ID list is a location where repetition occurs; an inference module that, when performing inference using the large-scale language model, inputs a text to be inferred into the large-scale language model to obtain a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, the i-th ID position including the location of each ID generated by the large-scale language model; adjusts the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID; and determines an ID at the i-th ID position and a corresponding text unit based on the third probability of each candidate ID; The present invention provides an optimization device for the iterative generation problem of a large-scale language model including

[0014] The method further includes a first acquiring unit selectively generating one first training data set based on one first prompt text, and acquiring a plurality of first training data sets generated based on a plurality of first prompt texts to obtain the first training set; Here, generating one first training data based on one first prompt text includes: inputting the first prompt text into a large-scale language model to obtain a second ID list generated based on a first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text; concatenating the first ID list with the second ID list to obtain a third ID list; generating a label for each ID in the first ID list to indicate that the corresponding ID should be ignored in the training process; generating a label for each ID in the second ID list to indicate whether the location of the corresponding ID is a position where repetitive generation occurs; concatenating the label for each ID in the first ID list with the label for each ID in the second ID list to obtain a label for the third ID list; and generating a first training data set based on the third ID list and the label for the third ID list.

[0015] Optionally, a first probability of each candidate ID at a specific ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. A recurrence probability that a recurrence occurs at a specific ID position is generated by a neural network based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. The fourth ID list is an ID list composed of IDs corresponding to each text unit in the inference target text.

[0016] Optionally, the inference module further comprises: Based on the second probability, the first probability of the first candidate ID is reduced to obtain a third probability of the first candidate ID, the first candidate ID being the ID with the largest first probability, the degree of reduction in the first probability of the first candidate ID being positively correlated with the second probability, and the rate of reduction in the first probability of other candidate IDs other than the first candidate ID being smaller than that of the first candidate ID.

[0017] Optionally, further comprising a fine-tuning module, the fine-tuning module comprising: Before performing inference using the large-scale language model, a second training set and a third training set are used to fine-tune the large-scale language model and / or the neural network, where the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data. During the fine-tuning process, if the input training data is the second training data, the second training data is used to train and fine-tune the large-scale language model and the neural network, and if the input training data is the first training data, the first training data is used to train the neural network.

[0018] According to a third aspect of the present invention, at least one embodiment provides an optimization apparatus for a large-scale language model iterative generation problem, including a processor, a memory, and a program or command stored in the memory and executable on the processor, the program or command, when executed by the processor, implementing steps of a method according to any one of the first aspects.

[0019] According to a fourth aspect of the present invention, at least one embodiment relates to a computer-readable recording medium having a program stored thereon, which, when executed by a processor, performs the steps of any one of the methods according to the first aspect.

[0020] According to a fifth aspect of the present invention, at least one embodiment provides a computer process product including computer instructions that, when executed by a processor, cause the steps of a method according to any one of the first aspects to be implemented. [Effects of the Invention]

[0021] Compared with the prior art, the method and apparatus for optimizing the iterative generation problem of a large-scale language model according to an embodiment of the present invention introduces a neural network into the large-scale language model inference process, and reduces the probability of ID / text units that are prone to iterative generation according to the occurrence probability of the iterative generation phenomenon, thereby improving the iterative generation problem of the large-scale language model, improving the readability of text generated by the large-scale language model, and improving the user experience. Furthermore, the embodiment of the present invention fine-tunes the large-scale language model and the neural network, allowing the large-scale language model to adapt to the adjustment of the neural network during the fine-tuning process, thereby mitigating the situation of infinite iterative generation. [Brief explanation of the drawings]

[0022] Various other benefits and advantages will become more apparent to those skilled in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention. Furthermore, like reference numerals refer to like elements throughout the drawings. [Figure 1] FIG. 1 is a flowchart of an optimization method for the iterative generation problem of a large-scale language model according to one embodiment of the present invention. [Figure 2] FIG. 2 is a flowchart of a method for optimizing the iterative generation problem of a large-scale language model according to another embodiment of the present invention. [Figure 3] FIG. 3 is a schematic diagram of the connections between a large-scale language model and a neural network according to an embodiment of the present invention. [Figure 4] FIG. 4 illustrates an example of a method for optimizing the problem of iterative generation of a large-scale language model according to an embodiment of the present invention. [Figure 5] FIG. 5 is a structural schematic diagram of an optimization device for the iterative generation problem of a large-scale language model according to the present invention. [Figure 6] FIG. 6 is another structural schematic diagram of the optimization device for the iterative generation problem of a large-scale language model according to the present invention. [Figure 7]FIG. 7 is another structural schematic diagram of the optimization device for the iterative generation problem of large-scale language models according to the present invention. [Figure 8] FIG. 8 is a structural schematic diagram of a question answering system according to an embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing an example of the hardware configuration of a computer according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram illustrating an example of the hardware configuration of a mobile terminal according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram illustrating a workflow of the question answering system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0023] In order to clarify the technical problems, technical solutions, and advantages of the present invention, a detailed description will be provided in combination with the accompanying drawings and specific embodiments. The following description of specific details, such as detailed layouts and units, is provided solely for the purpose of understanding the embodiments of the present invention. Therefore, it is obvious to those skilled in the art that various modifications and amendments to the described embodiments are possible within the spirit and scope of the present invention. Furthermore, for the sake of clarity and conciseness, descriptions of well-known functions and structures have been omitted.

[0024] It should be understood that the term "one embodiment" or "an embodiment" in the specification semantically indicates that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present invention. Therefore, the terms "in one embodiment" or "in one embodiment" in the specification do not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Terms such as "first," "second," and the like in the specification and claims of the present invention need not be used to describe a particular order or sequence, but are used to distinguish between similar objects. Furthermore, such terms may be interchangeable where appropriate, so that the embodiments described herein may be practiced, for example, in an order other than that illustrated or described herein. Furthermore, the terms "comprise" and "have" and any variations thereof are intended to cover a non-exclusive inclusion, for example, that a process, method, system, product, or device comprising a series of steps or units need not be limited to those steps or units expressly listed, but may include other steps or units not expressly listed or inherent to those processes, methods, products, or devices. "And / or" in this specification and claims means at least one of the connected objects.

[0025] In each embodiment of the present invention, the numbers of the following processes do not semantically indicate the order of execution, but the order of execution of each process should be determined by its function and inherent logic, and should not impose any limitations on the execution process of the embodiment of the present invention.

[0026] The following description is illustrative, not limiting, of the scope, applicability, or configuration set forth in the claims. Without departing from the spirit and scope of the present invention, the functions and arrangements of the elements under consideration can be changed. Various examples can appropriately omit, substitute, or add various regulations or components. For example, a method described in a different order than the described order can be executed, and various steps can be added, omitted, or combined. Further, features described with reference to one illustration can be combined with other illustrations.

[0027] The inference process of large language models such as question and answer, query, guidance, etc. usually interacts with the large language model through an interactive form. Therefore, the interpretability of the output content of the large language model is very important for users. As described in the background art, in the field of natural language generation, there may be a problem of iterative generation (also called infinite iterative generation) in the text generated by the large language model. Iterative generation means that the text generated by the large language model contains an iterative text block in which at least one text unit (token) occurs continuously. For example, when a certain text (Chinese) is "我已経準備好出発我知道我知道我知道……" (translation: "I have already prepared to set off. I know. I know. I know..."), multiple iterative text blocks of "我が知っている" appear continuously in the text. Here, since the first "我が知っている" is the first appearance of the iterative text block, iterative generation has not yet appeared until the first "我が知っている". Each "我が知っている" from the second "我が知っている" is an iterative generation of the first "我が知っている". Therefore, in the above text, the positions where iterative generation appears are the positions where each "我が知っている" starting from the second "我が知っている" is located. The iterative generation problem greatly affects the readability of the text generated by the large language model.

[0028] An embodiment of the present invention provides a method for optimizing the iterative generation problem of a large-scale language model by introducing a neural network, thereby improving the readability of text generated by the large-scale language model. As shown in Figure 1, the method includes:

[0029] In step S1, a neural network is obtained by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output is a hidden layer representation obtained by the large-scale language model based on an input ID list, the output of the neural network is the probability that repetition will occur at the location of the ID corresponding to the first output, the first training data includes an ID list and a label, the ID list is generated by the large-scale language model based on a prompt text, and the label is used to indicate whether the location of each ID in the ID list is a location where repetition will occur.

[0030] Here, an embodiment of the present invention introduces a neural network connected to a large-scale language model, the input of the neural network is the hidden layer representation obtained by the large-scale language model, and the output of the neural network is the probability that an iterative generation will occur at the location of the ID corresponding to the hidden layer representation.

[0031] In step S2, when performing inference using the large-scale language model, the text to be inferred is input into the large-scale language model to obtain a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, where the i-th ID position includes the location of each ID generated by the large-scale language model. Based on the second probability, the first probability of each candidate ID is adjusted to obtain a third probability of each candidate ID. Based on the third probability of each candidate ID, the ID at the i-th ID position and its corresponding text unit are determined.

[0032] Here, the neural network is used to obtain a second probability of repeated generation occurring at the location of each ID generated by the large-scale language model, and the second probability is further used to adjust the probability of candidate IDs for each ID generated by the large-scale language model, thereby reducing the probability of ID / text units that are prone to repeated generation, thereby improving the problem of duplicate generation in the large-scale language model, improving the readability of text generated by the large-scale language model, and improving the user experience.

[0033] Specifically, in step S2, the first probability of the first candidate ID can be reduced based on the second probability to obtain a third probability of the first candidate ID, wherein the first candidate ID is the ID with the largest first probability, the degree of reduction in the first probability of the first candidate ID is positively correlated with the second probability, and the rate of reduction in the first probability of other candidate IDs other than the first candidate ID is smaller than that of the first candidate ID.

[0034] Another embodiment of the present invention further provides a method for optimizing the large-scale language model iterative generation problem by generating one first training data set based on one first prompt text, obtaining multiple first training data sets generated based on multiple first prompt texts to obtain a first training set, and then training a neural network using the first training set. Note that the following method for constructing the first training data is merely an implementation method that can be adopted by the present invention and does not limit the present invention. The present invention can obtain the neural network by training only using the first training data required in the above-mentioned step S1.

[0035] Specifically, as shown in FIG. 2, the method includes:

[0036] In step 21, the first prompt text is input into a large-scale language model to obtain a second ID list generated based on the first ID list of the large-scale language model, where the first ID list includes IDs corresponding to each text unit in the first prompt text.

[0037] Here, the first prompt text may be extracted from a pre-constructed command dataset. The command dataset includes multiple command data, each of which includes one first prompt text (prompt) and one standard response (standard_response). The command dataset is also called a training dataset, and may in particular be a training dataset for a large-scale language model. An embodiment of the present invention uses the first prompt text in each command data to construct training data for training the neural network. First, the first prompt text is input into a large-scale language model. A large-scale language model generally includes a tokenizer or a related functional module with a tokenization function. An embodiment of the present invention uses the tokenizer or related functional module of the large-scale language model to divide the first prompt text into one or more text units (tokens) and convert each text unit into a corresponding ID, thereby obtaining a first ID list consisting of IDs corresponding to the text units. A text unit is a basic unit of text data and may be a word, a punctuation mark, a subword, or a character. Each text unit has a corresponding ID (also called an index) in the tokenizer, which may be in the form of a number. The ID list is sometimes referred to herein as the ID sequence.

[0038] The first ID list is then input to a large-scale language model, and a second ID list is generated using the large-scale language model. The second ID list can be converted into answer text, i.e., answers inferentially generated by the large-scale language model to the first prompt text. In a subsequent step, an embodiment of the present invention uses the second ID list and the first ID list to form training data for a neural network.

[0039] In step 22, the first ID list and the second ID list are concatenated to obtain a third ID list, a label is generated for each ID in the first ID list, which is used to indicate that the corresponding ID should be ignored in the training process, a label is generated for each ID in the second ID list, which is used to indicate whether the location of the corresponding ID is the location where iterative generation occurs, the label of each ID in the first ID list and the label of each ID in the second ID list are concatenated to obtain the label of the third ID list, and a first training data set is generated based on the third ID list and the labels of the third ID list.

[0040] Here, the first ID list and the second ID list are joined together to obtain the third ID list.

[0041] A corresponding label is generated for each ID in the first ID list, and the label is used to instruct the neural network to ignore the corresponding ID during the training process, e.g., -100, indicating that the corresponding position in the third ID list corresponds to no training required.

[0042] A corresponding label is generated for each ID in the second ID list, and the label is used to indicate whether the location of the corresponding ID is a location where repetitive generation occurs.

[0043] For example, if the location of the corresponding ID is where the repeated text block appears repeatedly, the ID may be labeled 1; if the location of the corresponding ID is not where a repetition occurs, the ID may be labeled 0. Taking the sentence "I've already prepared to leave, I know, I know, I know..." as an example, the generated labels are shown in Table 1. It should be noted that for ease of understanding, Table 1 uses the text units corresponding to each ID rather than the IDs. As can be seen from Table 1, each "I know" from the second "I know" is a repetition of the first "I know," and therefore, the IDs corresponding to each "I know" from the second "I know" are all labeled 1. Furthermore, the first "I know" is the first occurrence of the repeated text block, and no repetition has occurred at this point. Therefore, the IDs corresponding to the first "I know" are all labeled 0, and the remaining IDs are also all labeled 0.

[0044] [Table 1] For example, in an embodiment of the present invention, if the location of a corresponding ID is the repeated occurrence location of a repeated text block, the label of the ID can be labeled as 1; if the location of a corresponding ID is the first occurrence location of a repeated text block, the label of the ID can be labeled as 0.5; if the location of a corresponding ID is not the location of a repeated text block, the label of the ID can be labeled as 0, thereby obtaining the labels shown in Table 2.

[0045] [Table 2] Then, by using the third ID list as sample data and the labels of the third ID list as label data, one piece of first training data can be obtained. By generating multiple pieces of first training data based on multiple first prompt texts according to the above method, a first training set including multiple pieces of first training data can be obtained.

[0046] In step 23, a neural network is constructed, the input of which is the first output of the large-scale language model, and the output of the neural network is the probability that repeated generation will occur at the location of the ID corresponding to the first output, where the first output is a hidden layer representation obtained by the large-scale language model based on the input ID list, and the neural network is trained using a first training set including a plurality of first training data to obtain the trained neural network.

[0047] Here, the neural network may be a binary classification neural network whose input is the first output of a large-scale language model and whose output is the probability that a recursive generation occurs at the location of the ID corresponding to the first output. Specifically, the input / output dimension of the neural network may be (embedded_size, 1), where embedded_size indicates the size of the hidden layer of the large-scale language model and 1 indicates that a probability value is output. Through the above steps, the embodiment of the present invention can be obtained by training the first neural network.

[0048] FIG. 3 illustrates the connection relationship between a large-scale language model and a neural network. The input of the large-scale language model (LLM) 201 is a list of IDs, such as ID_0 to ID_4 in FIG. 3. In this case, the large-scale language model 201 obtains a hidden layer representation (also called an embedding representation) based on the input ID list. The hidden layer, as shown by 202_1 to 202_m in FIG. 3, indicates the probability value of each ID in the vocabulary obtained through the related processing of the large-scale language model, where m is the number of IDs in the vocabulary. Typically, the large-scale language model selects the ID with the largest probability value as the currently generated ID, i.e., ID_5. In this embodiment of the present invention, a neural network 203 is introduced. The input of the neural network 203 is the hidden layer representation obtained by the large-scale language model 201 based on the input ID list, as indicated by the line connected to the neural network 203 in FIG. 3. The output 204 of the neural network 203 is the probability of the occurrence of the repeated generation of the currently generated ID_5.

[0049] When training the neural network, the embodiment of the present invention freezes the parameters of the large-scale language model, inputs the third ID list in the first training data into the large-scale language model, and outputs the hidden layer representation of the large-scale language model to the neural network, which generates its output based on the hidden layer representation. In this way, a loss value can be calculated based on the output of the neural network and the labels in the third ID list, and the parameters of the neural network are continuously adjusted based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network.

[0050] In step S24, when performing inference using the large-scale language model, the text to be inferred is input into the large-scale language model to obtain a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, where the i-th ID position is the position of the i-th ID generated by the large-scale language model. Based on the second probability, the first probability of each candidate ID is adjusted to obtain a third probability of each candidate ID. Based on the third probability of each candidate ID, the ID at the i-th ID position and its corresponding text unit are determined.

[0051] Here, in the process of inference using the large-scale language model, the text to be inferred is input into the large-scale language model, and the large-scale language model generates a first probability for each candidate ID at each ID position. The candidate IDs usually include all IDs in the tokenizer of the large-scale language model, with different IDs corresponding to different text units. Prior art generally uses the first probability for each candidate ID at each ID position to determine the candidate ID corresponding to the highest probability value as the ID at that ID position, and then converts the ID into a corresponding text unit to obtain the text unit corresponding to that ID position. The answer text generated by the large-scale language model can be obtained by concatenating all the text units corresponding to the ID positions generated by the large-scale language model.

[0052] To reduce the effect of repetitive generation, an embodiment of the present invention obtains, in step 24, a first probability of each candidate ID at the ith ID position generated by the large-scale language model, where the ith ID position includes the position where each ID generated by the large-scale language model is located. Here, the first probability of each candidate ID at a certain ID position (for convenience of explanation, referred to as a specific ID position) is generated by the large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the inference target text. For example, the first probability of each candidate ID at the first ID position generated by the large-scale language model is generated based on the first ID in the fourth ID list, the first probability of each candidate ID at the second ID position generated by the large-scale language model is generated based on the first to second IDs in the fourth ID list, the first probability of each candidate ID at the third ID position generated by the large-scale language model is generated based on the first to third IDs in the fourth ID list, and so on.

[0053] Additionally, embodiments of the present invention utilize a previously trained neural network to determine a second probability that a repeat generation will occur at the i-th ID position, where the repeat probability that a repeat generation will occur at a particular ID position is generated by the neural network based on a partial ID up to the ID corresponding to the particular ID position in the fourth ID list. For example, the second probability that a repeat generation will occur at a first ID position generated by the neural network is generated based on the first ID in the fourth ID list, the second probability that a repeat generation will occur at a second ID position generated by the neural network is generated based on the first through second IDs in the fourth ID list, the second probability that a repeat generation will occur at a third ID position generated by the neural network is generated based on the first through third IDs in the fourth ID list, and so on.

[0054] Next, the first probability of each candidate ID is adjusted according to the second probability to obtain a third probability for each candidate ID. Here, the first probability of the first candidate ID needs to be reduced based on the second probability to obtain the third probability of the first candidate ID. The first candidate ID is the ID with the largest first probability, and the reduction in the first probability of the first candidate ID is positively correlated with the second probability. That is, the larger the second probability, the greater the reduction in the first probability value of the first candidate ID. The reduction is the difference between the first probability and the third probability of the first candidate ID. Furthermore, the reduction rate of the first probability of candidate IDs other than the first candidate ID is smaller than the reduction rate of the first probability of the first candidate ID. That is, the reduction rate of the first probability value of the first candidate ID is the largest. The reduction rate of the first probability of a certain candidate ID is the ratio of the difference between the first probability and the third probability of that candidate ID to the first probability of that candidate ID.

[0055] For example, the calculation method for reducing the first probability of the first candidate ID according to the second probability to obtain the third probability of the first candidate ID is as follows:

[0056] The third probability of the first candidate ID is calculated using the formula max_score-a*logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents a weighting factor, which is generally a positive number greater than 0, and the weighting factor is a hyperparameter or a learnable parameter, for example, learned during the process of simultaneous fine-tuning of a neural network and a large-scale language model.

[0057] Then, according to the third probability of each candidate ID, the candidate ID corresponding to the largest third probability is determined as the ID at the i-th ID position, and the text unit corresponding to the ID is determined as the i-th text unit generated by the large-scale language model. In this way, the embodiment of the present invention can obtain the text units generated by the large-scale language model, and can connect these text units to obtain the response text inferred and generated by the large-scale language model.

[0058] Through the above steps, the embodiment of the present invention introduces a neural network into the large-scale language model inference process, and reduces the probability of ID / text units that are prone to duplicate generation according to the occurrence probability of the duplicate generation phenomenon, thereby improving the duplicate generation problem of the large-scale language model, improving the readability of the text generated by the large-scale language model, and improving the user experience.

[0059] In addition, the above embodiment is performed by using the ID list generated by the large-scale language model in the process of training the neural network, which corresponds to the fact that the large-scale language model is not trained in the training process, and the parameters of the large-scale language model are frozen in the neural network training process.

[0060] In an embodiment of the present invention, the neural network is introduced into the inference process of the large-scale language model to adjust the generation results of the large-scale language model. Therefore, in order to maintain consistency between fine-tuning and training and improve the inference results of the large-scale language model, the embodiment of the present invention can fine-tune the large-scale language model and the neural network between steps 23 and 14, and the large-scale language model can adapt to the adjustment of the neural network in the fine-tuning process, thereby mitigating the situation of infinitely repeated generation.

[0061] Specifically, before performing inference using the large-scale language model, an embodiment of the present invention may fine-tune the large-scale language model and / or the neural network using a second training set and a third training set, where the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data. In the fine-tuning process, if the input training data is the second training data, the second training data is used to train and fine-tune the large-scale language model and the neural network, and if the input training data is the first training data, the first training data is used to train the neural network.

[0062] For example, when performing fine-tuning training on the large-scale language model and the neural network using a second training set, obtain a fourth probability of each candidate ID at the i-th ID position generated by a large-scale language model and a fifth probability of repeated generation occurring at the i-th ID position generated by the neural network; adjust the fourth probability of each candidate ID based on the fifth probability to obtain a sixth probability of each candidate ID; determine an ID at the i-th ID position based on the sixth probability of each candidate ID; and determine the text unit corresponding to the ID at the i-th ID position as the text unit at the i-th ID position generated by the large-scale language model; Based on the text units at each ID position generated by the large-scale language model, the predicted text generated by the large-scale language model is obtained, and the large-scale language model and the neural network are optimized according to the predicted text and the standard answer. For example, a loss value between the predicted text and the standard answer is calculated, and the loss value is used for backpropagation to optimize the parameters of the large-scale language model and the neural network until a preset termination condition is met. The relevant loss function and specific implementation method of the fine-tuning process can be referred to in the description of the prior art, and detailed description will be omitted in the embodiments of the present invention.

[0063] The above method is further explained below through more detailed exemplary flows.

[0064] As shown in Figure 4, the flow includes:

[0065] Step 41 presents data acquisition.

[0066] A command dataset containing multiple command data is acquired, and each command data generally includes one prompt data (prompt) and one standard response (standard_response). Table 3 shows an example of a command dataset consisting of several command data, and the command dataset can be prepared for a specific downstream task, such as a medical dataset, a financial dataset, a code dataset, etc.

[0067] [Table 3] In step 42, training data is obtained.

[0068] In step 41, one Prompt = (Prompt1, Prompt2, ..., Prompt n ) contains n characters, and each Prompt irepresents a character or word in Prompt. Input Prompt to the tokenizer of a large-scale language model to obtain an ID corresponding to each token in Prompt, and then create a list of IDs = (ID1, ID2, ..., ID m ) and obtain each ID i Assume that one token in Prompt represents an ID in the vocabulary of a large-scale language model, where Prompt is split into m tokens by the tokenizer of the large-scale language model. The length of the ID may differ from the length of the characters in Prompt, because the tokenizer of the large-scale language model may combine multiple characters into one token, such as "hello", or split one long word into multiple tokens.

[0069] The ID is input to the large-scale language model, and the large-scale language model generates an ID corresponding to a new token, which is written as ID_new. This indicates the content that the large-scale language model continues to generate for the given ID. At the same time, ID_new is added to the ID, and the updated ID = (ID1, ID2, ..., ID m ,ID_ new ) is obtained. The above operation is repeated, and one new ID_new is generated each time until ID_end is generated in the large-scale language model. ID_end is an end token, which indicates that the large-scale language model considers generation to be finished. Finally, the previous m IDs in ID are deleted, and only the contents generated by the large-scale language model are retained, and ID_response = (ID_new1, ID_new2, ..., ID_ newp ,ID_ end ), which is assumed to contain p IDs generated by large language models and one end ID.

[0070] Perform labeling on ID_response, mark the ID where the infinite iteration generation phenomenon appears as 1, the remaining IDs as 0, and the transition IDs between 0 and 1 as 0.5, and obtain the labeled label data (Label). Table 4 shows the labeling of infinite iteration generation data (the ID is converted to character format for easy understanding). (Note that the sentence starting with "No" in the first row of Table 4 is in Chinese, and its translation is "It is not recommended to eat raw cheese that eats raw cheese.")

[0071]

Table 4

[0072] A model that can identify the labels obtained in step 42 is trained. Here, a neural network is used as the model. Add a neural network nn to the large language model, and the dimension of nn is (embedded_size, 1), where embedded_size represents the size of the hidden layer of the large language model. In the training process, other parameters of the large language model are frozen and not involved in the training. In step 41, ID = (ID1, ID2,..., ID m , ID_ new1 , ID_ new2 ,..., ID_ newp , ID_ end ) is used as the input, and the Label obtained in step 42 is complemented with -100 to get Label = (-1001, -1002,..., -100 m , Label1, Label2,..., Label p , Label_ end) to match the length of the Label and ID, where -100 indicates that this position does not need to be trained. By training a neural network nn using pairs of IDs and corresponding labels, a large-scale language model can generate probabilities for all candidate IDs for each ID position, while also having nn predict the probability that an infinite number of repeated generations may occur at that ID position.

[0073] In step 44, the large-scale language model and the neural network are combined.

[0074] The nn obtained in step 43 is combined with the generation of a large-scale language model, and the specific method is as follows:

[0075] Logits_i=LLM(i-1) Max_value=max(Logits_i) Index=Logtis_i.index(Max_value) Logits_i[Index]=Max_value-a*nn(t-1) where Logits_i indicates the possible probability of all IDs in the vocabulary of the large-scale language model, given the i-th ID generated by the large-scale language model; LLM indicates the large-scale language model; and LLM(i-1) indicates the large-scale language model generates for the i-th ID based on the previous i-1 IDs.

[0076] Here, Max_value indicates the maximum value in Logits_i, and max(Logits_i) indicates how to obtain the maximum value for Logits_i.

[0077] Here, Index indicates the index in the vocabulary of the large-scale language model where the maximum value in Logits_i is found. Logits_i.index(Max_value) indicates the index at which the maximum value is obtained from Logits_i.

[0078] Here, Logits_i[Index] indicates that the maximum value in Logits_i is corrected. Max_value-a*nn(t-1) indicates that the corrected value is the prediction obtained by subtracting nn from the original maximum value. a indicates a learnable parameter to control the influence of nn.

[0079] The principle of the above combination is as follows: When generating a token, the large-scale language model will make a single prediction of the possibilities of all tokens in the vocabulary and usually select the token with the highest probability. Introducing NN means that when the large-scale language model generates a token, NN performs a probability score on the possibility of infinite repetition at the location of the token, and the large-scale language model uses this score to negatively affect the probability of the token with the highest prediction value in the vocabulary, thereby causing the large-scale language model to predict other tokens and alleviate the infinite repetition.

[0080] In step 45, decoding is performed.

[0081] Based on the Logits_i obtained in step 44, the token with the highest probability is selected as the prediction for the large-scale language model, and the calculation is as follows:

[0082] Pred=argmax(Logits_i) Here, Pred denotes an index based on which a token can be reversed from the vocabulary of a large-scale language model, and argmax denotes an index that selects the maximum value from Logtis_i.

[0083] The large-scale language model repeats this operation in the prediction process to alleviate the phenomenon of infinite repetitive generation; furthermore, the decoding method in step 44 is used to train the model in the training process, which can ensure the consistency of the large-scale language model in the training and inference stages.

[0084] Based on the above method, an embodiment of the present invention further provides an apparatus for implementing the above method. As shown in FIG. 5, the apparatus for optimizing the large-scale language model iterative generation problem provided in the embodiment of the present invention includes: a first training module 51, which constructs a neural network, the input of which is a first output of the large-scale language model, and the output of which is a probability of occurrence of repeated generation at the location of the ID corresponding to the first output, where the first output is a hidden layer representation obtained by the large-scale language model based on the input ID list; and trains the neural network using a first training set including a plurality of first training data to obtain the trained neural network; The inference module 52, when performing inference using the large-scale language model, includes: inputting the text to be inferred into the large-scale language model; obtaining a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, where the i-th ID position is the location of the i-th ID generated by the large-scale language model; adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID; and determining the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID.

[0085] Through the above modules, an embodiment of the present invention can use the probabilities obtained by the neural network to adjust the probability of each candidate ID generated by the large-scale language model, thereby improving the iterative generation problem of the large-scale language model.

[0086] Optionally, the first training module 51 freezes the parameters of the large-scale language model, inputs the ID list in the first training data into the large-scale language model, calculates a loss value based on the output of the neural network and the labels of the ID list, and continues to adjust the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network.

[0087] Optionally, a first probability of each candidate ID at a specific ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. A recurrence probability that a recurrence occurs at a specific ID position is generated by a neural network based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. The fourth ID list is an ID list composed of IDs corresponding to each text unit in the inference target text.

[0088] Optionally, the inference module 52 can further reduce the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID, wherein the first candidate ID is the ID with the largest first probability, and the reduction in the first probability of the first candidate ID is positively correlated with the second probability, and the reduction rate of the first probability of other candidate IDs other than the first candidate ID is smaller than that of the first candidate ID.

[0089] Optionally, the inference module 52 further calculates and obtains a third probability of the first candidate ID using the formula max_score-a*logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents a weighting, which is a hyperparameter or a learnable parameter.

[0090] Optionally, the device further includes:

[0091] A first training module may fine-tune the large-scale language model and / or the neural network using a second training set and a third training set before performing inference using the large-scale language model, wherein the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data, and during the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0092] As shown in FIG. 6, the apparatus for optimizing the large-scale language model iterative generation problem provided in the embodiment of the present invention includes: a first processing module 61 inputting a first prompt text into a large-scale language model to obtain a second ID list generated based on a first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text; a second processing module 62, which is used to concatenate the first ID list and the second ID list to obtain a third ID list, generate a tag (label) for each ID in the first ID list and indicate that the corresponding ID should be ignored during the training process, generate a tag for each ID in the second ID list and indicate whether the location of the corresponding ID is a location where repetitive generation occurs, concatenate the tag for each ID in the first ID list with the tag for each ID in the second ID list to obtain the tag for the third ID list, and generate a first training data set based on the third ID list and the tag for the third ID list; a first training module 63, in step S1, obtaining a neural network by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output is a hidden layer representation obtained by the large-scale language model based on an input ID list, the output of the neural network is a probability that repetition occurs at a location of an ID corresponding to the first output, the first training data includes an ID list and a tag, the ID list is generated by the large-scale language model based on a prompt text, and the tag is used to indicate whether a location of each ID in the ID list is a location where repetition occurs; and The inference module 64, when performing inference using the large-scale language model, includes: inputting the text to be inferred into the large-scale language model; obtaining a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, wherein the i-th ID position includes the location of each ID generated by the large-scale language model; adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID; and determining the ID at the i-th ID position and its corresponding text unit based on the third probability of each candidate ID.

[0093] With these modules, embodiments of the present invention can improve the problem of iterative generation of large-scale language models.

[0094] Optionally, the first training module 63 freezes the parameters of the large-scale language model, inputs a third ID list in the first training data into the large-scale language model, calculates a loss value based on the output of the neural network and the labels of the third ID list, and continues to adjust the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network.

[0095] Optionally, a first probability of each candidate ID at a specific ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. A recurrence probability that a recurrence occurs at a specific ID position is generated by a neural network based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. The fourth ID list is an ID list composed of IDs corresponding to each text unit in the inference target text.

[0096] Optionally, the inference module is further used to:

[0097] Based on the second probability, the first probability of the first candidate ID is reduced to obtain a third probability of the first candidate ID, the first candidate ID being the ID with the largest first probability, the degree of reduction in the first probability of the first candidate ID being positively correlated with the second probability, and the rate of reduction in the first probability of other candidate IDs other than the first candidate ID being smaller than that of the first candidate ID.

[0098] Optionally, the first probability of the first candidate ID is reduced according to the second probability to obtain a third probability of the first candidate ID, as described in detail below.

[0099] The third probability of the first candidate ID is calculated and obtained using the formula max_score-a*logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents a weighting factor, which is a hyperparameter or a learnable parameter.

[0100] Optionally, the device further includes:

[0101] A fine-tuning module may fine-tune the large-scale language model and / or the neural network using a second training set and a third training set before performing inference using the large-scale language model, wherein the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data, and during the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0102] It should be noted that each device / system provided by the above embodiments is a device corresponding to the method for optimizing the iterative generation problem of a large-scale language model, and the implementation forms in each of the above embodiments can be applied to the device embodiments to achieve the same technical effects. The above device provided by the embodiments of the present invention can implement all the method steps implemented by the above method embodiments and achieve the same technical effects, and the same parts and beneficial effects as those of the method embodiments in this embodiment will not be specifically described here.

[0103] Referring to FIG. 7, an embodiment of the present invention further provides a hardware structure block diagram of an optimization device for large-scale language model iterative generation problems. As shown in FIG. 7, the optimization device 700 for large-scale language model iterative generation problems includes: a processor 702; a memory 704 in which computer program commands are stored; Thus, the computer program commands, when executed by the processor, cause the processor 702 to perform the following steps: A neural network is obtained by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output is a hidden layer representation obtained by the large-scale language model based on an input ID list, the output of the neural network is a probability that repetition occurs at a location of an ID corresponding to the first output, the first training data includes an ID list and a tag, the ID list is generated by the large-scale language model based on a prompt text, and the tag is used to indicate whether a location of each ID in the ID list is a location where repetition occurs; and When performing inference using the large-scale language model, the text to be inferred is input into the large-scale language model to obtain a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, where the i-th ID position includes the location of each ID generated by the large-scale language model. Based on the second probability, the first probability of each candidate ID is adjusted to obtain a third probability of each candidate ID. Based on the third probability of each candidate ID, the ID at the i-th ID position and its corresponding text unit are determined.

[0104] It should be noted that the systems provided by the above embodiments are devices corresponding to the optimization methods for the iterative generation problem of large-scale language models, and the implementation forms in the above embodiments can all be applied to the device embodiments, achieving the same technical effects. The devices provided by the embodiments of the present invention can implement all the method steps implemented by the method embodiments and achieve the same technical effects, and the same parts and beneficial effects as those of the method embodiments in the present embodiments will not be specifically described here.

[0105] Furthermore, as shown in FIG. 7, the large-scale language model iterative generation problem optimization device 700 further includes a network interface 701 , an input device 703 , a hard disk 705 , and a display device 706 .

[0106] The above interfaces and devices are connected via a bus architecture. The bus architecture may include any number of buses and bridges interconnecting components. Specifically, various circuits, such as one or more central processors (CPUs) and / or graphics processors (GPUs) represented by processor 702, and one or more memories represented by memory 704, are connected. The bus architecture may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits. Needless to say, the bus architecture is used to realize connected communication between these units. It is well known in the art that the bus architecture may include a power bus, a control bus, and a status signal bus in addition to a data bus, and therefore a detailed description thereof will be omitted.

[0107] The network interface 701 can be connected to a network (such as the Internet or a local area network), receive data such as prompt text from the network, and store the received data in the hard disk 705 .

[0108] The input means 703 receives various commands input by an operator and sends them to the processor 702 for execution. The input means 703 includes a keyboard or a click device (such as a mouse, a trackball, a touch-sensitive plate, or a touch screen).

[0109] The display device 706 can display the results of the processor 702 executing commands, such as the progress of model training.

[0110] The memory 704 is used to store programs and data required for system operation, as well as data such as intermediate results obtained during the calculation process of the processor 702 .

[0111] As will be appreciated, memory 704 in embodiments of the invention may be volatile or nonvolatile memory, or may include both volatile and nonvolatile memory. Thus, nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM). Volatile memory may be random access memory (RAM), such as external cache. Memory 704 of the devices and methods described herein includes, but is not limited to, these and any other suitable types of memory.

[0112] In some embodiments, memory 704 stores the following elements: executable modules or data structures, or a subset or extension thereof: an operating system 7041 and application programs 7042 .

[0113] The operating system 7041 includes various system programs, such as a framework layer, a core layer, and a driver layer, for implementing various basic operations and hardware-based tasks. The application programs 7042 include various application programs, such as a browser, for implementing various application operations. A program for implementing the method of an embodiment of the present invention can be included in the application programs 7042.

[0114] The method according to the above embodiment of the present invention can be applied to or implemented by the processor 702. The processor 702 may be an integrated circuit chip capable of processing signals. In the implementation process, each step of the above method can be completed by a hardware integrated logic circuit in the processor 702 or by a command in software form. The processor 702 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-configurable logic array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the method, steps, and logic block diagrams disclosed in the present embodiment. The general-purpose processor may be a microprocessor or any conventional processor. The method steps disclosed in the embodiments of the present invention can be directly implemented by a hardware decoder processor or by a combination of hardware and software modules in the decoder processor. The software modules can be stored in a storage medium established in the art, such as a random memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, or a register. The recording medium is in the memory 704, and the processor 702 reads the information in the memory 704 and completes the steps of the above method in conjunction with the hardware.

[0115] It is understood that the embodiments described herein may be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing unit may be implemented as one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units, or a combination thereof, for performing the functions described in the present invention.

[0116] For a software implementation, the techniques herein may be implemented with modules (e.g., processes, functions, etc.) that perform the functions described herein. The software code may be stored in a memory and executed by a processor. The memory may be within the processor or external to the processor.

[0117] Specifically, the computer program can realize the following steps when executed by the processor 702: generating first training data based on the first prompt text before training the neural network using a first training set including a plurality of first training data; and obtaining a plurality of first training data generated based on a plurality of first prompt texts to obtain the first training set; Wherein, the step of generating one first training data based on one first prompt text includes: The first prompt text is input into a large-scale language model to obtain a second ID list generated based on the first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text; the first ID list and the second ID list are concatenated to obtain a third ID list; a tag is generated for each ID in the first ID list, which is used to indicate that the corresponding ID should be ignored during the training process; a tag is generated for each ID in the second ID list, which is used to indicate whether the location of the corresponding ID is a position where repeated generation occurs; the tag of each ID in the first ID list is concatenated with the tag of each ID in the second ID list to obtain the tag of the third ID list; and a first training data set is generated based on the third ID list and the tag of the third ID list.

[0118] Specifically, the computer program can realize the following steps when executed by the processor 702: The parameters of the large-scale language model are frozen, the ID list in the first training data is input into the large-scale language model, a loss value can be calculated based on the output of the neural network and the labels of the ID list, and the parameters of the neural network are continuously adjusted based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network.

[0119] Optionally, a first probability of each candidate ID at a specific ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. A recurrence probability that a recurrence occurs at a specific ID position is generated by a neural network based on partial IDs up to the ID corresponding to the specific ID position in a fourth ID list. The fourth ID list is an ID list composed of IDs corresponding to each text unit in the inference target text.

[0120] Specifically, the computer program can realize the following steps when executed by the processor 702: Based on the second probability, the first probability of the first candidate ID is reduced to obtain a third probability of the first candidate ID, the first candidate ID being the ID with the largest first probability, the degree of reduction in the first probability of the first candidate ID being positively correlated with the second probability, and the rate of reduction in the first probability of other candidate IDs other than the first candidate ID being smaller than that of the first candidate ID.

[0121] Specifically, the computer program can realize the following steps when executed by the processor 702: The third probability of the first candidate ID is calculated and obtained using the formula max_score-a*logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents a weighting factor, which is a hyperparameter or a learnable parameter.

[0122] Specifically, the computer program can realize the following steps when executed by the processor 702: Before performing inference using the large-scale language model, the large-scale language model and / or the neural network are fine-tuned using a second training set and a third training set, where the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data, and during the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0123] It should be noted that the above-mentioned apparatus provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned method embodiment and can achieve the same technical effects, and the same parts and beneficial effects as those of the method embodiment in this embodiment will not be specifically mentioned here.

[0124] FIG. 8 is a structural schematic diagram of a question answering system according to an embodiment of the present invention, which includes a client 801 and a large-scale language model execution device 802 .

[0125] Here, the client 801 may be a personal computer or a mobile terminal, or may be an application process running on the above-mentioned various terminals. The mobile terminal is a terminal device operated by a user. The mobile terminal may be a smartphone, a personal digital assistant (PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, and a terminal device in a next-generation communication system, such as a terminal device in an NR network or a terminal device in a future evolved public land mobile network (PLMN) network.

[0126] The large-scale language model execution device 802 may be a server system composed of one or more computers. The large-scale language model execution device 802 executes a large-scale language model 803 and a neural network 804. The neural network 804 may be obtained by training an optimization method / device based on an iterative generation problem of a large-scale language model, and the large-scale language model 803 may be generated by a fine-tuning process in the optimization method / device based on an iterative generation problem of a large-scale language model.

[0127] In the question answering system 800, a client 801 may be connected to a large-scale language model execution device 802 via a wired and / or wireless network.

[0128] The functions of both the client 801 and the large-scale language model execution device 802 may be distributed across multiple computers.

[0129] The following describes the hardware or software structure of the relevant devices, apparatus or functions.

[0130] <Hardware configuration> (computer) The client 801 and the large-scale language model execution device 802 are realized, for example, by a computer having the hardware configuration shown in Fig. 9. Fig. 9 is a diagram showing an example of the hardware configuration of a computer according to an embodiment of the present invention.

[0131] 9, a computer 500 includes an input device 501, a display device 502, an external I / F 503, a RAM 504, a ROM 505, a CPU 506, a communication I / F 507, and an HDD 508, all of which are connected to one another via a bus B. The following structure is allowed to be formed: The input device 501 and the display device 502 are connected as necessary.

[0132] The input device 501 includes a keyboard, a mouse, a touch panel, etc., and the user inputs various operation signals via these input devices. The display device 502 includes a display that displays the processing results of the computer 500, etc.

[0133] The communication I / F 507 is an interface that is installed to connect the computer 500 to various networks. Therefore, the computer 500 performs data communication via the communication I / F 507.

[0134] The HDD 508 is an example of a non-volatile storage device that stores programs and data. The stored data includes an operating system (OS), which is basic software that controls the entire computer 500, and application software (hereinafter simply referred to as "applications") that provides various functions in the OS. Instead of the HDD 508, the computer 500 may use a drive device that uses flash memory (for example, a solid-state drive (SSD)) as a storage medium.

[0135] The external I / F 503 is an interface for an external device. The external device is a recording medium 503a or the like. In this case, the computer 500 reads information from or writes information to the recording medium 503a via the external I / F 503. The recording medium 503a is a flexible disk, a CD, a DVD, an SD memory card, a USB memory, or the like.

[0136] The ROM 505 is a non-volatile semiconductor memory (storage device) that can retain programs and / or data even when the power is turned off. The ROM 505 stores programs and data for a basic input / output system (BIOS) that is executed when the computer 500 starts up, OS settings, network settings, etc. The RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily stores programs and / or data.

[0137] The CPU 506 is an arithmetic unit that reads programs and / or data from storage devices such as the ROM 505 and HDD 508. The read programs or read data execute processes to realize the overall control or functions of the computer 500.

[0138] The client 801 and the large-scale language model execution device 802 are realized, for example, by the hardware configuration of the computer 500 shown in FIG.

[0139] (Mobile device) The client 801 is realized, for example, by a hardware configuration as shown in Fig. 10. Fig. 10 is a diagram showing an example of the hardware configuration of a mobile terminal according to an embodiment of the present invention. The mobile terminal 12 shown in Fig. 10 includes a CPU 601, a ROM 602, a RAM 603, an EEPROM 604, a CMOS sensor 605, an acceleration sensor and orientation sensor 606, and a media drive 608.

[0140] The CPU 601 controls the overall operation of the mobile terminal 12. The ROM 602 stores basic input / output programs. The RAM 603 is used as a work area for the CPU 601. The EEPROM 604 reads or writes data that matches the control of the CPU 601. The CMOS sensor 605 captures and acquires image data that matches the control of the CPU 601. The acceleration and direction sensor 606 is an electro-magnetic compass, gyrocompass, acceleration sensor, or the like that detects the Earth's magnetic force.

[0141] The media drive 608 controls the reading and writing (storage) of data sent to and received from a recordable medium 607 such as a flash memory. Data already stored in the recordable medium 607 is read, or new data is written to the recordable medium 607. The recordable medium 607 is detachable from the media drive 608.

[0142] The EEPROM 604 stores the operating system (OS) executed by the CPU 601, related information required for network configuration, etc. Applications for executing various processes in the first embodiment are stored in the EEPROM 604, a recordable medium 607, etc.

[0143] The CMOS sensor 605 is a charge-coupled device that converts light into an electric charge and digitizes an image of an object. The CMOS sensor 605 may be realized by, for example, a charge-coupled device (CCD) sensor, as long as it can capture an image of the object.

[0144] The mobile terminal 12 also includes an audio input unit 609, an audio output unit 610, an antenna 611, a communication unit 612, a wireless LAN communication unit 613, a wireless communication antenna 614, a wireless communication unit 615, a display 616, a touchpad 617, and a bus 619.

[0145] The audio input unit 609 converts the sound into an audio signal. The audio output unit 610 converts the audio signal into a sound. The communication unit 612 uses an antenna 611 to communicate with the nearest base station device by wireless communication signal. The wireless LAN communication unit 613 performs wireless LAN communication with an access point that complies with the IEEE80411 standard. The wireless communication unit 615 performs wireless communication using a wireless communication antenna 614.

[0146] The display 616 is provided to display target images, various icons, etc. The display 616 is made up of a liquid crystal display, an organic electroluminescence display, etc. The touch panel 617 is attached to the display 616 and is made up of a pressure-sensitive or electrostatic panel. The touch position on the display 616 is detected by touch with a finger or a touch pen. The bus 619 is an address bus, a data bus, etc., and electrically connects the above-mentioned multiple units or components.

[0147] The client 801 includes a dedicated battery 618. The client 801 is powered by the battery 618. The audio input unit 609 includes a microphone for inputting sound. The audio output unit 610 includes a speaker for outputting sound.

[0148] The client 801 is realized by, for example, a hardware configuration as shown in FIG.

[0149] 11, an example of the workflow of the question answering system 800 is provided. In this workflow, a large-scale language model and a neural network obtained by training in the embodiment of the present invention are used to perform inference to provide answers to questions submitted by users, thereby solving the problem of iterative generation of large-scale language models. The process specifically includes the following steps:

[0150] In S801, a user sends a question text to a large-scale language model execution device 802 via a client 801, and the large-scale language model execution device 802 receives the question text sent from the client; specifically, the large-scale language model 803 can receive the question text sent from the client 801.

[0151] In S802, the large-scale language model 803 generates probability values ​​for candidate IDs at each ID position in the answer based on the question text entered by the user, uses a neural network to generate the probability of repeated generation at each ID position, and further uses the probabilities to adjust the probability values ​​of the candidate IDs to reduce the probability of IDs that are likely to be repeated. Based on the adjusted candidate ID probabilities, the model determines the final IDs and corresponding text units at each ID position, and then generates a final answer text composed of these text units.

[0152] In S803, the large-scale language model execution device 802 transmits the final answer text to the client 801. The client 801 can display the final answer text via its display device to obtain the answer to the question text. Specifically, the client 801 can display the final answer text via the interface of its display device.

[0153] Some embodiments of the present invention further provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, perform the following steps:

[0154] The first prompt text is input into a large-scale language model to obtain a second ID list generated based on the first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text.

[0155] The first ID list and the second ID list are concatenated to obtain a third ID list, a tag is generated for each ID in the first ID list, and this tag is used to indicate that the corresponding ID should be ignored during the training process, a tag is generated for each ID in the second ID list, and this tag is used to indicate whether the location of the corresponding ID is the location where repeated generation occurs, the tag of each ID in the first ID list is concatenated with the tag of each ID in the second ID list to obtain the tag of the third ID list, and a first training data set is generated based on the third ID list and the tag of the third ID list.

[0156] A neural network is constructed, the input of which is a first output of the large-scale language model, and the output of the neural network is the probability that repeated generation will occur at the location of the ID corresponding to the first output, where the first output is a hidden layer representation obtained by the large-scale language model based on the input ID list, and the neural network is trained using a first training set including a plurality of first training data to obtain the trained neural network.

[0157] When performing inference using the large-scale language model, the text to be inferred is input into the large-scale language model to obtain a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, where the i-th ID position is the location of the i-th ID generated by the large-scale language model. Based on the second probability, the first probability of each candidate ID is adjusted to obtain a third probability of each candidate ID. Based on the third probability of each candidate ID, the ID at the i-th ID position and its corresponding text unit are determined.

[0158] When this program is executed by a processor, it can realize all the implementation methods in the optimization method for the iterative generation problem of a large-scale language model described above, and can achieve the same technical effects. To avoid duplication, the description will be omitted here.

[0159] An embodiment of the present invention further provides a computer program product including computer commands, which, when executed by a processor, can realize each process of the embodiment of the optimization method for the large-scale language model iterative generation problem described above, and achieve the same technical effects. To avoid redundancy, they will not be described again here.

[0160] Those skilled in the art can combine the illustrated units and algorithm steps of the embodiments disclosed herein to realize electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the functions using different methods depending on each specific application, but this implementation should not be considered beyond the scope of the present invention.

[0161] For the convenience and brevity of the description, those skilled in the art can refer to the corresponding processes in the above method embodiments for the specific working processes of the above systems, devices and units, and no further details will be given here.

[0162] It goes without saying that the methods and devices described in the embodiments provided by the present invention can be implemented in other ways. For example, the device embodiments described above are merely exemplary, and the division of the modules is only a logical function division. In reality, other division methods are also possible. For example, multiple modules or units may be combined or aggregated into other systems, or certain features may be omitted or not implemented. Furthermore, the shown or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections via interfaces, devices, or modules, which may be electrical, mechanical, or other types.

[0163] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or may be distributed across multiple network units, and some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present invention.

[0164] In addition, each functional unit in each embodiment of the present invention may be integrated into one process unit, each unit may exist physically alone, or two or more units may be integrated into one unit.

[0165] The functions can be implemented in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. As described above, the essence of the technical solution of the present invention, or a portion of the technical solution that contributes to the prior art, can be realized in the form of a software product, which is stored in a storage medium and contains some commands to cause a computer device (which may be a personal computer, a server, or a network device) to perform all or some of the steps of the methods described in each embodiment of the present invention. The storage medium includes the following: a USB, a removable hard disk, a ROM, a RAM, a magnetic disk, an optical disk, or other media capable of storing various program codes.

[0166] The above is a specific implementation of the present invention, and the scope of protection of the present invention is not limited thereto. Those skilled in the art can easily imagine modifications and substitutions within the technical scope disclosed in the present invention, all of which are included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined based on the scope of protection of the claims.

Claims

1. 1. A computer-implemented method for optimizing an iterative generation problem of a large-scale language model, comprising: Obtaining a neural network by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output being a hidden layer representation obtained by the large-scale language model based on an input ID list, and the output of the neural network is a probability that repetition occurs at a location of an ID corresponding to the first output, the first training data including an ID list and a label, the ID list being generated by the large-scale language model based on a prompt text, and the label being used to indicate whether a location of each ID in the ID list is a location where repetition occurs; When performing inference using the large-scale language model, the method includes the steps of: inputting a text to be inferred into the large-scale language model; obtaining a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, the i-th ID position including the location of each ID generated by the large-scale language model; adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID; and determining an ID at the i-th ID position and its corresponding text unit based on the third probability of each candidate ID. A method characterized by:

2. Before obtaining the neural network by training using a first training set including the plurality of first training data, further generating a first training data set based on a first prompt text; obtaining a plurality of first training data generated based on a plurality of first prompt texts to obtain the first training set; The step of generating one first training data based on one first prompt text includes: inputting the first prompt text into a large-scale language model to obtain a second ID list generated based on a first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text; concatenating the first ID list and the second ID list to obtain a third ID list; generating a label for each ID in the first list of IDs, the label being used to indicate that the corresponding ID should be ignored in a training process; generating a label for each ID in the second ID list, the label being used to indicate whether the location of the corresponding ID is a location where recursive generation occurs; concatenating a label for each ID in the first ID list with a label for each ID in the second ID list to obtain a label for the third ID list; generating a first training data set based on the third ID list and the labels of the third ID list; 2. The method of claim 1.

3. Obtaining a neural network by training using a first training set including the plurality of first training data includes: freezing parameters of the large-scale language model, inputting the ID list in the first training data into the large-scale language model, calculating a loss value based on the output of the neural network and the labels of the ID list, and continuously adjusting the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining a trained neural network.

2. The method of claim 1.

4. a first probability of each candidate ID at a specific ID position is generated by the large-scale language model based on a partial ID up to the ID corresponding to the specific ID position in the fourth ID list; a repetition probability that a repetition occurs at a specific ID position is generated by a neural network based on a partial ID up to the ID corresponding to the specific ID position in the fourth ID list; The fourth ID list is an ID list composed of IDs corresponding to each text unit in the text to be inferred.

2. The method of claim 1.

5. adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID; a first probability of a first candidate ID is reduced based on the second probability to obtain a third probability of the first candidate ID, the first candidate ID being an ID with the largest first probability, the reduction in the first probability of the first candidate ID being positively correlated with the second probability, and the reduction in the first probability of other candidate IDs other than the first candidate ID being smaller than that of the first candidate ID; 2. The method of claim 1.

6. Reducing the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID, Calculate the third probability of the first candidate ID using the formula max_score-a*logits_infinite, max_score indicates a first probability of the first candidate ID, logits_infinite indicates the second probability, a indicates a weighting, and the weighting is a hyperparameter or a learnable parameter.

6. The method according to claim 5.

7. and, prior to performing inference using the large-scale language model, fine-tuning the large-scale language model and / or the neural network using a second training set and a third training set; the second training set includes a plurality of second training data, each of which is composed of a second prompt text and a standard answer; the third training set includes at least one of the first training data; and in the fine-tuning process, if the input training data is the second training data, the second training data is used to train and fine-tune the large-scale language model and the neural network; and if the input training data is the first training data, the first training data is used to train the neural network.

2. The method of claim 1.

8. An apparatus for optimizing a large-scale language model iterative generation problem, comprising: a first training module for obtaining a neural network by training using a first training set including a plurality of first training data, wherein the input of the neural network is a first output of the large-scale language model, the first output being a hidden layer representation obtained by the large-scale language model based on an input ID list, and the output of the neural network being a probability of repetition occurring at a location of an ID corresponding to the first output, the first training data including an ID list and a label, the ID list being generated by the large-scale language model based on a prompt text, and the label being used to indicate whether a location of each ID in the ID list is a location where repetition occurs; an inference module for, when performing inference using the large-scale language model, inputting a text to be inferred into the large-scale language model to obtain a first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability of repeated generation occurring at the i-th ID position obtained by the neural network, the i-th ID position including a location of each ID generated by the large-scale language model; adjusting the first probability of each candidate ID based on the second probability to obtain a third probability of each candidate ID; and determining an ID at the i-th ID position and a corresponding text unit based on the third probability of each candidate ID. An apparatus characterized in that

9. a first acquiring unit for generating one first training data set based on one first prompt text, and acquiring a plurality of first training data sets generated based on a plurality of first prompt texts to obtain the first training set; Generating a first training data set based on a first prompt text includes: inputting the first prompt text into a large-scale language model to obtain a second ID list generated based on a first ID list of the large-scale language model, the first ID list including IDs corresponding to each text unit in the first prompt text; concatenating the first ID list with a second ID list to obtain a third ID list; generating a label for each ID in the first ID list, the label indicating that the corresponding ID should be ignored in the training process; generating a label for each ID in the second ID list, the label indicating whether the location of the corresponding ID is a location where iterative generation occurs; concatenating the label of each ID in the first ID list with the label of each ID in the second ID list to obtain a label for the third ID list; and generating a first training data set based on the third ID list and the label of the third ID list.

9. The device according to claim 8, characterized in that

10. a first probability of each candidate ID at a specific ID position is generated by the large-scale language model based on a partial ID up to the ID corresponding to the specific ID position in the fourth ID list; a repetition probability that a repetition occurs at a specific ID position is generated by a neural network based on a partial ID up to the ID corresponding to the specific ID position in the fourth ID list; The fourth ID list is an ID list composed of IDs corresponding to each text unit in the text to be inferred.

9. The device according to claim 8, characterized in that

11. the inference module is further adapted to reduce the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID; the first candidate ID is an ID having the largest first probability, the extent of decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the rate of decrease in the first probability of candidate IDs other than the first candidate ID is smaller than that of the first candidate ID; 9. The device according to claim 8, characterized in that

12. a fine-tuning module that fine-tunes the large-scale language model and / or the neural network using a second training set and a third training set before performing inference using the large-scale language model; the second training set includes a plurality of second training data, each of which is composed of a second prompt text and a standard answer; the third training set includes at least one of the first training data; and in the fine-tuning process, if the input training data is the second training data, the second training data is used to train and fine-tune the large-scale language model and the neural network; and if the input training data is the first training data, the first training data is used to train the neural network.

9. The device according to claim 8, characterized in that

13. A program for causing a computer to execute the method according to any one of claims 1 to 7.

14. A computer-readable storage medium storing the program according to claim 13.

15. An apparatus for optimizing a large-scale language model iterative generation problem, comprising: Memory and a processor coupled to the memory; The memory stores a computer program; Apparatus, wherein the processor is configured to execute the computer program to implement the method of any one of claims 1 to 7.

Images