Optimization method and apparatus for the iterative generation problem of large-scale language models, storage medium and computer program

By integrating a neural network to adjust probabilities of repetitive text units, the iterative generation problem in large-scale language models is mitigated, enhancing text readability and user experience.

JP7948372B2Active Publication Date: 2026-11-11RICOH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2025113472
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2024-07-31
Filing Date
2025-07-04
Publication Date
2026-11-11
Estimated Expiration
2045-07-04

AI Technical Summary

Technical Problem

Large-scale language models suffer from the iterative generation problem, where they become overly dependent on previous content, leading to repetitive text blocks that degrade user experience and semantic analysis.

Method used

Introduce a neural network trained using a first training set containing ID lists and labels to adjust the probability of iterative generation, reducing the likelihood of repetitive text units by incorporating a neural network into the inference process of large-scale language models.

Benefits of technology

Improves the readability of generated text by reducing repetitive patterns and enhancing user experience through probability adjustment during the inference process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007948372000005
    Figure 0007948372000005
  • Figure 0007948372000006
    Figure 0007948372000006
  • Figure 0007948372000007
    Figure 0007948372000007
Patent Text Reader

Abstract

To provide a method and a device for optimizing a large language model repetition generation problem, a storage medium and a computer program.SOLUTION: The present invention introduces a neural network in the inference process of a large language model, and reduces the probability of an ID / text unit that is prone to repeated generation according to the occurrence probability of the repeated generation phenomenon, thereby improving the repeated generation problem of the large language model, improving the readability of the text generated by the large language model, and improving the user experience.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] This invention relates to the technical fields of machine learning and natural language processing (NLP), and more particularly to an optimization method and apparatus for large-scale language model iterative generation problems, a storage medium, and a computer program. [Background technology]

[0002] Large Language Models (LLMs) are models based on deep learning techniques and generally contain a large number of parameters for processing and generating natural language text. By performing pre-training on large amounts of text data, LLMs acquire powerful language understanding and generation capabilities by learning relationships between texts, semantic information, and patterns.

[0003] For large-scale language models, the structure of the large-scale language model is generally a decoder structure in a transformer, or at least includes a decoder. However, because the decoder structure generates the (t+1)th character from the previous t characters, this type of structure can cause the large-scale language model to become overly dependent on previous content, and consequently, can lead to the phenomenon of iterative generation (also known as infinite iterative generation). For example, if a user inputs "Q1", the response of the large-scale language model will be iteratively generated based on Q1 and the characters generated by each large-scale language model itself. If the response of the large-scale language model contains multiple consecutive repeating text blocks, it is considered an iterative generation phenomenon and has serious implications for user experience and semantic analysis.

[0004] To address the infinite iteration generation phenomenon, related techniques employ a method of sampling the logtis of the model output during the inference stage. However, instead of selecting the character with the highest probability each time, the technique improves diversity and reduces the infinite iteration generation phenomenon by selecting characters according to predetermined rules. However, even if the infinite iteration generation phenomenon is avoided as much as possible by using diversity, the problem is not fundamentally solved. [Overview of the project] [Problems that the invention aims to solve]

[0005] At least one embodiment of the present invention provides a method and apparatus, a storage medium and a computer program for optimizing the iterative generation problem of large-scale language models, which can improve the iterative generation problem of large-scale language models. [Means for solving the problem]

[0006] In one aspect of the present invention, embodiments of the present invention are: A neural network is obtained by training it using a first training set containing multiple first training data, wherein the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on the input ID list, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes one ID list and labels, the ID list is generated by the large language model based on one prompt text, and the labels are used to indicate whether the location of each ID in the ID list is a location where iterative generation occurs. When performing inference using the aforementioned large-scale language model, the steps include: inputting the text to be inferred into the large-scale language model; obtaining the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network, wherein the i-th ID position includes the location of each ID generated by the large-scale language model; adjusting the first probability of each candidate ID based on the second probability to obtain the third probability of each candidate ID; and determining the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID. This provides an optimization method for the iterative generation problem of large-scale language models, including those mentioned above.

[0007] Before obtaining a neural network by training it using a first training set containing multiple first training data sets, you can further... A step to generate one initial training data based on one initial prompt text, The steps include obtaining a first training set by acquiring a plurality of first training data generated based on a plurality of first prompt texts, Here, the step of generating one first training data based on one first prompt text is: The first prompt text is input into a large-scale language model to obtain a second ID list generated based on a first ID list of the large-scale language model, wherein the first ID list includes an ID corresponding to each text unit in the first prompt text. The first ID list and the second ID list are combined to obtain a third ID list. A step used to generate a label for each ID in the first ID list and to instruct the training process to ignore the corresponding ID, A step used to generate a label for each ID in the second ID list, indicating whether the location of the corresponding ID is a location where iterative generation occurs, The steps include: concatenating the labels of each ID in the first ID list with the labels of each ID in the second ID list to obtain the labels of the third ID list; The method includes the step of generating one first training data based on the third ID list and the labels of the third ID list.

[0008] The step of obtaining a neural network by training it using a first training set that includes multiple first training data sets, is as follows: The process includes freezing the parameters of the large-scale language model, inputting the ID list in the first training data into the large-scale language model, calculating a loss value based on the output of the neural network and the labels in the ID list, and continuously adjusting the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network.

[0009] Selectively, the first probability of each candidate ID at a particular ID position is generated by the large-scale language model based on the partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The recurrence probability at which repeated generation occurs at a particular ID position is generated by the neural network based on the partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the text to be inferred.

[0010] The step of selecting and adjusting the first probability of each candidate ID based on the second probability to obtain the third probability of each candidate ID is as follows: Based on the second probability, the first probability of the first candidate ID is reduced to obtain the third probability of the first candidate ID, the first candidate ID is the ID with the largest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs other than the first candidate ID includes smaller steps than that of the first candidate ID.

[0011] In the step of selecting and reducing the first probability of the first candidate ID based on the second probability to obtain the third probability of the first candidate ID, specifically, The third probability of the first candidate ID is calculated using the formula max_score - a * logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents the weight, which is a hyperparameter or learnable parameter.

[0012] Selectively, before performing inference using the large-scale language model, the method further: The process includes fine-tuning the large-scale language model and / or the neural network using a second training set and a third training set, wherein the second training set comprises a plurality of second training data consisting of a second prompt text and a standard response, and the third training set comprises at least one of the first training data, and the fine-tuning process includes the steps of training and fine-tuning the large-scale language model and the neural network using the second training data if the input training data is the second training data, and training the neural network using the first training data if the input training data is the first training data.

[0013] In a second aspect of the present invention, embodiments of the present invention are as follows: A neural network is obtained by training using a first training set containing multiple first training data, wherein the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on the input ID list, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes an ID list and labels, the ID list is generated by the large language model based on prompt text, and the labels are used to indicate whether the location of each ID in the ID list is a location where iterative generation occurs, and a first training module is provided. When performing inference using the aforementioned large-scale language model, the inference module inputs the text to be inferred into the large-scale language model, obtains the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network, the i-th ID position includes the location of each ID generated by the large-scale language model, adjusts the first probability of each candidate ID based on the second probability to obtain the third probability of each candidate ID, and determines the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID. This provides an optimization tool for the iterative generation problem of large-scale language models, including those mentioned above.

[0014] The system further includes a first acquisition unit that, optionally, generates one first training data based on one first prompt text, and acquires multiple first training data generated based on multiple first prompt texts to obtain the first training set, Here, generating one initial training data based on one initial prompt text is: inputting the first prompt text into a large language model, and obtaining a second ID list generated based on the first ID list of the large language model, wherein the first ID list includes an ID corresponding to each text unit in the first prompt text; concatenating the first ID list and the second ID list to obtain a third ID list; generating a label for each ID in the first ID list to instruct to ignore the corresponding ID in the training process; generating a label for each ID in the second ID list to indicate whether the position where the corresponding ID is located is a position where iterative generation occurs; concatenating the label of each ID in the first ID list and the label of each ID in the second ID list to obtain the label of the third ID list; and generating one piece of first training data based on the third ID list and the label of the third ID list.

[0015] Optionally, the first probability of each candidate ID at a specific ID position is generated by the large language model based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The iteration probability that iterative generation occurs at the specific ID position is generated by a neural network based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The fourth ID list is an ID list composed of IDs corresponding to respective text units in the inference target text.

[0016] Optionally, the inference module further is configured to reduce the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID, wherein the first candidate ID is the ID with the maximum first probability, the reduction amplitude of the first probability of the first candidate ID is positively correlated with the second probability, and the reduction rate of the first probability of other candidate IDs except the first candidate ID is smaller than that of the first candidate ID.

[0017] Optionally, the system further comprises a fine-tuning module, wherein the fine-tuning module Before performing inference using the large-scale language model, a second training set and a third training set are used to fine-tune the large-scale language model and / or the neural network, wherein the second training set includes a plurality of second training data consisting of second prompt texts and standard answers, and the third training set includes at least one of the first training data, and in the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0018] According to a third aspect of the present invention, at least one embodiment provides an optimization device for a large-scale language model iterative generation problem, which includes a processor, a memory, and a program or command stored in the memory and executable on the processor, wherein when the program or command is executed by the processor, a step of the method according to any one of the first aspects is realized.

[0019] According to a fourth aspect of the present invention, at least one embodiment relates to a computer-readable recording medium on which a program is stored, and when the program is executed by a processor, the steps of the method according to any one of the first aspects are realized.

[0020] According to a fifth aspect of the present invention, at least one embodiment provides a computer process product including a computer command, wherein a step of the method according to any one of the first aspects is realized when the computer command is executed by a processor. [Effects of the Invention]

[0021] Compared to conventional technologies, the method and apparatus for optimizing the iterative generation problem of large-scale language models according to the embodiment of the present invention improves the iterative generation problem of large-scale language models by introducing a neural network in the large-scale language model inference process and reducing the probability of ID / text units that are prone to iterative generation according to the probability of the occurrence of iterative generation phenomena, thereby improving the readability of the text generated by the large-scale language model and improving the user experience. Furthermore, the embodiment of the present invention allows the large-scale language model to adapt to the adjustment of the neural network during the fine-tuning process, thereby mitigating the situation of infinitely iterative generation. [Brief explanation of the drawing]

[0022] By reading the following detailed description of preferred embodiments, various other advantages and benefits will become clearer to those skilled in the art. The drawings are for illustrative purposes only of preferred embodiments and should not be construed as limiting the invention. Throughout the drawings, similar components are denoted by the same reference numerals. [Figure 1] Figure 1 is a flowchart of an optimization method for an iterative generation problem of a large-scale language model according to one embodiment of the present invention. [Figure 2] Figure 2 is a flowchart of an optimization method for the iterative generation problem of a large-scale language model according to another embodiment of the present invention. [Figure 3] Figure 3 is a schematic diagram of the connection relationship between a large-scale language model and a neural network according to an embodiment of the present invention. [Figure 4] Figure 4 illustrates an example of an optimization method for the iterative generation problem of a large-scale language model according to an embodiment of the present invention. [Figure 5] Figure 5 is a schematic diagram of the structure of the optimization device for the iterative generation problem of large-scale language models according to the present invention. [Figure 6] Figure 6 is a schematic diagram of another structure of the optimization device for the iterative generation problem of large-scale language models according to the present invention. [Figure 7]Figure 7 is a schematic diagram of another structure of the optimization device for the iterative generation problem of large-scale language models according to the present invention. [Figure 8] Figure 8 is a schematic diagram of the structure of a question answering system according to an embodiment of the present invention. [Figure 9] Figure 9 shows an example of a computer hardware configuration according to an embodiment of the present invention. [Figure 10] Figure 10 shows an example of the hardware configuration of a mobile terminal according to an embodiment of the present invention. [Figure 11] Figure 11 is an illustrative diagram of the workflow of a question answering system according to an embodiment of the present invention. [Modes for carrying out the invention]

[0023] To further clarify the technical problems, solutions, and advantages that this invention aims to solve, a detailed explanation is provided in combination with accompanying drawings and specific embodiments. The following descriptions of specific details such as detailed arrangements and units are provided solely for the purpose of understanding the embodiments of this invention. Therefore, it will be obvious to those skilled in the art that various changes and modifications can be made to the described embodiments within the scope of the spirit of this invention. Furthermore, for clarity and conciseness, descriptions of known functions and structures have been omitted.

[0024] Needless to say, the phrase "one embodiment" or "one example" in the specification semantically indicates that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in one example" in the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in one or more embodiments in any appropriate manner. The terms "first," "second," etc., in the specification and claims of the present invention do not need to be used to describe a specific order or sequence, but are used to distinguish similar subjects. The data used in this manner is interchangeable where appropriate, so that the embodiments described herein can be carried out, for example, in an order other than that illustrated or described herein. Furthermore, the terms “including” and “having” and any variations thereof are intended to cover a non-exclusive implication, for example, that a process, method, system, product, or device comprising a set of steps or units is not necessarily limited to those expressly listed steps or units, and may include other steps or units that are not expressly listed or are specific to those processes, methods, products, or devices. In this specification and the claims, “and / or” means at least one of the connected subjects.

[0025] In each embodiment of the present invention, the numbers of the processes described below do not semantically indicate the order in which they are executed. Rather, the execution order of each process should be determined by its function and inherent logic, and no limitations should be placed on the execution processes of the embodiments of the present invention.

[0026] The following description is illustrative rather than limiting the scope, applicability, or configuration set forth in the claims. Changes may be made in the function and arrangement of the elements discussed without departing from the spirit and scope of the present invention. Various examples may omit, substitute, or add various provisions or components as appropriate. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to one example may be combined in other examples.

[0027] The inference process of large language models for question answering, queries, guidance, and the like usually interacts with the large language model through an interactive form. Therefore, the interpretability of the output content of the large language model is of great importance to users. As described in the background art, in the field of natural language generation, the text generated by the large language model may suffer from a repetition generation problem (also referred to as infinite repetition generation). Repetition generation means that the text generated by the large language model contains consecutive repeated text blocks that include at least one text unit (token). For example, if a text (in Chinese) is "I am already ready to depart I know I know I know......", which corresponds to the Japanese translation "私は既に出発を準備した私が知っている私が知っている私が知っている......", a plurality of repeated text blocks "I know" appear consecutively in the text. Here, since the first "I know" is the first occurrence of the repeated text block, repetition generation has not yet occurred up to the first "I know". Each "I know" starting from the second "I know" is a repetition generation of the first "I know". Therefore, in the above text, the positions where repetition generation appears are the positions where each "I know" starting from the second "I know" is located. The repetition generation problem greatly affects the readability of the text generated by the large language model.

[0028] Embodiments of the present invention provide a method for optimizing the iterative generation problem of large-scale language models, thereby improving the readability of texts generated by large-scale language models by introducing neural networks. As shown in Figure 1, the method includes the following:

[0029] In step S1, a neural network is obtained by training using a first training set containing multiple first training data, where the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on the input ID list, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes an ID list and labels, the ID list is generated by the large language model based on prompt text, and the labels are used to indicate whether the location of each ID in the ID list is a location where iterative generation occurs.

[0030] In this embodiment of the present invention, a neural network connected to a large-scale language model is introduced, the input to the neural network is a hidden layer representation obtained by the large-scale language model, and the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the hidden layer representation.

[0031] In step S2, when performing inference using the large-scale language model, the text to be inferred is input to the large-scale language model, and the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network are obtained. The i-th ID position includes the location of each ID generated by the large-scale language model. Based on the second probability, the first probability of each candidate ID is adjusted to obtain a third probability for each candidate ID, and based on the third probability for each candidate ID, the ID at the i-th ID position and the corresponding text unit are determined.

[0032] Here, by using the aforementioned neural network, a second probability is obtained for the location of each ID generated by the large-scale language model where iterative generation occurs. Furthermore, by using this second probability to adjust the probability of candidate IDs for each ID generated by the large-scale language model, the probability of ID / text units prone to iterative generation is reduced, thereby improving the duplicate generation problem of the large-scale language model, enhancing the readability of the text generated by the large-scale language model, and improving the user experience.

[0033] Specifically, in step S2, the first probability of the first candidate ID can be reduced based on the second probability to obtain the third probability of the first candidate ID, the first candidate ID is the ID with the largest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs is smaller than that of the first candidate ID.

[0034] Another embodiment of the present invention further provides a method for optimizing a large-scale language model iterative generation problem by generating one first training data based on one first prompt text, obtaining a first training set by acquiring multiple first training data generated based on multiple first prompt texts, and further training a neural network using the first training set. The following methods for constructing the first training data are merely implementation methods that the present invention can employ and do not limit the present invention. The present invention can be used to train and obtain the neural network simply by using the first training data required in step S1 described above.

[0035] Specifically, as shown in Figure 2, the method includes the following:

[0036] In step 21, the first prompt text is input into the large language model to obtain a second ID list generated based on the first ID list of the large language model, the first ID list containing the ID corresponding to each text unit in the first prompt text.

[0037] Here, the first prompt text can be extracted from a pre-constructed command dataset. The command dataset contains multiple command data, each command data containing one first prompt text (prompt) and one standard response (standard_response). The command dataset is also called a training dataset, and may in particular be a training dataset for a large-scale language model. Embodiments of the present invention utilize the first prompt text in each command data to construct training data for training the neural network. First, the first prompt text is input into a large-scale language model, which generally includes a tokenizer or related functional module having tokenizing functionality. Embodiments of the present invention utilize the tokenizer or related functional module of the large-scale language model to divide the first prompt text into one or more text units (tokens), convert each text unit into a corresponding ID, thereby obtaining a first ID list consisting of IDs corresponding to the text units. A text unit is a basic unit in text data and may be a word, a punctuation mark, a subword, or a character. Each text unit has a corresponding ID (which may also be called an index) in the tokenizer, and the ID may be in numerical form. The ID list is sometimes referred to as the ID sequence in this specification.

[0038] Next, the first ID list is input into a large-scale language model, and the large-scale language model is used to generate a second ID list. The second ID list can be converted into response text, i.e., the response inferentially generated by the large-scale language model for the first prompt text. Embodiments of the present invention use the second ID list and the first ID list in subsequent steps to construct training data for a neural network.

[0039] In step 22, the first ID list and the second ID list are concatenated to obtain a third ID list, a label is generated for each ID in the first ID list and used to instruct the training process to ignore the corresponding ID, a label is generated for each ID in the second ID list and used to instruct whether the location of the corresponding ID is a location where iterative generation occurs, the labels of each ID in the first ID list and the labels of each ID in the second ID list are concatenated to obtain the label of the third ID list, and one first training data is generated based on the third ID list and the label of the third ID list.

[0040] Here, the first ID list and the second ID list are combined to obtain the third ID list.

[0041] Furthermore, a corresponding label is generated for each ID in the first ID list, and this label is used to instruct the neural network to ignore the ID corresponding to that label during the training process. For example, it can be represented as -100 to indicate that the corresponding position in the third ID list does not require training.

[0042] A corresponding label is generated for each ID in the second ID list, and this label is used to indicate whether the location of the corresponding ID is a location where iterative generation occurs.

[0043] For example, if the location of the corresponding ID is where the repeating text block appears repeatedly, the ID may be labeled with 1. If the location of the corresponding ID is not where the repetition generation occurs, the ID may be labeled with 0. Taking the preceding sentence, "I have already prepared to depart, I know, I know, I know...", as an example, the generated labels are shown in Table 1. What needs to be explained here is that, for ease of understanding, Table 1 does not display IDs, but uses text units corresponding to each ID. As can be seen from Table 1, each "I know" from the second "I know" is a repetition generation of the first "I know", and therefore, the IDs corresponding to each "I know" from the second "I know" are all labeled with 1. The first "I know" is the first appearance of the repeating text block, and at this point, repetition generation has not yet occurred, so all IDs corresponding to the first "I know" are labeled with 0, and all the remaining IDs are also labeled with 0.

[0044] [Table 1] Furthermore, in embodiments of the present invention, for example, if the location of the corresponding ID is a repeating occurrence of a repeating text block, the label of the ID can be labeled to 1; if the location of the corresponding ID is the first occurrence of a repeating text block, the label of the ID can be labeled to 0.5; and if the location of the corresponding ID is not a location of a repeating text block, the label of the ID can be labeled to 0, thereby obtaining the labels shown in Table 2.

[0045] [Table 2] Then, by using the third ID list as sample data and the labels of the third ID list as label data, one first training data can be obtained. Following the method described above, multiple first training data can be generated based on multiple first prompt texts, thereby obtaining a first training set containing multiple first training data.

[0046] In step 23, a neural network is constructed, the input to the neural network being the first output of the large-scale language model, and the output of the neural network being the probability that iterative generation occurs at the location of the ID corresponding to the first output, where the first output is a hidden layer representation obtained by the large-scale language model based on the input ID list, and the neural network is trained using a first training set containing multiple first training data to obtain the trained neural network.

[0047] Here, the neural network may be a binary classification neural network whose input is the first output of a large language model and whose output is the probability that iterative generation occurs at the location of the ID corresponding to the first output. Specifically, the input / output dimension of the neural network may be (embedded_size, 1), where embedded_size represents the size of the hidden layer of the large language model, and 1 represents the output of a probability value. By the above steps, an embodiment of the present invention can be obtained by training the first neural network.

[0048] Figure 3 provides the connection between a large-scale language model (LLM) and a neural network. The input to the LLM 201 is an ID list, such as ID_0 to ID_4 in Figure 3. In this case, the LLM 201 obtains a hidden layer representation (also called an embeddings representation) based on the input ID list. The hidden layer shows the probability value of each ID in the vocabulary obtained through the association processing of the LLM, as shown in 202_1 to 202_m in Figure 3, where m is the number of IDs in the vocabulary. Typically, the LLM takes the ID with the highest probability value as the currently generated ID, i.e., ID_5. In an embodiment of the present invention, a neural network 203 is introduced. The input to the neural network 203 is the hidden layer representation obtained by the LLM 201 based on the input ID list, and is shown by the lines connected to the neural network 203 in Figure 3. The output 204 of the neural network 203 is the probability that the currently generated ID_5 will be repeatedly generated.

[0049] When training a neural network, an embodiment of the present invention freezes the parameters of a large-scale language model, inputs a third ID list from the first training data into the large-scale language model, and thus outputs the hidden layer representation of the large-scale language model to the neural network, which then generates its output based on the hidden layer representation. In this way, a loss value can be calculated based on the output of the neural network and the labels of the third ID list, and based on the loss value, the parameters of the neural network are continuously adjusted until a preset training termination condition is met, thereby obtaining the trained neural network.

[0050] In step S24, when performing inference using the large-scale language model, the text to be inferred is input to the large-scale language model, and the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network are obtained. The i-th ID position is the location of the i-th ID generated by the large-scale language model. Based on the second probability, the first probability of each candidate ID is adjusted to obtain the third probability of each candidate ID, and based on the third probability of each candidate ID, the ID at the i-th ID position and the corresponding text unit are determined.

[0051] In the process of performing inference using the large-scale language model, the text to be inferred is input to the large-scale language model, and the large-scale language model generates a first probability for each candidate ID at each ID location. Candidate IDs typically include all IDs in the tokenizer of the large-scale language model, each having a different ID corresponding to a different text unit. Conventional techniques generally determine the candidate ID corresponding to the highest probability value based on the first probability of each candidate ID at each ID location, and use this as the ID at that ID location. Then, by converting this ID into the corresponding text unit, a text unit corresponding to that ID location can be obtained. By concatenating the text units corresponding to all ID locations generated by the large-scale language model, the answer text generated by the large-scale language model can be obtained.

[0052] To reduce the effects of iterative generation, embodiments of the present invention obtain, in step 24, the first probability of each candidate ID at the i-th ID position generated by the large-scale language model, where the i-th ID position includes the position where each ID generated by the large-scale language model is located. Here, the first probability of each candidate ID at a certain ID position (referred to as a specific ID position for convenience of explanation) is generated by the large-scale language model based on partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the inference target text. For example, the first probability of each candidate ID at the first ID position generated by the large-scale language model is generated based on the first ID in the fourth ID list, the first probability of each candidate ID at the second ID position generated by the large-scale language model is generated based on the first to second IDs in the fourth ID list, the first probability of each candidate ID at the third ID position generated by the large-scale language model is generated based on the first to third IDs in the fourth ID list, and so on.

[0053] Furthermore, embodiments of the present invention utilize a previously trained neural network to determine a second probability that iterative generation occurs at the i-th ID position. However, the recurrence probability that iterative generation occurs at a specific ID position is generated by the neural network based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. For example, the second probability that iterative generation occurs at the first ID position generated by the neural network is generated based on the first ID in the fourth ID list, the second probability that iterative generation occurs at the second ID position generated by the neural network is generated based on the first to second IDs in the fourth ID list, the second probability that iterative generation occurs at the third ID position generated by the neural network is generated based on the first to third IDs in the fourth ID list, and so on.

[0054] Next, the first probability of each candidate ID is adjusted according to the second probability to obtain the third probability of each candidate ID. Here, it is necessary to reduce the first probability of the first candidate ID based on the second probability to obtain the third probability of the first candidate ID, and the first candidate ID is the ID with the largest first probability, and the amount of reduction in the first probability of the first candidate ID is positively correlated with the second probability. That is, the larger the second probability, the greater the reduction in the first probability value of the first candidate ID. The reduction amount is the difference between the first probability and the third probability of the first candidate ID. Also, the reduction rate of the first probability of other candidate IDs is smaller than the reduction rate of the first probability of the first candidate ID. In other words, the reduction rate of the first probability value of the first candidate ID is the largest. The reduction rate of the first probability of a given candidate ID is the ratio of the difference between the first probability and the third probability of that candidate ID to the first probability of that candidate ID.

[0055] For example, the calculation method for obtaining the third probability of the first candidate ID by reducing the first probability of the first candidate ID according to the second probability is as follows:

[0056] The formula max_score - a * logits_infinite is used to calculate and obtain the third probability of the first candidate ID, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents the weight, which is generally a positive number greater than 0. The weight is a hyperparameter or a learnable parameter, which is learned, for example, in the process of simultaneously fine-tuning a neural network and a large language model.

[0057] Next, according to the third probability of each candidate ID, the candidate ID corresponding to the highest third probability is determined and set as the ID at the i-th ID position. Furthermore, the text unit corresponding to that ID is determined and set as the i-th text unit generated by the large-scale language model. In this way, the embodiment of the present invention can obtain text units generated by the large-scale language model, and by concatenating these text units, it is possible to obtain the response text inferred by the large-scale language model.

[0058] Through the steps described above, embodiments of the present invention can improve the duplicate generation problem of large-scale language models by introducing a neural network into the large-scale language model inference process and reducing the probability of duplicate generation for ID / text units that are prone to duplicate generation according to the probability of duplicate generation occurring, thereby improving the readability of the text generated by the large-scale language model and improving the user experience.

[0059] In addition, the above embodiment is performed using the ID list generated by the large-scale language model in the process of training the neural network, and is equivalent to not training the large-scale language model in the above training process, and the parameters of the large-scale language model being frozen in the neural network training process.

[0060] An embodiment of the present invention introduces the neural network in the inference process of a large-scale language model to adjust the generation results of the large-scale language model. To maintain consistency between fine-tuning and training and improve the inference results of the large-scale language model, the embodiment of the present invention allows for fine-tuning of the large-scale language model and the neural network between steps 23 and 14. The large-scale language model can adapt to the adjustment of the neural network during the fine-tuning process, mitigating the situation of infinitely iterative generation.

[0061] Specifically, before performing inference using the large-scale language model, embodiments of the present invention can fine-tune the large-scale language model and / or the neural network using a second training set and a third training set, wherein the second training set includes a plurality of second training data consisting of a second prompt text and a standard answer, and the third training set includes at least one of the first training data. Furthermore, in the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0062] For example, when performing fine-tuning training on the large-scale language model and the neural network using a second training set, The fourth probability of each candidate ID at the i-th ID position generated by the large-scale language model and the fifth probability of iterative generation occurring at the i-th ID position generated by the neural network are obtained, the fourth probability of each candidate ID is adjusted based on the fifth probability to obtain the sixth probability of each candidate ID, the ID at the i-th ID position is determined based on the sixth probability of each candidate ID, and the text unit corresponding to the ID at the i-th ID position is set as the text unit at the i-th ID position generated by the large-scale language model. Based on the text units at each ID location generated by the large-scale language model, the predicted text generated by the large-scale language model is obtained, and the large-scale language model and the neural network are optimized according to the predicted text and the standard response. For example, a loss value is calculated between the predicted text and the standard response, and the parameters of the large-scale language model and the neural network are optimized by backpropagation using the loss value until a preset termination condition is met. The relevant loss function and specific implementation method of the fine-tuning process can be found in the description of the prior art, and a detailed description of embodiments of the present invention is omitted.

[0063] The above method will be further explained below through a more detailed illustrative flow.

[0064] As shown in Figure 4, the flow includes the following:

[0065] Step 41 presents the data acquisition method.

[0066] A command dataset containing multiple command data sets is obtained, each command data set typically containing one prompt data (prompt) and one standard response (standard_response). Table 3 shows an example of a command dataset consisting of several command data sets, and command datasets can be prepared for specific downstream tasks, such as medical datasets, financial datasets, and coding datasets.

[0067] [Table 3] In step 42, the training data is obtained.

[0068] One Prompt in step 41 = (Prompt1, Prompt2, ..., Prompt n ) contains n characters, each Prompt irepresents a character or word within the Prompt. The Prompt is input into the tokenizer of a large language model to obtain the ID corresponding to each token in the Prompt, and then an ID containing m IDs = (ID1, ID2, ..., ID m ) obtain each ID i Here, one token in Prompt represents an ID in the lexicon of the large language model, and we assume that Prompt is divided into m tokens by the tokenizer of the large language model. The length of the ID may differ from the length of the character in Prompt, because the tokenizer of the large language model may combine multiple characters into one token, for example, "hello", and may also divide one long word into multiple tokens.

[0069] The ID is input into the large-scale language model, which generates an ID corresponding to the new token, denoting it as ID_new. This indicates what the large-scale language model continues to generate for the given ID, and simultaneously, ID_new is added to the ID, resulting in the updated ID = (ID1, ID2, ..., ID m ,ID_ new ) is obtained. The above operation is repeated, generating one new ID_new each time until ID_end is generated in the large language model, and it is shown that ID_end is the end token and the large language model has finished generating. Finally, the previous m IDs in ID are deleted, and only the content generated by the large language model is retained, so ID_response=(ID_new1,ID_new2,…,ID_ newp ,ID_ end ) obtain, where it is assumed that this includes IDs generated by p large language models and one termination ID.

[0070] Label the ID_response, mark the ID where the infinite iterative generation phenomenon appears as 1, the remaining IDs as 0, and the transition ID between 0 and 1 as 0.5, to obtain labeled label data (Label). Table 4 shows the labeling of infinite iterative generation data (IDs are converted to character form for ease of understanding) (note that the sentence starting with "不" in the first row of Table 4 is in Chinese, and its Japanese translation is "生チーズを食べる生チーズを食べるのはおすすめできません").

[0071] [Table 4] In step 43, the neural network is trained.

[0072] A model capable of identifying the labels obtained in step 42 is trained, where a neural network is used as the model. One neural network nn is added to the large language model, and the dimension of nn is (embedded_size, 1), where embedded_size indicates the size of the hidden layer of the large language model. During the training process, other parameters of the large language model are frozen and not involved in training. In step 41, ID=(ID1, ID2, ..., ID m , ID_ new1 , ID_ new2 , ..., ID_ newp , ID_ end ) is taken as input, the Label obtained in step 42 is complemented with -100, so that Label=(-1001, -1002, ..., -100 m , Label1, Label2, ..., Label p , Label_ endBy doing this, the length of the Label and the ID are matched, where -100 indicates that this position does not need to be trained. By training a neural network nn using pairs of IDs and corresponding Labels, a large-scale language model can generate probabilities of all candidate IDs for each ID position, while simultaneously allowing nn to predict the probability of infinite iterative generation occurring at that ID position.

[0073] In step 44, the large-scale language model and the neural network are combined.

[0074] The neural network (nn) obtained in step 43 is combined with the generation of a large-scale language model, and the specific method is as follows.

[0075] Logits_i=LLM(i-1) Max_value = max(Logits_i) Index=Logtis_i.index(Max_value) Logits_i[Index]=Max_value-a*nn(t-1) Here, Logits_i represents the possible probabilities of all IDs in the vocabulary of the large language model, where Logits_i is the i-th ID generated by the large language model. LLM represents the large language model. LLM(i-1) represents what the large language model generates for the i-th ID based on the previous i-1 IDs.

[0076] Here, Max_value represents the maximum value in Logits_i. max(Logits_i) shows how to obtain the maximum value for Logits_i.

[0077] Here, Index indicates the index of the maximum value in Logits_i within the vocabulary of the large language model. Logits_i.index(Max_value) indicates the index from which the maximum value is obtained from Logits_i.

[0078] Here, Logits_i[Index] indicates that the maximum value in Logits_i is corrected. Max_value-a*nn(t-1) indicates that the corrected value is the prediction obtained by subtracting nn from the original maximum value. a is a single learnable parameter to control the influence of nn.

[0079] The principle of the above coupling is as follows: When generating a token, the large-scale language model predicts the probability of all tokens in the vocabulary once and usually selects the token with the highest probability. Introducing nn means that when the large-scale language model generates a token, nn performs a probability score on the possibility of infinitely iterative generation occurring at the location of that token, and this score negatively affects the probability of the large-scale language model predicting the token with the highest value in the vocabulary, thereby causing the large-scale language model to predict other tokens and mitigating infinitely iterative generation.

[0080] In step 45, decoding is performed.

[0081] Based on the Logits_i obtained in step 44, the token with the highest probability is selected and used as the prediction for the large-scale language model, and the calculation is as follows.

[0082] Pred=argmax(Logits_i) Here, Pred indicates an index that allows us to retrieve tokens from the vocabulary of a large language model based on this index. argmax indicates the index used to select the maximum value from Logtis_i.

[0083] The large-scale language model repeats this operation in the prediction process to mitigate the infinite iterative generation phenomenon, and furthermore, the decoding method in step 44 is used to train the model in the training process, ensuring the consistency of the large-scale language model during the training and inference phases.

[0084] Based on the above method, the embodiment of the present invention further provides an apparatus for carrying out the above method, and as shown in Figure 5, the apparatus for optimizing the large-scale language model iteration generation problem provided in the embodiment of the present invention is A first training module 51 constructs a neural network, the input to the neural network is the first output of the large-scale language model, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, where the first output is a hidden layer representation obtained by the large-scale language model based on the input ID list, and the neural network is trained using a first training set containing multiple first training data to obtain the trained neural network, and An inference module 52, which, when performing inference using the large-scale language model, inputs the text to be inferred into the large-scale language model, obtains a first probability for each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability that iterative generation occurs at the i-th ID position obtained by the neural network, wherein the i-th ID position is the location of the i-th ID generated by the large-scale language model, adjusts the first probability of each candidate ID based on the second probability to obtain a third probability for each candidate ID, and determines the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID.

[0085] With the above modules, embodiments of the present invention can improve the iterative generation problem of large-scale language models by utilizing the probabilities obtained by the neural network to adjust the probabilities of each candidate ID generated by the large-scale language model.

[0086] Selectively, the first training module 51 can freeze the parameters of the large language model, input the ID list in the first training data into the large language model, calculate a loss value based on the output of the neural network and the labels in the ID list, and, based on the loss value, continue to adjust the parameters of the neural network until a preset training termination condition is met, thereby obtaining the trained neural network.

[0087] Selectively, the first probability of each candidate ID at a particular ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The recurrence probability at which repeated generation occurs at a particular ID position is generated by a neural network based on partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the text to be inferred.

[0088] Selectively, the inference module 52 can further reduce the first probability of the first candidate ID based on the second probability to obtain a third probability of the first candidate ID, wherein the first candidate ID is the ID with the largest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs is smaller than that of the first candidate ID.

[0089] Selectively, the inference module 52 may further calculate and obtain the third probability of the first candidate ID using the formula max_score - a * logits_infinite, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents the weight, which is a hyperparameter or learnable parameter.

[0090] The above device may optionally include the following:

[0091] A first training module is provided that, before performing inference using the large-scale language model, the large-scale language model and / or the neural network can be fine-tuned using a second training set and a third training set, wherein the second training set includes a plurality of second training data consisting of a second prompt text and a standard answer, and the third training set includes at least one of the first training data, and in the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0092] As shown in Figure 6, the apparatus for optimizing the large-scale language model iteration generation problem provided in the embodiment of the present invention is The first processing module 61 inputs the first prompt text into a large language model and obtains a second ID list generated based on the first ID list of the large language model, wherein the first ID list includes an ID corresponding to each text unit in the first prompt text. A second processing module 62 is used to concatenate the first ID list and the second ID list to obtain a third ID list, generate a tag (label) for each ID in the first ID list and instruct the system to ignore the corresponding ID during the training process, generate a tag for each ID in the second ID list and instruct whether the location of the corresponding ID is a location where iterative generation occurs, concatenate the tags of each ID in the first ID list and the tags of each ID in the second ID list to obtain the tags of the third ID list, and generate one first training data based on the third ID list and the tags of the third ID list. The first training module 63 acquires a neural network in step S1 by training with a first training set containing multiple first training data, wherein the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on the input ID list, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes an ID list and tags, the ID list is generated by the large language model based on prompt text, and the tags are used to indicate whether the location of each ID in the ID list is a location where iterative generation occurs, and An inference module 64, which, when performing inference using the large-scale language model, inputs the text to be inferred into the large-scale language model, obtains a first probability for each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability that iterative generation occurs at the i-th ID position obtained by the neural network, wherein the i-th ID position includes the location of each ID generated by the large-scale language model, adjusts the first probability of each candidate ID based on the second probability to obtain a third probability for each candidate ID, and determines the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID.

[0093] With the above modules, the embodiments of the present invention can improve the iterative generation problem of large-scale language models.

[0094] Selectively, the first training module 63 can freeze the parameters of the large language model, input the third ID list in the first training data into the large language model, calculate a loss value based on the output of the neural network and the labels of the third ID list, and continue to adjust the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network.

[0095] Selectively, the first probability of each candidate ID at a particular ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The recurrence probability at which repeated generation occurs at a particular ID position is generated by a neural network based on partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the text to be inferred.

[0096] Optionally, the inference module can be used for the following purposes:

[0097] Based on the second probability, the first probability of the first candidate ID is reduced to obtain the third probability of the first candidate ID, and the first candidate ID is the ID with the largest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs is smaller than that of the first candidate ID.

[0098] Selectively, the first probability of the first candidate ID is reduced according to the second probability to obtain the third probability of the first candidate ID, and the details are as follows.

[0099] The formula max_score - a * logits_infinite is used to calculate and obtain the third probability of the first candidate ID, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents the weight, which is a hyperparameter or a learnable parameter.

[0100] The above device may optionally include the following:

[0101] A fine-tuning module is provided that, before performing inference using the large-scale language model, the large-scale language model and / or the neural network can be fine-tuned using a second training set and a third training set, wherein the second training set includes a plurality of second training data consisting of a second prompt text and a standard answer, and the third training set includes at least one of the first training data, and in the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0102] Furthermore, each apparatus / system provided in the above embodiments corresponds to an optimization method for the iterative generation problem of the large-scale language model described above, and the implementation forms in each of the above embodiments can all be applied to the embodiments of the apparatus and achieve similar technical effects. The apparatus provided in the embodiments of the present invention can implement all the method steps implemented in the embodiments of the above method and achieve the same technical effects, and the parts that are the same as those in the embodiments of the method in these embodiments and the beneficial effects will not be specifically described here.

[0103] Referring to Figure 7, an embodiment of the present invention further provides a hardware structure block diagram of an optimization device for large-scale language model iteration generation problems, and as shown in Figure 7, the optimization device 700 for large-scale language model iteration generation problems is Processor 702 and, Includes memory 704 where computer program commands are stored, Therefore, when the computer program command is executed by the processor, the processor 702 is instructed to perform the following steps, that is, A neural network is obtained by training using a first training set containing multiple first training data, where the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on the input ID list, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes an ID list and tags, the ID list is generated by the large language model based on prompt text, the tags are used to indicate whether the location of each ID in the ID list is a location where iterative generation occurs, and When performing inference using the aforementioned large-scale language model, the text to be inferred is input to the large-scale language model, and the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network are obtained. The i-th ID position includes the location of each ID generated by the large-scale language model, and based on the second probability, the first probability of each candidate ID is adjusted to obtain a third probability for each candidate ID, and based on the third probability for each candidate ID, the ID at the i-th ID position and the corresponding text unit are determined.

[0104] Furthermore, each system provided in the above embodiments is a device corresponding to the optimization method for the iterative generation problem of the large-scale language model described above, and the implementation forms in each of the above embodiments can all be applied to embodiments of the device and achieve similar technical effects. The device provided in the embodiments of the present invention can implement all the method steps implemented in the embodiments of the above method and achieve the same technical effects, and the same parts and beneficial effects as those in the embodiments of the method in this embodiment will not be specifically described here.

[0105] Furthermore, as shown in Figure 7, the large-scale language model iterative generation problem optimization device 700 further includes a network interface 701, an input device 703, a hard disk 705, and a display device 706.

[0106] Each of the above interfaces and devices are connected via a bus architecture. The bus architecture can include any number of interconnected buses and bridges. Specifically, it connects one or more central processors (CPUs) and / or graphics processors (GPUs), represented by processor 702, and various circuits of one or more memories, represented by memory 704. The bus architecture can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits. Needless to say, the bus architecture is used to enable communication between these units. It is well known in the art that the bus architecture includes a data bus, a power bus, a control bus, and a status signal bus, and a detailed explanation is omitted.

[0107] The network interface 701 can connect to a network (such as the Internet or a local area network), receive data such as prompt text from the network, and save the received data to the hard disk 705.

[0108] The input means 703 receives various commands entered by the worker and sends them to the processor 702 for execution. The input means 703 includes a keyboard or a click device (e.g., a mouse, a trackball, a contact sensor, or a touchscreen).

[0109] The display device 706 can display the results obtained by the processor 702 executing commands, for example, the progress of model training.

[0110] The memory 704 is used to store programs and data essential for system operation, as well as data such as intermediate results in the calculation process of the processor 702.

[0111] To ensure clarity, the memory 704 in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Therefore, the non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), or electrically erasable programmable read-only memory (EEPROM). The volatile memory may be random-access memory (RAM), which is used as an external cache. The memory 704 of the apparatus and methods described herein includes, but is not limited to, these and any other suitable types of memory.

[0112] In some embodiments, the memory 704 stores the following elements, namely, executable modules or data structures, or subsets or extensions thereof: an operating system 7041 and an application program 7042.

[0113] Therefore, the operating system 7041 includes various system programs, such as a framework layer, core layer, and drive layer, to realize various basic operations and hardware-based tasks. The application program 7042 includes various application programs, such as a browser, to realize various application operations. A program that realizes the method of the embodiment of the present invention can be included in the application program 7042.

[0114] The method according to the above embodiment of the present invention can be applied to or implemented by a processor 702. The processor 702 may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware within the processor 702 or by a command in software form. The processor 702 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-configurable circuit array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components that may implement or execute the method, steps, and logic block diagram disclosed in this embodiment. The general-purpose processor may be a microprocessor or any common processor. Directly, the steps of the method disclosed from the embodiment of the present invention can be completed by execution from a hardware decoder processor, or by execution from a combination of hardware and software modules within the decoder processor. The software modules may be located in mature recording media in the art, such as random memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, or registers. The recording medium is located in memory 704, and the processor 702 reads the information in memory 704 and combines it with the hardware to complete the steps of the above method.

[0115] It can be understood that these embodiments described herein can be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may implement one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing units (DSPDs), programmable logic units (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present invention, or a combination thereof.

[0116] Regarding the implementation of the software, the techniques described herein can be implemented by modules (e.g., processes, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. Memory can be implemented within or outside the processor.

[0117] Specifically, when the computer program is executed on the processor 702, the following steps can be achieved, namely, A step of generating first training data based on first prompt text before obtaining a neural network through training using a first training set containing multiple first training data; and The step is to obtain a first training set by acquiring multiple first training data generated based on multiple first prompt texts, Here, the step of generating one first training data based on one first prompt text includes the following: The first prompt text is input to a large-scale language model to obtain a second ID list generated based on the first ID list of the large-scale language model, the first ID list containing IDs corresponding to each text unit in the first prompt text, the first ID list and the second ID list are concatenated to obtain a third ID list, a tag is generated for each ID in the first ID list and used to instruct the model to ignore the corresponding ID during the training process, a tag is generated for each ID in the second ID list and used to instruct whether the location of the corresponding ID is a location where iterative generation occurs, the tags of each ID in the first ID list and the tags of each ID in the second ID list are concatenated to obtain the tags of the third ID list, and one first training data is generated based on the third ID list and the tags of the third ID list.

[0118] Specifically, when the computer program is executed on the processor 702, the following steps can be achieved, namely, The parameters of a large-scale language model are frozen, an ID list in the first training data is input to the large-scale language model, a loss value can be calculated based on the output of the neural network and the labels in the ID list, and based on the loss value, the parameters of the neural network are continuously adjusted until a preset training termination condition is met, thereby obtaining the trained neural network.

[0119] Selectively, the first probability of each candidate ID at a particular ID position is generated by a large-scale language model based on partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The recurrence probability at which repeated generation occurs at a particular ID position is generated by a neural network based on partial IDs up to the ID corresponding to the particular ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the text to be inferred.

[0120] Specifically, when the computer program is executed on the processor 702, the following steps can be achieved, namely, Based on the second probability, the first probability of the first candidate ID is reduced to obtain the third probability of the first candidate ID, and the first candidate ID is the ID with the largest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs is smaller than that of the first candidate ID.

[0121] Specifically, when the computer program is executed on the processor 702, the following steps can be achieved, namely, The formula max_score - a * logits_infinite is used to calculate and obtain the third probability of the first candidate ID, where max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, and a represents the weight, which is a hyperparameter or a learnable parameter.

[0122] Specifically, when the computer program is executed on the processor 702, the following steps can be achieved, namely, Before performing inference using the large-scale language model, the large-scale language model and / or the neural network are fine-tuned using a second training set and a third training set, wherein the second training set includes a plurality of second training data consisting of a second prompt text and a standard answer, and the third training set includes at least one of the first training data. Furthermore, during the fine-tuning process, if the input training data is the second training data, the large-scale language model and the neural network are trained and fine-tuned using the second training data, and if the input training data is the first training data, the neural network is trained using the first training data.

[0123] Furthermore, the apparatus provided in the embodiment of the present invention can implement all the method steps realized in the embodiment of the method described above and achieve the same technical effects. Therefore, the same parts and beneficial effects as those in the embodiment of the method described here will not be specifically described here.

[0124] Figure 8 is a schematic diagram of the structure of a question answering system according to an embodiment of the present invention, and includes a client 801 and a large-scale language model execution device 802.

[0125] Here, the client 801 may be a personal computer or a mobile terminal, or it may be an application process running on any of the above-mentioned terminals. A mobile terminal is a terminal device operated by a user. A mobile terminal may be a smartphone, a personal digital assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, or a next-generation communication system, such as a terminal device in an NR network or a terminal device in a future advanced public land mobile network (PLMN) network.

[0126] The large-scale language model execution device 802 may be a server system composed of one or more computers. The large-scale language model execution device 802 executes the large-scale language model 803 and the neural network 804. The neural network 804 may be obtained by training an optimization method / device based on the iterative generation problem of the large-scale language model, and the large-scale language model 803 may be generated by a fine-tuning process in the optimization method / device based on the iterative generation problem of the large-scale language model.

[0127] In the question answering system 800, the client 801 may be connected to the large-scale language model execution device 802 via a wired and / or wireless network.

[0128] The functions of both the client 801 and the large-scale language model execution device 802 may be distributed across multiple computers.

[0129] The following describes the hardware or software structure of the relevant devices, apparatus, or functions.

[0130] <Hardware Configuration> (computer) The client 801 and the large-scale language model execution device 802 are implemented by a computer having the hardware configuration shown in Figure 9. Figure 9 is a diagram showing an example of the hardware configuration of a computer according to an embodiment of the present invention.

[0131] Referring to Figure 9, the computer 500 includes an input device 501, a display device 502, an external I / F 503, RAM 504, ROM 505, a CPU 506, a communication I / F 507, an HDD 508, etc., all interconnected via bus B. The following configuration is permitted: The input device 501 and the display device 502 are connected as needed.

[0132] The input device 501 includes a keyboard, mouse, touch panel, etc., through which the user inputs various operation signals. The display device 502 includes a display, etc., that shows the processing results from the computer 500.

[0133] Communication I / F 507 is an interface installed to connect computer 500 to various networks. Therefore, computer 500 performs data communication via communication I / F 507.

[0134] The HDD 508 is an example of a non-volatile storage device for storing programs and data. The stored data includes the operating system (OS), which is the basic software that controls the entire computer 500, and application software (hereinafter simply referred to as "applications") that provides various functions within the OS. The computer 500 may use a drive device that uses flash memory (e.g., a solid-state drive (SSD)) as its storage medium instead of the HDD 508.

[0135] External I / F 503 is an interface for an external device. The external device is a recording medium 503a, etc. In this case, the computer 500 reads information from or writes information to the recording medium 503a via the external I / F 503. The recording medium 503a is a flexible disk, CD, DVD, SD memory card, USB memory, etc.

[0136] ROM 505 is a non-volatile semiconductor memory (storage device) that can still retain programs and / or data even when the power is turned off. ROM 505 stores programs and data for the Basic Input / Output System (BIOS), OS settings, network settings, etc., which are executed when the computer 500 starts up. RAM 504 is an example of a volatile semiconductor memory (storage device) that temporarily stores programs and / or data.

[0137] The CPU 506 is an arithmetic unit that reads programs and / or data from storage devices such as the ROM 505 and HDD 508. The read programs or data execute processes to realize the overall control or functionality of the computer 500.

[0138] The client 801 and the large-scale language model execution device 802 are implemented, for example, by the hardware configuration of the computer 500 shown in Figure 8.

[0139] (Mobile device) The client 801 is implemented, for example, by a hardware configuration as shown in Figure 10. Figure 10 is a diagram showing an example of a hardware configuration of a mobile terminal according to an embodiment of the present invention. The mobile terminal 12 shown in Figure 10 includes a CPU 601, a ROM 602, a RAM 603, an EEPROM 604, a CMOS sensor 605, an acceleration sensor and a compass sensor 606, and a media drive 608.

[0140] The CPU 601 controls the overall operation of the mobile terminal 12. The ROM 602 stores basic input / output programs. The RAM 603 is used as the work area for the CPU 601. The EEPROM 604 reads or writes data that matches the control of the CPU 601. The CMOS sensor 605 captures image data that matches the control of the CPU 601 and acquires image data. The acceleration and orientation sensors 606 are electromagnetic compasses, gyrocompasses, acceleration sensors, etc., that detect the Earth's magnetic field.

[0141] The media drive 608 controls the reading or writing (storage) of data to and from the recordable medium 607, such as flash memory. Data already stored in the recordable medium 607 is read, or new data is written to the recordable medium 607. The recordable medium 607 is detachable from the media drive 608.

[0142] The EEPROM 604 stores the operating system (OS) executed by the CPU 601, as well as related information necessary for network configuration. The applications for executing the various processes in the first embodiment are stored in the EEPROM 604 or on the recordable medium 607, etc.

[0143] The CMOS sensor 605 is a charge-coupled device that converts light into electric charge and digitizes an image of the target. The CMOS sensor 605 may be implemented by, for example, a charge-coupled device (CCD) sensor, as long as it can capture an image of the target.

[0144] The mobile terminal 12 also includes an audio input unit 609, an audio output unit 610, an antenna 611, a communication unit 612, a wireless LAN communication unit 613, a wireless communication antenna 614, a wireless communication unit 615, a display 616, a touchpad 617, and a bus 619.

[0145] The audio input unit 609 converts sand into an audio signal. The audio output unit 610 converts the audio signal back into sand. The communication unit 612 uses the antenna 611 to communicate with the nearest base station equipment via wireless communication signals. The wireless LAN communication unit 613 performs wireless LAN communication with an access point compliant with the IEEE 80411 standard. The wireless communication unit 615 performs wireless communication using the wireless communication antenna 614.

[0146] The display 616 is provided for displaying target images, various icons, etc. The display 616 is composed of liquid crystal or organic EL. The touch panel 617 is attached to the display 616 and is composed of a pressure-sensitive or electrostatic panel. The touch position on the display 616 is detected by touch with a finger or stylus. The bus 619 is an address bus, data bus, etc., and electrically connects the above-mentioned multiple units or components.

[0147] Client 801 includes a dedicated battery 618. Client 801 is powered by the battery 618. The audio input 609 includes a microphone for inputting sound. The audio output 610 includes a speaker for outputting sound.

[0148] Client 801 can be implemented, for example, by a hardware configuration as shown in Figure 10.

[0149] Referring to Figure 11 below, an example of the workflow of the question answering system 800 described above is provided, in which inference is performed using the large-scale language model and neural network trained in the embodiment of the present invention to provide answers in response to questions submitted by the user and improve the iterative generation problem of the large-scale language model. The process specifically includes the following:

[0150] In S801, the user sends a question text to the large-scale language model execution device 802 via the client 801, and the large-scale language model execution device 802 receives the question text sent from the client. Specifically, the large-scale language model 803 can receive the question text sent from the client 801.

[0151] In S802, the large-scale language model 803 generates probability values ​​for candidate IDs at each ID position in the answer based on the question text entered by the user, generates the probability of repeated generation occurring at each ID position using a neural network, further adjusts the probability values ​​of candidate IDs using this probability to reduce the probability of IDs that are prone to repeated generation, determines the final ID and the corresponding text unit at each ID position based on the adjusted candidate ID probabilities, and further generates the final answer text composed of these text units.

[0152] In S803, the large-scale language model execution unit 802 sends the final answer text to the client 801. The client 801 can display the final answer text via its display device in order to obtain the answer to the question text. Specifically, the client 801 can display the final answer text via the interface of its display device.

[0153] In some embodiments of the present invention, a computer-readable storage medium is provided on which computer program commands are stored, and the following steps are performed when the program is executed by a processor.

[0154] The first prompt text is input into a large language model to obtain a second ID list generated based on the first ID list of the large language model, the first ID list containing the ID corresponding to each text unit in the first prompt text.

[0155] The first ID list and the second ID list are combined to obtain a third ID list, a tag is generated for each ID in the first ID list and used to instruct the system to ignore the corresponding ID during the training process, a tag is generated for each ID in the second ID list and used to instruct the system whether the location of the corresponding ID is a location where iterative generation occurs, the tags of each ID in the first ID list and the tags of each ID in the second ID list are combined to obtain the tags of the third ID list, and one first training data is generated based on the third ID list and the tags of the third ID list.

[0156] A neural network is constructed, the input to the neural network is the first output of the large-scale language model, and the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, where the first output is a hidden layer representation obtained by the large-scale language model based on the input ID list, and the neural network is trained using a first training set containing multiple first training data to obtain the trained neural network.

[0157] When performing inference using the aforementioned large-scale language model, the text to be inferred is input to the large-scale language model, and the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network are obtained. The i-th ID position is the location of the i-th ID generated by the large-scale language model, and based on the second probability, the first probability of each candidate ID is adjusted to obtain the third probability of each candidate ID, and based on the third probability of each candidate ID, the ID at the i-th ID position and the corresponding text unit are determined.

[0158] The program, when executed by the processor, can implement all the implementation methods for optimizing the iterative generation problem of the large-scale language model described above, and achieve the same technical effect. Therefore, to avoid duplication, the explanation is omitted here.

[0159] Embodiments of the present invention further provide a computer program product including computer commands, which, when executed by a processor, can implement each process of the embodiment of the optimization method for the large-scale language model iterative generation problem described above and achieve the same technical effects, and will not be described again here to avoid redundancy.

[0160] Those skilled in the art will be able to realize these functions by combining the illustrated units and algorithmic steps of the embodiments disclosed herein with electronic hardware or computer software. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may realize these functions using different methods depending on the specific application, but such realization should not be considered beyond the scope of the present invention.

[0161] For those skilled in the art, in order to make the explanation convenient and concise, the specific working processes of the above-described systems, apparatuses and units can be referenced to the corresponding processes in the above-described embodiment of the method, and no further explanation is provided here.

[0162] It goes without saying that the methods and apparatus described in the embodiments provided by the present invention can also be implemented in other ways. For example, the apparatus embodiments described above are illustrative only, and for instance, the module partitions are only logic function partitions, and in practice, they can be implemented in other partitioning ways. For example, multiple modules or units may be combined or integrated into other systems, or certain features may be ignored or not performed. Furthermore, the shown or discussed coupling, direct coupling or communication connection between entities may also be indirect coupling or communication connection via interfaces, devices or modules, and may be in electrical, mechanical or other forms.

[0163] The units described as separating members may or may not be physically separated, and the members shown as units may or may not be physical units; that is, they may be located in one place, or they may be distributed as multiple network units. Depending on the actual needs, some or all of these units can be selected to achieve the objectives of the embodiments of the present invention.

[0164] Furthermore, while it is possible to integrate each functional unit in each embodiment of the present invention into a single process unit, each unit can also exist physically independently, or two or more units can be integrated into a single unit.

[0165] The functions are implemented in the form of software function units and, when sold or used as independent products, can be stored on a single computer-readable storage medium. Thus, the essence of the technical solution of the present invention, or a part of the contribution to the prior art, or a part of the technical solution can be realized in the form of a software product, which is stored on a recording medium and contains some commands to cause a computer device (which may be a personal computer, server, or network equipment, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The recording medium includes: various media capable of storing program code, such as USB, portable hard disks, ROM, RAM, magnetic disks, or optical disks.

[0166] The above describes a specific method of implementing the present invention, and the scope of protection of the present invention is not limited thereto. Those skilled in the art can easily conceive of variations and substitutions within the scope of the art disclosed in the present invention, and all of these fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for optimizing the iterative generation problem of large-scale language models performed by a computer, A neural network is obtained by training it using a first training set containing multiple first training data sets, the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on a second ID list that has been input, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes a second ID list and a label, the second ID list is generated by the large language model based on a single prompt text, and the label is used to indicate whether the location of each ID in the second ID list is a location where iterative generation occurs, When performing inference using the aforementioned large-scale language model, the process includes the steps of: inputting the text to be inferred into the large-scale language model; obtaining the first probability of each candidate ID at the i-th ID position obtained by the large-scale language model and the second probability of iterative generation occurring at the i-th ID position obtained by the neural network, wherein the i-th ID position includes the location of each ID generated by the large-scale language model; adjusting the first probability of each candidate ID based on the second probability to obtain the third probability of each candidate ID; and determining the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID. A method characterized by the following features.

2. Before obtaining a neural network by training using the first training set containing the aforementioned multiple first training data, A step to generate one initial training data based on one initial prompt text, The steps include obtaining a first training set by acquiring a plurality of first training data generated based on a plurality of first prompt texts, The step of generating one initial training data based on one initial prompt text is: The steps include inputting the first prompt text into a large-scale language model to obtain the second ID list generated based on the first ID list of the large-scale language model, wherein the first ID list includes an ID corresponding to each text unit in the first prompt text, The steps include: combining the first ID list and the second ID list to obtain a third ID list; A step of generating a label for each ID in the first ID list, and using the label to instruct the training process to ignore the corresponding ID, A step of generating a label for each ID in the second ID list, the label being used to indicate whether the location of the corresponding ID is a location where iterative generation occurs, The steps include: concatenating the labels of each ID in the first ID list with the labels of each ID in the second ID list to obtain the labels of the third ID list; The step of generating one first training data based on the third ID list and the labels of the third ID list includes: The method according to claim 1, characterized in that

3. Obtaining a neural network by training it using the first training set which includes the aforementioned multiple first training data sets is possible. This includes freezing the parameters of the large-scale language model, inputting the second ID list in the first training data into the large-scale language model, calculating a loss value based on the output of the neural network and the labels in the second ID list, and continuously adjusting the parameters of the neural network based on the loss value until a preset training termination condition is met, thereby obtaining the trained neural network. The method according to claim 1, characterized in that

4. The first probability of each candidate ID at a specific ID position is generated by the large-scale language model based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The repetition probability at which repeated generation occurs at a specific ID position is generated by a neural network based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the text to be inferred. The method according to claim 1, characterized in that

5. Based on the aforementioned second probability, adjusting the first probability of each candidate ID to obtain the third probability of each candidate ID is: Based on the aforementioned second probability, the first probability of the first candidate ID is reduced to obtain the third probability of the first candidate ID, the first candidate ID is the ID with the largest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs other than the first candidate ID is smaller than that of the first candidate ID. The method according to claim 1, characterized in that

6. Based on the aforementioned second probability, reducing the first probability of the first candidate ID to obtain the third probability of the first candidate ID is: The third probability of the first candidate ID is calculated using the formula max_score - a*logits_infinite. max_score represents the first probability of the first candidate ID, logits_infinite represents the second probability, a represents the weight, and the weight is a hyperparameter or learnable parameter. The method according to claim 5, characterized in that

7. Before performing inference using the aforementioned large-scale language model, further, This includes the step of fine-tuning the large-scale language model and / or the neural network using the second and third training sets, The second training set includes a plurality of second training data sets consisting of a second prompt text and a standard response, the third training set includes at least one of the first training data sets, and the fine-tuning process includes, if the input training data is the second training data, using the second training data to train and fine-tune the large-scale language model and the neural network, and if the input training data is the first training data included in the third training set, using the first training data to train the neural network. The method according to claim 1, characterized in that

8. A device for optimizing the iterative generation problem of large-scale language models, A neural network is obtained by training using a first training set containing multiple first training data sets, the input to the neural network is the first output of the large language model, the first output is a hidden layer representation obtained by the large language model based on a second ID list that has been input, the output of the neural network is the probability that iterative generation occurs at the location of the ID corresponding to the first output, the first training data includes the second ID list and labels, the second ID list is generated by the large language model based on prompt text, and the labels are used to indicate whether the location of each ID in the second ID list is a location where iterative generation occurs, and a first training module An inference module that, when performing inference using the aforementioned large-scale language model, inputs the text to be inferred into the large-scale language model, obtains a first probability for each candidate ID at the i-th ID position obtained by the large-scale language model and a second probability that iterative generation occurs at the i-th ID position obtained by the neural network, wherein the i-th ID position includes the location of each ID generated by the large-scale language model, adjusts the first probability of each candidate ID based on the second probability to obtain a third probability for each candidate ID, and determines the ID at the i-th ID position and the corresponding text unit based on the third probability of each candidate ID, includes: An apparatus characterized by the following features.

9. The system further includes a first acquisition unit that generates one first training data based on one first prompt text, and acquires multiple first training data generated based on multiple first prompt texts to obtain the first training set, To generate one initial training data based on one initial prompt text, The process includes inputting the first prompt text into a large-scale language model to obtain the second ID list generated based on the first ID list of the large-scale language model, wherein the first ID list includes IDs corresponding to each text unit in the first prompt text; concatenating the first ID list and the second ID list to obtain a third ID list; generating a label for each ID in the first ID list, where the label instructs to ignore the corresponding ID during the training process; generating a label for each ID in the second ID list, where the label indicates whether the location of the corresponding ID is a location where iterative generation occurs; concatenating the labels for each ID in the first ID list and the labels for each ID in the second ID list to obtain the labels for the third ID list; and generating one first training data based on the third ID list and the labels for the third ID list. The apparatus according to claim 8, characterized in that

10. The first probability of each candidate ID at a specific ID position is generated by the large-scale language model based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The repetition probability at which repeated generation occurs at a specific ID position is generated by a neural network based on the partial IDs up to the ID corresponding to the specific ID position in the fourth ID list. The fourth ID list is an ID list consisting of IDs corresponding to each text unit in the text to be inferred. The apparatus according to claim 8, characterized in that

11. The inference module is further used to reduce the first probability of the first candidate ID based on the second probability, in order to obtain the third probability of the first candidate ID. The first candidate ID is the ID with the highest first probability, the decrease in the first probability of the first candidate ID is positively correlated with the second probability, and the decrease in the first probability of other candidate IDs is smaller than that of the first candidate ID. The apparatus according to claim 8, characterized in that

12. The system further includes a fine-tuning module that uses a second and a third training set to fine-tune the large-scale language model and / or the neural network before performing inference using the large-scale language model, The second training set includes a plurality of second training data sets consisting of a second prompt text and a standard response, the third training set includes at least one of the first training data sets, and the fine-tuning process includes, if the input training data is the second training data, using the second training data to train and fine-tune the large-scale language model and the neural network, and if the input training data is the first training data included in the third training set, using the first training data to train the neural network. The apparatus according to claim 8, characterized in that

13. A program for causing a computer to perform the method described in any one of claims 1 to 7.

14. A computer-readable storage medium storing the program described in claim 13.

15. A device for optimizing the iterative generation problem of large-scale language models, Memory and A processor connected to the aforementioned memory, The aforementioned memory stores computer programs. The apparatus wherein the processor is configured to realize the method according to any one of claims 1 to 7 by executing the computer program.

Citation Information

Patent Citations

  • JPP7520275B

  • Similarity-based generative ai output filtering

    US20240160902A1

  • Machine learning structured result generation

    WO2024137128A1