An automatic parallel language model text generation method

Abstract

The present invention belongs to the technical field of language model text generation, and relates to an automatic parallel language model text generation method, which comprises the following steps: 1) constructing structured training data; 2) fine-tuning structured instructions; 3) automatic parallel generation; 4) traversing paragraph trees. The present invention uses structured data to fine-tune the instructions of the language model, so that during the generation process, the language model automatically issues parallel generation instructions to the reasoning system in combination with the language model's own understanding of the generated content structure, generates multiple parallel generation streams, and thus increases the parallel degree of the language model generation.

Description

Technical Field

[0001] The invention belongs to the technical field of language model generation, and relates to a language model text generation method, in particular to an automatic parallel language model text generation method. Background Art

[0002] Chatbots powered by language models have become a core part of modern AI applications and research. The widespread use of these language models has put forward higher requirements for efficient model deployment in order to provide low latency and high throughput services. However, the autoregressive generation process of these language models presents significant challenges in meeting these performance requirements.

[0003] A core problem stems from the process of autoregressive generation, where each new token is generated sequentially based on all previously generated tokens. This step-by-step decoding process results in suboptimal generation speeds, primarily because each generation step requires access to a large number of model parameters. Therefore, when the generation batch size is not sufficiently large, this process is memory-bound and the GPU’s computational power is inefficiently utilized. Second, in the Transformer architecture, the computational complexity of the attention operation grows linearly with the sequence length in autoregressive generation. However, in high-throughput scenarios, the generation process is computationally bottlenecked. For longer generated responses, the increased computation hinders further improvements in generation throughput. In addition, the key-value (KV) cache of generated tokens, despite advances in memory-efficient algorithms such as paging attention, also scales linearly with the sequence length, limiting the number of concurrent requests the system can handle.

[0004] Therefore, the decoding optimization of language models is a research hotspot. In terms of computing process optimization, previous methods include optimization of operators and computational graphs. In actual deployment, model compression is also widely used, such as quantization and pruning. There is also work that modifies the model architecture, including efficient attention that optimizes computational complexity and multi-query attention that optimizes memory access.

[0005] Another type of work is to improve the parallelism of the decoding process, which is mainly based on improving generation efficiency by amortizing memory access time. Dynamic batching and paging attention improve the maximum generation throughput through more reasonable scheduling strategies. Speculative decoding verifies multiple inferred identifiers in parallel, thereby reducing generation latency. Non-autoregressive generation samples multiple generation identifiers in parallel, but the effect is usually not ideal in dialogue models.

[0006] Therefore, in view of the defects existing in the above-mentioned prior art, it is necessary to develop a new language model text generation method. Summary of the invention

[0007] In order to overcome the defects of the prior art, the present invention proposes an automatic parallel language model text generation method, which uses structured data to fine-tune the instructions of the language model, so that during the generation process, the language model can automatically send parallel generation instructions to the reasoning system in combination with the language model's own understanding of the structure of the generated content, thereby generating multiple parallel generation streams, thereby increasing the degree of parallelism in the generation of the language model.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] An automatic parallel language model text generation method, characterized by comprising the following steps:

[0010] 1) Construct structured training data: construct the original training corpus into structured training data in the form of paragraph trees;

[0011] 2) Structural instruction fine-tuning: Based on the structured training data in the form of the paragraph tree, the language model is fine-tuned using tree attention to obtain a language model after instruction fine-tuning, so that the language model after instruction fine-tuning issues a parallel output call at a location where a parallel structure appears in the generated content;

[0012] 3) Automatic parallel generation: input the prompt string into the language model after the instruction fine-tuning, and the language model after the instruction fine-tuning automatically and parallelly decodes the prompt string and its corresponding generated content in a loop to obtain a paragraph tree of the prompt string and its corresponding generated content;

[0013] 4) Traversing the paragraph tree: traversing the paragraph tree of the prompt string and its corresponding generated content to form linear generated content.

[0014] Preferably, in step 1), if the original training corpus is unstructured sequential data, the original training corpus is extracted into structured training data in the form of a paragraph tree using rules; if the original training corpus is structured data, it is directly converted into structured training data in the form of a paragraph tree.

[0015] Preferably, in step 1), the rule is a regular expression.

[0016] Preferably, in the step 1), in the process of constructing the original training corpus into structured training data in the form of a paragraph tree, two control identifiers are inserted to indicate a parallelizable structure, wherein the [FORK] control identifier indicates that a parallelizable structure appears here, and the [CHILD] control identifier indicates that the subsequent content is one level lower than the content where the [FORK] control identifier is located.

[0017] Preferably, in step 2), when fine-tuning the instruction, each directed extremely long path starting from the root node in the structured training data in the form of a paragraph tree is represented as a training sample.

[0018] Preferably, in step 3), during the automatic parallel decoding process, if the language model after the instruction fine-tuning issues a parallel generation call, the prompt string prefix is ​​shared, a new parallel generation stream is established, and a control identifier is added for parallel generation.

[0019] Preferably, in step 3), the prompt string and its corresponding generated content are constructed into a sequence group G, and an identifier string is generated for each unfinished sequence in the sequence group G.

[0020] Preferably, in the step 3), when the automatic parallel decoding is performed cyclically, the automatic parallel generation is repeatedly applied to the sequence group G until all the generated streams are finished.

[0021] Preferably, in the step 4), when traversing the paragraph tree of the prompt string and its corresponding generated content to form linear generated content, the paragraph tree of the prompt string and its corresponding generated content is recursively accessed in the order of "root-first child-next brother", and the accessed identifier strings are concatenated to obtain the linear generated content.

[0022] Compared with the prior art, the automatic parallel language model text generation method of the present invention has one or more of the following beneficial technical effects:

[0023] 1. The present invention uses structured data to fine-tune the language model instructions, so that during the generation process, the language model automatically sends parallel generation instructions to the reasoning system in combination with the language model's own understanding of the generated content structure, generating multiple parallel generation streams, thereby increasing the degree of parallelism in the generation of the language model.

[0024] 2. In the memory access bottleneck scenario, the present invention achieves a speedup ratio of about 2 times on multiple types of tasks, and can be combined with a speculative decoding algorithm to achieve a speedup ratio of about 4 times.

[0025] 3. In the computing bottleneck scenario, the present invention can be combined with the existing page-based attention method to improve the generation throughput by 20%-70% and reduce the generation time by 20%-35%.

[0026] 4. Unlike previous work, the present invention does not modify the operator or model architecture, but adopts a tree-like attention structure for the structured output content to reduce computation. Therefore, it is orthogonal to the existing technologies and can be applied in combination with them. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the automatic parallel language model text generation method of the present invention.

[0028] Figure 2 This is an example diagram of data in the training phase of the present invention.

[0029] Figure 3 This is a comparison chart of the generation speed on different benchmarks and categories.

[0030] Figure 4 This is a comparison chart of throughput and latency in a computing bottleneck scenario. DETAILED DESCRIPTION

[0031] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components set forth in the following description or illustrated in the following figures. The present invention can have other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used herein are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and variations thereof herein is intended to cover the items and their equivalents and additional items displayed below.

[0032] Furthermore, in the disclosure of the present invention, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0033] The present invention proposes an automatic parallel language model text generation method, which uses the language model's own understanding and organization capabilities of generated content after instruction fine-tuning to automatically initiate parallel generation flows, thereby increasing the degree of parallelism in language model generation; and can be combined with paging attention and speculative decoding to improve generation efficiency in memory access bottleneck and computing bottleneck scenarios.

[0034] The text generation of the language model can be formally defined as, given a prompt string (ie, user question, which is an identifier string) p and a language model Θ, sampling an identifier string x from the distribution P of the prompt string p as the generated content of the language model.

[0035] Among them, the identifier string x can be regarded as a sequence group composed of multiple identifier sequences. Therefore, in the present invention, the identifier sequence generated sequentially by the language model is organized into a paragraph tree according to the linguistic meaning of the sequence, and the paragraph tree contains at least one node. Each node of the paragraph tree has three attributes, namely the root, the first child and the next brother. The root is an identifier string. If the node is a leaf node, the root is the detailed content of the paragraph, otherwise the root is an introduction or summary of the paragraph. The first child is a nullable pointer pointing to the detailed content of the paragraph where the root is located. The next brother is a nullable pointer pointing to the next paragraph at the same level as the root.

[0036] Figure 1 FIG. 4 is a flow chart showing the automatic parallel language model text generation method of the present invention. Figure 1 As shown, the automatic parallel language model text generation method of the present invention can be summarized into two parts: a training phase and a generation phase.

[0037] The training phase is used to train the existing language model. In order to apply the method proposed in the present invention to the existing language model, it is first necessary to use structured training data in the form of paragraph trees to fine-tune the instructions of the existing language model (i.e., train it) so that the language model triggers automatic parallel generation when encountering parallelizable output content.

[0038] The generation phase uses the language model fine-tuned by the instructions to generate corresponding answers to the input prompt string (that is, user questions).

[0039] In the present invention, each language model Θ only needs to be applied once in the training phase, while in the generation phase, the language model Θ' obtained in the training phase can be repeatedly applied to all prompt strings p to generate generated content corresponding to the prompt string p.

[0040] Specifically, the automatic parallel language model text generation method of the present invention comprises the following steps:

[0041] 1. Construct structured training data.

[0042] That is, the original training corpus D is constructed into structured training data in the form of a paragraph tree. If the original training corpus is unstructured sequential data, the original training corpus is extracted into structured training data in the form of a paragraph tree using rules; if the original training corpus is structured data, it is directly converted into structured training data in the form of a paragraph tree.

[0043] Specifically, suppose there is a dialogue dataset (which includes multiple original training corpora D, each of which is a sequence), where each sequence includes a user question and an AI assistant's answer. Figure 2(a) shows an original sequence.

[0044] A rule-based method can be used to extract parallel structures in the answers in the original sequence. The present invention uses a regular expression (\d+\.)\s+([\s\S]+?):([\s\S]+?)(?=(?:\d+\.)\s+[\s\S]+?:) to extract the main content and detailed content in the answers of the points, and integrate the answers of the points into a paragraph tree. Figure 2 (b) where p i The nodes in the paragraph tree are represented by the dashed box, the content in the dashed box represents the root of each node, and the blue and red arrows respectively indicate the pointers to the first child and next sibling attributes (null pointers are indicated if not drawn). Applying rules to extract each original training corpus in the dialogue dataset can convert the parallelizable structures in each original training corpus in the dialogue dataset into a paragraph tree.

[0045] In addition, other rules can be used to extract more complex structures. For example, for a paragraph with a general-specific structure, the first sentence can be used as the root for construction.

[0046] In the process of extracting the paragraph tree, the present invention inserts two control identifiers to indicate a parallelizable structure. The [FORK] control identifier indicates that a parallelizable structure appears here, and the [CHILD] control identifier indicates that the subsequent content is one level lower than the content where the [FORK] identifier is located. Therefore, when the language model Θ encounters [FORK][CHILD] appearing consecutively in the prefix, it indicates the detailed content of the paragraph, while [FORK] without [CHILD] indicates the subsequent content of the same level.

[0047] If an original training corpus does not conform to any rule, it forms a paragraph tree with a unique node, the root of which is the original training corpus itself.

[0048] 2. Fine-tune structured instructions.

[0049] That is, based on the structured training data in the form of the paragraph tree, the language model is fine-tuned using tree attention to obtain a fine-tuned language model, so that the fine-tuned language model issues parallel output calls at locations where parallel structures appear in the generated content.

[0050] Specifically, in this stage, the structured training data in the form of paragraph tree is input into the language model θ for instruction fine-tuning. Each directed extremely long path starting from the root node in the structured training data in the form of paragraph tree represents a training sample.

[0051] by Figure 2Take the paragraph tree in (b) as an example, the training samples it contains include “p 0 –p 1 ", "p 0 –p 2 –p 3 " and "p 0 –p 2 –p 4 "Three, and its attention is Figure 2 (c) is shown. Figure 2 In (c), each row represents what a node can attend to, and the triangle represents causal self-attention. The default parameters of the instruction fine-tuning process are shown in Table 1.

[0052] Table 1. Default parameters for the fine-tuning process

[0053] Language Model Vicuna-7B / Vicuna-13B Batch size 128 Optimizer Adam Learning Rate 2e-5 Preheating ratio 0.03 Training steps 2000

[0054] The language model after fine-tuning with structured instructions is denoted as Θ'.

[0055] 3. Automatic parallel generation.

[0056] The prompt string is input into the language model after the instruction fine-tuning, and the language model after the instruction fine-tuning automatically and parallelly decodes the prompt string and its corresponding generated content loop to obtain a paragraph tree of the prompt string and its corresponding generated content.

[0057] After the prompt string (that is, the user question) is input into the fine-tuned language model, the fine-tuned language model will automatically find the generated content corresponding to the prompt string. After finding the generated content corresponding to the prompt string, the fine-tuned language model determines whether to issue a parallel generation call based on the structure of the generated content, that is, whether there is a parallelization structure in the generated content. When there is a parallelization structure in the generated content, the fine-tuned language model issues a parallel generation call, shares the prompt string prefix, establishes a new parallel generation stream, and adds a control identifier for parallel generation.

[0058] Specifically, in the present invention, the concepts of sequence and sequence group are introduced. A sequence is an identifier string, and a sequence group is a collection of sequences. Each generated prompt string and its corresponding generated content paragraph tree corresponds to a sequence group, and each sequence in the sequence group corresponds to a leaf node of the paragraph tree, that is, corresponds to a generated stream.

[0059] For the input prompt string p, it is constructed as a sequence group G = {p}. The following algorithm represents the decoding process generated by automatic parallelization.

[0060] Input: Sequence group G.

[0061] Description: Generate the next identifier string for each unfinished sequence in the sequence group G.

[0062] process:

[0063] Traverse each sequence s in the sequence group G.

[0064] If sequence s is completed, skip this sequence and continue with the next sequence.

[0065] Sample a new identifier x using the model Θ' and the current sequence s.

[0066] If the last token of the current sequence s is the control identifier [Fork], perform the following steps:

[0067] 1. Create a new subsequence s′ that copies the state of the parent sequence, that is, the current sequence s.

[0068] 2. Add the control identifier [CHILD] to the subsequence s′.

[0069] 3. Add the subsequence s′ to the sequence group G.

[0070] 4. Create new paragraph tree leaf nodes n and n′, and set the start positions to the lengths of s and s′ respectively.

[0071] 5. Update the end position, first child, and next sibling of the corresponding leaf node in sequence s, setting them to the length of s, pointing to n, and pointing to n', respectively.

[0072] 6. Update the current nodes of s and s′ to n and n′ respectively.

[0073] Add the just generated identifier x to the sequence s.

[0074] If the generated marker x is the end marker or the length limit has been reached, the marker sequence s is complete and the key-value vector cache belonging only to s is released.

[0075] Returns the sequence group G.

[0076] Automatic parallel generation operates on sequence groups and updates the status of sequence groups. Figure 2 As shown, after the sequence group is initialized, the automatic parallel generation is repeatedly applied to the sequence group until all generation streams are completed.

[0077] 4. Traverse the paragraph tree.

[0078] That is, the paragraph tree of the prompt string and its corresponding generated content is traversed to form linear generated content.

[0079] After all generation flows are completed, we hope to obtain linear generated content from the paragraph tree corresponding to the sequence group. We recursively access the generated paragraph tree in the order of "root-first child-next brother" and concatenate the visited identifier strings to obtain the final generated content. Figure 2 The paragraph tree in (b) can be obtained by 0 –p 1 –p 2 –p 3 –p 4 " (ignoring the control identifier) ​​and restore the original sequence.

[0080] In order to prove the effect of the present invention, experiments were conducted, including the efficiency in the memory access bottleneck scenario, the efficiency in the computing bottleneck scenario and the generation quality. The experimental results are as follows:

[0081] 1. Efficiency in Memory Access Bottleneck Scenarios

[0082] In the memory access bottleneck scenario, the present invention uses two dialogue test benchmark data sets, Vicuna-Bench and MT-Bench, to test the acceleration effect of the automatic parallel language model text generation method proposed in the present invention on different categories.

[0083] To avoid memory bottleneck, the decoding batch size is set to 1. Original means no generation acceleration method is used, APAR refers to the method of the present invention, and Medusa means using Medusa speculative decoding for acceleration. A total of 4 groups of experiments were conducted, in which Vanilla-APAR means using only the automatic parallel generation method proposed by the present invention, and Medusa-APAR means using both the automatic parallel generation method proposed by the present invention and Medusa speculative decoding. The experimental results are shown in Figure 2. Figure 3 As shown, Figure 3 The speedup ratio in the figure is a comparison between the experimental setting using the present invention and the one not using the generation acceleration method. On average, the present invention can accurately identify whether the generated content can be accelerated in parallel and can significantly improve the generation speed. For example, in general and knowledge types, the method of the present invention has a significant acceleration effect; while in types such as code and mathematics that require step-by-step reasoning, the present invention will not bring additional acceleration because they do not have structures that can be generated in parallel.

[0084] 2. Calculate the efficiency in bottleneck scenarios.

[0085] In the computing bottleneck scenario, the present invention randomly selected 1000 real user requests from the ShareGPT dataset for the experiment. In this scenario, both the generated throughput and speed need to be considered. Figure 4The experimental results shown in the figure show that the parallel generation method proposed in the present invention significantly reduces the average delay of the generation process, and because it can reduce the average attention length and release the part-time cache in advance, it improves the throughput under the same key-value cache usage. Figure 4 Error bars indicate 25% and 75% quantiles, and dots indicate means.

[0086] 3. Generate quality.

[0087] The automatic parallel generation method proposed in the present invention requires retraining of the language model, so it is also necessary to evaluate the impact of the generation method on the quality of generated responses. The present invention conducts generation quality experiments on the dialogue benchmark datasets Vicuna-Bench and MT-Bench, and the results are summarized in Tables 2 and 3 below. The experiment shows that the dialogue content generated by the automatic parallel generation method of the present invention is similar to the quality before fine-tuning, and has little impact on the generation quality.

[0088] Table 2: Generation quality on MT-Bench

[0089]

[0090] Table 3: Build quality on Vicuna-Bench

[0091]

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can modify or replace the technical solution of the present invention according to the idea of ​​the present invention without departing from the essence and scope of the technical solution of the present invention.

Claims

1. An automatic parallel language model text generation method, characterized in that: The following steps are involved: 1) Construct structured training data: construct the original training corpus into structured training data in the form of paragraph trees; 2) Structural instruction fine-tuning: Based on the structured training data in the form of the paragraph tree, the language model is fine-tuned using tree attention to obtain a language model after instruction fine-tuning, so that the language model after instruction fine-tuning issues a parallel output call at a location where a parallel structure appears in the generated content; 3) Automatic parallel generation: input the prompt string into the language model after the instruction fine-tuning, and the language model after the instruction fine-tuning automatically decodes the prompt string and its corresponding generated content in parallel to obtain a paragraph tree of the prompt string and its corresponding generated content; wherein, after the prompt string is input into the language model after the instruction fine-tuning, the language model after the instruction fine-tuning will automatically find the generated content corresponding to the prompt string. After finding the generated content corresponding to the prompt string, the language model after the instruction fine-tuning determines whether to issue a parallel generation call based on the structure of the generated content, that is, whether there is a parallelization structure in the generated content. When there is a parallelization structure in the generated content, the language model after the instruction fine-tuning issues a parallel generation call, shares the prompt string prefix, establishes a new parallel generation stream, and adds a control identifier for parallel generation; 4) Traversing the paragraph tree: traversing the paragraph tree of the prompt string and its corresponding generated content to form linear generated content.

2. The automatic parallel language model text generation method according to claim 1, characterized in that: In the step 1), if the original training corpus is unstructured sequential data, the original training corpus is extracted into structured training data in the form of a paragraph tree using rules; If the original training corpus is structured data, it is directly converted into structured training data in the form of a paragraph tree.

3. The automatic parallel language model text generation method according to claim 2, characterized in that: In the step 1), the rule is a regular expression.

4. The automatic parallel language model text generation method according to claim 3, characterized in that: In the step 1), in the process of constructing the original training corpus into structured training data in the form of a paragraph tree, two control identifiers are inserted to indicate a parallelizable structure, wherein the [FORK] control identifier indicates that a parallelizable structure appears here, and the [CHILD] control identifier indicates that the subsequent content is one level lower than the content where the [FORK] control identifier is located.

5. The automatic parallel language model text generation method according to claim 4, characterized in that: In the step 2), when fine-tuning the instruction, each directed extremely long path starting from the root node in the structured training data in the form of the paragraph tree is represented as a training sample.

6. The automatic parallel language model text generation method according to claim 5, characterized in that: In the step 3), during the automatic parallel generation process, the prompt string is constructed into a sequence group G, and an identifier string is generated for each unfinished sequence in the sequence group G.

7. The automatic parallel language model text generation method according to claim 6, characterized in that: In the step 3), when the automatic parallel decoding is performed in a loop, the automatic parallel generation is repeatedly applied to the sequence group G until all the generated streams are finished.

8. The automatic parallel language model text generation method according to claim 7, characterized in that: In the step 4), when traversing the paragraph tree of the prompt string and its corresponding generated content to form a linear generated content, the paragraph tree of the prompt string and its corresponding generated content is recursively accessed in the order of "root-first child-next brother", and the accessed identifier strings are concatenated to obtain the linear generated content.

Images