A long text generation method based on large language model context compression

Abstract

The application provides a long text generation method based on context compression of a large language model, which comprises the following steps: obtaining context text to be compressed and prompt word text, and performing compression-based encoding processing to obtain corresponding compressed vectors and prompt word embedding vectors; splicing the compressed vectors and the prompt word embedding vectors, and performing self-recurrence-based decoding processing on the spliced fusion features to obtain corresponding token identifiers; mapping the token identifiers to text strings one by one according to a preset word table, and composing the text strings into compressed context text. Through the application, the context long text processed by the large language model is compressed, thereby solving the technical problem that a huge amount of model calculation resources and data storage resources are consumed when a semantic model processes long context text in the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a long text generation method based on context compression of a large language model. BACKGROUND

[0002] With the success of chat language models such as ChatGPT, language models based on Transformers are playing an increasingly important role in human life, such as knowledge Q&A, copywriting, code generation, etc. However, due to the inherent limitations of current language models based on Transformers, they need to retain all context information in memory, so there is a problem of huge consumption of computing resources and storage resources when processing long context text, which limits the application of language models in long context. In the application of vertical fields, language models often need to face a large amount of professional knowledge text. For example, in some scenarios, the model needs to answer legal knowledge text according to a large number of laws and regulations, and the length of these legal texts often far exceeds the input context window size of the language model, so the language model in the prior art is difficult to complete such tasks, or needs to consume huge model computing resources and data storage resources. SUMMARY

[0003] The present application provides a long text generation method based on context compression of a large language model to solve the technical problem that the large language model in the prior art needs to consume huge model computing resources and data storage resources when processing long context text.

[0004] The present application provides a long text generation method based on context compression of a large language model, which comprises the following steps:

[0005] Obtain the context text to be compressed and the prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for a large language model to perform a text processing task;

[0006] Splice the compression vector and the prompt word embedding vector corresponding to the prompt word text, and perform autoregressive decoding processing on the fused features obtained by splicing to obtain a plurality of target token identifiers;

[0007] According to a predetermined word table, map the target token identifiers one by one to text strings, and compose the text strings into a compressed context text.

[0008] In some embodiments, the compression-based encoding processing of the context text to be compressed to obtain a corresponding compression vector comprises:

[0009] convert each text character of the context text to be compressed into a token identifier, to obtain a corresponding token list;

[0010] segment the token list according to a preset sequence length, and sequentially store the segmented sub-sequences into a queue;

[0011] select a sub-sequence from the queue and input it into an encoder network for compression for encoding processing, and splice the vector obtained by the encoding processing to obtain a spliced vector;

[0012] input the spliced vector into a multi-layer perception for feature dimension mapping processing, to obtain a corresponding compressed vector.

[0013] In some embodiments, the fused feature spliced is subjected to autoregressive-based decoding processing to obtain a plurality of target token identifiers, comprising:

[0014] obtain a current cumulative predicted token identifier;

[0015] splice the fused feature spliced and a cumulative embedding vector corresponding to the cumulative predicted token identifier, and input the spliced feature into a decoding model for autoregressive decoding processing to obtain a corresponding target token identifier;

[0016] if the current target token identifier is a terminator, stop autoregressive decoding; if the current target token identifier is not a terminator, add the current target token identifier as the cumulative predicted token identifier.

[0017] In some embodiments, the compressed context text is compressed by a text compression model, the text compression model comprising a text compression unit model, a compressed vector mapping model, and a compressed vector decoding model, and a training method of the text compression model comprising:

[0018] obtain a context text training dataset, and divide the context text training dataset into a short text dataset and a long text dataset according to a preset text length threshold;

[0019] perform first-stage training of the text compression model through the short text dataset to obtain a first text compression model, wherein during the first-stage training, the parameters of the text compression unit model, the compressed vector mapping model, and the compressed vector decoding model in the first text compression model are updated;

[0020] performing second stage training on the first text compression model through the long text dataset to obtain a second text compression model, wherein during the second stage training, parameters of the text compression unit model of the first text compression model are frozen, and parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

[0021] In some embodiments, the short text dataset includes short text samples, first prompt word samples, and first token prompt samples, and the first stage training of the text compression model through the short text dataset to obtain a first text compression model includes:

[0022] inputting the short text samples into the text compression unit model of the text compression model for encoding processing to obtain corresponding first compression vectors, and inputting the first compression vectors into the compression vector mapping model in the text compression model for feature mapping to obtain text compression vectors.

[0023] determining a first embedding vector of the first prompt word sample and a second embedding vector of the first token prompt sample;

[0024] concatenating the text compression vectors, the first embedding vector, and the second embedding vector in a sequence direction to obtain first concatenated features;

[0025] inputting the first concatenated features into the compression vector decoding model of the text compression model for decoding processing to obtain token identifiers of first to-be-predicted texts;

[0026] constructing a first cross-entropy loss function according to the token identifiers of the first to-be-predicted texts and the first token prompt samples;

[0027] performing back propagation in the text compression model through the first cross-entropy loss function, and updating parameters of the text compression unit model, the compression vector mapping model, and the compression vector decoding model of the text compression model during the back propagation to obtain a first text compression model.

[0028] In some embodiments, the short text dataset includes long text samples, second prompt word samples, and second token prompt samples, and the second stage training of the first text compression model through the long text dataset to obtain a second text compression model includes:

[0029] inputting the long text samples, the second prompt word samples, and the second token prompt samples into the first text compression model for forward propagation to obtain token identifiers of second to-be-predicted texts;

[0030] constructing a second cross-entropy loss function according to the token identifier of the second to-be-predicted text and the second token prompt sample;

[0031] performing back propagation in the first text compression model through the second cross-entropy loss function, and updating parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model in the process of back propagation, while freezing parameters of the text compression unit model of the first text compression model.

[0032] The application further provides a long text generation device based on context compression of a large language model, which comprises the following modules:

[0033] An encoding module is configured to acquire a context text to be compressed and a prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for a large language model to perform a text processing task.

[0034] A decoding module is configured to splice the compression vector and a prompt word embedding vector corresponding to the prompt word text, and perform autoregressive decoding processing on the fused features obtained through splicing to obtain a plurality of target token identifiers.

[0035] A generation module is configured to map the target token identifiers to text strings one by one according to a preset word table, and compose the text strings into a compressed context text.

[0036] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the long text generation method based on context compression of a large language model according to any one of the above-mentioned methods when executing the computer program.

[0037] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the long text generation method based on context compression of a large language model according to any one of the above-mentioned methods.

[0038] The application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the long text generation method based on context compression of a large language model according to any one of the above-mentioned methods.

[0039] The long text generation method based on large language model context compression provided by the application can save text data storage and computing resources by performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector. Next, the compression vector and the prompt word embedding vector are decoded into token identifiers through decoding processing, and are mapped back to a text string one by one as compressed context text. By compressing the text, the large language model can reduce the consumption of model computing resources and data storage resources when performing text processing tasks. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description one by one. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0041] Figure 1 is a flowchart of the long text generation method based on large language model context compression provided by the application.

[0042] Figure 2 is a principle diagram of the long text generation method based on large language model context compression provided by the application.

[0043] Figure 3 is a processing process diagram of text compression provided by the application.

[0044] Figure 4 is a processing diagram of compression vector decoding provided by the application.

[0045] Figure 5 is a process diagram of model training provided by the application.

[0046] Figure 6 is a structure diagram of the long text generation device based on large language model context compression provided by the application.

[0047] Figure 7 is a structure diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0049] The application provides a long text generation method and device based on context compression of a large language model. Figures 1-6 The application provides a long text generation method and device based on context compression of a large language model.

[0050] Figure 1 The application provides a long text generation method and device based on context compression of a large language model. Figure 1 The method comprises the following steps 101 to 103.

[0051] In step 101, the context text to be compressed and the prompt word text are obtained, and the context text to be compressed is subjected to compression-based encoding processing to obtain a corresponding compression vector.

[0052] In some embodiments, for example in a text question and answer scenario, the large language model needs to perform a text processing task, such as text question and answer and chat, according to the input context text and the prompt word text. The context text to be compressed is the text used by the large language model to perform the text processing task, and these texts all belong to long texts, for example, texts exceeding a certain character quantity threshold are referred to as long texts. The prompt word text is the text that helps the large language model to perform the text processing task.

[0053] Next, the context text to be compressed is subjected to compression-based encoding processing to obtain a corresponding compression vector, which is described below.

[0054] First, each text character of the context text to be compressed is converted into a token identifier to obtain a corresponding token list. The conversion process can be according to a preset vocabulary, and the token identifier is also called token id. The token list is obtained by combining the converted token ids. Since the context text to be compressed is a long text, the token list is segmented according to a preset sequence length, and the segmented subsequences are stored in a queue in sequence. The preset sequence length can be N, and the last cutting sequence length is less than N, and the compression vector is used to fill the token to make the length of each subsequence N. The obtained subsequences are stored in the queue in sequence, and then the subsequences are selected from the queue and input into the encoder network for encoding processing, and the vector obtained by the encoding processing is spliced to obtain a spliced vector. Finally, the spliced vector is input into a multi-layer perception to perform feature dimension mapping processing to obtain a corresponding compression vector. Here, the encoder network can be a Transformer model or a Mamba model, and the subsequence is stored in the queue and selected to prevent the order of the subsequence in the context text to be compressed from being disturbed, which affects the semantic relationship between texts. The feature dimension mapping processing is to ensure that the feature dimension of the compression vector is consistent with the input feature dimension of the subsequent encoding model, which is convenient for subsequent decoding.

[0055] For example, refer to Figure 2 , Figure 2 The principle diagram of the long text generation method based on context compression of a large language model provided by the application is as follows. In the input layer, the obtained long context text (N*S tokens) is subjected to subsequence division and Token embedding, and the divided subsequences are input into the text compression layer. In the text compression layer, the divided subsequences are input into the encoder network through multiple hidden layers in sequence to perform encoding processing, and the vectors obtained by the encoding processing are compressed through the hidden layers, and finally the compressed compression vectors are obtained in sequence, spliced and output.

[0056] The processing process of the text compression layer can be seen in Figure 3 , such as Figure 3As shown, first, the long context text is converted into a token id sequence (i.e., token list) through a vocabulary table, then the token id (i.e., token identifier) sequence is segmented according to a length N, and the sequences with insufficient length are padded to N using padding characters (i.e., compressed vector pre-padding tokens), and then the segmented multiple sequences are stored in a queue according to the position order. Next, it is determined whether there are still elements in the queue. If so, a token sequence is read from the queue using the first-in first-out method and input into the encoder network, and the compressed vector of the current token sequence is obtained through encoding processing, and then it is determined whether there are still elements in the queue, and all token sequences stored in the queue are sequentially encoded into compressed vectors in this way. When it is determined that there are no elements in the queue, it means that all token sequence encoding has been completed, and all compressed vectors are spliced along the queue order direction to obtain the final compressed vector.

[0057] Step 102, splice the compressed vector and the prompt word embedding vector corresponding to the prompt word text, and perform self-recursive decoding processing on the fusion features obtained by splicing to obtain a plurality of target token identifiers.

[0058] When performing sub-sequence segmentation in step 101, padding is needed by using compressed vector pre-padding tokens, so the compressed vector pre-padding tokens (S tokens) and the corresponding prompt word text (M tokens) also need to perform corresponding token embedding. The prompt word text is converted into corresponding prompt word tokens through a vocabulary table and embedded into corresponding prompt word embedding vectors, and the compressed vector pre-padding tokens (S tokens) are also embedded into corresponding embedding vectors and merged into the compressed vector.

[0059] Then, the compressed vector and the prompt word embedding vector corresponding to the prompt word text are spliced to obtain fusion features for subsequent decoding process. In addition, in the decoding process, the present embodiment adopts a self-recursive decoding mechanism, which re-enters the result of the previous decoding output into the decoding model to help the decoding model perform subsequent decoding process. Therefore, in the present embodiment, the current cumulative predicted token identifier is obtained at each decoding, and these identifiers are target token identifiers output by the decoding model, and then embedding processing is also needed to obtain a cumulative embedding vector.

[0060] In each decoding process, the fused feature obtained by splicing is spliced with the accumulated embedding vector corresponding to the accumulated predicted token identifier, and then the spliced feature obtained by splicing is input into the decoding model for autoregressive decoding processing to obtain the corresponding target token identifier. Here, the decoding model can also be a Transformer model. It should be noted that in the first decoding, the accumulated predicted token identifier does not exist, and the fused feature obtained by splicing is directly input into the decoding model for autoregressive decoding processing.

[0061] After the decoding model is decoded by autoregression, the corresponding target token identifiers are output in succession, which are used for subsequent generation of the corresponding compressed text. According to the autoregressive decoding mechanism, it is necessary to judge each target token identifier at this time. If the current target token identifier is a terminator, it means that the decoding process is complete, and the autoregressive decoding is stopped. If the current target token identifier is not a terminator, it means that the decoding process is still continuing, and the current target token identifier is added as an accumulated predicted token identifier for subsequent decoding process, and the target token identifier is continuously generated until the current target token identifier is a terminator.

[0062] Step 103, according to the preset word table, the target token identifier is mapped to a text string one by one, and the text string is composed into a compressed context text.

[0063] After the decoding model outputs all the target token identifiers through step 102, that is, after the terminator is output, according to the preset word table, the target token identifier is mapped to a text string one by one, that is, the token identifier is restored to the corresponding text string by querying the word table, and finally the text string is composed into a compressed context text. This compressed context text is the final result of the compression processing of the context text to be compressed, which can be directly used for text processing tasks of a large language model.

[0064] For example, see Figure 2The result output by the text compression layer enters the compressed information mapping layer, and after feature dimension mapping by a multi-layer perception of a linear mapping vector mapping unit, the final compressed vector is output. Then the compressed vector enters the compressed information decoding layer, and after pre-filling tokens (S tokens) and corresponding prompt word texts (M tokens) by the compressed vector embedding, the hidden layer is input to the decoding model (i.e., the Transformer language model) for decoding processing, and the corresponding target token id is output. In the last output layer, according to the preset word table, the target token id is mapped to the word table to generate the corresponding target text as the compressed context text.

[0065] The specific process of the decoding processing can be referred to Figure 4 The prompt word token id list and the accumulated predicted token id list are embedded into embedding vectors a and b respectively, and then spliced with the text compression vector along the sequence direction to obtain the corresponding spliced features, which are input into the Transformer language model for decoding processing to obtain the token id (i.e., target token identifier) for generating a new word. Then it is judged whether the generated token id is a terminator. If not, the token id is added to the accumulated predicted token id list for subsequent decoding process, i.e., the next decoding process. If it is judged that the generated token id is a terminator, the decoding process is completed, and the self-recurrence decoding is stopped. All newly generated token ids are mapped to text characters by the word table and output, so as to obtain the target text, i.e., the compressed context text.

[0066] In the embodiment of the application, the context text to be compressed is subjected to compression-based encoding processing to obtain a corresponding compressed vector, so that the text data storage and computing resources can be saved. Next, the compressed vector and the prompt word embedding vector are decoded into token identifiers by decoding processing, which are mapped back to text strings one by one as compressed context texts. By compressing the text, the large language model can reduce the consumption of model computing resources and data storage resources when performing text processing tasks.

[0067] Of course, the above-mentioned embodiments of the present application combine the encoder network, the multi-layer perceptron and the decoding model to jointly complete the compression task of long text, solving the technical problem that the large language model needs to consume huge model computing resources and data storage resources when processing long context text. However, considering that the encoder network, the multi-layer perceptron and the decoding model process the context text to be compressed, and these network models also need to be trained, if the long context text to be compressed is directly used as training data, it will greatly increase the burden and difficulty of model training, and the final compression result will not be ideal, which cannot meet the basic requirements of long text compression.

[0068] Based on the above problems, the embodiment of the present application further proposes a model training method, which first takes the encoder network, the multi-layer perceptron and the decoding model as three processing modules of a text compression unit model, a compression vector mapping model and a compression vector decoding model respectively, and then integrates the three processing modules into a text compression model. The text compression model is used to perform text compression on the context text to be compressed, and the compressed context text obtained in step 103 is obtained. The embodiment of the present application adopts a two-stage training strategy to train the text compression model, which will be described in detail below.

[0069] Firstly, a context text training data set is obtained, which includes text sequences of various lengths. Considering that the text compression unit model directly processes the long text to be compressed, it needs to have very good text compression capability, so its parameters are the most difficult to train. Therefore, in the two-stage training strategy, the data sets used in the two stages are different. In the first stage, the text compression capability of the text compression unit model is first trained through some short text data set.

[0070] Therefore, after obtaining the context text training data set, a text length threshold (for example, 5000 character lengths) can be preset, and then the context text training data set is divided into a short text data set and a long text data set according to the preset text length threshold. Specifically, the text samples included in the short text data set have a character length of not more than 5000 characters, and the text samples included in the long text data set have a character length of more than 5000 characters.

[0071] The first text compression model is obtained by performing the first stage training on the text compression model through the short text data set. In the first stage training process, the parameters of the text compression unit model, the compression vector mapping model and the compression vector decoding model in the first text compression model are updated, and the first stage training process will be described in detail below.

[0072] First, the short text dataset includes short text samples, first prompt word samples, and first token prompt samples. The first token prompt sample is the token obtained by compressing and decoding the short text sample; it serves as the corresponding real label for the short text sample.

[0073] During the forward propagation of training, short text samples are input into the text compression unit model of the text compression model for encoding to obtain the corresponding first compressed vector. However, before encoding, these vectors must be converted into corresponding token identifiers. Then, the first compressed vector is input into the compression vector mapping model of the text compression model for feature mapping to obtain the text compressed vector. Simultaneously, the first embedding vector of the first prompt word sample and the second embedding vector of the first token prompt sample are determined. Here, the first prompt word sample is first converted into its corresponding token identifier before embedding, while the first token prompt sample is directly embedded.

[0074] Next, the text compression vector, the first embedding vector, and the second embedding vector are concatenated along the sequence direction to obtain the first concatenated feature. The first concatenated feature is then input into the compression vector decoding model of the text compression model for decoding processing to obtain the token identifier of the first text to be predicted, which serves as the prediction result output by the text compression model after forward propagation.

[0075] Then, based on the token identifier of the first text to be predicted and the first token prompt sample, a first cross-entropy loss function is constructed. The purpose of the cross-entropy loss function here is to maximize the prediction probability of the correct string obtained by mapping the token identifier used to generate the first text to be predicted.

[0076] This is to ensure the accuracy of the token identifier prediction obtained by the text compression model during decoding.

[0077] When constructing the loss function, multiple token identifiers at the end of the token identifier sequence of the first text to be predicted are removed, and then the same number of token identifiers at the beginning of the first token prompt sample are removed. The corresponding loss function is then constructed, and the specific formula is as follows:

[0078] (1)

[0079] In formula (1) above, c represents the first token prompt sample after removing the leading token identifier. wherein represents the text compression vector corresponding to the token identifier of the first to-be-predicted text, p represents the first embedding vector of the first prompt word sample, and P represents the prediction probability of the correct text string predicted according to the token identifier of the first to-be-predicted text, wherein represents the model parameter of the compression vector decoding model.

[0080] Finally, the first cross-entropy loss function is calculated by The text compression model is back-propagated, and the parameters of the text compression unit model, the compression vector mapping model and the compression vector decoding model of the text compression model are updated in the process of back-propagation, to obtain the first text compression model.

[0081] In each iteration training process, the first cross-entropy loss function is calculated by forward propagation The gradient of the text compression model is calculated by back-propagation, and Adam (Adaptive Moment Estimation) is used as the gradient descent algorithm to optimize the gradient, update the model parameters of the text compression model, and the text compression unit model, the compression vector mapping model and the compression vector decoding model all participate in the parameter update.

[0082] According to the two-stage training strategy, in the second stage, the first text compression model trained in the first stage is trained in the second stage by using a long text data set to obtain a second text compression model. Because the text compression unit model of the first text compression model has very good text compression capability through the training of the short text data set in the first stage. In the second stage, the text compression unit model of the first text compression model does not need to be trained again, so the parameters of the text compression unit model of the first text compression model are frozen, only the parameters of the compression vector mapping model and the compression vector decoding model of the first text compression model are updated, and the training process of the second stage is specifically explained as follows.

[0083] Similar to the short text data set, the long text data set specifically includes long text samples, second prompt word samples and second token prompt symbol samples. When the training starts, the long text samples, the second prompt word samples and the second token prompt symbol samples are input into the first text compression model for forward propagation to obtain the token identifier of the second to-be-predicted text.

[0084] The forward propagation process is: inputting the long text sample into the text compression unit model of the first text compression model for encoding processing to obtain a corresponding second compression vector, but before the encoding processing, the second prompt word sample is also converted into a corresponding token identifier for encoding. Then the second compression vector is input into the compression vector mapping model in the first text compression model for feature mapping to obtain a text compression vector. At the same time, a third embedding vector of the second prompt word sample and a fourth embedding vector of the second token prompt sample are determined. Here, the second prompt word sample is first converted into a corresponding token identifier and then embedded, while the second token prompt sample is directly embedded. Then the text compression vector, the third embedding vector and the fourth embedding vector are spliced in the sequence direction to obtain a second spliced feature; next, the second spliced feature is input into the compression vector decoding model of the first text compression model for decoding processing to obtain a token identifier of the second to-be-predicted text.

[0085] Similarly to the first stage, a second cross-entropy loss function is constructed according to the token identifier of the second to-be-predicted text and the second token prompt sample, and the construction process is similar to that of the first cross-entropy loss function, which will not be repeated here.

[0086] Finally, the second cross-entropy loss function is used to perform back propagation in the first text compression model, and the second cross-entropy loss function is used to train the prediction generation capability of the text compression model. Therefore, in the process of back propagation, the parameters of the text compression unit model of the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated. That is, the parameters of the text compression unit model of the first text compression model are fixed, and only the parameters of the compression vector mapping model and the compression vector decoding model behind are updated during training.

[0087] For example, the training process of the second stage can refer to Figure 5, first sample the training set, divide the long context text therein, and convert it into a token id list, then input it into the compression vector unit (i.e. the text compression unit model of the first text compression model) for encoding processing, and input the result obtained by the encoding processing into the compression vector mapping unit (i.e. the compression vector mapping model in the first text compression model) to obtain the corresponding text compression vector, while the second prompt word sample is also converted into a corresponding prompt word token id list, then embedded to obtain an embedding vector a, and the token id list of the sentence to be predicted (i.e. the second token prompt sample) is also embedded into an embedding vector b, then the text compression vector, the embedding vector a and the embedding vector b are spliced along the sequence direction, and the obtained spliced features are input into the Transformer language model (i.e. the compression vector decoding model of the first text compression model) for decoding processing to obtain the token id of the sentence to be predicted (i.e. the token identifier of the second text to be predicted).

[0088] The token id of the sentence to be predicted needs to remove the multiple tokens at the end, and calculate the cross-entropy loss according to the corresponding logit vector and the token of the sentence to be predicted (remove the token at the beginning), wherein the logit vector represents the text compression vector corresponding to the decoded token id of the sentence to be predicted. During training, the cross-entropy loss is back propagated in the model, and it is judged whether the loss converges. When it does not converge, the next round of iterative training is continued according to the sampled data set. If it starts to converge, the training is terminated, and the model weight is obtained, i.e. the model parameters are trained.

[0089] The trained text compression model can directly compress the context text to be compressed to obtain the compressed context text obtained in step 103, which can effectively guarantee the compression effect of the text and meet the text compression demand of the large language model when performing the text processing task.

[0090] In the embodiment of the application, a two-stage training strategy is used to train the text compression model. In the first stage, the text compression unit model is trained by short text to ensure the text compression capability of the text compression model. In the second stage, the text compression unit model is frozen when training with long text, which can greatly reduce the memory occupation of the model during training, and ensure that the text compression model has the ability to predict and generate. Therefore, by using the two-stage training strategy, the training burden and difficulty of the model can be effectively reduced while ensuring that the model has basic compression processing capability.

[0091] The long text generation device based on large language model context compression provided by the present application is described below. The long text generation device based on large language model context compression described below can be correspondingly referred to the long text generation method based on large language model context compression described above.

[0092] As shown in Figure 6 The long text generation device based on large language model context compression provided by the present application includes an encoding module 601, a decoding module 602, and a generation module 603. Specifically, the encoding module 601 is configured to obtain a context text to be compressed and a prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for a large language model to perform a text processing task. The decoding module 602 is configured to concatenate the compression vector with a prompt word embedding vector corresponding to the prompt word text, and perform self-recursive decoding processing on the fused features obtained by concatenation to obtain a plurality of target token identifiers. The generation module 603 is configured to map the target token identifiers to text strings one by one according to a preset word table, and compose the text strings into a compressed context text.

[0093] It should be noted that the beneficial effects of the long text generation device based on large language model context compression described herein can be correspondingly referred to the beneficial effects of the long text generation method based on large language model context compression described above, and thus the beneficial effects of the long text generation device based on large language model context compression will not be described again.

[0094] Figure 7 An example of an entity structure diagram of an electronic device is shown in Figure 7As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logic instruction in the memory 730 to execute a long text generation method based on large language model context compression, which includes: obtaining context text to be compressed and prompt word text, and performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is text used for a large language model to perform a text processing task; concatenating the compression vector with a prompt word embedding vector corresponding to the prompt word text, and performing autoregressive decoding processing on the fused features obtained by concatenation to obtain a plurality of target token identifiers; mapping the target token identifiers one by one to text strings according to a preset word table, and composing the text strings into compressed context text.

[0095] In addition, the logic instruction in the memory 730 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0096] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer-readable storage medium, and the computer program can be executed by a processor to enable a computer to perform the long text generation method based on context compression of a large language model provided by the above-mentioned methods, which comprises: obtaining context text to be compressed and prompt word text, and performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is text used for a large language model to perform a text processing task; concatenating the compression vector with a prompt word embedding vector corresponding to the prompt word text, and performing autoregressive-based decoding processing on the fused features obtained by concatenation to obtain a plurality of target token identifiers; mapping the target token identifiers to text strings one by one according to a preset word table, and composing the text strings into compressed context text.

[0097] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a long text generation method based on context compression of a large language model provided by the above-mentioned methods, which comprises: obtaining context text to be compressed and prompt word text, and performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is text used for a large language model to perform a text processing task; concatenating the compression vector with a prompt word embedding vector corresponding to the prompt word text, and performing autoregressive-based decoding processing on the fused features obtained by concatenation to obtain a plurality of target token identifiers; mapping the target token identifiers to text strings one by one according to a preset word table, and composing the text strings into compressed context text.

[0098] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0099] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0100] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating long text based on context compression of a large language model, characterized in that, The method includes: Obtain the context text to be compressed and the prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain the corresponding compression vector. The context text to be compressed is the text used by the large language model to perform text processing tasks. The compressed vector is concatenated with the prompt word embedding vector corresponding to the prompt word text, and the concatenated fusion feature is subjected to autoregressive decoding processing to obtain multiple corresponding target token identifiers. According to a preset vocabulary, the target token identifiers are mapped one by one to text strings, and the text strings are combined into compressed context text; The compressed context text is obtained by calling a text compression model, which includes a text compression unit model, a compression vector mapping model, and a compression vector decoding model. The training method of the text compression model includes: Obtain the context text training dataset, and divide the context text training dataset into a short text dataset and a long text dataset according to a preset text length threshold; The text compression model is trained in the first stage using the short text dataset to obtain a first text compression model. During the first stage of training, the parameters of the text compression unit model, the compression vector mapping model, and the compression vector decoding model in the first text compression model are updated. The first text compression model is trained in the second stage using the long text dataset to obtain the second text compression model. During the second stage of training, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

2. The long text generation method based on large language model context compression according to claim 1, characterized in that, The step of performing compression-based encoding on the context text to be compressed to obtain the corresponding compressed vector includes: Each text character of the context text to be compressed is converted into a token identifier to obtain the corresponding token list; The token list is divided according to a preset sequence length, and the resulting sub-sequences are stored sequentially in a queue. Subsequences are selected from the queue and input into an encoder network for compression for encoding processing. The vectors obtained from the encoding processing are then concatenated to obtain a concatenated vector. The concatenated vector is input into a multilayer perceptron for feature dimension mapping to obtain the corresponding compressed vector.

3. The long text generation method based on large language model context compression according to claim 1, characterized in that, The fused features obtained by splicing are then subjected to autoregressive decoding to obtain multiple corresponding target token identifiers, including: Get the current cumulative prediction token identifier; The fused feature obtained by splicing is spliced ​​with the cumulative embedding vector corresponding to the cumulative predicted token identifier, and the spliced ​​feature obtained by splicing is input into the decoding model for autoregressive decoding processing to obtain the corresponding target token identifier. If the current target token identifier is a terminator, stop autoregressive decoding; if the current target token identifier is not a terminator, add the current target token identifier as the cumulative predicted token identifier.

4. The long text generation method based on large language model context compression according to claim 1, characterized in that, The short text dataset includes short text samples, first prompt word samples, and first token prompt symbol samples. The first stage of training the text compression model using the short text dataset to obtain the first text compression model includes: The short text sample is input into the text compression unit model of the text compression model for encoding processing to obtain the corresponding first compression vector. The first compression vector is then input into the compression vector mapping model of the text compression model for feature mapping to obtain the text compression vector. Determine the first embedding vector of the first prompt word sample and the second embedding vector of the first token prompt sample; The text compression vector, the first embedding vector, and the second embedding vector are concatenated along the sequence direction to obtain the first concatenation feature; The first concatenated feature is input into the compression vector decoding model of the text compression model for decoding processing to obtain the token identifier of the first text to be predicted; Based on the token identifier of the first text to be predicted and the first token prompt sample, a first cross-entropy loss function is constructed; The first text compression model is obtained by backpropagating through the first cross-entropy loss function in the text compression model and updating the parameters of the text compression unit model, compression vector mapping model and compression vector decoding model in the text compression model during the backpropagation process.

5. The long text generation method based on large language model context compression according to claim 1, characterized in that, The short text dataset includes long text samples, second prompt word samples, and second token prompt symbol samples. The second text compression model is obtained by training the first text compression model using the long text dataset in a second stage, including: The long text sample, the second prompt word sample, and the second token prompt sample are input into the first text compression model for forward propagation to obtain the token identifier of the second text to be predicted. Based on the token identifier of the second text to be predicted and the second token prompt sample, a second cross-entropy loss function is constructed; Backpropagation is performed in the first text compression model using the second cross-entropy loss function. During the backpropagation process, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

6. A long text generation device based on context compression of a large language model, characterized in that, The device includes: The encoding module is used to obtain the context text to be compressed and the prompt word text, and to perform compression-based encoding processing on the context text to be compressed to obtain the corresponding compression vector. The context text to be compressed is the text used by the large language model to perform text processing tasks. The decoding module is used to concatenate the compressed vector with the prompt word embedding vector corresponding to the prompt word text, and perform autoregressive decoding processing on the concatenated fusion features to obtain multiple corresponding target token identifiers; The generation module is used to map the target token identifiers one by one into text strings according to a preset vocabulary, and to assemble the text strings into compressed context text; The compressed context text is obtained by calling a text compression model, which includes a text compression unit model, a compression vector mapping model, and a compression vector decoding model. The training method of the text compression model includes: Obtain the context text training dataset, and divide the context text training dataset into a short text dataset and a long text dataset according to a preset text length threshold; The text compression model is trained in the first stage using the short text dataset to obtain a first text compression model. During the first stage of training, the parameters of the text compression unit model, the compression vector mapping model, and the compression vector decoding model in the first text compression model are updated. The first text compression model is trained in the second stage using the long text dataset to obtain the second text compression model. During the second stage of training, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the long text generation method based on large language model context compression as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the long text generation method based on large language model context compression as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the long text generation method based on large language model context compression as described in any one of claims 1 to 5.

Images