Long text generation method based on context compression of large language model

Abstract

The invention provides a long text generation method based on context compression of a large language model, and the method comprises the steps: obtaining a to-be-compressed context text and a cue word text, and carrying out compression-based coding processing to obtain a corresponding compression vector and a cue word embedding vector; splicing the compression vector and the cue word embedding vector, and performing autoregression-based decoding processing on a fused feature obtained by splicing to obtain a plurality of corresponding token identifiers; and according to a preset word list, mapping the token identifiers into text character strings one by one, and forming the text character strings into a compressed context text. According to the method and the device, the context long text processed by the large language model is compressed, so that the technical problem that huge model computing resources and data storage resources need to be consumed when the long context text is processed by a semantic model in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a long text generation method based on large language model context compression. Background Art

[0002] With the success of chat language models such as ChatGPT, Transformer-based language models are playing an increasingly important role in human life, such as knowledge question and answer, copywriting, code generation, etc. However, due to the inherent limitations of the current Transformer-based language model, it needs to keep all context information in memory, so there is a huge consumption of computing resources and storage resources when processing text with long context, which limits the application of language models in long context. In vertical field applications, language models usually need to face a large amount of professional knowledge text. For example, in some scenarios, the model is required to conduct text questions and answers on legal knowledge based on numerous laws and regulations, and the text length of these laws and regulations often far exceeds the input context window size of the language model. Therefore, the language model in the prior art is difficult to complete such a task, or it needs to consume huge model computing resources and data storage resources. Summary of the invention

[0003] The present invention provides a long text generation method based on large language model context compression, which is used to solve the technical problem in the prior art that huge model computing resources and data storage resources are consumed when a large language model processes long context text.

[0004] The present invention provides a long text generation method based on large language model context compression, the method comprising the following steps: Acquire context text to be compressed and prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for the large language model to perform a text processing task; The compressed vector is concatenated with the prompt word embedding vector corresponding to the prompt word text, and the concatenated fusion features are subjected to autoregressive decoding processing to obtain corresponding multiple target token identifiers; According to a preset vocabulary, the target token identifiers are mapped into text strings one by one, and the text strings are composed into compressed context text.

[0005] In some embodiments, performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector includes: Convert each text character of the context text to be compressed into a token identifier to obtain a corresponding token list; The token list is segmented according to a preset sequence length, and the segmented subsequences are sequentially stored in a queue; Selecting a subsequence from the queue and inputting the subsequence into an encoder network for compression for encoding, and concatenating vectors obtained by the encoding process to obtain a concatenated vector; The concatenated vector is input into a multi-layer perceptron for feature dimension mapping processing to obtain a corresponding compressed vector.

[0006] In some embodiments, the concatenated fusion features are subjected to autoregressive decoding processing to obtain corresponding multiple target token identifiers, including: Get the current cumulative prediction token identifier; Concatenating the concatenated fusion features with the accumulated embedding vector corresponding to the accumulated predicted token identifier, and inputting the concatenated features into a decoding model for autoregressive decoding processing to obtain a corresponding target token identifier; If the current target token identifier is a terminal character, the autoregressive decoding is stopped; if the current target token identifier is not a terminal character, the current target token identifier is added as the cumulative predicted token identifier.

[0007] In some embodiments, the compressed context text is obtained by compressing a text compression model, the text compression model includes a text compression unit model, a compression vector mapping model and a compression vector decoding model, and the training method of the text compression model includes: Acquire a context text training data set, and divide the context text training data set into a short text data set and a long text data set according to a preset text length threshold; Performing a first-stage training on the text compression model using the short text data set to obtain a first text compression model, wherein during the first-stage training process, parameters of a text compression unit model, a compression vector mapping model, and a compression vector decoding model in the first text compression model are updated; The first text compression model is trained in the second stage using the long text data set to obtain a second text compression model, wherein during the second stage training process, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

[0008] In some embodiments, the short text dataset includes a short text sample, a first prompt word sample, and a first token prompt sample, and the first stage training of the text compression model using the short text dataset to obtain the first text compression model includes: The short text sample is input into the text compression unit model of the text compression model for encoding processing to obtain a corresponding first compression vector, and the first compression vector is input into the compression vector mapping model in the text compression model for feature mapping to obtain a text compression vector.

[0009] Determining a first embedding vector for the first prompt word sample and a second embedding vector for the first token prompt sample; Splicing the text compression vector, the first embedding vector, and the second embedding vector along a sequence direction to obtain a first splicing feature; Inputting the first concatenated feature into the compression vector decoding model of the text compression model for decoding processing to obtain a token identifier of the first text to be predicted; Constructing a first cross entropy loss function according to the token identifier of the first to-be-predicted text and the first token prompt sample; The first cross entropy loss function is back-propagated in the text compression model, and the parameters of the text compression unit model, the compression vector mapping model and the compression vector decoding model in the text compression model are updated during the back-propagation process to obtain a first text compression model.

[0010] In some embodiments, the short text dataset includes a long text sample, a second prompt word sample, and a second token prompt sample, and the second stage training of the first text compression model using the long text dataset to obtain the second text compression model includes: Inputting the long text sample, the second prompt word sample and the second token prompt sample into the first text compression model for forward propagation to obtain a token identifier of the second text to be predicted; Constructing a second cross entropy loss function according to the token identifier of the second to-be-predicted text and the second token prompt sample; Back propagation is performed in the first text compression model through the second cross entropy loss function, and in the process of back propagation, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

[0011] The present invention also provides a long text generation device based on large language model context compression, the device comprising the following modules: An encoding module, used to obtain context text to be compressed and prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for the large language model to perform text processing tasks; A decoding module, used for concatenating the compression vector with the prompt word embedding vector corresponding to the prompt word text, and performing autoregressive decoding processing on the concatenated fusion features to obtain corresponding multiple target token identifiers; The generating module is used to map the target token identifiers into text strings one by one according to a preset vocabulary, and compose the text strings into compressed context text.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the long text generation method based on large language model context compression as described in any one of the above-mentioned methods is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for generating long text based on large language model context compression.

[0014] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for generating long text based on large language model context compression.

[0015] The long text generation method based on large language model context compression provided by the present invention obtains a corresponding compression vector by performing compression-based encoding processing on the context text to be compressed, thereby saving text data storage and computing resources. Next, through decoding processing, the compression vector and the prompt word embedding vector are decoded into token identifiers, which are mapped back to the text string one by one as the compressed context text. By compressing the text, the large language model can reduce the consumption of model computing resources and data storage resources when performing text processing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a flowchart of the long text generation method based on large language model context compression provided by the present invention.

[0018] Figure 2 It is a schematic diagram of the principle of the long text generation method based on large language model context compression provided by the present invention.

[0019] Figure 3 It is a schematic diagram of the processing process of text compression provided by the present invention.

[0020] Figure 4 It is a processing schematic diagram of compressed vector decoding provided by the present invention.

[0021] Figure 5 It is a schematic diagram of the model training process provided by the present invention.

[0022] Figure 6 It is a structural schematic diagram of a long text generation device based on large language model context compression provided by the present invention.

[0023] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Combine the following Figure 1-Figure 6 The present invention describes a long text generation method and device based on large language model context compression.

[0026] Figure 1 is a flow chart of a long text generation method based on large language model context compression provided by the present invention, such as Figure 1 As shown, the method includes the following steps 101 to 103.

[0027] Step 101: Obtain the context text to be compressed and the prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector.

[0028] In some embodiments, such as in a text question-answering scenario, the large language model needs to perform text processing tasks, such as text question-answering, chatting, etc., based on the input context text and prompt word text. The context text to be compressed is the text used by the large language model to perform text processing tasks. These texts are all long texts. For example, texts that exceed a certain character count threshold are called long texts. The prompt word text is the text that helps the large language model perform text processing tasks.

[0029] Next, the context text to be compressed is subjected to compression-based encoding processing to obtain a corresponding compression vector, which is described in detail below.

[0030] First, each text character of the context text to be compressed is converted into a token identifier to obtain a corresponding token list. The conversion process can be performed according to a preset vocabulary. The token identifier is also called a token id. The token ids obtained by the conversion are combined to obtain a token list. Since the context text to be compressed is a long text, the token list is segmented according to the preset sequence length, and the subsequences obtained by the segmentation are stored in the queue in sequence. The preset sequence length can be N. When the sequence length is less than N during the final cutting, the compressed vector is used to pre-fill the token to make it complete, so that the length of each subsequence is N. When segmenting, the obtained subsequences are stored in the queue in sequence, and then the subsequences are selected from the queue and input into the encoder network for compression for encoding processing, and the vectors obtained by the encoding processing are spliced ​​to obtain a spliced ​​vector. Finally, the spliced ​​vector is input into a multi-layer perceptron for feature dimension mapping processing to obtain the corresponding compressed vector. Here, the encoder network can be a Transformer model or a Mamba model, and the subsequences are stored and selected by the queue to prevent the order of the subsequences in the context text to be compressed from being disrupted, which affects the semantic connection between the texts. The feature dimension mapping process is to ensure that the feature dimension of the compressed vector is consistent with the input feature dimension of the subsequent encoding model, which facilitates subsequent decoding.

[0031] For example, see Figure 2 , Figure 2The principle diagram of the long text generation method based on large language model context compression provided by the present invention is that in the input layer, the obtained long context text (N*S tokens) is subjected to subsequence division and Token embedding, and the corresponding subsequences obtained by division enter the text compression layer. In the text compression layer, the subsequences obtained by division are successively input into the encoder network through multiple hidden layers for encoding processing, and the vectors obtained by the encoding processing are vector compressed through the hidden layers, and finally the compressed vectors obtained by compression are successively spliced ​​and output.

[0032] The processing of the text compression layer can be found in Figure 3 ,like Figure 3 As shown in the figure, the long context text is first converted into a token id sequence (i.e., a token list) through a vocabulary, and then the token id (i.e., token identifier) ​​sequence is segmented according to the length N. If the sequence length is insufficient, fill characters (i.e., compressed vector pre-filled tokens) are used to fill it up to N, and then the multiple segments are stored in a queue in position order. Next, it is determined whether there are still elements in the queue. If so, a token sequence is read from the queue using the first-in-first-out method and input into the encoder network. The compressed vector of the current token sequence is obtained through encoding processing, and then it is determined whether there are still elements in the queue. All token sequences stored in the queue are repeatedly encoded into compressed vectors in sequence. When it is determined that there are no elements in the queue, it means that the encoding of all token sequences has been completed, and all compressed vectors are spliced ​​along the queue order to obtain the final compressed vector.

[0033] Step 102: concatenate the compressed vector with the prompt word embedding vector corresponding to the prompt word text, and perform autoregressive decoding processing on the concatenated fusion features to obtain corresponding multiple target token identifiers.

[0034] When performing subsequence segmentation in step 101, it is necessary to pre-fill tokens through the compressed vector for completion. Therefore, the compressed vector pre-filled tokens (S tokens) and the corresponding prompt word texts (M tokens) also need to perform corresponding token embedding. The prompt word text is converted into the corresponding prompt word token through the vocabulary and embedded into the corresponding prompt word embedding vector. The compressed vector pre-filled tokens (S tokens) are also embedded into the corresponding embedding vector and integrated into the compressed vector.

[0035] Then the compressed vector is concatenated with the prompt word embedding vector corresponding to the prompt word text to obtain a fusion feature for the subsequent decoding process. In addition, during the decoding process, the embodiment of the present invention adopts an autoregressive decoding mechanism to re-input the result of the previous decoding output into the decoding model to help the decoding model perform the subsequent decoding process. Therefore, the embodiment of the present invention obtains the current cumulative prediction token identifiers at each decoding. These identifiers are the target token identifiers obtained by the output of the decoding model, and then it is also necessary to perform embedding processing to obtain the cumulative embedding vector.

[0036] In each decoding process, the concatenated fusion features are concatenated with the accumulated embedding vector corresponding to the accumulated predicted token identifier, and then the concatenated features are input into the decoding model for autoregressive decoding to obtain the corresponding target token identifier. Here, the decoding model can also be a Transformer model. It should be noted that during the first decoding, the accumulated predicted token identifier does not exist, and the concatenated fusion features can be directly input into the decoding model for autoregressive decoding.

[0037] After the decoding model passes the autoregressive decoding, it outputs the corresponding target token identifiers one after another. These target token identifiers are used to generate the corresponding compressed text later. According to the autoregressive decoding mechanism, it is necessary to judge each target token identifier at this time. If the current target token identifier is a terminator, it means that the decoding process is completed and the autoregressive decoding is stopped. If the current target token identifier is not the terminal character, it means that the decoding process is still continuing. Then the current target token identifier is added as the cumulative predicted token identifier for the subsequent decoding process, and the target token identifier is continued to be generated until the current target token identifier is a terminator.

[0038] Step 103: According to a preset vocabulary, target token identifiers are mapped to text strings one by one, and the text strings are combined into compressed context text.

[0039] Through step 102, after the decoding model outputs all the target token identifiers, that is, after outputting the terminator, it maps the target token identifiers to text strings one by one according to the preset vocabulary, that is, by querying the word vocabulary, the token identifiers are restored to the corresponding text strings, and finally the text strings are composed into compressed context text. This compressed context text is the final result of the compression processing of the context text to be compressed, and can be directly used for the large language model to perform text processing tasks.

[0040] For examples, see Figure 2, the output of the text compression layer will enter the compression information mapping layer, and after feature dimension mapping through the multi-layer perceptron of the compression vector mapping unit (linear mapping), the final compression vector is output. Then the compression vector enters the compression information decoding layer, and the compression vector pre-filled with tokens (S tokens) and the corresponding prompt word text (M tokens) combined with the token embedding is input into the decoding model (i.e. Transformer language model) through the hidden layer for decoding processing, and the corresponding target token id is output. In the final output layer, according to the preset vocabulary, the target tokenid is mapped to the vocabulary to generate the corresponding target text as the compression context text.

[0041] The specific process of decoding can be found in Figure 4 , the prompt word token id list and the cumulative predicted token id list are embedded into embedding vector a and embedding vector b respectively, and then spliced ​​with the text compression vector along the sequence direction to obtain the corresponding splicing features, which are input into the Transformer language model for decoding to obtain the token id (i.e., the target token identifier) ​​used to generate new words. Then, it is determined whether the generated token id is a terminator. If not, this token id is added to the cumulative predicted token id list for the subsequent decoding process, i.e., the next decoding process. If the generated token id is determined to be a terminator, the decoding process is completed, the autoregressive decoding is stopped, and all newly generated token ids are mapped to text characters through the vocabulary and output, thereby obtaining the target text, i.e., the compressed context text.

[0042] In the embodiment of the present invention, the context text to be compressed is subjected to compression-based encoding processing to obtain a corresponding compression vector, which can save text data storage and computing resources. Next, through decoding processing, the compression vector and the prompt word embedding vector are decoded into token identifiers, which are mapped back to the text string one by one as the compressed context text. By compressing the text, the large language model can reduce the consumption of model computing resources and data storage resources when performing text processing tasks.

[0043] Of course, the above-mentioned embodiment of the present invention combines the encoder network, the multi-layer perceptron and the decoding model to jointly complete the compression task of the long text, and solves the technical problem that the large language model needs to consume huge model computing resources and data storage resources when processing the long context text. However, considering that the objects processed by the encoder network, the multi-layer perceptron and the decoding model are the context text to be compressed, and these network models themselves also need to be trained, if the long context text to be compressed is directly used as training data, the burden and difficulty of model training will be greatly increased, so that the final compression result is not ideal and cannot meet the basic requirements of long text compression.

[0044] Based on the above problems, the embodiment of the present invention further proposes a model training method, firstly, the encoder network, the multi-layer perceptron and the decoding model are respectively used as three processing modules of the text compression unit model, the compression vector mapping model and the compression vector decoding model, and then the three processing modules are integrated into a text compression model, and the context text to be compressed is compressed by the text compression model to obtain the compressed context text obtained in step 103. The embodiment of the present invention adopts a two-stage training strategy to train the text compression model, which is described in detail below.

[0045] First, obtain the context text training data set, which includes text sequences of various lengths. Considering that the text compression unit model directly processes long texts to be compressed, it needs to have excellent text compression capabilities, so its parameters are the most difficult to train. Therefore, in the two-stage training strategy, the data sets used for the two-stage training are different. The first stage is to train the text compression capabilities of the text compression unit model through some short text data sets.

[0046] Therefore, after obtaining the context text training data set, a text length threshold (e.g., 5000 character string length) can be preset, and then the context text training data set can be divided into a short text data set and a long text data set according to the preset text length threshold. Specifically, the character string length of the text samples included in the short text data set does not exceed 5000 character string lengths, while the character string length of the text samples included in the long text data set exceeds 5000 character string lengths.

[0047] The first stage of training of the text compression model is carried out using a short text dataset to obtain a first text compression model. During the first stage of training, the parameters of the text compression unit model, the compression vector mapping model and the compression vector decoding model in the first text compression model are updated. The training process of the first stage is described in detail below.

[0048] First, the short text dataset includes short text samples, first prompt word samples and first token prompt samples. The first token prompt sample is the token prompt obtained by text compression encoding and decoding of the short text sample, which is the real label corresponding to the short text sample.

[0049] During the forward propagation process of training, the short text sample is input into the text compression unit model of the text compression model for encoding processing to obtain the corresponding first compression vector, but it needs to be converted into the corresponding token identifier before encoding. Then the first compression vector is input into the compression vector mapping model in the text compression model for feature mapping to obtain the text compression vector. At the same time, the first embedding vector of the first prompt word sample and the second embedding vector of the first token prompt sample are determined. Here, the first prompt word sample is first converted into the corresponding token identifier and then embedded, while the first token prompt sample is directly embedded.

[0050] Next, the text compression vector, the first embedding vector and the second embedding vector are concatenated in the sequence direction to obtain a first concatenated feature. The first concatenated feature is input into the compression vector decoding model of the text compression model for decoding processing to obtain a token identifier of the first text to be predicted as the prediction result output by the text compression model after forward propagation.

[0051] Then, according to the token identifier of the first text to be predicted and the first token prompt sample, the first cross entropy loss function is constructed. The purpose of the cross entropy loss function here is to maximize the prediction probability of the correct string obtained by mapping the token identifier of the first text to be predicted.

[0052] This is the only way to ensure the prediction accuracy of the token identifier decoded by the text compression model.

[0053] When constructing the loss, remove multiple token identifiers at the end of the token identifier sequence of the first text to be predicted, then remove the same number of token identifiers at the beginning of the first token prompt sample, and then construct the corresponding loss function. The specific formula is as follows: (1) In the above formula (1), c represents the first token prompt sample without the initial token identifier. represents the text compression vector corresponding to the token identifier of the first text to be predicted, p represents the first embedding vector of the first prompt word sample, and P represents the prediction probability of obtaining the correct text string according to the token identifier of the first text to be predicted. Represents the model parameters of the compressed vector decoding model.

[0054] Finally, through the first cross entropy loss function Back propagation is performed in the text compression model, and the parameters of the text compression unit model, the compression vector mapping model and the compression vector decoding model in the text compression model are updated during the back propagation process to obtain a first text compression model.

[0055] In each iterative training process, the first cross entropy loss function is calculated by forward propagation. , the gradient of the text compression model is calculated through back propagation, and Adam (Adaptive Moment Estimation) is used as the gradient descent algorithm to optimize the gradient and update the model parameters of the text compression model. The text compression unit model, the compression vector mapping model, and the compression vector decoding model all participate in the parameter update.

[0056] According to the two-stage training strategy, in the second stage, the first text compression model obtained by the first stage training is trained by the long text data set to obtain the second text compression model. Because the text compression unit model in the first text compression model has excellent text compression capabilities through the training of the short text data set in the first stage. In the second stage, there is no need to train the text compression unit model of the first text compression model, so the parameters of the text compression unit model in the first text compression model are frozen, and only the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated. The training process of the second stage is described in detail below.

[0057] Similar to the short text dataset, the long text data specifically includes long text samples, second prompt word samples, and second token prompt samples. At the beginning of training, the long text samples, the second prompt word samples, and the second token prompt samples are input into the first text compression model for forward propagation to obtain the token identifier of the second text to be predicted.

[0058] Among them, the forward propagation process is: input the long text sample into the text compression unit model of the first text compression model for encoding processing to obtain the corresponding second compression vector, but before the encoding processing, it must also be converted into the corresponding token identifier and then encoded. Then the second compression vector is input into the compression vector mapping model in the first text compression model for feature mapping to obtain the text compression vector. At the same time, determine the third embedding vector of the second prompt word sample and the fourth embedding vector of the second token prompt sample. Here, the second prompt word sample is first converted into the corresponding token identifier and then embedded, while the second token prompt sample can be directly embedded. Then the text compression vector, the third embedding vector and the fourth embedding vector are spliced ​​along the sequence direction to obtain the second splicing feature; next, the second splicing feature is input into the compression vector decoding model of the first text compression model for decoding processing to obtain the token identifier of the second text to be predicted.

[0059] Similar to the first stage, a second cross entropy loss function is constructed according to the token identifier of the second text to be predicted and the second token prompt sample. The construction process is similar to the first cross entropy loss function and will not be repeated here.

[0060] Finally, back propagation is performed in the first text compression model through the second cross entropy loss function, and the second cross entropy loss function is used to train the prediction generation ability of the text compression model. Therefore, in the process of back propagation, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated. In other words, the parameters of the text compression unit model in the first text compression model are fixed, and only the parameters of the subsequent compression vector mapping model and the compression vector decoding model are updated during training.

[0061] For example, the second stage of training process can be seen in Figure 5, first sample the training set, divide the long context text therein, and convert it into a token id list, then input the compression vector unit (that is, the text compression unit model of the first text compression model) for encoding processing, and input the result of the encoding processing into the compression vector mapping unit (that is, the compression vector mapping model in the first text compression model) to obtain the corresponding text compression vector, and the second prompt word sample is also converted into the corresponding prompt word token id list, and then embedded to obtain the embedding vector a, and the tokenid list of the sentence to be predicted (that is, the second token prompt sample) is also embedded to obtain the embedding vector b, and then the text compression vector, the embedding vector a and the embedding vector b are spliced ​​along the sequence direction, and the obtained splicing features are input into the Transformer language model (that is, the compression vector decoding model of the first text compression model) for decoding processing to obtain the corresponding token id of the sentence to be predicted (that is, the token identifier of the second text to be predicted).

[0062] The token id of the sentence to be predicted needs to remove multiple tokens at the end, and the cross entropy loss is calculated based on the corresponding logit vector and the token of the sentence to be predicted (excluding the token at the beginning), where the logit vector represents the text compression vector corresponding to the token id of the sentence to be predicted obtained by decoding. During training, the cross entropy loss is used to perform back propagation in the model and determine whether the loss converges. If it does not converge, the next round of iterative training is continued based on the sampled data set. If it starts to converge, the training is terminated and the model weights, that is, the model parameters, are obtained through training.

[0063] The trained text compression model can directly perform text compression on the context text to be compressed to obtain the compressed context text obtained in step 103, which can effectively ensure the compression effect of the text and meet the text compression requirements when the large language model performs text processing tasks.

[0064] The embodiment of the present invention adopts a two-stage training strategy to train the text compression model. In the first stage, the text compression unit model is trained by short text to ensure the text compression capability of the text compression model. When the long text is used for training in the second stage, the text compression unit model is frozen, which can greatly reduce the video memory usage during the training of the model and ensure that the text compression model has the prediction generation capability. Thus, through the two-stage training strategy, the training burden and difficulty of the model can be effectively reduced while ensuring that the model has basic compression processing capabilities.

[0065] The following is a description of the long text generation device based on large language model context compression provided by the present invention. The long text generation device based on large language model context compression described below and the long text generation method based on large language model context compression described above can refer to each other.

[0066] like Figure 6 As shown, the long text generation device based on large language model context compression provided by the present invention includes an encoding module 601, a decoding module 602, and a generation module 603. Specifically, the encoding module 601 is used to obtain the context text to be compressed and the prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for the large language model to perform text processing tasks; the decoding module 602 is used to splice the compression vector with the prompt word embedding vector corresponding to the prompt word text, and perform autoregressive decoding processing on the spliced ​​fusion features to obtain corresponding multiple target token identifiers; the generation module 603 is used to map the target token identifiers to text strings one by one according to a preset vocabulary, and compose the text strings into compressed context text.

[0067] It should be noted that the beneficial effects of the long text generation device based on large language model context compression here and the long text generation method based on large language model context compression mentioned above can correspond to each other, so the beneficial effects of the long text generation device based on large language model context compression are not repeated here.

[0068] Figure 7 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communication interface 720 and the memory 730 communicate with each other through the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a long text generation method based on context compression of a large language model, the method comprising: obtaining a context text to be compressed and a prompt word text, and performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text for a large language model to perform a text processing task; splicing the compression vector with the prompt word embedding vector corresponding to the prompt word text, and performing autoregressive decoding processing on the spliced ​​fusion features to obtain a corresponding plurality of target token identifiers; mapping the target token identifiers to text strings one by one according to a preset vocabulary, and composing the text strings into a compressed context text.

[0069] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0070] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the long text generation method based on large language model context compression provided by the above methods, the method including: obtaining context text to be compressed and prompt word text, and performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for a large language model to perform text processing tasks; splicing the compression vector with the prompt word embedding vector corresponding to the prompt word text, and performing autoregressive decoding processing on the spliced ​​fusion features to obtain corresponding multiple target token identifiers; according to a preset vocabulary, mapping the target token identifiers one by one to text strings, and composing the text strings into compressed context text.

[0071] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the long text generation method based on large language model context compression provided by the above-mentioned methods, the method comprising: obtaining context text to be compressed and prompt word text, and performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for a large language model to perform text processing tasks; concatenating the compression vector with the prompt word embedding vector corresponding to the prompt word text, and performing autoregressive decoding processing on the concatenated fusion features to obtain corresponding multiple target token identifiers; according to a preset vocabulary, mapping the target token identifiers one by one to text strings, and composing the text strings into compressed context text.

[0072] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0073] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A long text generation method based on large language model context compression, characterized in that: The method comprises: Acquire context text to be compressed and prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for the large language model to perform a text processing task; The compressed vector is concatenated with the prompt word embedding vector corresponding to the prompt word text, and the concatenated fusion features are subjected to autoregressive decoding processing to obtain corresponding multiple target token identifiers; According to a preset vocabulary, the target token identifiers are mapped into text strings one by one, and the text strings are composed into compressed context text.

2. The long text generation method based on large language model context compression according to claim 1 is characterized in that: The step of performing compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector includes: Convert each text character of the context text to be compressed into a token identifier to obtain a corresponding token list; The token list is segmented according to a preset sequence length, and the segmented subsequences are sequentially stored in a queue; Selecting a subsequence from the queue and inputting the subsequence into an encoder network for compression for encoding, and concatenating vectors obtained by the encoding process to obtain a concatenated vector; The concatenated vector is input into a multi-layer perceptron for feature dimension mapping processing to obtain a corresponding compressed vector.

3. The long text generation method based on large language model context compression according to claim 1 is characterized in that: The concatenated fusion features are decoded based on autoregression to obtain corresponding multiple target token identifiers, including: Get the current cumulative prediction token identifier; Concatenating the concatenated fusion features with the accumulated embedding vector corresponding to the accumulated predicted token identifier, and inputting the concatenated features into a decoding model for autoregressive decoding processing to obtain a corresponding target token identifier; If the current target token identifier is a terminal character, the autoregressive decoding is stopped; if the current target token identifier is not a terminal character, the current target token identifier is added as the cumulative predicted token identifier.

4. The long text generation method based on large language model context compression according to claim 1 is characterized in that: The compressed context text is obtained by compressing a text compression model, wherein the text compression model includes a text compression unit model, a compression vector mapping model, and a compression vector decoding model. The training method of the text compression model includes: Acquire a context text training data set, and divide the context text training data set into a short text data set and a long text data set according to a preset text length threshold; Performing a first-stage training on the text compression model using the short text data set to obtain a first text compression model, wherein during the first-stage training process, parameters of a text compression unit model, a compression vector mapping model, and a compression vector decoding model in the first text compression model are updated; The first text compression model is trained in the second stage using the long text data set to obtain a second text compression model, wherein during the second stage training process, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

5. The long text generation method based on large language model context compression according to claim 4 is characterized in that: The short text data set includes short text samples, first prompt word samples and first token prompt samples. The first stage training of the text compression model using the short text data set to obtain the first text compression model includes: The short text sample is input into the text compression unit model of the text compression model for encoding processing to obtain a corresponding first compression vector, and the first compression vector is input into the compression vector mapping model in the text compression model for feature mapping to obtain a text compression vector. Determining a first embedding vector for the first prompt word sample and a second embedding vector for the first token prompt sample; Splicing the text compression vector, the first embedding vector, and the second embedding vector along a sequence direction to obtain a first splicing feature; Inputting the first concatenated feature into the compression vector decoding model of the text compression model for decoding processing to obtain a token identifier of the first text to be predicted; Constructing a first cross entropy loss function according to the token identifier of the first to-be-predicted text and the first token prompt sample; The first cross entropy loss function is back-propagated in the text compression model, and the parameters of the text compression unit model, the compression vector mapping model and the compression vector decoding model in the text compression model are updated during the back-propagation process to obtain a first text compression model.

6. The long text generation method based on large language model context compression according to claim 4 is characterized in that: The short text data set includes a long text sample, a second prompt word sample, and a second token prompt sample. The second stage training of the first text compression model using the long text data set to obtain the second text compression model includes: Inputting the long text sample, the second prompt word sample and the second token prompt sample into the first text compression model for forward propagation to obtain a token identifier of the second text to be predicted; Constructing a second cross entropy loss function according to the token identifier of the second to-be-predicted text and the second token prompt sample; Back propagation is performed in the first text compression model through the second cross entropy loss function, and in the process of back propagation, the parameters of the text compression unit model in the first text compression model are frozen, and the parameters of the compression vector mapping model and the compression vector decoding model in the first text compression model are updated.

7. A long text generation device based on large language model context compression, characterized in that: The device comprises: An encoding module, used to obtain context text to be compressed and prompt word text, and perform compression-based encoding processing on the context text to be compressed to obtain a corresponding compression vector, wherein the context text to be compressed is a text used for the large language model to perform text processing tasks; A decoding module, used for concatenating the compression vector with the prompt word embedding vector corresponding to the prompt word text, and performing autoregressive decoding processing on the concatenated fusion features to obtain corresponding multiple target token identifiers; The generating module is used to map the target token identifiers into text strings one by one according to a preset vocabulary, and compose the text strings into compressed context text.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the long text generation method based on large language model context compression is implemented as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the long text generation method based on large language model context compression is implemented as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the long text generation method based on large language model context compression is implemented as described in any one of claims 1 to 6.

Images