Processing methods for large language models and their related applications

Abstract

A processing method for a large language model (LLM) and related applications thereof are proposed. The method comprises: generating a current round vector based on the content of the current round in a multi-round dialogue; feeding the current round vector into a memory network containing information of previous rounds of the multi-round dialogue to generate a context vector; fusing the context vector with the feature value transmitted in the LLM during the reasoning process of the LLM for a new query; and generating a reply to the new query based on the output of the LLM. The processing method disclosed in the present invention can enable the LLM to use only the current user query as a model input, and solve the problem of the limitation of the long context splicing length of the LLM in multi-round dialogues by incorporating the limited length context vector generated by the memory network in the reasoning process.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a processing method for a large language model and related applications thereof. Background Art

[0002] Large Language Model (LLM) refers to Natural Language Processing (NLP) models with large parameters and a large amount of training data.

[0003] Large language models have made significant progress in conducting multi-turn conversations with users, where the model remembers previous context and generates responses based on the user's input. This capability has led to their widespread use in virtual assistants, customer service chatbots, educational applications, and more. Summary of the invention

[0004] According to a first aspect of the present disclosure, a processing method for LLM is proposed, comprising: generating a current round vector based on the content of a current round in a multi-round dialogue; sending the current round vector into a memory network containing information of previous rounds of the multi-round dialogue to generate a context vector; in the reasoning process of the LLM for a new query, fusing the context vector with a feature value transmitted in the LLM; and generating a reply to the new query based on the output of the LLM.

[0005] Optionally, generating a current round vector based on current round context content in a multi-round dialogue includes: merging all previous round contents and current round content into current round context content; and encoding the current round context content to obtain the current round vector.

[0006] Optionally, sending the current round vector into a memory network containing information about previous rounds of the multi-round conversations to generate a context vector includes: sending the current round vector into a memory network containing information about previous rounds of the multi-round conversations, and generating an intermediate output vector based on an output of the memory network; and generating the context vector based on the intermediate output vector.

[0007] Optionally, generating the context vector based on the intermediate output vector includes: fusing the intermediate output vector with the current round vector to generate the context vector.

[0008] Optionally, generating the context vector based on the intermediate output vector includes: using a linear transformation layer to map the intermediate output vector into a shape that can be spliced ​​with the feature value transmitted in the model.

[0009] Optionally, the memory network includes multiple parallel memory units, each memory unit is a recurrent neural network with the same structure and different parameters, and sending the current round vector into the memory network containing the previous round information of the multi-round dialogue to generate an intermediate output vector includes: sending the current round vector into the multiple parallel memory units to obtain multiple memory unit outputs; and fusing the multiple memory unit outputs to obtain the intermediate output vector.

[0010] Optionally, each memory cell corresponds to a recurrent neural network including a gating mechanism.

[0011] Optionally, each memory unit corresponds to a gated recurrent unit, and the gated recurrent unit only includes an update gate.

[0012] Optionally, during the reasoning process of the LLM for a new query, fusing the context vector with the feature value transmitted in the LLM includes: concatenating the context vector with the feature value transmitted by the LLM for reasoning for the new query, and using them for subsequent calculations of the model.

[0013] Optionally, in the reasoning process of the LLM for a new query, fusing the context vector with the feature value transmitted in the LLM includes: fusing the context vector with the feature value transmitted in the current layer in multiple layers of the LLM.

[0014] According to a second aspect of the present disclosure, a multi-round dialogue system is proposed, including an input-output module, a context vector generation module and a reasoning module, wherein the context vector generation module is used to: generate a current round context vector based on the current round content in the multi-round dialogue; send the current round context vector to a memory network containing the information of the previous round of the multi-round dialogue to generate a context vector, and the reasoning module is arranged with a large language model (LLM), and is used to: in the reasoning process of the LLM for a new query, fuse the context vector with the feature value transmitted in the model; and generate a reply to the new query based on the output of the LLM, and the input-output module is used to: obtain the query of each round, and output the reply generated by the LLM.

[0015] According to a third aspect of the present disclosure, a terminal device is proposed, comprising: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method based on the first aspect.

[0016] According to a fourth aspect of the present disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method based on the first aspect.

[0017] According to a fifth aspect of the present disclosure, a computer program product is proposed, comprising computer program instructions, characterized in that when the computer program instructions are executed by a processor, the method based on the first aspect is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0019] Figure 1 A schematic flowchart of a processing method for LLM according to an embodiment of the present disclosure is shown.

[0020] Figure 2 Shows Figure 1 Schematic diagram of the operation of the method shown.

[0021] Figure 3 An example of generating a context vector is shown.

[0022] Figure 4 A schematic diagram of the structure of GRU is shown.

[0023] Figure 5 An example of concatenating the context vector with the feature value during LLM inference is shown.

[0024] Figure 6 A schematic diagram of the composition of a multi-round dialogue system according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0025] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0026] At present, some large language models have performed well in natural language understanding, text generation, translation and other tasks, partly because they have learned extensive language knowledge and patterns through pre-training on large-scale corpora. Pre-training is a technique for training models on large-scale text data, which enables the model to learn the statistical structure, grammatical rules, semantic information, etc. of the language. This pre-trained model has shown certain language understanding and generation capabilities without fine-tuning for specific tasks.

[0027] During the pre-training phase, the model usually uses self-supervised learning methods. This means that the model learns without explicit labels by leveraging the intrinsic structure and contextual information in the text. For example, the model may be asked to predict missing words based on the context (e.g., MLM (Masked Language Model) task), which has enabled the model to make significant progress in understanding context and language relations.

[0028] Once pre-trained, the model can be fine-tuned to adapt to specific tasks, such as text classification, machine translation, or question answering. The purpose of fine-tuning is to train the model additionally with a small amount of labeled data based on the pre-trained model to make it better suited to specific tasks and fields.

[0029] This two-stage training process enables large language models to fully utilize large-scale unlabeled data and improve their performance on various natural language processing tasks.

[0030] These large language models play an important role in various applications and have made significant progress in conducting multi-round conversations with users. This application is often referred to as conversational AI. Large language models are designed to generate coherent responses in natural language or other modalities (such as pictures, videos, machine code, etc.) to respond to various questions, requests, or instructions raised by users in conversations.

[0031] Here, we can think of "generative large language model" as a language model that is trained on a large-scale conversation dataset and directly generates a response after receiving input. It is essentially a prediction of the output sequence based on the input sequence (such as text, pictures, audio, etc.). Each word, character or word in the output sequence is obtained based on the previous input sequence and the output sequence that has been output. This allows LLM to understand the input sequence and generate the output that best matches the model probability based on the input sequence.

[0032] In multiple rounds of conversation, the model remembers the previous context and generates appropriate responses based on the user's input. This ability has made them widely used in virtual assistants, customer service chatbots, educational applications and other fields.

[0033] Large language models usually process contextual information based on internal attention mechanisms to remember previous contexts across multiple rounds of conversations. The attention mechanism allows the model to focus on different parts at different positions in the input sequence while processing the input sequence. Specifically, in a conversation, the model receives a sequence of historical contexts, including the user's questions, the model's previous answers, and so on. When the model needs to generate a response, it uses the attention mechanism to determine which parts should be paid more attention to when generating the response based on the current input and previous context. In this way, the model is able to dynamically adjust its focus to better adapt to the development of the conversation.

[0034] For example, if the user mentioned a specific topic in the previous round of dialogue, the model can pay more attention to the topic through the attention mechanism when generating a response to ensure that the generated answer is consistent with the context. The attention mechanism can make the model more flexible in handling long sequences and multi-round dialogues, and better capture important contextual information, thereby generating more accurate and coherent responses.

[0035] Assume there is a conversation like this between a user and a model:

[0036]

[0037] Here, the user input is called a query, and the model's response is called an answer. It should be understood that the user input can be a question as shown above, or a response to the model output. For example, in a new query, the user answers "I like riding a bike." Regardless of the form of input (question or statement) given by the user, it is considered a query to the LLM, that is, a query to the model.

[0038] In a multi-round conversation, one question and one answer is usually counted as one round. Whenever a user asks a question and gets an answer, this is considered a round of conversation. Therefore, each user's question and the corresponding model reply are considered as part of a round of conversation. At this point, the above conversation can be written as follows after marking the rounds:

[0039]

[0040] Here, Q represents the query input by the user, and A represents the answer output by the model. When the user inputs a new query "I like to ride a bike" based on the second round of answers, the content of the first two rounds can be used as the "previous information" or "background information" of the new query, which can also be called "context" based on the custom in the field.

[0041] In the current LLM, when implementing multiple rounds of dialogue, the question-answer pairs of the previous dialogue will be spliced ​​before the query as historical information and fed into the model as input. For example, in the above example, when the user enters a new query "I like riding a bike.", the model will use all the content of the previous two rounds as model input before the new query content, and perform model reasoning based on this, giving the output of the answer to the new query (at this point, the third round ends). That is, the new query entered by the user is just "I like riding a bike", but the actual input used by LLM for reasoning is "Q: Hello, can you tell me what the weather is like today? A: Of course, it's sunny today. Q: Great! Do you have any outdoor activities to recommend? A: On sunny days, you have many options for outdoor sports, such as cycling, hiking, jogging, and picnics. Q: I like riding a bike."

[0042] Although this allows the model to notice thematic information in the above text, such as sunny weather, outdoor activities, and riding a bicycle, it is obvious that as the conversation deepens and the number of conversation turns increases, the model input will become longer and longer until it exceeds the maximum effective input length of the model, thereby truncating the earlier conversation content. In order to increase the number of conversation turns and the length of the conversation content that the model can handle, the number of input parameters of LLM needs to be expanded, but the expansion of the number of input parameters often causes a sharp increase in the number of model parameters, which greatly increases the capacity and computing power required for the model.

[0043] In order to solve the context capacity problem more reasonably and efficiently, the present disclosure proposes a processing method for LLM and a multi-round dialogue system using the method. The present disclosure introduces a memory network module to obtain the encoded context vector, and realizes the output of the model for new queries based on context information by fusing the context vector into each feature extraction layer in the LLM reasoning process for new queries. Through this method, the input token length required for context information in multi-round dialogues can be greatly reduced, thereby reducing the complexity of the model and achieving a longer historical dialogue depth.

[0044] Figure 1 FIG. 1 is a schematic flow chart of a processing method for LLM according to an embodiment of the present disclosure. For ease of understanding, Figure 2 Shows Figure 1 Schematic diagram of the operation of the method shown. Figure 1The processing method shown is suitable for generating model responses for multi-round dialogues in a scenario where a dialogue system is implemented using LLM. Here, LLM can be considered a generative language model (rather than a retrieval-based model).

[0045] In step S110, a current round vector is generated based on the content of the current round in the multi-round dialogue, and in step S120, the current round vector is sent to a memory network containing information of previous rounds of the multi-round dialogue to generate a context vector. Figure 2 As shown, a context vector generation module (the part in the dotted box in the figure) independent of the LLM can generate a context vector based on the content of all rounds that have occurred (including the current round and the previous round). Specifically, the context vector generation module may include an encoder and a memory module. The encoder is used to implement the operation of step S110, that is, to encode the input current round content into a vector. The memory module is used to implement step S120, and the memory module includes a memory network, so that a context vector can be generated based on the current round vector and the previous round information stored in the memory network.

[0046] Thus, in step S130, during the process of LLM reasoning for the new query, the context vector output by the memory network can be used to merge with the feature value transmitted in the LLM. Subsequently, in step S140, a response to the new query can be generated based on the output of the LLM. For example, the output of the LLM can be directly used as a response to the new query. Since the inter-layer feature values ​​merge the context vector output by the memory network during the reasoning process of the LLM, it can be considered that the output generated by the LLM is a response to the new query given by taking the context into consideration.

[0047] For example, when the current round is the second round in the above example (i.e., LLM completes the output of "A: On sunny days, you have many options for outdoor sports, such as biking, hiking, jogging, and picnicking."), since the context vector generation module can integrate the content information of each existing round, when the user enters a new query ("I like to ride a bike."), there is no need to incorporate all the historical sequence content into the user's new query as the input of LLM. At this time, the input of LLM is only the new query content entered by the user, and only in the process of LLM reasoning about the new query, the context vector can be fused with the feature value based on the new query passed between the layers of LLM (for example, the concatenate operation described in detail below).

[0048] Therefore, the processing method disclosed in the present invention can enable LLM to only use the current user query (i.e., the "new query" as described above) as the model input, and solve the problem of the limitation of the long context splicing length of LLM in multi-round dialogues by incorporating the limited length context vector generated by the memory network during the reasoning process.

[0049] In one embodiment, the context vector generation module can input only the updated content of the current round of multiple rounds of dialogue each time. For example, when the current round is the second round in the above example, only the question and answer content of the second round "Q: Great! Are there any outdoor activities recommended? A: On sunny days, you have many options for outdoor sports, such as cycling, hiking, jogging, and picnics." can be input into the encoder, thereby obtaining the current round vector generated only based on the updated content in step S110, and then sending the current round vector to the memory module that stores the previous content information to generate the context vector. This operation can reduce the input length that the encoder needs to encode, as well as the processing capacity of the memory module, but it requires a higher ability for the memory module to handle the correlation between each round.

[0050] In another embodiment, the context vector generation module inputs the context of multiple rounds of dialogue up to the current round each time, that is, each round of dialogue so far can be input into the encoder (such as Figure 2 As shown, the context is input into the encoder). At this time, step S110 may include: merging all previous round contents and current round contents into current round context content (i.e., the context is input into the encoder); and encoding the current round context content to obtain the current round vector (i.e., the "current round vector" output by the encoder corresponds to the "current round context content"). For example, when the current round is the second round in the above example, the context input to the encoder includes all the question and answer content of the first two rounds, that is, including: "Q: Hello, can you tell me what the weather is like today? A: Of course, it's sunny today. Q: Great! Are there any outdoor activities recommended? A: On sunny days, you have many options for outdoor sports, such as cycling, hiking, jogging, and picnics." The current round vector output by the encoder corresponds to the current round context vector (in order to distinguish it from the "context vector" output by the context vector generation module, the output of the encoder may be referred to as a "sentence vector" below). Therefore, the memory module can update the output of the context vector based on the sentence vector and the previous round information stored in its network. Since each input to the memory network is the complete context of the current round, the memory network can more easily extract the content that should be paid attention to in the context after the current round.

[0051] In the present disclosure, "memory network" refers to a neural network that has "memory" of previous inputs. In other words, the current output of the network is not only related to the current input, but also to the previous state. "Memory network" is usually implemented by a recurrent neural network (RNN). RNN is a neural network structure for processing sequence data. Its main feature is the introduction of recurrent connections, which allows the network to keep memory of historical information when processing sequence data.

[0052] Specifically, RNN maintains memory of past information by introducing hidden states in the recurrent connections of the neural network. This hidden state is updated at each time step so that the network can retain memory of past information when processing new inputs. For example, after each round, the encoding vector corresponding to the current context is output to the RNN. In the above example, after the first round ends (that is, after the model generates "Of course, it is sunny today."), the RNN obtains the vector corresponding to the query content of the first round as input, and outputs the context vector cv 1 And contains hidden state h 1 . Context vector cv 1 Can be fed into the LLM so that when the LLM receives a new query “Great! Any outdoor activities to recommend?”, it can combine the context vector in the reasoning process. cv 1 After the second round (i.e., after the model generates "On a sunny day, you have many options for outdoor sports, such as biking, hiking, jogging, and picnicking"), the RNN takes the vectors corresponding to the encodings of the two rounds of content as input, and calculates the output based on the input and the hidden state. h 1 Output context vector cv 2 And contains hidden state h 2 . Context vector cv 2 It can be fed into the LLM for the LLM to output for the new query content (i.e., "I like riding a bike.") (for example, the model's answer at this time is "Great! Riding a bike is a very pleasant outdoor activity that can both exercise the body and enjoy the surrounding scenery. Here are some suggestions and recommendations on riding a bike..."). After the third round, the RNN takes the vector corresponding to the content of rounds 1-3 as input, and according to the current input and hidden state h 2 Output context vector cv 3 And contains hidden state h 3 . Context vector cv3 It can then be used for LLM reasoning on new queries (e.g., questions in round 4).

[0053] Because conventional RNNs have the problem of gradient vanishing or gradient exploding when processing long sequences, it is sometimes difficult to effectively capture long-term dependencies. To this end, the memory network disclosed in the present invention can use an improved RNN, such as an RNN with a gating mechanism. The gating mechanism usually determines whether information should be passed or updated by learning, so as to more effectively capture long-term dependencies in sequence data. The implementation of RNN with a gating mechanism may include a long short-term memory network (LSTM) and a gated recurrent unit (GRU), which will be described in detail below.

[0054] Figure 3 An example of generating a context vector is shown. The operation of this diagram can be viewed as Figure 2 A specific implementation of the context vector generation module shown.

[0055] As shown in the figure, the context content (i.e., including the current and previous round content) can be fed into the encoder as input based on the current round content. Specifically, the context content can be converted into an embedding representation based on the LLM vocabulary, and encoded by the encoder to obtain a sentence vector. The encoder can take the form of the encoder part of the Transformer (transformer model) or the self-encoding feature extractor of a large-scale pre-trained model such as BERT, which is used to convert the current context text into a sentence vector that can be calculated by the downstream model.

[0056] In one embodiment, in order to ensure the diversity of context memory dimensions and expand the memory capacity of the memory module, a multi-head memory mechanism can be adopted, that is, multiple memory units are established to process the input sentence vectors respectively to obtain memory information of different focus dimensions. At this time, the memory network includes multiple parallel memory units, each memory unit is a recurrent neural network with the same structure and different parameters, and step S120 may include: sending the current round vector to the multiple parallel memory units to obtain multiple memory unit outputs; and fusing the multiple memory unit outputs to obtain the intermediate output vector. Each memory unit may correspond to an RNN with a gating mechanism, such as multiple gated recurrent units (GRU) shown in the figure. Here, gating refers to various thresholds to control the amount of information in and out. It should be understood that the number of GRU units can be selected as needed. For example, the number of GRU units can be set as a hyperparameter.

[0057] Figure 4The GRU structure diagram is shown in Figure 1. The GRU structure contains two gate control units: update gate and reset gate. The reset gate can be used to forget information, and the update gate can be used to update information to achieve memory. Memory is stored in the hidden state. In the case of the next context input, it will be combined with Complete interaction, update to form new memories.

[0058] The update gate determines whether to update the past hidden state Update to the current hidden state The gate vector generated by a sigmoid function To control the update, the calculation formula is:

[0059] (1)

[0060] in, is the weight matrix of the update gate, is the input of the GRU (e.g., the sentence vector output by the encoder), is the bias vector of the update gate, is the sigmoid activation function (e.g., a fully connected layer with an activation function).

[0061] The reset gate determines the past hidden state How to use the current input Combined. The gating vector generated by a sigmoid function To control the reset, the calculation formula is:

[0062] (2)

[0063] in, is the weight matrix of the reset gate, is the input of the GRU (e.g., the sentence vector output by the encoder), is the bias vector for the reset gate, is the sigmoid activation function (e.g., a fully connected layer with an activation function).

[0064] The new candidate hidden state can be calculated based on the update gate and reset gate (also called the intermediate hidden state), and then use the update gate to integrate the current hidden state and candidate hidden states To get the final hidden state The specific calculation formula is:

[0065] (3)

[0066] (4)

[0067] Among them, tanh represents the hyperbolic tangent activation function used to calculate the candidate hidden state, ⊙ represents element-by-element multiplication, is the weight matrix used for hidden state update, is the bias vector.

[0068] When the context vector generation model disclosed in the present invention uses the GRU as a memory unit to update the context vector, the threshold value can be first calculated based on equations (1) and (2): and (It should be understood that the sentence vector output by the encoder can be regarded as the input in equations (1) and (2) ). Then, the candidate hidden state is calculated based on formula (3) . Contains the current input information , and the remaining after the reset gate selection Information (In some embodiments, the GRU unit may not need to reset the gate and only retain the update gate. That is, each GRU unit only includes an update gate). Then, according to formula (4), the previous hidden state is fused and the current candidate hidden state , complete the hidden state Update (in a model embodiment, it can be given or When weights are attached, this allows the GRU to decide whether to input more or less prior memory states in one inference. Finally, the output of the GRU unit is calculated. The output can be directly obtained using formula (4) , you can also Then do some nonlinear transformations.

[0069] Multiple GRU units complete the update independently, and then merge and superimpose to generate the output vector of the memory network module. The output vector contains context information.

[0070] In some cases, the vector output by the memory network is not used directly as a context vector, but as an intermediate output vector required to generate the context vector of the LLM. The intermediate output vector can be combined with other vectors, such as the sentence vector output by the encoder, to avoid losing too much information of the current round of dialogue, or undergo other operations to obtain the context vector. At this time, step S120 may include: sending the current round context vector to the memory network containing the information of the previous round of the multi-round dialogue, and generating an intermediate output vector based on the output of the memory network; and generating the context vector based on the intermediate output vector.

[0071] In order to further improve the memory depth of the memory module and reduce the loss of original information, Figure 3 As shown, a shortcut across layers is used to directly superimpose the original context information with the output processed by the memory network in a certain proportion, thereby generating the context vector. Therefore, generating the context vector based on the intermediate output vector may include: fusing the intermediate output vector with the current round context vector (i.e., the sentence vector shown in the figure) to generate the context vector.

[0072] Furthermore, as shown in the figure, the output of the memory network (or the output of the memory network with the sentence vector superimposed as above) can be mapped to the same size as the LLM intermediate layer through a linear transformation layer. At this time, generating the context vector based on the intermediate output vector includes: using a linear transformation layer to map the intermediate output vector into a shape that can be spliced ​​with the eigenvalues ​​transmitted in the model. Specifically, the above mapping can be a dimensionality-upgrading mapping. In other words, the data dimension operated in the context vector generation module can be smaller than the dimension of the eigenvalues ​​transmitted in the LLM, and the linear transformation layer shown in the figure is used for dimensionality upgrading when output to the LLM, thereby reducing the number of parameters in the context vector generation module and reducing the amount of calculation.

[0073] The context vector generated by the context vector generation module may, in different embodiments, be integrated with the feature value passed by the LLM during the reasoning process for the new query in different ways.

[0074] In neural networks, eigenvalues ​​usually refer to intermediate calculated values ​​generated in the middle layers of the network. These intermediate calculated values ​​are the representation of the input data after a series of linear and nonlinear transformations in the network, and can also be understood as the representation or activation of the data in a specific layer. These eigenvalues ​​contain various characteristic information of the input data, which helps the network to perform tasks such as classification and regression of the data. In deep learning, each neuron in the hidden layer calculates a set of eigenvalues, and the extraction process of these eigenvalues ​​is achieved through the linear combination of weight parameters and the nonlinear transformation of the activation function. These eigenvalues ​​are abstract representations of the data learned by the network, with different semantic meanings, and can be used to represent various attributes and characteristics of the input data.

[0075] The dimension of the eigenvalue depends on the structure of the network and the settings of each layer. In deep neural networks, the dimension of the eigenvalue is usually determined by the number of neurons in the hidden layer of the network. The dimension of the eigenvalue is a dynamic concept in neural networks and depends on the structure and parameter settings of the network. Generally, the dimension of the eigenvalue increases with the depth and width of the network, reflecting the network's more abstract and higher-level representation of the input data. In other words, in different layers, the eigenvalues ​​usually have different sizes.

[0076] Here, it should be understood that whether it is a "feature value" or a "context vector", it is usually a tensor with multiple dimensions. In a neural network, a tensor is a multidimensional array structure and is one of the most basic data representations in a deep learning model. Tensors can have different ranks and shapes. The dimension of a tensor refers to the number of axes of the tensor. For example, a one-dimensional tensor is a vector (in the computer's storage consciousness, a vector is actually a one-dimensional quantity, which has a value in only one dimension. In actual operation, in order to facilitate computer storage, a multidimensional tensor is usually "flattened" into a one-dimensional vector. For example, the context vector of the present disclosure actually corresponds to a multidimensional tensor, usually a three-dimensional tensor), a two-dimensional tensor is a matrix, and a three-dimensional tensor can be regarded as a three-dimensional array, and so on. The shape of a tensor describes the size of the tensor in each dimension. For example, a tensor with a shape of (3, 4) represents a two-dimensional array with 3 rows and 4 columns of elements. Two tensors with different dimensions and / or shapes can be considered to have different sizes.

[0077] The feature values ​​passed in LLM are usually multi-dimensional and the last dimension corresponding to the hidden dimension usually has the same size, such as 768. At this time, in an LLM based on Transformer, for example, the shape of the feature value is (?,?,768), the value of the first question mark can represent the batch size B, the value of the second question mark can represent the sequence length T (for example, the current input has 50 tokens), and 768 indicates that the hidden dimension is 768. When LLM conducts multiple rounds of dialogue, since B is 1 at this time, the feature value passed in LLM can be a two-dimensional matrix of T×768.

[0078] In one embodiment, the context vector can be fused with the feature values ​​in only one specific layer of the LLM. Figure 3The linear transformation layer shown converts the intermediate output vector of the merged sentence vector into a context vector of a size that can be fused with the feature value of this layer. In one embodiment, the fusion is splicing (e.g., addition of corresponding dimensions). In this case, the dimension of the context vector is the same as the dimension of the feature value of this layer, and the last dimension is also a tensor of 768 (i.e., the shape does not need to be exactly the same, only the dimension and the last dimension need to be the same size). For example, when B is 1, the context vector and the feature value of this layer can both be two-dimensional, the shape of the context vector is (a, 768), and the shape of the feature value of this layer is (b, 768), then the shape of the fused vector obtained after splicing is (a+b, 768).

[0079] In other embodiments, in step S130, the context vector and the feature value transmitted in the current layer may be fused in multiple layers of the LLM. The fusion may be concatenation. Figure 5 An example of concatenating the context vector with the feature value in the LLM reasoning process is shown. The context vector on the left side of the linear transformation layer shown in the figure can correspond to Figure 3 The intermediate output vector of the sentence vector is added (i.e., corresponding to the context vector without linear transformation). Since the context vector and the feature value need to be concatenated for multiple layers of LLM (usually the hidden layers of the model), and the dimensions of the feature values ​​transmitted in the hidden layers may be different, the linear transformation layer shown in the figure can transform the context vector into different shapes for different layers. The ?×?×768 of the context vector in the figure indicates that the context vector can be transformed into a 3-dimensional tensor with the last dimension being 768. The values ​​of the first two ?×? can be determined according to the original shape of the context vector without linear transformation and the shape of the feature value to be concatenated. Similarly, the ?×?×768 of the feature value in the figure indicates that the feature value transmitted during the LLM reasoning process is also a 3-dimensional tensor with the last dimension being 768. The values ​​of the first two ?×? can be determined according to the structure of the previous hidden layer and the input query.

[0080] In addition, it should be understood that, due to the existence of the memory module, the dimension of the context vector generated by the context vector generation module of the present invention will not increase significantly as the context content of the input encoder increases, but will be maintained in a relatively small range.

[0081] Therefore, in order to more reasonably solve the problem of multi-round context capacity, the present disclosure introduces a memory network module, uses a multi-head recurrent neural network as a memory unit, extracts multi-dimensional context information, and uses a direct connection method to compensate for the degradation problem caused by too many rounds. The sampled and encoded historical information can then be integrated into each feature extraction layer of the large model to achieve context fusion. This can avoid the need to splice the above multi-round dialogue information into the current query, making the model more economical, the input query content can also be longer, and there is no need to expand the input layer parameters to achieve deeper context, avoiding the problem of long context being truncated due to fixed input length.

[0082] The present disclosure may also be implemented as a multi-round dialogue system. Figure 6 A schematic diagram of the composition of a multi-round dialogue system according to an embodiment of the present disclosure is shown.

[0083] As shown in the figure, the multi-round dialogue system 600 includes an input-output module 610, a context vector generation module 620, and a reasoning module 630. The context vector generation module and the reasoning module can cooperate to perform the processing method for LLM disclosed above. Specifically, the input-output module is used to obtain queries of each round and output the corresponding responses generated by the LLM.

[0084] The context vector generation module 620 is used to generate a current round context vector based on the current round content in the multi-round dialogue; the current round context vector is sent to the memory network containing the information of the previous round of the multi-round dialogue to generate a context vector. The reasoning module 630 is arranged with an LLM, and is used to merge the context vector with the feature value transmitted in the model during the reasoning process of the LLM for a new query; and generate a reply to the new query based on the output of the LLM.

[0085] In one embodiment, the present disclosure may also be implemented as a terminal device, including: a processor; and a memory, on which executable codes are stored, and when the executable codes are executed by the processor, the processor executes the method described above.

[0086] In one embodiment, the present disclosure may also be implemented as a non-transitory machine-readable storage medium having executable codes stored thereon. When the executable codes are executed by a processor of an electronic device, the processor is caused to execute the method described above.

[0087] In one embodiment, the present disclosure may also be implemented as a computer program product, including computer program instructions, characterized in that when the computer program instructions are executed by a processor, the method described above is implemented.

[0088] The present disclosure proposes a context management method for LLM, which realizes context fusion by encoding the context history and sampling it using a memory network module, and then integrating the sampled and encoded context information into each feature extraction layer of LLM. As a result, the length of the tokens required to input multiple rounds of dialogue in the existing LLM can be greatly reduced, thereby reducing the model complexity of LLM and achieving a longer historical dialogue depth.

[0089] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0090] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system and method according to multiple embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or code, and the part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0091] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A processing method for a large language model LLM, comprising: Generate a current round vector based on the current round content in a multi-round conversation; Sending the current turn vector to a memory network containing information of previous turns of the multi-round dialogue to generate a context vector, wherein the memory network is independent of the LLM; In the reasoning process of the LLM for a new query as input, the context vector is merged with the feature value based on the new query transmitted in the LLM; as well as A reply to the new query is generated based on the output of the LLM.

2. The method of claim 1, wherein: Generating the current round vector based on the current round context content in a multi-round dialogue includes: Merge all previous round contents and current round contents into current round context contents; and The current round context content is encoded to obtain the current round vector.

3. The method of claim 1, wherein: Sending the current round vector into a memory network containing information of previous rounds of the multi-round dialogue to generate a context vector includes: Sending the current round vector to a memory network containing information of previous rounds of the multi-round dialogue, and generating an intermediate output vector based on an output of the memory network; and The context vector is generated based on the intermediate output vector.

4. The method of claim 3, wherein: The generating the context vector based on the intermediate output vector comprises: The intermediate output vector is merged with the current round vector to generate the context vector.

5. The method of claim 3, wherein: Generating the context vector based on the intermediate output vector comprises: A linear transformation layer is used to map the intermediate output vector into a shape that can be concatenated with the eigenvalue passed in the LLM.

6. The method of claim 3, wherein: The memory network includes a plurality of parallel memory units, each memory unit is a recurrent neural network with the same structure and different parameters, and the current round vector is sent to the memory network containing the information of the previous round of the multi-round dialogue to generate an intermediate output vector including: Sending the current round vector to the plurality of parallel memory units to obtain a plurality of memory unit outputs; and The plurality of memory unit outputs are fused to obtain the intermediate output vector.

7. The method of claim 6, wherein: Each memory cell corresponds to a recurrent neural network including a gating mechanism.

8. The method of claim 6, wherein: Each memory cell corresponds to a gated recurrent unit, and the gated recurrent unit only includes an update gate.

9. The method of claim 1, wherein: In the reasoning process of the LLM for a new query, fusing the context vector with the feature value transmitted in the LLM includes: The context vector is concatenated with the feature value passed by the LLM for reasoning about the new query and used for subsequent calculations of the LLM.

10. The method of claim 1, wherein: In the reasoning process of the LLM for a new query, fusing the context vector with the feature value transmitted in the LLM includes: The context vector is fused with the feature value transmitted in the current layer in multiple layers of the LLM.

11. A multi-turn dialogue system, comprising an input-output module, a context vector generation module and a reasoning module, wherein: The context vector generation module is used for: Generate a current round context vector based on the current round content in a multi-round conversation; The current round context vector is fed into a memory network containing information of previous rounds of the multi-round dialogue to generate a context vector, and The reasoning module is arranged with a large language model LLM and is used to: In the reasoning process of the LLM for a new query as input, the context vector is merged with the feature value based on the new query transmitted in the LLM; as well as generating a response to the new query based on the output of the LLM, The input and output modules are used for: Get the queries for each round and output the answers generated by the LLM.

12. A terminal device, comprising: processor; as well as A memory having executable codes stored therein, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 10.

13. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 10.

14. A computer program product comprising computer program instructions, characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images