Large language model retrieval fine tuning method based on memory enhancement

Abstract

The invention discloses a memory enhancement-based large language model retrieval fine tuning method. The method comprises the following steps of: performing layer-by-layer dense vector representation on text sequences in a text sequence library; performing layer-by-layer alignment analysis to obtain an alignment loss value of each layer; performing layer-by-layer uniformity analysis to obtain a uniformity loss value of each layer; obtaining an optimal alignment layer and an optimal uniform layer according to the alignment loss value of each layer and the uniformity loss value of each layer; inputting the optimal alignment layer and the optimal uniform layer into a retrieval fine tuning module to obtain a final output vector; and substituting the final output vector into the cross entropy training loss function, and obtaining model parameters of the retrieval fine tuning module by minimizing the value of the cross entropy training loss function. According to the method, effective text retrieval is carried out while internal parameters of the large language model are not finely adjusted and the original generation capability of the large language model is kept, model compatibility of text retrieval and text generation is completed, and a technical basis is provided for constructing a long-term memory module of the large language model.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing and information retrieval, and in particular relates to a large language model retrieval fine-tuning method based on memory enhancement. Background Art

[0002] In the blueprint for building a general artificial intelligence framework, the memory module is regarded as one of the core components of general artificial intelligence. It can effectively retain information that needs to be retained for a long time, and help large language models (LargeLanguage Models) to better plan and make decisions in various tasks in the form of memory. In recent years, many methods have emerged that can be used to enhance the short-term memory of large language models. These methods enable large language models to use historical information by improving the attention mechanism of Transformer or adding a recurrent network mechanism. However, the scale of historical information that can be used by these methods is limited, and it is difficult to apply them to the long-term memory stage.

[0003] At present, the mainstream method to enhance the long-term memory of large language models is dense retrieval (DR), which can compress a large amount of information that has been processed or generated by the large language model into dense vectors for long-term storage, and support the rapid retrieval of memory information related to the current text generation task from large-scale memory vectors. However, due to the heterogeneity of the text retrieval and text generation task paradigms, the DR model and the large language model generation model are relatively independent in most existing studies. It usually requires re-fine-tuning a large language model as a specific retrieval module to support the generation function of the large language model, but this training method is costly; in addition, the large language model will also degenerate into a retrieval task-specific model, making it incompatible with the generation paradigm of the large language model for processing various natural language tasks, limiting the vision of developing the large language model into a universal foundation for various natural language processing and generation tasks. Summary of the invention

[0004] The technical problem solved by the present invention is: to overcome the shortcomings of the prior art, to provide a large language model retrieval fine-tuning method based on memory enhancement, to achieve effective text retrieval without fine-tuning the internal parameters of the large language model and retaining its original generation capability, to achieve model compatibility between text retrieval and text generation, and to provide a technical basis for constructing a long-term memory module of a large language model.

[0005] The object of the present invention is achieved through the following technical scheme: a large language model retrieval fine-tuning method based on memory enhancement, comprising: mapping a text sequence X in a preset text sequence library through a large language model to obtain a first layer-by-layer vector sequence, performing average pooling processing on the first layer-by-layer vector sequence to obtain a first layer-by-layer dense vector; mapping a related text sequence X of the text sequence X in the preset text sequence library to obtain a first layer-by-layer dense vector;+ A second layer-by-layer vector sequence is obtained through mapping with a large language model, and the second layer-by-layer vector sequence is processed by average pooling to obtain a second layer-by-layer dense vector; a text sequence Y in a preset text sequence library is mapped by a large language model to obtain a third layer-by-layer vector sequence, and the third layer-by-layer vector sequence is processed by average pooling to obtain a third layer-by-layer dense vector; an alignment loss value of each layer is obtained according to the first layer-by-layer dense vector and the second layer-by-layer dense vector; a uniformity loss value of each layer is obtained according to the first layer-by-layer dense vector and the third layer-by-layer dense vector; an optimal alignment vector sequence of the text sequence X, an optimal uniform vector sequence of the text sequence X, and a text sequence X are obtained according to the alignment loss value of each layer and the uniformity loss value of each layer. + The optimal alignment vector sequence and text sequence X + The optimal uniform vector sequence of the text sequence Y, the optimal alignment vector sequence of the text sequence Y, and the optimal uniform vector sequence of the text sequence Y are input into the retrieval fine-tuning module, and the output vector sequence of the text sequence X is output. The output vector sequence of the text sequence X is processed by average pooling to obtain the final output vector of the text sequence X. + The optimal alignment vector sequence and the optimal uniform vector sequence are input into the retrieval fine-tuning module, and the output text sequence X + The output vector sequence of the text sequence X + The output vector sequence of is processed by average pooling to obtain the text sequence X + The optimal alignment vector sequence and the optimal uniform vector sequence of the sequence text Y are input into the retrieval fine-tuning module, and the output vector sequence of the text sequence Y is output. The output vector sequence of the text sequence Y is processed by average pooling to obtain the final output vector of the text sequence Y. The final output vector of the text sequence X and the sequence text X are + The final output vector of and the final output vector of the text sequence Y are brought into the cross entropy training loss function, and the model parameters of the retrieval fine-tuning module are obtained by minimizing the value of the cross entropy training loss function.

[0006] In the above memory-enhanced large language model retrieval fine-tuning method, for any text sequence X, there are manually annotated related text sequences X + ; Define the preset text sequence library except text sequence X and text sequence X + All other text sequences except are sequence text Y.

[0007] In the above-mentioned memory-enhanced large language model retrieval fine-tuning method, the alignment loss value of each layer is obtained by the following formula:

[0008]

[0009] in, is the alignment loss value of each layer, x l is the first layer-by-layer dense vector, x +,l is the second layer-by-layer dense vector, To calculate the expected function, An operation that calculates the square of the Euclidean norm (the second norm).

[0010] In the above-mentioned memory-enhanced large language model retrieval fine-tuning method, the uniformity loss value of each layer is obtained by the following formula:

[0011]

[0012] in, is the uniformity loss value of each layer, x l is the first layer-by-layer dense vector, y l is the third layer-by-layer dense vector, To calculate the expected function, An operation that calculates the square of the Euclidean norm (the second norm).

[0013] In the above-mentioned memory-enhanced large language model retrieval fine-tuning method, the cross entropy training loss function is:

[0014]

[0015] Among them, O X is the final output vector of the text sequence X, For sequence text X + The final output vector, O Y is the final output vector of the text sequence Y, θ is the model parameter of the retrieval fine-tuning module, Large language models predict large model parameters.

[0016] A memory-enhanced large language model retrieval fine-tuning system comprises: a first module for mapping a text sequence X in a preset text sequence library through a large language model to obtain a first layer-by-layer vector sequence, and processing the first layer-by-layer vector sequence through average pooling to obtain a first layer-by-layer dense vector; a second module for mapping a related text sequence X of the text sequence X in the preset text sequence library to a first layer-by-layer dense vector; +A second layer-by-layer vector sequence is obtained by mapping the large language model, and the second layer-by-layer vector sequence is processed by average pooling to obtain a second layer-by-layer dense vector; a third module is used to map the text sequence Y in the preset text sequence library through the large language model to obtain a third layer-by-layer vector sequence, and the third layer-by-layer vector sequence is processed by average pooling to obtain a third layer-by-layer dense vector; a fourth module is used to obtain a per-layer alignment loss value according to the first layer-by-layer dense vector and the second layer-by-layer dense vector; a fifth module is used to obtain a per-layer uniformity loss value according to the first layer-by-layer dense vector and the third layer-by-layer dense vector; a sixth module is used to obtain the optimal alignment vector sequence of the text sequence X, the optimal uniform vector sequence of the text sequence X, and the text sequence X according to the per-layer alignment loss value and the per-layer uniformity loss value. + The optimal alignment vector sequence and text sequence X + The optimal uniform vector sequence of the text sequence X, the optimal alignment vector sequence of the text sequence Y, and the optimal uniform vector sequence of the text sequence Y; the seventh module is used to input the optimal alignment vector sequence and the optimal uniform vector sequence of the text sequence X into the retrieval fine-tuning module to output the output vector sequence of the text sequence X, and the output vector sequence of the text sequence X is processed by average pooling to obtain the final output vector of the text sequence X; the eighth module is used to + The optimal alignment vector sequence and the optimal uniform vector sequence are input into the retrieval fine-tuning module, and the output text sequence X + The output vector sequence of the text sequence X + The output vector sequence of is processed by average pooling to obtain the text sequence X + The ninth module is used to input the optimal alignment vector sequence and the optimal uniform vector sequence of the sequence text Y into the retrieval fine-tuning module, output the output vector sequence of the text sequence Y, and obtain the final output vector of the text sequence Y by average pooling processing on the output vector sequence of the text sequence Y; the tenth module is used to convert the final output vector of the text sequence X and the sequence text X + The final output vector of and the final output vector of the text sequence Y are brought into the cross entropy training loss function, and the model parameters of the retrieval fine-tuning module are obtained by minimizing the value of the cross entropy training loss function.

[0017] In the above memory-enhanced large language model retrieval fine-tuning system, for any text sequence X, there are manually annotated related text sequences X + ; Define the preset text sequence library except text sequence X and text sequence X + All other text sequences except are sequence text Y.

[0018] In the above memory-enhanced large language model retrieval fine-tuning system, the alignment loss value of each layer is obtained by the following formula:

[0019]

[0020] in, is the alignment loss value of each layer, x l is the first layer-by-layer dense vector, x +，l is the second layer-by-layer dense vector, To calculate the expected function, An operation that calculates the square of the Euclidean norm (the second norm).

[0021] In the above memory-enhanced large language model retrieval fine-tuning system, the uniformity loss value of each layer is obtained by the following formula:

[0022]

[0023] in, is the uniformity loss value of each layer, x l is the first layer-by-layer dense vector, y l is the third layer-by-layer dense vector, To calculate the expected function, An operation that calculates the square of the Euclidean norm (the second norm).

[0024] In the above memory-enhanced large language model retrieval fine-tuning system, the cross-entropy training loss function is:

[0025]

[0026] Among them, O X is the final output vector of the text sequence X, For sequence text X + The final output vector, O Y is the final output vector of the text sequence Y, θ is the model parameter of the retrieval fine-tuning module, Large language models predict large model parameters.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] (1) The present invention provides an analysis method for the spatial alignment and uniformity of generative large language models, which realizes efficient analysis and evaluation of the spatial alignment and uniformity of large language models;

[0029] (2) The present invention realizes effective text retrieval without fine-tuning the internal parameters of the large language model and retaining its original generation capability, completes the model compatibility of text retrieval and text generation, and provides a technical basis for constructing the long-term memory module of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 Schematic diagram of the connection between the large language model and the retrieval fine-tuning module provided by an embodiment of the present invention;

[0032] Figure 2 It is a schematic diagram of a retrieval fine-tuning module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] This embodiment provides a large language model retrieval fine-tuning method based on memory enhancement, the method comprising:

[0035] The text sequence X in the preset text sequence library is mapped by the large language model to obtain the first layer-by-layer vector sequence The first layer-by-layer vector sequence The first layer-by-layer dense vector is obtained by average pooling

[0036] The related text sequence X of the text sequence X in the preset text sequence library + The second layer-by-layer vector sequence is obtained through mapping of the large language model The second layer-by-layer vector sequence The second layer-by-layer dense vector is obtained by average pooling

[0037] The text sequence Y in the preset text sequence library is mapped by the large language model to obtain the third layer-by-layer vector sequence The third layer-by-layer vector sequence The third layer-by-layer dense vector is obtained by average pooling

[0038] Obtaining the alignment loss value of each layer according to the first layer-by-layer dense vector and the second layer-by-layer dense vector;

[0039] Obtaining a uniformity loss value of each layer according to the first layer-by-layer dense vector and the third layer-by-layer dense vector;

[0040] The optimal alignment vector sequence of the text sequence X is obtained according to the alignment loss value of each layer and the uniformity loss value of each layer. The optimal uniform vector sequence of the text sequence X is Text sequence X + The optimal alignment vector sequence Text sequence X + The optimal uniform vector sequence The optimal alignment vector sequence for the text sequence Y The optimal uniform vector sequence of the text sequence Y

[0041] The optimal alignment vector sequence of the text sequence X and the optimal uniform vector sequence Input into the retrieval fine-tuning module, output the output vector sequence of the text sequence X Output vector sequence for text sequence X The final output vector O of the text sequence X is obtained by average pooling X ;

[0042] Sequence text X + The optimal alignment vector sequence and the optimal uniform vector sequence Input into the retrieval fine-tuning module and output the text sequence X + The output vector sequence For the text sequence X + The output vector sequence The text sequence X is obtained by average pooling + The final output vector

[0043] The optimal alignment vector sequence of sequence text Y and the optimal uniform vector sequence Input into the retrieval fine-tuning module, output the output vector sequence of the text sequence Y Output vector sequence for text sequence Y The final output vector O of the text sequence Y is obtained by average pooling Y ;

[0044] The final output vector O of the text sequence X X , sequence text X + The final output vector And the final output vector O of the text sequence Y YSubstitute it into the cross entropy training loss function and obtain the model parameter θ of the retrieval fine-tuning module by minimizing the value of the cross entropy training loss function.

[0045] For any text sequence X, there is a manually annotated related text sequence X + ; Define the preset text sequence library except text sequence X and text sequence X + All other text sequences except are sequence text Y.

[0046] The alignment loss value of each layer is obtained by the following formula:

[0047]

[0048] in, is the alignment loss value of each layer, x l is the first layer-by-layer dense vector, x +,l is the second layer-by-layer dense vector, To calculate the expected function, An operation that calculates the square of the Euclidean norm (the second norm).

[0049] The uniformity loss value of each layer is obtained by the following formula:

[0050]

[0051] in, is the uniformity loss value of each layer, x l is the first layer-by-layer dense vector, y l is the third layer-by-layer dense vector, To calculate the expected function, An operation that calculates the square of the Euclidean norm (the second norm).

[0052] The cross entropy training loss function is:

[0053]

[0054] Among them, O X is the final output vector of the text sequence X, For sequence text X + The final output vector, O Y is the final output vector of the text sequence Y, θ is the model parameter of the retrieval fine-tuning module, Large language models predict large model parameters.

[0055] Specifically, the method comprises the following steps:

[0056] Step 1: Perform layer-by-layer dense vector representation on the text sequences in the text sequence library

[0057] The preset text sequence library D contains multiple text sequences. For any text sequence X, there is a manually annotated related text sequence X + , define (X, X + ) is a pair of positive samples, that is, (X, X + )~D. In addition, for any text sequence X, define the text sequence library except X and X + All other text sequences except the positive sample are Y, and (X, Y) is defined as any sample pair other than the positive sample, that is, (X, Y) ~ D\X + .

[0058] (1) The text sequence X is passed through a large language model to obtain a layer-by-layer vector sequence Then the layer-by-layer vector sequence Through the average pooling operation f(·), we get the layer-by-layer dense vector The text sequence is represented as X = {x1, ..., x t , …, x n}, x1 is the first character, x n is the nth character; layer-by-layer vector sequence Expressed as is the vector representation of the first character x1 of the text sequence X in the lth layer of the large language model, The nth character x of the text sequence X is the lth layer of the large language model n The vector representation of ;

[0059] (2) The related text sequence X of the text sequence X + Mapped into layer-by-layer vector representations of related text sequences by a large language model Then the layer-by-layer vector sequence Through the average pooling operation f(·), we get the layer-by-layer dense vector Among them, the related text sequence of text sequence X is expressed as is the first character, is the nth character; the layer-by-layer vector sequence of the relevant text is represented as For the lth layer of the large language model, the text sequence X + The first character of The vector representation of For the lth layer of the large language model, the text sequence X + The nth character of The vector representation of ;

[0060] (3) Map the text sequence Y into a layer-by-layer vector sequence through a large language model Then the layer-by-layer vector sequence Through the average pooling operation f(·), we get the layer-by-layer dense vector Among them, the text sequence Y is represented by Y = {y1,…,y t ,…,y n}, y1 is the first character, y n is the nth character; the layer-by-layer vector sequence is represented as is the vector representation of the first character y1 of the text sequence Y in the lth layer of the large language model, The nth character y of the text sequence Y is the lth layer of the large language model n The vector representation of ;

[0061] Step 2: Layer-by-layer alignment analysis of large language models

[0062] For each positive text sequence sample pair (X,X + )The layer-by-layer vector (x) obtained in step 1 l ,x +,l ) through the alignment loss function The alignment loss value of each layer is calculated.

[0063] The alignment loss is used to measure the pair of positive text sequence samples (X, X + ), the distance between X and X + The closer the distance between them, the smaller the alignment loss and the better the alignment.

[0064] Step 3: Layer-by-layer uniformity analysis of large language models

[0065] For each text sequence sample X and all other text sequences Y in its corresponding text library, the layer-by-layer vector x obtained in step 1 is l and l Through the uniformity loss function The uniformity loss value of each layer is calculated.

[0066] Uniformity loss is used to measure the average distance between all sample pairs (X, Y) in the text sequence library. The larger the average vector distance between all samples, the smaller the uniformity loss and the better the uniformity.

[0067] An ideal text representation space should have both good alignment and uniformity. Therefore, a retrieval fine-tuner is built on the optimal alignment layer and the optimal uniformity layer of the large language model.

[0068] Step 4: Select the optimal alignment layer and optimal uniform layer for the large language model

[0069] According to the alignment loss value and uniformity loss value of each layer calculated layer by layer in step 3 and step 4, for any text sequence, the optimal alignment layer of the large language model is defined as the a-th layer, and the optimal uniform layer is defined as the u-th layer.

[0070] Specifically, the optimal alignment vector sequence of the text sequence X is defined as The optimal uniform vector sequence is Text sequence X + The optimal alignment vector sequence is The optimal uniform vector sequence is The optimal alignment vector sequence of the text sequence Y is The optimal uniform vector sequence is

[0071] Step 5: Input to the retrieval fine-tuning module for the large language model

[0072] Establish a retrieval fine-tuning module, and then use the optimal alignment vector sequence of the text sequence X and the optimal uniform vector sequence Input into the retrieval fine-tuning module, the output vector sequence of the retrieval fine-tuning module for the text sequence X is Then, through the average pooling operation f(·), the final output vector O of the text sequence X is obtained X ;

[0073] Similarly, the sequence text X + The optimal alignment vector sequence and the optimal uniform vector sequence Input into the retrieval fine-tuning module to obtain the retrieval fine-tuning module for the text sequence X + The output vector sequence Then, through the average pooling operation f(·), we get the text sequence X + The final output vector is O X+ ;

[0074] The optimal alignment vector sequence of sequence text Y and the optimal uniform vector sequence Input into the retrieval fine-tuning module to obtain the output vector sequence of the retrieval fine-tuning module for the text sequence Y Then, through the average pooling operation f(·), the final output vector of the text sequence Y is obtained as O Y .

[0075] The connection between the large language model and the retrieval fine-tuner is as follows Figure 1 shown.

[0076] Step 6: Train and optimize the parameters of the retrieval fine-tuner

[0077] The final output vector O of the text sequence X X , sequence text X + The final output vector And the final output vector O of the text sequence Y Y Substitute into the following cross entropy training loss function,

[0078]

[0079] By minimizing the cross entropy training loss value, iteratively optimize the parameters θ of the retrieval fine-tuner and the large model parameters It remains fixed during training.

[0080] Figure 2 is a schematic diagram of a search fine-tuning module provided by an embodiment of the present invention. Figure 2 As shown in the figure, the retrieval fine-tuner is a multi-layer neural network structure, each layer includes 7 sub-layers, namely:

[0081] (1) Bidirectional Self-Attention Sublayer (Self-Attention)

[0082] Using the bidirectional attention mechanism, we can identify and capture the contextual features of text sequences from the perspective of alignment:

[0083]

[0084] Among them, the attention matrix Q a , K a and V a By the optimal alignment layer H a Mapping obtained.

[0085] (2) The first residual connection and normalization sublayer

[0086] (3) Cross Bi-Attention Sublayer

[0087] By leveraging the bidirectional attention mechanism to build intrinsic connections between the alignment and uniform layers of the large language model, the retrieval fine-tuner for the large language model can narrow the gap between them:

[0088]

[0089] Among them, K u and V u By the optimal uniform layer H u Projection obtains the query matrix Q s Obtained from the output mapping of the bidirectional self-attention sub-layer of the previous retrieval fine-tuner for large language models.

[0090] (4) Second residual connection and normalization sublayer

[0091] (5) Feedforward neural network sublayer

[0092] (6) The third residual connection and normalization sublayer

[0093] (7) Pooling and normalization sublayer.

[0094] This embodiment also provides a memory-enhanced large language model retrieval fine-tuning system, which includes: a first module, used to map a text sequence X in a preset text sequence library through a large language model to obtain a first layer-by-layer vector sequence, and to process the first layer-by-layer vector sequence through average pooling to obtain a first layer-by-layer dense vector; a second module, used to map a text sequence X in a preset text sequence library to a related text sequence X + A second layer-by-layer vector sequence is obtained by mapping the large language model, and the second layer-by-layer vector sequence is processed by average pooling to obtain a second layer-by-layer dense vector; a third module is used to map the text sequence Y in the preset text sequence library through the large language model to obtain a third layer-by-layer vector sequence, and the third layer-by-layer vector sequence is processed by average pooling to obtain a third layer-by-layer dense vector; a fourth module is used to obtain a per-layer alignment loss value according to the first layer-by-layer dense vector and the second layer-by-layer dense vector; a fifth module is used to obtain a per-layer uniformity loss value according to the first layer-by-layer dense vector and the third layer-by-layer dense vector; a sixth module is used to obtain the optimal alignment vector sequence of the text sequence X, the optimal uniform vector sequence of the text sequence X, and the text sequence X according to the per-layer alignment loss value and the per-layer uniformity loss value. + The optimal alignment vector sequence and text sequence X + The seventh module is used to input the optimal uniform vector sequence of the text sequence X and the optimal uniform vector sequence into the retrieval fine-tuning module, output the output vector sequence of the text sequence X, and obtain the final output vector of the text sequence X by average pooling the output vector sequence of the text sequence X; the eighth module is used to pool the sequence text X + The optimal alignment vector sequence and the optimal uniform vector sequence are input into the retrieval fine-tuning module, and the output text sequence X + The output vector sequence of the text sequence X + The output vector sequence of is processed by average pooling to obtain the text sequence X + The ninth module is used to input the optimal alignment vector sequence and the optimal uniform vector sequence of the sequence text Y into the retrieval fine-tuning module, output the output vector sequence of the text sequence Y, and obtain the final output vector of the text sequence Y by average pooling processing on the output vector sequence of the text sequence Y; the tenth module is used to convert the final output vector of the text sequence X and the sequence text X +The final output vector of and the final output vector of the text sequence Y are brought into the cross entropy training loss function, and the model parameters of the retrieval fine-tuning module are obtained by minimizing the value of the cross entropy training loss function.

[0095] This embodiment has efficient zero-shot retrieval performance: This embodiment selects the best alignment layer and uniform layer, uses bidirectional cross attention and bidirectional self-attention mechanisms, etc., and the retrieval fine-tuner for the large language model can effectively improve the performance of the large language model in the zero-shot retrieval task while completely fixing the internal parameters of the large language model and without destroying the text generation function of the large language model. The NDCG@10 scores on six evaluation datasets show that when the retrieval fine-tuner for the large language model is applied to the GPT series large language model, its average NDCG@10 score is significantly improved compared to the existing retrieval baseline model (for example, 3% and 9% higher than BM25 and DPR).

[0096] This embodiment has high parameter efficiency: the standard version of the retrieval fine-tuner for large language models (GPT-j-6B) has only 13% of the parameters of the large language model, while the reduced-dimensional version of the retrieval fine-tuner for large language models (a two-layer fully connected feedforward network is used to reduce the hidden dimension of the large language model to one-fourth of the original before the large language model is connected) has only 2% of the parameters of the large language model. Despite the significant reduction in the number of parameters, the average retrieval performance of the retrieval fine-tuner for large language models after the reduced-dimensionality test on six zero-shot retrieval evaluation datasets only decreased by 1%.

[0097] This embodiment has high training efficiency: since the retrieval fine-tuner for the large language model avoids propagating gradients back to the large language model during training, on a single A100 GPU (40G), the single-step training time of the standard retrieval fine-tuner for the large language model (GPT-j-6B) is only 14% of that of directly fine-tuning the large language model, and the retrieval fine-tuner for the large language model after dimensionality reduction compresses the training time to 4% of the time required for directly fine-tuning the large language model.

[0098] This embodiment has high reasoning efficiency: thanks to the mechanism that the retrieval fine-tuner for large language models directly uses the hidden state generated by the large language model to store and retrieve historical information during the reasoning process, its reasoning cost only needs to consider its own parameter calculation overhead. The standard version of the retrieval fine-tuner for large language models achieves retrieval performance similar to that of GTR-XXL, but its retrieval task-specific parameter amount is only 18% of that of GTR-XXL. The parameter amount of the retrieval fine-tuner for large language models after dimensionality reduction is similar to that of DPR, but its retrieval performance is 7% higher than that of DPR.

[0099] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A large language model retrieval fine-tuning method based on memory enhancement, characterized in that include: Mapping the text sequence X in the preset text sequence library through the large language model to obtain a first layer-by-layer vector sequence, and processing the first layer-by-layer vector sequence through average pooling to obtain a first layer-by-layer dense vector; The related text sequence X of the text sequence X in the preset text sequence library + A second layer-by-layer vector sequence is obtained through mapping of the large language model, and the second layer-by-layer vector sequence is processed by average pooling to obtain a second layer-by-layer dense vector; Mapping the text sequence Y in the preset text sequence library through the large language model to obtain a third layer-by-layer vector sequence, and processing the third layer-by-layer vector sequence through average pooling to obtain a third layer-by-layer dense vector; Obtaining the alignment loss value of each layer according to the first layer-by-layer dense vector and the second layer-by-layer dense vector; Obtaining a uniformity loss value of each layer according to the first layer-by-layer dense vector and the third layer-by-layer dense vector; According to the alignment loss value of each layer and the uniformity loss value of each layer, the optimal alignment vector sequence of the text sequence X, the optimal uniform vector sequence of the text sequence X, and the text sequence X are obtained. + The optimal alignment vector sequence and text sequence X + The optimal uniform vector sequence of , the optimal alignment vector sequence of text sequence Y, and the optimal uniform vector sequence of text sequence Y; The optimal alignment vector sequence and the optimal uniform vector sequence of the text sequence X are input into the retrieval fine-tuning module, and the output vector sequence of the text sequence X is output. The output vector sequence of the text sequence X is processed by average pooling to obtain the final output vector of the text sequence X. Sequence text X + The optimal alignment vector sequence and the optimal uniform vector sequence are input into the retrieval fine-tuning module, and the output text sequence X + The output vector sequence of the text sequence X + The output vector sequence of is processed by average pooling to obtain the text sequence X + The final output vector of The optimal alignment vector sequence and the optimal uniform vector sequence of the sequence text Y are input into the retrieval fine-tuning module, and the output vector sequence of the text sequence Y is output. The output vector sequence of the text sequence Y is processed by average pooling to obtain the final output vector of the text sequence Y. The final output vector of the text sequence X, the sequence text X + The final output vector of and the final output vector of the text sequence Y are brought into the cross entropy training loss function, and the model parameters of the retrieval fine-tuning module are obtained by minimizing the value of the cross entropy training loss function.

2. The memory-enhanced large language model retrieval fine-tuning method according to claim 1, characterized in that: For any text sequence X, there is a manually annotated related text sequence X + ; Define the preset text sequence library except text sequence X and text sequence X + All other text sequences except are sequence text Y.

3. The memory-enhanced large language model retrieval fine-tuning method according to claim 1, characterized in that: The alignment loss value of each layer is obtained by the following formula: in, is the alignment loss value of each layer, x l is the first layer-by-layer dense vector, x +，l is the second layer-by-layer dense vector, To calculate the expected function, An operation that computes the square of the Euclidean norm.

4. The memory-enhanced large language model retrieval fine-tuning method according to claim 1, characterized in that: The uniformity loss value of each layer is obtained by the following formula: in, is the uniformity loss value of each layer, x l is the first layer-by-layer dense vector, y l is the third layer-by-layer dense vector, To calculate the expected function, An operation that computes the square of the Euclidean norm.

5. The memory-enhanced large language model retrieval fine-tuning method according to claim 1, characterized in that: The cross entropy training loss function is: Among them, O X is the final output vector of the text sequence X, For sequence text X + The final output vector, O Y is the final output vector of the text sequence Y, θ is the model parameter of the retrieval fine-tuning module, is the large prediction model parameter.

6. A large language model retrieval fine-tuning system based on memory enhancement, characterized by include: The first module is used to map the text sequence X in the preset text sequence library through the large language model to obtain a first layer-by-layer vector sequence, and to obtain a first layer-by-layer dense vector by average pooling the first layer-by-layer vector sequence; The second module is used to convert the related text sequence X of the text sequence X in the preset text sequence library into + A second layer-by-layer vector sequence is obtained through mapping of the large language model, and the second layer-by-layer vector sequence is processed by average pooling to obtain a second layer-by-layer dense vector; The third module is used to map the text sequence Y in the preset text sequence library through the large language model to obtain a third layer-by-layer vector sequence, and process the third layer-by-layer vector sequence through average pooling to obtain a third layer-by-layer dense vector; A fourth module is used to obtain a per-layer alignment loss value according to the first layer-by-layer dense vector and the second layer-by-layer dense vector; A fifth module is used to obtain a uniformity loss value of each layer according to the first layer-by-layer dense vector and the third layer-by-layer dense vector; The sixth module is used to obtain the optimal alignment vector sequence of the text sequence X, the optimal uniform vector sequence of the text sequence X, and the optimal uniform vector sequence of the text sequence X according to the alignment loss value of each layer and the uniformity loss value of each layer. + The optimal alignment vector sequence and text sequence X + The optimal uniform vector sequence of , the optimal alignment vector sequence of text sequence Y, and the optimal uniform vector sequence of text sequence Y; The seventh module is used to input the optimal alignment vector sequence and the optimal uniform vector sequence of the text sequence X into the retrieval fine-tuning module, output the output vector sequence of the text sequence X, and obtain the final output vector of the text sequence X by average pooling the output vector sequence of the text sequence X; The eighth module is used to convert the sequence text X + The optimal alignment vector sequence and the optimal uniform vector sequence are input into the retrieval fine-tuning module, and the output text sequence X + The output vector sequence of the text sequence X + The output vector sequence of is processed by average pooling to obtain the text sequence X + The final output vector of The ninth module is used to input the optimal alignment vector sequence and the optimal uniform vector sequence of the sequence text Y into the retrieval fine-tuning module, output the output vector sequence of the text sequence Y, and obtain the final output vector of the text sequence Y by performing average pooling on the output vector sequence of the text sequence Y; The tenth module is used to convert the final output vector of the text sequence X, the sequence text X + The final output vector of and the final output vector of the text sequence Y are brought into the cross entropy training loss function, and the model parameters of the retrieval fine-tuning module are obtained by minimizing the value of the cross entropy training loss function.

7. The memory-enhanced large language model retrieval fine-tuning system according to claim 6, characterized in that: For any text sequence X, there is a manually annotated related text sequence X + ; Define the preset text sequence library except text sequence X and text sequence X + All other text sequences except are sequence text Y.

8. The memory-enhanced large language model retrieval fine-tuning system according to claim 6, characterized in that: The alignment loss value of each layer is obtained by the following formula: in, is the alignment loss value of each layer, x l is the first layer-by-layer dense vector, x +，l is the second layer-by-layer dense vector, To calculate the expected function, An operation that computes the square of the Euclidean norm.

9. The memory-enhanced large language model retrieval fine-tuning system according to claim 6, characterized in that: The uniformity loss value of each layer is obtained by the following formula: in, is the uniformity loss value of each layer, x l is the first layer-by-layer dense vector, y l is the third layer-by-layer dense vector, To calculate the expected function, An operation that computes the square of the Euclidean norm.

10. The memory-enhanced large language model retrieval fine-tuning system according to claim 6, characterized in that: The cross entropy training loss function is: Among them, O X is the final output vector of the text sequence X, is the final output vector of the sequence text X+, O Y is the final output vector of the text sequence Y, θ is the model parameter of the retrieval fine-tuning module, is the large prediction model parameter.

Images