Long document question and answer generation method and system based on large language model

Abstract

The invention relates to the technical field of artificial intelligence and natural languages, in particular to a long document question and answer generation method based on a large language model.The method comprises the steps that approximate nearest neighbor search is conducted through an HNSW index, N most relevant triads are retrieved, and for each position in the retrieved triads, a fixed-size window such as [-M, M] is obtained; all (k, v) pairs within a range of 1, 2, 3, 4, 5, 6, 7, 7, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, 9, a new K'matrix and a new V 'matrix are formed by splicing (k, v) pairs obtained by retrieval and expansion with K and V matrixes of a current input sequence, attention is calculated by using a standard zoom dot product attention mechanism, and only vectors corresponding to an input problem sequence are reserved, so that the global view is ensured, unnecessary calculation is avoided, and the accuracy of the attention calculation is improved. Therefore, the serious problems of calculation efficiency and memory use during long sequence processing are solved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and natural language processing, and in particular to a method and system for generating long document questions and answers based on a large language model. Background Art

[0002] As the complexity of natural language processing tasks continues to increase, the need to process long documents and long conversations becomes increasingly prominent. However, traditional Transformer models face serious computational efficiency and memory usage issues when processing long sequences. Summary of the invention

[0003] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a long document question and answer generation method based on a large language model, which helps to avoid the serious computational efficiency and memory usage problems faced by the traditional Transformer model when processing long sequences.

[0004] In order to implement the above technical solution, in a first aspect, the present invention provides a long document question-answering generation method based on a large language model, comprising the following steps: Step 1: Build a prompt project template; Step 2: Obtain the user question, insert the user question into the constructed prompt engineering template, and convert the prompt engineering template with the inserted user question into a question token sequence; Step 3: Record the length of the question token sequence QLEN; Step 4: Embed the question token sequence to obtain the embedding vector sequence E; Step 5: Based on the obtained embedding vector sequence E, forward propagation calculation is performed layer by layer in the large language model, and the calculation result of the last layer is standardized to generate a standardized layer output vector sequence; Step 6: Generate the probability distribution of tokens based on the obtained normalized layer output vector sequence, and select a token based on the token probability distribution to add to the question token sequence; Step 7: Determine whether at least one of the following stop conditions is met. If so, collect all tokens to form the answer text. If not, repeat steps 2 to 7. Step 8: Process the constructed answer text to generate the final answer.

[0005] Furthermore, before step 1, the method further includes: pre-building an HNSW index and a location index.

[0006] Furthermore, the step five comprises: C1: Input the obtained embedding vector sequence E into the first layer of the large language model, perform forward propagation calculation in the first layer to obtain the layer output vector sequence OUT1 and input it into the second layer of the large language model; C2: For the mth layer, where m ≥ 2 and m is a positive integer, forward propagation calculation is performed based on the vector sequence output by the m-1th layer of the large language model to obtain the layer output vector sequence OUT m ; C3: Determine whether the mth layer of the large language model is the last layer of the large language model. If so, output the vector sequence OUT for the layer m Normalize to obtain the standardized layer output vector sequence OUT. If not, the layer output vector sequence OUT m As the input of the m+1 layer, the forward propagation calculation is continued in the m+1 layer.

[0007] Further, the pre-building of the HNSW index and the location index includes: Receive long documents input by users; Construct a prompt project template, input the long document content received from the user into the created prompt project, convert the prompt project with the long document content inserted into a document token sequence, and record the length LEN of the document token sequence; Based on the converted document token sequence, attention calculation is performed layer by layer in the large language model to create a triple of each token in the document token sequence, where the triple of each token is (k i , v i , i), where i represents the i-th token in the document token sequence; k i represents the Key vector corresponding to the i-th token, v i Represents the Value vector corresponding to the i-th token; Create a list of triples for each layer of the large language model; Based on the triple storage list of each layer of the large language model, HNSW index and position index are constructed.

[0008] Furthermore, the step C2 comprises: Receive the output vector sequence OUT of the previous layer m-1 , use the layer normalization function LN to output the vector sequence OUT m-1 Standardize to get the standardized vector sequence E m , where E m =LN(OUT m-1 ); Normalize the vector sequence E mPerform self-attention calculation and linearly transform the calculated self-attention to obtain the linearly transformed layer output vector sequence OUT m .

[0009] Furthermore, the normalized vector sequence E m Perform self-attention calculation and linearly transform the calculated self-attention to obtain the linearly transformed layer output vector sequence OUT m include: Normalize the vector sequence E m Perform a linear transformation to generate the query vector matrix Q of the user's question m , key vector matrix K m Sum value vector matrix V m ; Use the query vector matrix Q m The query vector in the last column retrieves the most similar TOP -N Key vectors from the pre-built HNSW index structure at the mth level; Get the position of the token corresponding to the TOP -N key vectors above; Use the position index structure to obtain all Value vectors and Key vectors within the maximum number of neighbors M of the token position corresponding to each Key vector; According to the position order of the token corresponding to each Key vector in the token sequence, all the retrieved Key vectors are pieced together to obtain the matrix K m _hist, and concatenate all retrieved Value vectors together to get the matrix V m _hist; The resulting matrix K m _hist and key vector matrix K m Splice and get the matrix K' m , and the resulting matrix V m _hist and value vector matrix V m Splice and get V m '; Use the rotation position encoding to encode the query vector matrix Q m and matrix K' m Add position information to get the query vector matrix Q' m and matrix K' m ; Based on the matrix V' m And the query vector matrix Q' with added location information m and matrix K' m , perform attention calculation, and perform linear transformation on the attention calculation result to obtain the linear transformation vector sequence Om ; Based on the obtained linear transformation vector sequence O m and the normalized vector sequence E m , perform residual connection to obtain the residual vector sequence A m ; For the obtained residual vector sequence A m Standardize to get the standardized residual vector sequence X m ; The standardized residual vector sequence X is calculated by the following formula m Perform linear calculations on a two-layer feedforward neural network;

[0010] The output of the feedforward neural network is combined with the standardized residual vector sequence A m Add together to get the layer output vector sequence OUT m . In a second aspect, the present invention provides a long document question-answering generation system based on a large language model, comprising: The human-computer interaction module is used to receive long documents and user questions input by users, and display the final answers; A sequence conversion module is used to convert the prompt engineering module inserted with user questions and document content into a question token sequence and a document token sequence, and record the length of the question token sequence and the length of the document token sequence; The question embedding module is used to embed the question token sequence to obtain the embedding vector sequence E; A propagation calculation module is used to perform forward propagation calculation layer by layer in the large language model based on the obtained embedding vector sequence E, and to normalize the calculation result of the last layer to generate a normalized layer output vector sequence; A judgment module, used to judge whether the stop condition is met; The answer text generation module is used to collect all tokens to form a complete answer text; An answer text processing module is used to process the constructed answer text to generate a final answer; The prompt project template building module is used to build the prompt project template.

[0011] Furthermore, the system further comprises: HNSW index and location index building modules are used to build HNSW index and location index.

[0012] In a third aspect, the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the question and answer generation method based on the large language model described above.

[0013] The beneficial effects of the present invention are: (1) The present invention constructs an independent vector database for each layer of the large language model to store (k, v, i) triples. The HNSW (Hierarchical Navigable Small World) algorithm is used to construct an efficient approximate nearest neighbor search structure with the k vector as the index. At the same time, a position-based digital index is created to achieve fast position retrieval, which can speed up the search speed. It is also highly scalable and suitable for processing large-scale data. It can complete the approximate nearest neighbor search within the O(log n) time complexity, greatly improving the efficiency of subsequent retrieval.

[0014] (2) By performing an approximate nearest neighbor search in the HNSW index, the most relevant N triplets are retrieved, so that the historical information most relevant to the current token can be retrieved in constant time, rather than linearly scanning the entire sequence. For each position in the retrieved triplet, all (k, v) pairs in a fixed-size window around it (such as [-M, +M]) are obtained. By expanding the context window, local coherence is guaranteed and information fragmentation that may be caused by relying solely on single-point retrieval is avoided. By concatenating the (k, v) pairs obtained by retrieval and expansion with the K and V matrices of the current input sequence to form new K' and V' matrices, the standard scaled dot product attention mechanism is used to calculate the attention and only retain the vector corresponding to the input problem sequence. This not only ensures a global view but also avoids unnecessary calculations, thereby solving the serious computational efficiency and memory usage problems faced when processing long sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0016] Figure 1 The present invention is a flowchart of the preparation stage of a question and answer generation method based on a large language model.

[0017] Figure 2 The present invention is a flowchart of the processing stage of a question and answer generation method based on a large language model. DETAILED DESCRIPTION

[0018] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0019] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further explanation of the present invention. Unless otherwise specified, each technical and scientific term used in this embodiment has the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0020] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0021] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0022] In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. Relevant scientific research or technical personnel in this field can determine the specific meanings of the above terms in the present invention according to specific circumstances, and they should not be understood as limiting the present invention.

[0023] Embodiment 1: This embodiment provides a long document question-answering generation method based on a large language model, comprising the following steps: 1. Preparation stage: Figure 1 As shown, the following steps are included: S1: receiving a long document input by a user, wherein the long document may include multiple texts.

[0024] S2: Construct a prompt project template, input the long document content received from the user into the created prompt project, convert the prompt project with the long document content inserted into a document token sequence, and record the length LEN of the document token sequence.

[0025] Specifically, the created prompt project (prompt) template is as follows: The following is a long document, please read it carefully and be prepared to answer questions: {Document Content} Based on the above document, please answer the following questions.

[0026] S3: Based on the converted document token sequence, perform row attention calculation layer by layer in the large language model to obtain the Key vector and Value vector corresponding to each token in the document token sequence at each layer of the large language model; and create (k i , v i , i) triple, where i represents the i-th token in the document token sequence; k i represents the Key vector corresponding to the i-th token, v i Represents the Value vector corresponding to the i-th token.

[0027] Among them, the position of each token in the document token sequence is actually the order of the token in the document token sequence, that is, the token number.

[0028] S4: Create a triple storage list for each layer of the large language model to store the created triples.

[0029] Each triple storage list includes LEN triples.

[0030] It should be noted that if the large language model has L layers, L triple storage lists can be obtained in the end.

[0031] S5: Build the HNSW index based on the triple storage list of each layer of the large language model.

[0032] The specific steps include: S51: Based on the triple storage list of the large language model, construct an HNSW index structure, and set parameters of the HNSW index structure, the parameters including: a maximum number of neighbors M and a number of candidates to be searched during construction efConstruction.

[0033] S52: Obtain the triple storage list of each layer of the large language model, add all the Key vectors in the triple storage list, as well as the Value vector information corresponding to each Key vector and the position information in the token sequence to the HNSW index structure to generate an HNSW index.

[0034] S53: Save the generated HNSW index into the database.

[0035] It should be noted that HNSW is a graphical index structure that organizes high-dimensional vector space into a multi-layer navigable small-world graph. For each Key vector, its distance to other Key vectors is calculated and connections are established in the graph.

[0036] S6: Build a position index based on the triple storage list of each layer of the large language model.

[0037] The specific steps include: S61: Create a dictionary or data structure based on the triple storage list of the large language model.

[0038] S62: Map each token position in the large language model triple storage list to its corresponding Key vector and Value vector to generate a position index.

[0039] S63: Save the generated location index into the database.

[0040] (ii) Processing stage, such as Figure 2 As shown, the following steps are included: S7: Obtain the user question, and insert the user question into the constructed prompt engineering template to convert the prompt engineering template into a question token sequence.

[0041] Specifically, the created user question prompt project (prompt) template is as follows: {User Question} Specifically, for example, a tokenizer using a large language model (e.g., Llama 2 model) converts a prompt engineering template that is inserted into a user question into a sequence of question tokens.

[0042] S8: Record the length QLEN of the question token sequence.

[0043] It should be noted that the length of the token sequence is actually the number of tokens in the token sequence.

[0044] S9: Embed the question token sequence to obtain the embedding vector sequence E.

[0045] Among them, the shape of the embedding vector sequence E is [QLEN, d model ],d model is the dimension of the embedding vector sequence, which is a parameter set in the large language model configuration file.

[0046] S10: Based on the obtained embedding vector sequence E, forward propagation calculation is performed layer by layer in the large language model, and the calculation result of the last layer is normalized to generate a standardized layer output vector sequence.

[0047] It should be noted that different large language models have different numbers of layers. For example, the Llama2-7B large language model has 32 layers, the Llama2-13B large language model has 40 layers, and the Qwen2.5-3B large language model has 36 layers.

[0048] The large language model in this embodiment adopts the Llama2 large language model, assuming that the large language model has L layers as an example; L is an integer greater than or equal to 2. The forward propagation calculation is performed layer by layer in the large language model, specifically including the following steps: S10-1: First, the obtained embedding vector sequence E is input into the first layer of the large language model, and the following operations are performed in the first layer: A1: Use the layer normalization function LN to normalize the embedded vector sequence E to obtain the normalized vector sequence E1, where E1=LN(E).

[0049] A2: Perform self-attention calculation on the standardized vector sequence E1, and linearly transform the calculated self-attention to obtain linearly transformed self-attention.

[0050] The specific steps include: A2-1: Perform linear transformation on the standardized vector sequence E1 to generate the query vector matrix Q1, key vector matrix K1 and value vector matrix V1 of the user question.

[0051] Among them, Q1 = Linearq(E1); K1 = Lineark(E1); V1 = Linearv(E1); the shapes of Q1, K1, and V1 are all [QLEN, d k ], where d k The dimensions are preset.

[0052] A2-2: Use the query vector in the last column of the query vector matrix Q1 to retrieve the most similar TOP -N Key vectors in the pre-built first-layer HNSW index structure.

[0053] A2-3: Get the positions of the tokens corresponding to the TOP -N key vectors mentioned above.

[0054] A2-4: Use the position index structure to obtain all Value vectors and Key vectors within the maximum number of neighbors M of the token position corresponding to each Key vector.

[0055] Specifically, assuming that the token position corresponding to one of the Key vectors is i, all Value vectors and Key vectors in the position range [iM, i+M] are obtained.

[0056] The value of M is pre-set according to actual needs.

[0057] Wherein, N is a predefined parameter, which can be set according to actual needs.

[0058] A2-5: According to the position order of the token corresponding to each Key vector in the token sequence, all the retrieved Key vectors are pieced together to obtain the matrix K1_hist (whose shape is [L_hist, d k ]), and all retrieved Value vectors are concatenated together to obtain the matrix V1_hist (also of shape [L_hist, d k ]).

[0059] A2-6: Concatenate the obtained matrix K1_hist with the key vector matrix K1 to obtain the matrix K'1, and concatenate the obtained matrix V1_hist with the value vector matrix V1 to obtain V'1.

[0060] Specifically, the obtained matrix K1_hist is concatenated to the front of the key vector matrix K1 to obtain the matrix K'1, and the matrix V1_hist is concatenated to the front of the value vector matrix V1 to obtain the matrix V'1.

[0061] Among them, the shapes of matrices K'1 and V'1 are [QLEN+L_hist, d k ].

[0062] A2-7: Use rotational position encoding (e.g., PoPE) to add position information to the query vector matrix Q1 and the matrix K'1 to obtain the query vector matrix Q'1 and the matrix K'1.

[0063] Specifically, adding rotational position encoding to the query vector matrix Q1 and matrix K'1 can ensure that the relative position information between tokens is included when calculating self-attention, thereby achieving a more comprehensive and complete semantic understanding.

[0064] A2-8: Perform attention calculation based on matrix V'1 and query vector matrix Q'1 and matrix K'1 with added position information, and perform linear transformation on the attention calculation result to obtain linear transformation vector sequence O1.

[0065] The attention calculation formula is as follows:

[0066] Perform a linear transformation on the attention calculation result to obtain the formula of the linear transformation vector sequence O1 as follows:

[0067] A2-9: Based on the obtained linear transformation vector sequence O1 and the standardized vector sequence E1, residual connection is performed to obtain a residual vector sequence A1; Among them, the residual connection formula is as follows: A1=E1+O1.

[0068] A2-10: Normalize the obtained residual vector sequence A1 to obtain a standardized residual vector sequence X1.

[0069] Among them, the formula for standardizing the obtained residual vector sequence A1 is as follows: X1=LN(A1).

[0070] A2-11: Perform a two-layer feedforward neural network linear calculation on the standardized residual vector sequence X1 using the following formula;

[0071] A2-12: Add the output of the feedforward neural network to the standardized residual vector sequence A1 to obtain the layer output vector sequence OUT1, and use the layer output vector sequence OUT1 as the input of the second layer.

[0072] Specifically, OUT1=FFN(X1)+A1.

[0073] S10-2: For the mth layer, where L≥m≥2 and m is a positive integer, perform the following operations: B1: Receive the output vector sequence OUT of the previous layer m-1 , use the layer normalization function LN to output the vector sequence OUT m-1 Standardize to get the standardized vector sequence E m , where E m =LN(OUT m-1 ).

[0074] B2: Normalize the vector sequence E m Perform self-attention calculation and linearly transform the calculated self-attention to obtain the linearly transformed layer output vector sequence OUT m .

[0075] Specifically, the steps include: B2-1: Normalize the vector sequence E m Perform a linear transformation to generate the query vector matrix Q of the user's question m , key vector matrix K m Sum value vector matrix V m .

[0076] Among them, Q m = Linearq (E m );K m = Linear k (E m );V m = Linear v (E m );Q m , K m 、V m The shape is [QLEN, d k ], where d k The dimensions are preset.

[0077] B2-2: Using the query vector matrix Q m The query vector in the last column retrieves the most similar TOP -N Key vectors from the pre-built HNSW index structure at the mth level.

[0078] B2-3: Get the position of the token corresponding to the TOP -N key vectors mentioned above.

[0079] B2-4: Use the position index structure to obtain all Value vectors and Key vectors within the maximum number of neighbors M of the token position corresponding to each Key vector.

[0080] Specifically, assuming that the token position corresponding to one of the Key vectors is i, all Value vectors and Key vectors in the position range [iM,i+M] are obtained.

[0081] The value of M is pre-set according to actual needs.

[0082] Wherein, N is a predefined parameter, which can be set according to actual needs.

[0083] B2-5: According to the position order of the token corresponding to each Key vector in the token sequence, all the retrieved Key vectors are pieced together to obtain the matrix K m _hist (whose shape is [L_hist, d k ]), and concatenate all retrieved Value vectors together to get the matrix V m _hist (also of shape [L_hist, d k ]).

[0084] B2-6: The obtained matrix K m _hist and key vector matrix K m Splice and get the matrix K' m , and the resulting matrix V m_hist and value vector matrix V m Splice and get V m '.

[0085] Specifically, the obtained matrix K m _hist is concatenated to the key vector matrix K m The matrix K' is obtained above m , the matrix V m _hist is concatenated to the value vector matrix V m The matrix V' is obtained above m .

[0086] Among them, the matrix K' m and V' m The shape is [QLEN+L_hist, d k ].

[0087] B2-7: Use rotational position encoding (e.g., PoPE) to encode the query vector matrix Q m and matrix K' m Add position information to get the query vector matrix Q' m and matrix K' m .

[0088] Specifically, for the query vector matrix Q m and matrix K' m Adding rotational position encoding can ensure that the relative position information between tokens is included when calculating self-attention, thereby achieving a more comprehensive and complete semantic understanding.

[0089] B2-8: Based on matrix V' m And the query vector matrix Q' with added location information m and matrix K' m , perform attention calculation, and perform linear transformation on the attention calculation result to obtain the linear transformation vector sequence O m .

[0090] The attention calculation formula is as follows:

[0091] Perform a linear transformation on the attention calculation result to obtain the linear transformation vector sequence O m The formula is as follows:

[0092] B2-9: Based on the obtained linear transformation vector sequence O m and the normalized vector sequence E m , perform residual connection to obtain the residual vector sequence A m ; Among them, the residual connection formula is as follows: A m =E m +O m .

[0093] B2-10: The obtained residual vector sequence A m Standardize to get the standardized residual vector sequence X m .

[0094] Among them, the formula for standardizing the obtained residual vector sequence A is as follows: X m =LN(A m ).

[0095] B2-11: Standardize the residual vector sequence X by the following formula m Perform linear calculations on a two-layer feedforward neural network;

[0096] B2-12: Combine the output of the feedforward neural network with the standardized residual vector sequence A m Add together to get the layer output vector sequence OUT m . Specifically, OUT m =FFN(X m )+A m .

[0097] B3: Determine whether layer m is the last layer of the large language model. If so, output the vector sequence OUT for the layer m Normalize to obtain the standardized layer output vector sequence OUT. If not, the layer output vector sequence OUT m As the input of the m+1 layer, steps B1 to B2 are performed in the m+1 layer.

[0098] S11: Based on the obtained normalized layer output vector sequence OUT, generate a probability distribution of tokens and select a token according to the token probability distribution, and add the selected token to the question token sequence.

[0099] Specifically, the normalization layer output vector sequence OUT is passed to the large language model head to generate the probability distribution result of the next token. Based on the generated probability distribution result of the next token, a sampling strategy (such as greedy decoding or beam search) selects the next token and adds the selected token to the question token sequence.

[0100] S12: Determine whether at least one of the following stop conditions is met. If so, collect all tokens to form a complete answer text; if not, repeat steps S8 to S12.

[0101] The stop conditions include: whether an end symbol is generated and whether the problem token sequence reaches a predefined token sequence length. S13: Process the constructed answer text to generate a final answer.

[0102] The processing of the constructed answer text includes: removing special tokens and formatting.

[0103] Embodiment 2: This embodiment provides a long document question-answering generation system based on a large language model, including: (I) Human-computer interaction module, used to receive long documents and user questions input by users, and display the final answers.

[0104] (ii) HNSW index and position index construction module, used to construct the HNSW index and position index of the large language model.

[0105] The HNSW index and location index building blocks include: The prompt project building module is used to build the prompt project template.

[0106] The project conversion unit is used to convert the prompt project template into a document token sequence and record the length of the document token sequence.

[0107] The triple creation unit is used to perform attention calculation layer by layer in the large language model based on the converted document, so as to obtain the Key vector and Value vector corresponding to each token in the document token sequence in each layer of the large language model; and based on the position of each token in the document token sequence, create (k i , v i ,i) Triplet.

[0108] A storage list creation module is used to create a triple storage list for each layer of the large language model to store the created (k i , v i ,i) Triplet.

[0109] HNSW index structure building unit, used to build HNSW index structure; The HNSW index structure parameter setting unit is used to set the parameters of the HNSW index structure.

[0110] The HNSW index construction unit obtains the triple storage list of each layer of the large language model, and adds all the Key vectors in the triple storage list, as well as the Value vector information corresponding to each Key vector and the position information in the token sequence to the HNSW index structure to generate an HNSW index.

[0111] Positional index structure building block used to create a dictionary or data structure. The position index creation unit is used to map each token position in the triple storage list of each layer of the large language model to its corresponding Key vector and Value vector to generate a position index.

[0112] Database, used to store location index and HNSW index.

[0113] (iii) a sequence conversion module, which is used to convert the prompt engineering module inserted with user questions or document content into a question token sequence and record the length of the question token sequence; (iv) Question embedding module, used to embed the question token sequence to obtain the embedding vector sequence E; (V) A propagation calculation module, which is used to perform forward propagation calculation layer by layer in the large language model based on the obtained embedding vector sequence E, and to normalize the calculation result of the last layer to generate a standardized layer output vector sequence.

[0114] (vi) A sequence selection module, which is used to generate a token probability distribution based on the obtained normalized layer output vector sequence, select a token according to the token probability distribution, and add the selected token to the question token sequence.

[0115] (VII) a judgment module, used to judge whether the stop condition is met; (8) Answer text generation module, used to collect all tokens to form a complete answer text; (ix) An answer text processing module is used to process the constructed answer text to generate a final answer.

[0116] (10) A prompt engineering template construction module, used to construct a prompt engineering template.

[0117] Embodiment 3: This embodiment provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute a long document question and answer generation method based on a large language model described in Example 2.

[0118] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the terminal embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.

[0119] In the several embodiments provided by the present invention, it should be understood that the disclosed system and method can It can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, it should be noted that the flowchart in the accompanying drawings shows the method of the embodiment of the present disclosure. In the corresponding description in the flowchart or block diagram in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in an order different from that disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be performed substantially in parallel, or sometimes in the opposite order, which may depend on the functions involved. Each block in the block diagram and / or flow chart, and the combination of blocks in the block diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A long document question answering generation method based on a large language model, characterized in that: The following steps are involved: Step 1: Build a prompt project template; Step 2: Obtain the user question, insert the user question into the constructed prompt engineering template, and convert the prompt engineering template with the inserted user question into a question token sequence; Step 3: Record the length of the question token sequence QLEN; Step 4: Embed the question token sequence to obtain the embedding vector sequence E; Step 5: Based on the obtained embedding vector sequence E, forward propagation calculation is performed layer by layer in the large language model, and the calculation result of the last layer is standardized to generate a standardized layer output vector sequence; Step 6: Generate the probability distribution of tokens based on the obtained normalized layer output vector sequence, and select a token based on the token probability distribution to add to the question token sequence; Step 7: Determine whether at least one of the following stop conditions is met. If so, collect all tokens to form the answer text. If not, repeat steps 2 to 7. Step 8: Process the constructed answer text to generate the final answer.

2. The long document question-answering generation method based on a large language model according to claim 1, characterized in that: Before step 1, the method further includes: pre-building an HNSW index and a location index.

3. The long document question-answering generation method based on a large language model according to claim 2, characterized in that: The step five comprises: C1: Input the obtained embedding vector sequence E into the first layer of the large language model, perform forward propagation calculation in the first layer to obtain the layer output vector sequence OUT1 and input it into the second layer of the large language model; C2: For the mth layer, where m ≥ 2 and m is a positive integer, forward propagation calculation is performed based on the vector sequence output by the m-1th layer of the large language model to obtain the layer output vector sequence OUT m ; C3: Determine whether the mth layer of the large language model is the last layer of the large language model. If so, output the vector sequence OUT for the layer m Normalize to get the standardized layer output vector sequence OUT. If not, then the layer output vector sequence OUT m As the input of the m+1 layer, the forward propagation calculation is continued in the m+1 layer.

4. The long document question-answering generation method based on a large language model according to claim 3 is characterized in that: The pre-built HNSW index and location index include: Receive long documents input by users; Construct a prompt project template, input the long document content received from the user into the created prompt project, convert the prompt project with the long document content inserted into a document token sequence, and record the length LEN of the document token sequence; Based on the converted document token sequence, attention calculation is performed layer by layer in the large language model to create a triple of each token in the document token sequence, where the triple of each token is (k i , v i , i), where i represents the i-th token in the document token sequence; k i represents the Key vector corresponding to the i-th token, v i Represents the Value vector corresponding to the i-th token; Create a list of triples for each layer of the large language model; Based on the triple storage list of each layer of the large language model, HNSW index and position index are constructed.

5. The long document question-answering generation method based on a large language model according to claim 4, characterized in that: The step C2 comprises: Receive the output vector sequence OUT of the previous layer m-1 , use the layer normalization function LN to output the vector sequence OUT m-1 Standardize to get the standardized vector sequence E m , where E m =LN(OUT m-1 ); Normalize the vector sequence E m Perform self-attention calculation and linearly transform the calculated self-attention to obtain the linearly transformed layer output vector sequence OUT m .

6. The long document question-answering generation method based on a large language model according to claim 5, characterized in that: The normalized vector sequence E m Perform self-attention calculation and linearly transform the calculated self-attention to obtain the linearly transformed layer output vector sequence OUT m include: Normalize the vector sequence E m Perform a linear transformation to generate the query vector matrix Q of the user's question m , key vector matrix K m Sum value vector matrix V m ; Use the query vector matrix Q m The query vector in the last column retrieves the most similar TOP -N Key vectors from the pre-built HNSW index structure at the mth level; Get the position of the token corresponding to the TOP -N key vectors above; Use the position index structure to obtain all Value vectors and Key vectors within the maximum number of neighbors M of the token position corresponding to each Key vector; According to the position order of the token corresponding to each Key vector in the token sequence, all the retrieved Key vectors are pieced together to obtain the matrix K m _hist, and concatenate all retrieved Value vectors together to get the matrix V m _hist; The resulting matrix K m _hist and key vector matrix K m Splice and get the matrix K' m , and the resulting matrix V m _hist and value vector matrix V m Splice and get V m '; Use the rotation position encoding to encode the query vector matrix Q m and matrix K' m Add position information to get the query vector matrix Q' m and matrix K' m ; Based on the matrix V' m And the query vector matrix Q' with added location information m and matrix K' m , perform attention calculation, and perform linear transformation on the attention calculation result to obtain the linear transformation vector sequence O m ; Based on the obtained linear transformation vector sequence O m and the normalized vector sequence E m , perform residual connection to obtain the residual vector sequence A m ; For the obtained residual vector sequence A m Standardize to get the standardized residual vector sequence X m ; The standardized residual vector sequence X is calculated by the following formula m Perform linear calculations on a two-layer feedforward neural network; The output of the feedforward neural network is combined with the standardized residual vector sequence A m Add together to get the layer output vector sequence OUT m .

7. A long document question-answering generation system based on a large language model, characterized in that: include: The human-computer interaction module is used to receive long documents and user questions input by users, and display the final answers; A sequence conversion module is used to convert the prompt engineering module inserted with user questions and document content into a question token sequence and a document token sequence, and record the length of the question token sequence and the length of the document token sequence; The question embedding module is used to embed the question token sequence to obtain the embedding vector sequence E; A propagation calculation module is used to perform forward propagation calculation layer by layer in the large language model based on the obtained embedding vector sequence E, and to normalize the calculation result of the last layer to generate a normalized layer output vector sequence; A judgment module, used to judge whether the stop condition is met; The answer text generation module is used to collect all tokens to form a complete answer text; An answer text processing module is used to process the constructed answer text to generate a final answer; The prompt project template building module is used to build the prompt project template.

8. The long document question-answering generation system based on a large language model according to claim 7, characterized in that: Also includes: HNSW index and location index building modules are used to build HNSW index and location index.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the question and answer generation method based on a large language model as described in any one of claims 1 to 6.

Images