Document question and answer processing method and system, electronic equipment, storage medium and computer program product

Abstract

The invention discloses a document question and answer processing method and system, electronic equipment, a storage medium and a computer program product, and relates to the technical field of large model technologies and document question and answer technologies. The method comprises the following steps: according to an input question, obtaining a recalled document and a key value cache corresponding to the recalled document; generating an initial text sequence by using the input question and the recalled document; based on a mixed key value strategy, key value filling calculation is carried out on the initial text sequence, a target text sequence is obtained, the mixed key value strategy is used for representing key value confirmation mechanisms corresponding to a plurality of lexical elements in different model layers, and the key value confirmation mechanisms comprise one of a key value multiplexing mechanism based on key value cache and a key value reusing mechanism based on key value cache; a hybrid mechanism of a key value multiplexing mechanism and a key value calculation mechanism; and performing reasoning calculation by using the target text sequence to obtain a target answer. The technical problem that a model is difficult to consider the reasoning speed and the reasoning precision due to a calculation mode completely depending on key value multiplexing in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the fields of large model technology and document question and answer processing technology, and specifically, to a document question and answer processing method, system, electronic device, storage medium and computer program product. Background Art

[0002] In the application scenarios of Large Language Model (LLM) (such as retrieval enhancement, document question answering, etc.), some text documents are usually recalled for input questions, and then the long text input of LLM is generated based on the input questions and text documents, so that LLM generates the output result corresponding to the input question. However, according to related technologies, in the reasoning process of LLM, in order to speed up the reasoning speed, the key-value cache generated for the long text input in the pre-filling stage is simply and mechanically reused, but this will greatly reduce the reasoning accuracy of LLM.

[0003] Therefore, how to improve the pre-filling scheme to balance the enhancement of the reasoning speed and reasoning accuracy of LLM has become one of the important technical issues in the relevant technical field. Currently, no effective solution has been proposed for the above problems. Summary of the invention

[0004] The embodiments of the present application provide a document question and answer processing method, system, electronic device, storage medium and computer program product to at least solve the technical problem in the related technology that the calculation method that completely relies on key-value reuse makes it difficult for the model to take into account both reasoning speed and reasoning accuracy.

[0005] According to one aspect of an embodiment of the present application, a document question and answer processing method is provided, including: obtaining a recalled document and a key-value cache corresponding to the recalled document according to an input question; generating an initial text sequence using the input question and the recalled document; performing key-value filling calculations on the initial text sequence based on a hybrid key-value strategy to obtain a target text sequence, wherein the hybrid key-value strategy is used to characterize a key-value confirmation mechanism corresponding to multiple word elements in different model layers, and the key-value confirmation mechanism includes one of the following: a key-value reuse mechanism based on a key-value cache, and a hybrid mechanism of a key-value reuse mechanism and a key-value calculation mechanism; performing reasoning calculations using the target text sequence to obtain a target answer.

[0006] According to another aspect of an embodiment of the present application, a document question and answer processing method is also provided, including: obtaining a document question and answer processing request through a first application programming interface, wherein the request data carried in the document question and answer processing request includes: an input question; returning a document question and answer processing response through a second application programming interface, wherein the response data carried in the document question and answer processing response includes: a target answer, and the target answer is obtained according to any one of the above-mentioned document question and answer processing methods.

[0007] According to another aspect of an embodiment of the present application, a document question and answer processing method is also provided, including: responding to a document question and answer processing instruction on an operation interface, uploading an input question; responding to an image processing instruction on an operation interface, displaying a target answer on the operation interface, wherein the target answer is obtained according to any one of the above-mentioned document question and answer processing methods.

[0008] According to another aspect of an embodiment of the present application, a document question and answer processing system is also provided, including: a client, used to send input questions to be processed, wherein the input questions are used to obtain recalled documents and a key value cache corresponding to the recalled documents; a server, connected to the client, used to generate an initial text sequence using the input questions and recalled documents, based on a hybrid key value strategy, perform key value filling calculations on the initial text sequence to obtain a target text sequence, and perform inference calculations using the target text sequence to obtain a target answer, wherein the hybrid key value strategy is used to characterize a key value confirmation mechanism corresponding to multiple word elements in different model layers, and the key value confirmation mechanism includes one of the following: a key value reuse mechanism based on a key value cache, a hybrid mechanism of a key value reuse mechanism and a key value calculation mechanism; the client is also used to output a target answer.

[0009] According to another aspect of an embodiment of the present application, there is also provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein when the program is running, any one of the document question and answer processing methods described above is executed.

[0010] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned document question and answer processing methods.

[0011] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements any of the above-mentioned document question and answer processing methods when executed by a processor.

[0012] In an embodiment of the present application, according to an input question, a recalled document and a key-value cache corresponding to the recalled document are obtained; an initial text sequence is generated using the input question and the recalled document; based on a hybrid key-value strategy, a key-value filling calculation is performed on the initial text sequence to obtain a target text sequence, wherein the hybrid key-value strategy is used to characterize the key-value confirmation mechanism corresponding to multiple word units in different model layers, and the key-value confirmation mechanism includes one of the following: a key-value reuse mechanism based on a key-value cache, a hybrid mechanism of a key-value reuse mechanism and a key-value calculation mechanism; reasoning calculation is performed using the target text sequence to obtain a target answer. It is easy to notice that before the reasoning calculation, the present application determines the key-value information of the word unit through a hybrid key-value strategy, rather than simply and mechanically reusing the key-value cache, thereby improving the accuracy of the reasoning calculation. In addition, compared with the solution of simply performing key-value calculation on all word units, determining the key-value information of the word unit through a hybrid key-value strategy can also improve the key-value confirmation efficiency, thereby improving the reasoning speed. Therefore, the present application achieves the purpose of confirming word key information by introducing a hybrid key value strategy, improving the reasoning speed while ensuring the reasoning accuracy, thereby achieving the technical effect of balancing the improvement of reasoning speed and reasoning accuracy in the document question and answer processing process, and further solves the technical problem in the related technology that the calculation method that completely relies on key value reuse makes it difficult for the model to balance reasoning speed and reasoning accuracy.

[0013] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1 It is a schematic diagram of an application scenario of a document question-and-answer processing method according to an embodiment of the present application;

[0016] Figure 2 is a flowchart of a document question and answer processing method according to an embodiment of the present application;

[0017] Figure 3 is a schematic diagram of an optional pre-filling stage according to an embodiment of the present application;

[0018] Figure 4 is a schematic diagram of an optional word unit screening process according to an embodiment of the present application;

[0019] Figure 5 is a schematic diagram of an optional attention mechanism calculation according to an embodiment of the present application;

[0020] Figure 6 is a flowchart of another document question and answer processing method according to an embodiment of the present application;

[0021] Figure 7 is a flowchart of another document question and answer processing method according to an embodiment of the present application;

[0022] Figure 8 is a structural block diagram of a document question and answer processing device according to an embodiment of the present application;

[0023] Fig. 9 is a structural block diagram of another document question and answer processing device according to an embodiment of the present application;

[0024] Fig.10 is a structural block diagram of another document question and answer processing device according to an embodiment of the present application;

[0025] Fig.11 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] The technical solution provided in this application is mainly implemented using large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than 10 trillion model parameters. The large model can also be called a cornerstone model / foundation model (Foundation Model). The large model is pre-trained through large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large language model (LLM, also known as a large model), a multi-modal pre-training model, etc.

[0029] It should be noted that when the large model is actually used, the pre-trained model can be fine-tuned through a small number of samples so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields, and can be specifically applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, etc. It can also be widely used in text-based sentiment classification, text summary generation, machine translation and other natural language processing tasks. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present application, the document question and answer processing using a large language model in the document question and answer scenario is used as an example for explanation.

[0030] First, some nouns or terms that appear in the process of describing the embodiments of the present application are subject to the following explanations.

[0031] Prefill: refers to the starting stage of the large language model reasoning process. In the prefill stage, the model will create a key-value cache for each input token and generate the first answer token, which serves as the beginning of the output answer. The prefill stage can improve the efficiency of the model's deep reasoning and ensure that the model can smoothly and quickly perform context understanding and generation tasks in the steps after the prefill stage.

[0032] Key-value cache (KV Cache): refers to the key-value data pre-calculated and stored for documents. Specifically, the key-value cache includes key (Key, K for short) vectors and value (Value, V for short) vectors corresponding to multiple words in the document, that is, each word corresponds to a cache key-value pair. Through the key-value cache, the computational burden of the model inference stage can be reduced. In the attention mechanism calculation, for a certain word, the cache key-value pair corresponding to the word can be reused to avoid recalculating the key-value information of the word.

[0033] According to an embodiment of the present application, a document question and answer processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] Considering the huge number of model parameters of the large model and the limited computing resources of the mobile terminal, the above method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 In the application scenario shown, the large model is deployed in the server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 here may include but is not limited to: a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to implement the call of the large model, thereby implementing the method provided in the embodiment of the present application.

[0035] In an embodiment of the present application, a system consisting of a client device and a server can execute the following steps: the client device uploads an input question to the server, the server obtains a recalled document and a key value cache corresponding to the recalled document based on the input question, generates an initial text sequence using the input question and the recalled document, performs key value filling calculation on the initial text sequence based on a hybrid key value strategy to obtain a target text sequence, performs inference calculation using the target text sequence to obtain a target answer, and further returns the target answer to the client device.

[0036] It should be noted that, with the rapid development of high-performance computing units, in other application scenarios, the above method provided in the embodiment of the present application can also be applied to the model all-in-one machine. In an optional embodiment, a plurality of models are built into the model all-in-one machine, and the user can choose to adjust a model as needed to obtain the user's own model, so that the high-performance computing unit built into the model all-in-one machine can directly call the adjusted model to execute the above method provided in the embodiment of the present application. In another optional embodiment, a trained model is built into the large model all-in-one machine, so that the high-performance computing unit built into the model all-in-one machine can directly call the model to execute the above method provided in the embodiment of the present application.

[0037] Furthermore, when users need to train their own models, they can also upload their own data sets through the client, which are sent by the client to the server, so that the server can adjust the pre-trained model with the data set to obtain the user's own model and then deploy it to the production environment. In order to facilitate users' needs for model adjustment, the server can provide complete adjustment tools, development frameworks and processes, and can support multiple adjustment strategies, so that the adjusted model can better adapt to applications in different fields and achieve a high degree of customization.

[0038] Under the above operating environment, this application provides Figure 2 The document question and answer processing method shown. Figure 2 1 is a flowchart of a document question-answering processing method according to an embodiment of the present application. Figure 2 As shown, the method may include the following steps S202 to S208.

[0039] Step S202: Obtain the recalled document and the key value cache corresponding to the recalled document according to the input question.

[0040] The document question answering method can be implemented by a document question answering model or a document question answering algorithm. The document question answering model can be implemented by a large language model (e.g., a language model using a Transformer architecture). In addition, the document question answering model can also be other types of models, such as a recurrent neural network model, a long short-term memory network model, a bidirectional Transformer model, a graph neural network model, etc.

[0041] The above input question can be a specific question asked by the user or the system to the document question answering model. The input question serves as the starting point of the document question answering process and can also serve as the basis for the document question answering model to reason and generate answers. The input question can be expressed in natural language in various forms (such as text, pictures, voice, video, etc.).

[0042] The above input questions can be basic questions (e.g., what is the revenue of Group A in 2024?), instructions and questions (e.g., find out the revenue of Group A in 2024 from the most recent financial report), background and questions (e.g., how will Group A perform in 2024 under the current market environment?) and open questions (e.g., what is the future development trend of Group A?).

[0043] The above input questions can be used to guide the acquisition of recalled documents and the construction of initial text sequences, and can also be used to assist in pre-filling text sequences and assist in reasoning calculations to generate target answers.

[0044] The recall documents may include a series of text documents related to the input question retrieved from the document database. These recall documents may contain answers to the input question or context information related to the input question. The recall documents may serve as a basis for constructing the initial text sequence.

[0045] The key-value cache may be key-value data pre-calculated and cached for the recalled document. Specifically, the key-value cache may include the K vector and V vector of the recalled document at each model layer and each attention head. The cache key-value pair of a token in the key-value cache may be reused as the key-value pair of the token in the attention mechanism calculation.

[0046] In the application scenario, after receiving the input question, the document question answering model can use the recall algorithm to retrieve text documents related to the input question and obtain recalled documents. At the same time, the key-value caches corresponding to these recalled documents are also recalled for reuse in the attention mechanism calculations of subsequent model layers.

[0047] By obtaining recalled documents and key-value caches, rich document information and pre-calculated cache key-value pairs are prepared for subsequent calculation processes. This not only speeds up the response speed of the model, but also ensures that when processing long text inputs, the model can quickly access the key representation of the document, thereby improving reasoning efficiency.

[0048] Step S204: Generate an initial text sequence using the input question and the recalled document.

[0049] The above initial text sequence can be a text sequence generated based on the input question, the recalled document, and other instructions (such as question mark, document segmentation mark, etc.). The initial text sequence can be used as the input prompt text of the document question answering model. Exemplarily, the content of the input question, the content of the recalled document, and the content of the instruction are spliced ​​to generate an input format that can be processed by the model to obtain the initial text sequence.

[0050] By constructing the initial text sequence, the document question answering model is provided with complete contextual input, ensuring that the model can consider all the information of user questions and related documents at the same time during the reasoning process, thereby enhancing the model's understanding ability and the accuracy of the answers.

[0051] Step S206, based on the hybrid key value strategy, perform key value filling calculation on the initial text sequence to obtain the target text sequence, wherein the hybrid key value strategy is used to characterize the key value confirmation mechanism corresponding to multiple words in different model layers, and the key value confirmation mechanism includes one of the following: a key value reuse mechanism based on key value cache, a hybrid mechanism of the key value reuse mechanism and the key value calculation mechanism.

[0052] The document question-answering model may include multiple model layers. Based on the above hybrid key-value strategy, each model layer of the multiple model layers may use a key-value reuse mechanism to confirm the key values ​​of multiple words in the current model layer; each model layer of the multiple model layers may also use a hybrid mechanism to confirm the key values ​​of multiple words in the current model layer, that is, for multiple words in each model layer, a part of the words use a key-value reuse mechanism, and another part of the words use a key-value calculation mechanism. In addition, based on the above hybrid key-value strategy, for a part of the model layers in the multiple model layers (such as the first 5 model layers), each word in each model layer of these model layers uses a key-value reuse mechanism to determine the word-word key value; and for another part of the model layers (such as other model layers except the first 5 model layers), multiple words in each model layer of these model layers use a hybrid mechanism to determine the word-word key value (that is, a part of the words use key-value reuse, and another part of the words use key-value calculation). The key-value filling calculation process of the document question-answering model may involve iterative calculations of multiple model layers, that is, for the first model layer, the input sequence is the initial text sequence, and for the subsequent model layers, the input sequence is the output sequence of the previous model layer. In some model layers, the key-value reuse mechanism may be used for all multiple tokens in the input sequence; while in other model layers, the key-value reuse mechanism may be used for some of the multiple tokens in the input sequence, and the key-value calculation mechanism may be used for another part of the multiple tokens. In the application scenario, the document question-answering model can determine in which model layers the key-value reuse mechanism is used, and in which model layers the key-value calculation mechanism is used for which tokens according to specific logic (other tokens in these model layers will also reuse the key-value cache).

[0053] The above key-value filling calculation can be a calculation or processing action performed in the pre-filling stage. In a certain model layer, after determining to perform key-value reuse or key-value calculation on a certain token, the key-value pair corresponding to the token can be further determined, and the key-value pair can be filled into the key-value data of the current layer corresponding to the model layer. Furthermore, the output sequence will be calculated based on the key-value data of the current layer in the model layer.

[0054] That is to say, after the key-value filling calculation, the output sequence corresponding to each model layer and the key-value data of this layer can be obtained, and the output sequence of the last model layer of the document question-answering model is determined as the target text sequence.

[0055] It should be noted that the output sequence of a model layer and the key-value data of this layer can be transmitted to the next model layer at the same time. The output sequence serves as the input sequence of the next model layer, and the key-value data of this layer can be used to assist the next model layer in word filtering (that is, in the next model layer, the tokens to be calculated for key values ​​and the tokens to be reused for key values ​​are distinguished from the multiple tokens contained in the input sequence).

[0056] In addition, in some application scenarios, the text sequence (including the initial text sequence and the target text sequence) transmitted between model layers can be understood as a data set, that is, the text sequence can contain both a sequence of multiple tokens and the key-value data of this layer.

[0057] It is easy to notice that, although the scheme of calculating the key value of each token in each model layer can achieve higher reasoning accuracy, the calculation cost is high and the reasoning speed is slow; the scheme of reusing the key value of each token in each model layer can save the calculation cost and improve the reasoning speed, but it will bring about the loss of reasoning accuracy, resulting in reduced answer quality. In the embodiment of the present application, by introducing the above-mentioned key-value hybrid strategy, it is possible to balance the computational efficiency and output accuracy of the document question and answer model. By selectively reusing key-value caches and recalculating key-value pairs for different word units, the document question and answer model can efficiently process long text inputs while ensuring that key information is fully considered and accurately processed, thereby generating more relevant and accurate answers.

[0058] Step S208, use the target text sequence to perform inference calculation to obtain the target answer.

[0059] The target answer is the answer text generated by the document question answering model based on the target text sequence. The target answer can be used as a direct response to the input question, that is, the target answer is displayed in the graphical user interface to provide to the user. The target answer can include the answer content to the input question and can also include contextual extension content to the input question.

[0060] In the application scenario, the document question answering model receives the target text sequence as input and performs forward propagation to calculate the target answer. The forward propagation process may include attention mechanism calculation (used to generate the first answer token) and sequence generation calculation (used to continue to generate subsequent answer tokens until the complete target answer is generated).

[0061] Through the above method steps, when the target text sequence is used for reasoning and calculation to obtain the target answer, not only the reasoning speed is significantly improved, but also the reasoning accuracy is effectively guaranteed. Applying the above method steps, the model can efficiently process a large number of documents and complex questions in the document question and answer scenario, provide fast, accurate and detailed answers, and enhance the user experience and system question and answer processing capabilities.

[0062] Based on the above steps S202 to S208, in an embodiment of the present application, according to the input question, the recalled document and the key value cache corresponding to the recalled document are obtained; the input question and the recalled document are used to generate an initial text sequence; based on the hybrid key value strategy, the initial text sequence is subjected to key value filling calculation to obtain a target text sequence, wherein the hybrid key value strategy is used to characterize the key value confirmation mechanism corresponding to multiple word units in different model layers, and the key value confirmation mechanism includes one of the following: a key value reuse mechanism based on a key value cache, a hybrid mechanism of a key value reuse mechanism and a key value calculation mechanism; the target text sequence is used to perform reasoning calculation to obtain a target answer. It is easy to notice that before the reasoning calculation, the present application determines the key value information of the word unit through a hybrid key value strategy, rather than simply and mechanically reusing the key value cache, thereby improving the accuracy of the reasoning calculation. In addition, compared with the scheme of simply performing key value calculation on all word units, determining the key value information of the word unit through a hybrid key value strategy can also improve the key value confirmation efficiency, thereby improving the reasoning speed. Therefore, the present application achieves the purpose of confirming word key information by introducing a hybrid key value strategy, improving the reasoning speed while ensuring the reasoning accuracy, thereby achieving the technical effect of balancing the improvement of reasoning speed and reasoning accuracy in the document question and answer processing process, and further solves the technical problem in the related technology that the calculation method that completely relies on key value reuse makes it difficult for the model to balance reasoning speed and reasoning accuracy.

[0063] The following further describes other optional method steps included in the above-mentioned document question and answer processing method of the embodiment of the present application in combination with application scenarios.

[0064] In an optional embodiment, the document database includes multiple candidate documents and a candidate key value cache pre-calculated based on the multiple candidate documents; in step S202, according to the input question, the recalled document and the key value cache corresponding to the recalled document are obtained, including the following method steps:

[0065] Step S221, obtaining a recalled document from a document database according to the relevance between the input question and a plurality of candidate documents;

[0066] Step S222: retrieve the key value cache corresponding to the recalled document from the candidate key value cache in the document database.

[0067] The document database may be a database storing a large number of candidate documents and candidate key value caches. The candidate documents may be pre-collected and organized according to a specific topic or a specific field to respond to possible future inquiries or questions. The candidate key value cache is a key value cache corresponding to each candidate document. The key value cache is usually calculated by the token sequence of the candidate document through the multi-head attention mechanism of the model. By pre-calculating and storing the key value caches of these candidate documents, the amount of calculation of the model in the actual reasoning process for these recalled documents can be significantly reduced when some candidate documents are recalled.

[0068] When the system receives an input question, it starts the recall algorithm, which selects the most relevant one or more candidate documents from the document database as recall documents based on the similarity or relevance between the input question and the quality inspection of each candidate document in the document database. This step ensures that the system can find answers from the most relevant document set, improving the pertinence and accuracy of the answers.

[0069] By efficiently screening and acquiring recalled documents, the system can quickly locate the set of documents that are most likely to contain the answer from the document database, thereby accelerating the subsequent answer generation process.

[0070] Furthermore, after obtaining the recalled documents, the system will further retrieve the key-value cache corresponding to these recalled documents so as to reuse these pre-computed key-value results in subsequent model reasoning.

[0071] In the application scenario, according to the above document question and answer processing method, the following is provided: Figure 3 A pre-filled calculation process is shown. As shown in the figure, after receiving the input question, the document database and the key-value cache are recalled according to the input question. Specifically, according to the relevance between the input question and each candidate document in the document database, the recalled document is obtained from the document database. The recalled document may include one or more candidate documents. Further, the key-value cache corresponding to the recalled document is obtained from the database for storing the key-value cache.

[0072] Through the above steps S221 to S222, the embodiment of the present application can achieve efficient and accurate information retrieval and answer generation in the intelligent document question and answer scenario by combining candidate documents in the document database with the key-value cache, and the process of recalling documents and key-value cache retrieval.

[0073] In an optional embodiment, in step S206, based on the hybrid key value strategy, key value filling calculation is performed on the initial text sequence to obtain the target text sequence, including the following method steps:

[0074] Step S261, using the target language model to perform key-value filling calculation on the initial text sequence to obtain a target text sequence, wherein the target language model is constructed based on the hybrid key-value strategy.

[0075] The target language model can be a large language model optimized by a hybrid key-value strategy, which is particularly suitable for document question-answering scenarios. The target language model can process long text inputs more efficiently in the pre-filling stage while maintaining the high quality of generated answers.

[0076] In the application scenario, based on the hybrid key-value strategy, the key-value confirmation strategy corresponding to multiple word units in each model layer of the large language model can be executed by defining hyperparameters. For example, by defining hyperparameters, it is specified that multiple word units in the first 5 model layers of the LLM adopt a key-value reuse mechanism, and multiple word units in other model layers except the first 5 model layers in the LLM adopt a hybrid mechanism. Based on this, the pre-trained LLM is adjusted according to the pre-defined hyperparameters and hybrid key-value strategy to construct the above-mentioned target language model. The above-mentioned target text sequence may include the output sequence (including multiple tokens) of the last model layer in the multiple model layers of the target language model, and may also include the key-value information corresponding to the output sequence.

[0077] In the application scenario, based on the hybrid key-value strategy, a large language model is designed and optimized so that the large language model can dynamically determine in which model layers to adopt the key-value reuse mechanism and in which model layers to adopt the key-value calculation mechanism for which tokens (other tokens in these model layers will also reuse the key-value cache) according to the characteristics of the input sequence.

[0078] The target language model built based on the hybrid key-value strategy can significantly improve the reasoning speed and reduce the consumption of computing resources while maintaining the ability to process long texts. At the same time, it maintains a high level of answer accuracy by screening layer by layer and supplementing cross-attention.

[0079] Furthermore, the initial text sequence is input into the target language model, which optimizes the key-value pair of each token in each model layer according to the hybrid key-value strategy (i.e., decides whether to reuse the cache or recalculate), and finally calculates the target text sequence based on the optimized key-value information.

[0080] The above key-value filling calculation ensures that the target language model can quickly process long text sequences in the pre-filling stage. At the same time, by selectively reusing cache and supplementary calculations, the accuracy loss is reduced and the quality of the answer is improved.

[0081] Through the above steps S261 to S262, the embodiment of the present application can achieve the dual optimization of reasoning acceleration and result accuracy in the document question and answer scenario as a whole by constructing a target language model based on a hybrid key-value strategy and implementing key-value filling calculations. Specifically, the application of the hybrid key-value strategy enables the model to intelligently balance computing efficiency and answer accuracy when processing long text pre-filling, avoiding the decline in answer quality that may be caused by simple reuse of cache. The construction of the target language model ensures that the model can adapt to the specific needs of document question and answer. Through the optimized decoding strategy, the target language model can quickly generate a complete answer after generating the first answer token, while maintaining a high output quality.

[0082] In an optional embodiment, in a document question-answering processing method, based on the stacking direction of multiple model layers in a target language model, the multiple model layers include: a first part model layer and a second part model layer, wherein:

[0083] The first partial model layer includes continuous partial model layers starting from the first model layer among the multiple model layers. The key value confirmation mechanism corresponding to the first partial model layer is a key value reuse mechanism.

[0084] The second partial model layer includes a partial model layer that is arranged after and continuous with the first partial model layer, and the key value confirmation mechanism corresponding to the second partial model layer is a hybrid mechanism.

[0085] The target language model may include multiple Transformer layers, each of which is used to perform specific feature extraction and information processing tasks. These Transformer layers adopt a layer stacking structure. The output of the previous Transformer layer serves as the input of the next Transformer layer. When constructing the target language model, multiple layers of Transformer are connected in a specific order (i.e., the stacking direction mentioned above), and this specific order determines the flow path of information within the target language of the model.

[0086] The first part of the model layers can be the first few model layers in the target language model, that is, the shallow model layers. Each model layer in the first part of the model layers preferentially uses the key-value reuse mechanism in the pre-filling phase to directly obtain key-value pairs from the key-value cache of the recalled document to accelerate the initial attention calculation.

[0087] The second part model may be a series of continuous model layers located after the first part model layer in the target language model. Each model layer in the second part model layer uses a hybrid mechanism in the pre-filling stage, that is, in each model layer in the second part model layer, key value calculation is performed on some tokens among the multiple tokens contained in the input sequence, and key value reuse is performed on other tokens among the multiple tokens.

[0088] It should be noted that, based on the above hybrid mechanism, the target language model will dynamically decide whether to use the cached key-value pair in the key-value cache as the key-value pair of the token in the current model layer, or to recalculate the key-value pair of the token in the current model layer according to the importance of each token.

[0089] It should be noted that the first part of the model layer and the second part of the model layer in the target language model are set by setting and adjusting the hyper parameters of the target language model.

[0090] It should be noted that the first part of the model layer may also include one or more model layers after the second part of the model layer.

[0091] Through the architectural design of the above-mentioned target language model, the embodiment of the present application can achieve efficient acceleration of the reasoning process in the intelligent document question and answer scenario by implementing the key-value reuse mechanism and the hybrid mechanism in a layered manner in the target language model, while reducing the accuracy loss caused by computational optimization, ensuring that the system can generate high-quality answers quickly and accurately, and significantly improving the user experience and system performance.

[0092] In an optional embodiment, in step S262, the target language model is used to perform key-value filling calculation on the initial text sequence to obtain the target text sequence, including the following method steps:

[0093] Step S2621, in the first model layer, according to the key value reuse mechanism and the key value cache, determine the word-unit key value information of the initial text sequence, and calculate the output sequence of the current layer based on the word-unit key value information;

[0094] Step S2622, in the model layers other than the first model layer in the first part of the model layers, determine the word-unit key value information of the input sequence of the current layer according to the key-value reuse mechanism and the key-value cache, and calculate the output sequence of the current layer based on the word-unit key value information;

[0095] Step S2623, in the second part of the model layer, based on the hybrid mechanism and the key value cache, determine the word unit key value information of the input sequence of the current layer, and calculate the output sequence of the current layer based on the word unit key value information;

[0096] Step S2624, determining the output sequence corresponding to the last model layer of the target language model as the target text sequence;

[0097] Among them, for any model layer except the last model layer, the output sequence of the current layer is used as the input sequence corresponding to the next model layer.

[0098] The first model layer is the first model layer in the first part of the model layers in the target language model. The input sequence of the first model layer is the initial text sequence, and the input of the first model layer also includes a key-value cache.

[0099] The above-mentioned final model layer is the last model layer in the target language model, and the output sequence of the final model layer is used as the final output of the entire target language model, that is, the target text sequence. It should be noted that the above-mentioned final model layer can belong to the second part model layer (that is, using a hybrid mechanism) or the first part model layer (that is, using a key-value reuse mechanism).

[0100] In the application scenario, when the target language model is used to perform key-value filling calculations on the initial text sequence, the initial text sequence and the key-value cache are input into the first model layer. In the first model layer, the word-value information of the initial text sequence is determined based on the key-value reuse mechanism and the key-value cache. The word-value information includes the key-value pairs of each token in the initial text sequence in the current layer. In this example, the first model layer directly uses the key-value cache of the recalled document to determine the key-value pairs of multiple tokens in the current layer and obtain the word-value information. Furthermore, the output sequence of the current layer is calculated based on the word-value information.

[0101] In the first part of the model layers, except for the first model layer, the input sequence of other model layers is the output sequence of the previous model layer. The key-value pairs of multiple tokens in the input sequence in the current layer are still determined based on the key-value reuse mechanism and key-value cache, and the output sequence of the current layer is further generated.

[0102] In the second part, the model layer, a hybrid mechanism is introduced. The model layer dynamically determines which tokens are key-value calculated in the current layer and reuses the key-value cache for other tokens based on the difference between the input sequence and the key-value cache. After determining the key-value pairs of multiple tokens in the current layer, the output sequence of the current layer is further generated.

[0103] Still like Figure 3 As shown in FIG, after obtaining the recall document and the key-value cache, the input question, the recall document and the key-value cache are used to generate an initial text sequence, which is used as the input of the first model layer of the large language model (i.e., the target language model mentioned above).

[0104] like Figure 3 As shown, in the process of key-value filling calculation of a large language model, it is determined whether the current layer needs to be screened (that is, whether it belongs to the second part of the model layer mentioned above) according to the hyperparameters. If it does not need to be screened (that is, it belongs to the first part of the model layer), the attention mechanism calculation is performed, and the calculated output sequence is used as the input sequence of the next layer, and the next layer calculation is entered; if screening is required, the difference calculation process is entered.

[0105] Furthermore, if Figure 3 As shown, after the large language model completes the calculation of multiple model layers to obtain the target text sequence, it is confirmed that the pre-filling stage is completed.

[0106] Through the above-mentioned steps S2621 to S2626, the embodiment of the present application achieves efficient acceleration of the reasoning process in the document question and answer scenario through the implementation of hierarchical calculation and optimization strategies, while greatly reducing the loss of precision, generating answers that are both fast and accurate, and significantly improving user experience and system performance.

[0107] Specifically, by using the key-value reuse mechanism in the first few layers of the target language model (i.e., the first model layer), the system can quickly start the reasoning process, avoid repeated calculations of recalled documents, and significantly improve the processing speed. The hybrid mechanism introduced in the second model layer enables the model to dynamically select which words to perform key-value calculations on and which words to reuse the key-value cache, supplementing key information and reducing the loss of precision caused by simple reuse. As the model layer goes deeper, from the first model layer to the last model layer, the representation of the input information is gradually enriched, and the target text sequence finally generated contains a deep understanding and accurate answer to the question, ensuring the high quality of the system output.

[0108] In an optional embodiment, in step S2623, based on the hybrid mechanism and the key value cache, determining the word-value information of the input sequence includes the following method steps:

[0109] Step S271, performing key value difference calculation on the input sequence and the key value cache to obtain a difference calculation result;

[0110] Step S272, determining a first part of word-grams from a plurality of word-grams in the input sequence based on the difference calculation result;

[0111] Step S273, performing key value calculation on the first part of the word units according to the key value calculation mechanism to obtain a key value calculation result;

[0112] Step S274, performing key-value reuse on the second part of the word-grams according to the key-value reuse mechanism and the key-value cache to obtain a key-value reuse result, wherein the second part of the word-grams are other word-grams in the plurality of word-grams except the first part of the word-grams;

[0113] Step S275, obtaining word-unit key value information based on the key value calculation result and the key value reuse result.

[0114] The first part of the word-grams is the word-gram to be calculated for key-value. According to the key-value difference calculation result, the word-grams with large differences between the cached key-value pairs in the input sequence and the key-value cache are determined as the word-grams that need to recalculate the key-value pairs, and the first part of the word-grams are obtained. By recalculating the key-value pairs, important information required in the attention mechanism calculation can be supplemented, reducing the loss of model reasoning accuracy.

[0115] In order to ensure the integrity of the word-value information of the text sequence, the words other than the first part of the words are determined as the second part of the words. These second part of the words are the words to be reused for key value. The difference between the second part of the words and the cache key-value pairs in the key-value cache is small, and the cache key-value pairs can be used as the key-value pairs of these second part of the words in the current layer, saving computing resources (avoiding key value calculation for the second part of the words) and computing time.

[0116] The above-mentioned word-unit key-value information may include multiple key-value pairs corresponding to all the multiple word-units (including the first part of the word-units and the second part of the word-units) in the input sequence. The above-mentioned word-unit key-value information serves as the basis for performing attention mechanism calculation and generating output sequence in the current layer.

[0117] In the application scenario, in any model layer of the second part of the target language model, the difference between the key value information of each word in the input sequence and the key value information stored in the key value cache is calculated to obtain the key value difference calculation to determine which words need to recalculate the key value pairs. According to the result of the key value difference calculation, the words in the current layer that need to recalculate the key value pairs are screened out to obtain the first part of the words. Furthermore, the complete key value pair calculation steps are performed on the first part of the words to obtain the key value calculation results to ensure that the key information of this part of the words is accurately represented, so as to assist in the subsequent generation of high-quality answers. For the second part of the words, the key value pairs in the key value cache are directly used to obtain the key value reuse results, avoiding repeated calculations and saving computing resources and time. The key value calculation results of the first part of the words and the key value reuse results of the second part of the words are integrated to form the word key value information of the input sequence, laying the foundation for subsequent attention calculations and answer generation.

[0118] Based on the above steps S271 to S275, the embodiment of the present application accelerates the reasoning process in the document question and answer scenario through key value difference calculation, key value calculation of the first part of the word units, key value reuse of the second part of the word units, and integration of the final word unit key value information. At the same time, by supplementing key information, the loss of precision is reduced, the high quality of the system output is ensured, and the practical application value and user satisfaction of the system are improved.

[0119] In an optional embodiment, in step S271, performing key value difference calculation on the input sequence and the key value cache to obtain a difference calculation result includes the following method steps:

[0120] Step S2711, performing attention mechanism mapping based on the input sequence to determine first key value data corresponding to a plurality of word-units;

[0121] Step S2712, determining second key value data corresponding to the plurality of word units based on the key value cache;

[0122] Step S2713, performing difference norm calculation on the first key value data and the second key value data to obtain a difference calculation result.

[0123] The first key-value data is a key-value pair corresponding to a word in the input sequence obtained by mapping through the attention mechanism. Since the multiple model layers of the target language model adopt a layer stacking structure, the input sequence of the latter model layer is the output sequence of the previous model layer, and the output sequence of the previous model layer is generated based on the key-value pairs corresponding to multiple words in the previous model layer. Therefore, the key-value pairs of the first key-value data in the current layer correspond to the key-value pairs of multiple words in the previous layer). The first key-value data can reflect the understanding and representation of the text sequence by the previous layer of the current layer.

[0124] The second key-value data is a cache key-value pair in the key-value cache corresponding to the word in the input sequence. The second key-value data can reflect the representation of the recalled document obtained by preprocessing.

[0125] The above difference norm is used to characterize the difference between two vectors. In this example, the two vectors can be K vectors corresponding to multiple tokens obtained by mapping the input sequence (ie, the first key value data) and K vectors corresponding to multiple tokens in the key value cache (ie, the second key value data).

[0126] In the application scenario, in any model layer of the second part of the target language model, on the one hand, the input sequence is mapped by the attention mechanism, and the key-value pair of each word is calculated to form the first key-value data; on the other hand, the key-value pair of each word is retrieved from the key-value cache to form the second key-value data; further, the difference between the first key-value data and the second key-value data is calculated, and the norm is used to quantify the difference, and the result is used for subsequent word filtering and key-value pair updating.

[0127] Based on the above steps S2711 to S2713, the embodiment of the present application accelerates the reasoning process in the document question and answer scenario through attention mechanism mapping, key-value cache utilization and difference norm calculation. At the same time, by retaining the key-value pair calculation of key information, it reduces the loss of precision, ensures the high quality and fast response of the system output, and improves the user experience and system performance.

[0128] In an optional embodiment, the difference calculation result includes a plurality of difference norms corresponding to the plurality of word-grams respectively; in step S272, based on the difference calculation result, determining the first part of word-grams from the plurality of word-grams of the input sequence includes the following method steps:

[0129] Step S2721, sorting the multiple word units by word unit number according to the multiple difference norms to obtain a sorting result;

[0130] Step S2722: using the sorting result, select the first part of word-grams from the plurality of word-grams.

[0131] The above difference calculation results include a difference norm (eg, L1 norm or L2 norm) corresponding to each word. The higher the value of the difference norm corresponding to a word, the greater the difference between the key-value pair of the word and the key-value cache in the recalled document.

[0132] In the application scenario, in any model layer of the second part of the target language model, multiple difference norms in the difference calculation results are sorted, and based on this, it is determined which words have the largest key-value pairs and need to be recalculated first; specifically, based on the results of the word number sorting, the model selects a certain proportion of words with the largest difference norm as the first part of the words, and these words will recalculate the key-value pairs to ensure the accurate expression of key information and reduce the accuracy loss of the answer.

[0133] Still like Figure 3 As shown in the figure, during the key-value filling calculation of the large language model, if it is determined that the current layer needs to be screened, the norm difference between the word key value of the input sequence and the key-value cache is calculated, and a certain proportion of words are selected according to the difference sorting to obtain the first part of the word. Based on the first part of the word, the Q vector, K vector and V vector of the input sequence are processed by dimensionality reduction and then the attention mechanism is calculated to obtain the output sequence of the current layer.

[0134] It should be noted that after completing the attention mechanism calculation of the current layer, if all the model layers of the large language model are calculated, the target text sequence is obtained; otherwise, the calculated output sequence is used as the input sequence of the next layer and enters the next layer calculation.

[0135] It should be noted that the schematic diagram of the word unit screening process in the second part of the model layer is as follows Figure 4 As shown. Figure 4 As shown, the input sequence of the current layer can contain multiple key-value caches ( Figure 4 In the cache C1, cache C2 and cache C3), each key-value cache can contain cache key-value pairs of a portion of word units. Based on these key-value caches, word units are screened according to the above method steps to obtain word unit screening results. Figure 4 As shown, in the word unit screening result, the first part of the word units are represented as diagonal shadows. That is, in the input sequence, multiple word units in the first part of the word units may be distributed discretely or continuously.

[0136] Furthermore, based on the word-unit screening results, the following can be done: Figure 5 The attention mechanism shown in Figure 1 is ready for calculation. Figure 5 As shown in the figure, for the input sequence, the query (Q) vector of the input sequence is calculated, and the Q vector is multiplied by the K vector of the current layer to obtain the attention matrix of the current layer. Based on the attention matrix (combined with the V vector of the current layer), the attention mechanism calculation can be performed to obtain the output sequence of the current layer. It should be noted that in Figure 5 In the figure, the white squares in the K vector represent the key elements (key) of the word units corresponding to the key-value reuse mechanism, and the gray squares in the K vector represent the key elements of the word units corresponding to the key-value calculation mechanism.

[0137] Based on the above steps S2721 to S2722, the embodiment of the present application achieves acceleration and accuracy maintenance of the reasoning process in the intelligent document question-answering scenario through word unit numbering and dynamic screening strategies based on difference norms. The model can quickly identify and process word units containing key information, and cache and reuse word units with smaller differences, which not only ensures the accuracy and high quality of the answers, but also improves the processing speed, realizes the efficient operation of the intelligent dialogue system, and improves the user experience and practical application value.

[0138] In an optional embodiment, in step S273, according to the key value calculation mechanism, key value calculation is performed on the first part of word units to obtain the key value calculation result, including the following method steps:

[0139] Step S2731, extracting semantic features of the first part of the word-grams to obtain an embedding representation corresponding to the first part of the word-grams;

[0140] Step S2732, based on the weight matrix corresponding to the current layer, a linear transformation calculation is performed on the embedded representation to obtain a key value calculation result, wherein the key value calculation result includes at least a key vector and a value vector.

[0141] The above-mentioned embedding representation refers to the feature representation of a word in the vector space, which is used to reflect the characteristics of the word at the semantic level. The above-mentioned weight matrix is ​​the parameter matrix corresponding to each model layer, which is used to determine the weights in the linear transformation calculation, that is, to control the degree of attention and processing method of the model layer on the embedded representation. The above-mentioned linear transformation calculation refers to the multiplication and addition operations of the embedding representation of the word using the weight matrix to generate a new vector representation. The above-mentioned key-value calculation results may include K vectors and V vectors. The key-value calculation results are the key outputs in the attention mechanism calculation, which are used to characterize the information of the word in a specific context and the degree of influence on the subsequent output.

[0142] In the application scenario, in any model layer of the second part of the target language model, in the process of key-value calculation of the first part of the word units, each word unit is firstly deeply processed, the semantic information of the word unit is extracted from the context, and the word unit is converted into an embedded representation in the vector space; further, in the model, the embedded representation of the first part of the word units is linearly transformed and calculated using the specific weight matrix corresponding to the current layer to generate a key vector and a value vector to obtain the key-value calculation result.

[0143] It should be noted that, during the above key value calculation process, the query (Query, abbreviated as Q) vector corresponding to each word unit can also be calculated synchronously.

[0144] Based on the above steps S2731 to S2732, the embodiment of the present application achieves accurate supplementation and in-depth understanding of key information in the document question-answering scenario through semantic feature extraction and linear transformation calculation based on weight matrix, reduces the precision loss caused by cache reuse, and maintains the efficiency of the reasoning process. In the embodiment of the present application, the model can generate high-quality answers that are both accurate and relevant, significantly improving the user experience and system performance, and demonstrating the flexibility and efficiency of the intelligent dialogue system in handling complex document question-answering tasks.

[0145] In an optional embodiment, in step S208, the target text sequence is used to perform reasoning calculation to obtain the target answer, including the following method steps:

[0146] Step S281, using the target language model to decode and sample the target text sequence to generate the first answer word;

[0147] Step S282, performing autoregressive inference based on the first answer word to generate a target answer.

[0148] According to the aforementioned method steps, the pre-filling stage is also completed in the process of generating the target text sequence. Further, the target language model begins to decode and sample the target text sequence, and predicts and generates the first answer word based on the internal state of the model and the input key value information as the starting point for answer generation.

[0149] Furthermore, the target language model incorporates the first answer word into the currently generated answer sequence, and performs the next step of prediction and generation based on the existing sequence information. This process is carried out in an autoregressive manner, gradually constructing a complete answer until the target answer is generated.

[0150] Based on the above steps S281 to S282, the embodiment of the present application achieves speed improvement and quality assurance of answer generation in the document question-answering scenario through decoding sampling and autoregressive reasoning. The target language model can quickly generate fluent answers while maintaining high accuracy, greatly improving the practical application value and user satisfaction of the intelligent dialogue system.

[0151] In summary, the embodiment of the present application proposes a document question-answering processing method, which focuses on optimizing the reasoning speed of the pre-filling stage of large language models when facing long text inputs, while maintaining high-precision answer generation. The core technical innovation lies in combining the efficient reuse of document key-value caches with layer-by-layer screening strategies to achieve intelligent dimensionality reduction of model input dimensions, supplement and enhance the cross-attention calculations in the attention mechanism, thereby significantly reducing end-to-end accuracy loss. Compared with the existing solutions, the above-mentioned technical solution provided in the embodiment of the present application achieves an average reasoning acceleration of 1.421 times on the basis of ensuring that the answer accuracy is only lost within 5%, showing excellent performance improvement. In addition, the above-mentioned technical solution can also design multiple adjustable hyperparameters to support users to flexibly control the acceleration effect and calculation logic, while only making lightweight modifications to the attention mechanism, ensuring compatibility with large language model deployment frameworks and computing operators, and enhancing the practicality and ease of use of the above-mentioned technical solution.

[0152] In the aforementioned operating environment, the present application also provides Figure 6 A document question and answer processing method is shown. Figure 6 is a flowchart of another document question and answer processing method according to an embodiment of the present application, such as Figure 6 As shown, the document question and answer processing method includes:

[0153] Step S601, obtaining a document question and answer processing request through a first application programming interface, wherein the request data carried in the document question and answer processing request includes: an input question;

[0154] Step S602: Return a document question and answer processing response through a second application programming interface, wherein the response data carried in the document question and answer processing response includes: a target answer.

[0155] The above target answer is obtained according to any of the aforementioned document question and answer processing methods.

[0156] The document question and answer processing method of the embodiment of the present application can be run in a cloud server to provide a document question and answer processing cloud service for the client. The client calls the first application programming interface to send a document question and answer processing request, and after the cloud server obtains the document question and answer processing request through the first application programming interface, it generates a target answer according to any of the above-mentioned document question and answer processing methods, and further returns a document question and answer processing response to the client through the second application programming interface.

[0157] The first application programming interface and the second application programming interface may be the same application programming interface or different application programming interfaces. In an optional embodiment, the interface parameters in the first application programming interface and the second application programming interface may include but are not limited to: interface global identifier, interface signature key, interface timestamp, interface request identifier, system call credential identifier, etc. The first application programming interface may use GET or POST as the interface request method to obtain the file processing request. The second application programming interface may use JSON format to feedback the file processing response.

[0158] In an embodiment of the present application, a document question and answer processing request is obtained through a first application programming interface, wherein the request data carried in the document question and answer processing request includes: an input question; a document question and answer processing response is returned through a second application programming interface, wherein the response data carried in the document question and answer processing response includes: a target answer. The above-mentioned target answer is obtained according to any of the aforementioned document question and answer processing methods. As a result, the present application achieves the purpose of confirming word-meta key information by introducing a hybrid key-value strategy, thereby improving the reasoning speed while ensuring the reasoning accuracy, thereby achieving the technical effect of balancing the improvement of the reasoning speed and the reasoning accuracy in the document question and answer processing process, and further solving the technical problem in the related technology that the calculation method that completely relies on key-value reuse makes it difficult for the model to balance the reasoning speed and the reasoning accuracy.

[0159] It should be noted that the preferred implementation of the above steps S601 to S602 can be found in the above-mentioned related descriptions and will not be repeated here.

[0160] In the aforementioned operating environment, this application provides the following Figure 7 A document question and answer processing method is shown. Figure 7 is a flowchart of another document question-and-answer processing method according to an embodiment of the present application. Figure 7 As shown, the document question and answer processing method includes:

[0161] Step S701, responding to the document question and answer processing instruction on the operation interface, uploading the input question;

[0162] Step S702, responding to the image processing instruction acting on the operation interface, and displaying the target answer on the operation interface.

[0163] The above target answer is obtained according to any of the aforementioned document question and answer processing methods.

[0164] The document question and answer processing method provided in the embodiment of the present application can use the aforementioned target language model to implement a visual document question and answer processing solution, which is convenient for human-computer interaction. The user inputs a document question and answer processing dialogue request, triggering the system to generate a target answer using the target language model and displaying the target answer in the graphical user interface.

[0165] According to the above method steps, a visualization scheme for document question and answer processing function is provided. The terminal device provides a graphical user interface, and at least one image document question and answer processing scene is displayed in the graphical user interface. The display content of the graphical user interface also includes an input component (such as a text input box, a voice input control, etc.) and a display component (such as a text display window, an image display window, etc.). The user inputs a document question and answer processing dialogue request through the input component to specify the input question in the document question and answer processing task. After detecting the user's input behavior, the target language model is used to obtain the target answer, and further, the target answer is played through the display component in the graphical user interface.

[0166] In an embodiment of the present application, in response to a document question and answer processing instruction acting on the operation interface, an input question is uploaded; in response to an image processing instruction acting on the operation interface, a target answer is displayed on the operation interface. The above target answer is obtained according to any of the aforementioned document question and answer processing methods. As a result, the present application achieves the purpose of confirming word-meta key information by introducing a hybrid key value strategy, improving the reasoning speed while ensuring the reasoning accuracy, thereby achieving the technical effect of balancing the reasoning speed and reasoning accuracy in the document question and answer processing process, and further solving the technical problem in the related technology that the calculation method that completely relies on key value reuse makes it difficult for the model to balance the reasoning speed and reasoning accuracy.

[0167] It should be noted that the preferred implementation of the above steps S701 to S702 can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0169] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM, RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0171] According to an embodiment of the present application, a device embodiment for implementing the above-mentioned document question and answer processing method is also provided, such as Figure 8 As shown, the document question and answer processing device includes: an acquisition module 801, which is used to acquire recalled documents and key-value caches corresponding to recalled documents according to input questions; a generation module 802, which is used to generate an initial text sequence using the input questions and recalled documents; a calculation module 803, which is used to perform key-value filling calculations on the initial text sequence based on a hybrid key-value strategy to obtain a target text sequence, wherein the hybrid key-value strategy is used to characterize the key-value confirmation mechanism corresponding to multiple word elements in different model layers, and the key-value confirmation mechanism includes one of the following: a key-value reuse mechanism based on a key-value cache, a hybrid mechanism of a key-value reuse mechanism and a key-value calculation mechanism; an inference module 804, which is used to perform inference calculations using the target text sequence to obtain a target answer.

[0172] Optionally, the document database includes multiple candidate documents and a candidate key value cache pre-calculated based on the multiple candidate documents; the above-mentioned acquisition module 801 is also used to: obtain recalled documents from the document database based on the relevance between the input question and the multiple candidate documents; retrieve the key value cache corresponding to the recalled document from the candidate key value cache in the document database.

[0173] Optionally, the calculation module 803 is further used to: perform key-value filling calculation on the initial text sequence using the target language model to obtain a target text sequence.

[0174] Optionally, in the above-mentioned document question and answer processing device, based on the stacking direction of multiple model layers in the target language model, the multiple model layers include: a first partial model layer and a second partial model layer, wherein the first partial model layer includes continuous partial model layers starting from the first model layer in the multiple model layers, and the key value confirmation mechanism corresponding to the first partial model layer is a key value reuse mechanism, and the second partial model layer includes a partial model layer that is arranged after the first partial model layer and is continuous, and the key value confirmation mechanism corresponding to the second partial model layer is a hybrid mechanism.

[0175] Optionally, the above-mentioned calculation module 803 is also used to: in the first model layer, determine the word-gram key value information of the initial text sequence according to the key-value reuse mechanism and the key-value cache, and calculate the output sequence of the current layer based on the word-gram key value information; in the other model layers in the first part of the model layer except the first model layer, determine the word-gram key value information of the input sequence of the current layer according to the key-value reuse mechanism and the key-value cache, and calculate the output sequence of the current layer based on the word-gram key value information; in the second part of the model layer, determine the word-gram key value information of the input sequence of the current layer based on the hybrid mechanism and the key-value cache, and calculate the output sequence of the current layer based on the word-gram key value information; determine the output sequence corresponding to the last model layer of the target language model as the target text sequence; wherein, for any model layer other than the last model layer, the output sequence of the current layer is used as the input sequence corresponding to the next model layer.

[0176] Optionally, the above-mentioned calculation module 803 is also used to: perform key value difference calculation on the input sequence and the key value cache to obtain a difference calculation result; based on the difference calculation result, determine a first part of the word elements from multiple word elements in the input sequence; according to the key value calculation mechanism, perform key value calculation on the first part of the word elements to obtain a key value calculation result; according to the key value reuse mechanism and the key value cache, perform key value reuse on the second part of the word elements to obtain a key value reuse result, wherein the second part of the word elements are other word elements in the multiple word elements except the first part of the word elements; based on the key value calculation result and the key value reuse result, obtain the word element key value information.

[0177] Optionally, the above-mentioned calculation module 803 is also used to: perform attention mechanism mapping based on the input sequence to determine the first key-value data corresponding to multiple word-grams; determine the second key-value data corresponding to multiple word-grams based on the key-value cache; perform difference norm calculation on the first key-value data and the second key-value data to obtain the difference calculation result.

[0178] Optionally, the difference calculation result includes multiple difference norms corresponding to multiple word units respectively; the above-mentioned calculation module 803 is also used to: sort the multiple word units by word unit number according to the multiple difference norms to obtain the sorting result; and use the sorting result to select the first part of word units from the multiple word units.

[0179] Optionally, the above-mentioned calculation module 803 is also used to: extract semantic features of the first part of the words to obtain the embedded representation corresponding to the first part of the words; based on the weight matrix corresponding to the current layer, perform linear transformation calculation on the embedded representation to obtain a key-value calculation result, wherein the key-value calculation result includes at least a key vector and a value vector.

[0180] Optionally, the above-mentioned reasoning module 804 is also used to: use the target language model to decode and sample the target text sequence to generate a first answer word; perform autoregressive reasoning based on the first answer word to generate a target answer.

[0181] The acquisition module 801, generation module 802, calculation module 803 and reasoning module 804 correspond to steps S202 to S208 in the above embodiment. The four modules have the same instances and application scenarios as the corresponding steps, but are not limited to the contents disclosed in the above embodiment.

[0182] According to an embodiment of the present application, another device embodiment for implementing the above document question and answer processing method is also provided, such as Fig. 9 As shown, the document question and answer processing device includes:

[0183] The request module 901 is used to obtain a document question and answer processing request through a first application programming interface, wherein the request data carried in the document question and answer processing request includes: an input question;

[0184] The response module 902 is used to return a document question and answer processing response through a second application programming interface, wherein the response data carried in the document question and answer processing response includes: a target answer, and the target answer is obtained according to any of the above-mentioned document question and answer processing methods.

[0185] The request module 901 and the response module 902 correspond to steps S601 to S602 in the above embodiment. The two modules and the corresponding steps implement the same instances and application scenarios, but are not limited to the contents disclosed in the above embodiment.

[0186] According to an embodiment of the present application, another device embodiment for implementing the above document question and answer processing method is also provided, such as Fig.10 As shown, the document question and answer processing device includes:

[0187] The upload module 1001 is used to respond to the document question and answer processing instruction on the operation interface and upload the input question;

[0188] The display module 1002 is used to respond to the image processing instruction acting on the operation interface and display the target answer on the operation interface, wherein the target answer is obtained according to any one of the document question and answer processing methods mentioned above.

[0189] The above-mentioned upload module 1001 and display module 1002 correspond to steps S701 to S702 in the above-mentioned embodiment. The examples and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiment.

[0190] It should be noted that the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above modules may also be run as part of the device in the server provided in the above embodiment.

[0191] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in the above embodiments, as well as the application scenario and implementation process, but is not limited to the scheme provided in the above embodiments.

[0192] An embodiment of the present application can provide a document question and answer processing system, including: a client, used to send input questions to be processed, wherein the input questions are used to obtain recalled documents and a key-value cache corresponding to the recalled documents; a server, connected to the client, used to use the input questions and recalled documents to generate an initial text sequence, based on a hybrid key-value strategy, perform key-value filling calculations on the initial text sequence to obtain a target text sequence, and perform inference calculations using the target text sequence to obtain a target answer, wherein the hybrid key-value strategy is used to characterize a key-value confirmation mechanism corresponding to multiple word elements in different model layers, and the key-value confirmation mechanism includes one of the following: a key-value reuse mechanism based on a key-value cache, a hybrid mechanism of a key-value reuse mechanism and a key-value calculation mechanism; the client is also used to output a target answer.

[0193] An embodiment of the present application may provide an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein when the program is running, any one of the document question and answer processing methods described above is executed.

[0194] Fig.11 is a structural block diagram of an electronic device according to an embodiment of the present application. Fig.11 As shown, the electronic device 110 may include: one or more (only one is shown in the figure) processors 112, a memory 114, a storage controller, and a peripheral interface.

[0195] The above-mentioned electronic device can be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a personal computer (PC for short), a model all-in-one machine, etc., and the electronic device can be pre-installed with the above-mentioned model in the above-mentioned embodiment of the present application.

[0196] Specifically, the electronic device can preset various types of models, including but not limited to models in the fields of natural language processing, visual processing, speech processing, code processing, multimodal task processing, etc., so as to provide a variety of model selection. In different product forms, the electronic device can support one or more model usage methods, including but not limited to model training, model calling, model fine-tuning, model deployment, model reasoning and application, etc. In some product forms, the electronic device also supports model management, including but not limited to multi-type model management (supporting the management of multiple types of models such as discriminants and generators), model version control (supporting the control of different model versions), model evaluation (based on model evaluation tools, evaluating the performance and effect of the model), etc. In other product forms, the electronic device can also create applications based on the model, provide application programming interface (Application Programming Interface, referred to as API) calling capabilities, can call the model to the created application through the API interface, and provide application management tools to achieve management and monitoring of the application.

[0197] Furthermore, the electronic device may also include data management (supporting the creation and management of model tuning data sets), a training center (providing rich training resources to help users learn and master artificial intelligence (AI) technology), and basic management and control capabilities (providing enterprise-level basic management and control capabilities to ensure the security and efficient operation of the system). Through the above functions, a comprehensive, integrated AI development, training, deployment and application device is provided.

[0198] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the document question and answer processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the document question and answer processing method in the above embodiment is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0199] The processor can call the executable program stored in the memory through the transmission device to execute any one of the document question and answer processing methods in the above embodiments.

[0200] Those skilled in the art will understand that Fig.11The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, and a mobile Internet device (Mobile Internet Devices, MID for short). Fig.11 The structure of the electronic device is not limited. For example, the electronic device 110 may also include Fig.11 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Fig.11 Different configurations are shown.

[0201] A person of ordinary skill in the art may understand that all or part of the steps in the various document question and answer processing methods of the above-mentioned embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0202] The embodiment of the present application also provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the document question and answer processing method provided in the above embodiment.

[0203] Optionally, in this embodiment, the above storage medium may be located in an electronic device.

[0204] Optionally, in this embodiment, the computer-readable storage medium is configured to store an executable program, and when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the document question and answer processing methods in the above embodiments.

[0205] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and when the computer program is executed by a processor, the document question and answer processing method provided in the embodiment is implemented.

[0206] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the document question-and-answer processing method provided in the embodiments is implemented.

[0207] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the document question and answer processing method provided in the above embodiment is implemented.

[0208] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0209] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0210] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0212] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the document question and answer processing method in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, ROM, RAM, mobile hard disks, magnetic disks or optical disks.

[0213] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A document question-answering processing method, characterized in that: include: According to the input question, obtain the recalled document and the key-value cache corresponding to the recalled document; Generate an initial text sequence using the input question and the recalled document; Based on the hybrid key value strategy, the initial text sequence is subjected to key value filling calculation to obtain the target text sequence, wherein the hybrid key value strategy is used to characterize the key value confirmation mechanism corresponding to multiple word elements in different model layers, and the key value confirmation mechanism includes one of the following: a key value reuse mechanism based on the key value cache, a hybrid mechanism of the key value reuse mechanism and a key value calculation mechanism; The target text sequence is used to perform reasoning calculations to obtain a target answer.

2. The document question-answering processing method according to claim 1, characterized in that: The document database includes a plurality of candidate documents and a candidate key value cache pre-calculated based on the plurality of candidate documents; According to the input question, obtaining the recalled document and the key-value cache corresponding to the recalled document includes: Acquire the recalled document from the document database according to the relevance between the input question and the plurality of candidate documents; The key-value cache corresponding to the recalled document is retrieved from the candidate key-value cache in the document database.

3. The document question-answering processing method according to claim 1, characterized in that: Based on the hybrid key value strategy, The initial text sequence is subjected to key value filling calculation to obtain the target text sequence, which includes: The target language model is used to perform key-value filling calculation on the initial text sequence to obtain the target text sequence, wherein the target language model is constructed based on the hybrid key-value strategy.

4. The document question-answering processing method according to claim 3, characterized in that: Based on the stacking direction of multiple model layers in the target language model, the multiple model layers include: a first part of model layers and a second part of model layers, wherein: The first partial model layer includes continuous partial model layers starting from the first model layer among the multiple model layers, and the key value confirmation mechanism corresponding to the first partial model layer is the key value reuse mechanism. The second partial model layer includes a partial model layer that is arranged after and continuous with the first partial model layer, and the key value confirmation mechanism corresponding to the second partial model layer is the hybrid mechanism.

5. The document question-answering processing method according to claim 4, characterized in that: The target language model is used to perform key value filling calculation on the initial text sequence to obtain the target text sequence, including: In the first model layer, according to the key-value reuse mechanism and the key-value cache, the word-unit key-value information of the initial text sequence is determined, and the output sequence of the current layer is calculated based on the word-unit key-value information; In the model layers other than the first model layer in the first part of the model layers, determining the word-unit key value information of the input sequence of the current layer according to the key-value reuse mechanism and the key-value cache, and calculating the output sequence of the current layer based on the word-unit key value information; In the second part of the model layer, based on the hybrid mechanism and the key value cache, the word unit key value information of the input sequence of the current layer is determined, and the output sequence of the current layer is calculated based on the word unit key value information; Determine the output sequence corresponding to the last model layer of the target language model as the target text sequence; Among them, for any model layer other than the last model layer, the output sequence of the current layer is used as the input sequence corresponding to the next model layer.

6. The document question-answering processing method according to claim 5, characterized in that: Based on the hybrid mechanism and the key value cache, determining the word-value information of the input sequence includes: Performing key value difference calculation on the input sequence and the key value cache to obtain a difference calculation result; Based on the difference calculation result, determining a first part of word-grams from a plurality of word-grams of the input sequence; According to the key value calculation mechanism, performing key value calculation on the first part of word units to obtain a key value calculation result; According to the key-value reuse mechanism and the key-value cache, key-value reuse is performed on the second part of word-units to obtain a key-value reuse result, wherein the second part of word-units are other word-units in the multiple word-units except the first part of word-units; The word-unit key value information is obtained based on the key value calculation result and the key value reuse result.

7. The document question-answering processing method according to claim 6, characterized in that: The key value difference calculation is performed on the input sequence and the key value cache to obtain the difference calculation result, including: Performing attention mechanism mapping based on the input sequence to determine first key-value data corresponding to the multiple word-units; Based on the key-value cache, determining second key-value data corresponding to the plurality of word-units; A difference norm calculation is performed on the first key-value data and the second key-value data to obtain the difference calculation result.

8. The document question-answering processing method according to claim 6, characterized in that: The difference calculation result includes a plurality of difference norms corresponding to the plurality of word units respectively; Based on the difference calculation result, determining the first part of word-grams from the plurality of word-grams of the input sequence comprises: According to the plurality of difference norms, the plurality of word units are sorted by word unit number to obtain a sorting result; The first part of word-grams is selected from the plurality of word-grams by using the sorting result.

9. A document question-answering processing method, characterized in that: include: Obtaining a document question and answer processing request through a first application programming interface, wherein the request data carried in the document question and answer processing request includes: an input question; A document question and answer processing response is returned through a second application programming interface, wherein the response data carried in the document question and answer processing response includes: a target answer, and the target answer is obtained according to the document question and answer processing method described in any one of claims 1 to 8.

10. A document question-answering processing method, characterized in that: include: Respond to the document question and answer processing instructions on the operation interface and upload the input questions; In response to an image processing instruction acting on the operation interface, a target answer is displayed on the operation interface, wherein the target answer is obtained according to the document question and answer processing method according to any one of claims 1 to 9.

11. A document question-answering processing system, characterized in that: include: A client, used for sending an input question to be processed, wherein the input question is used to obtain a recalled document and a key-value cache corresponding to the recalled document; The server is connected to the client and is used to generate an initial text sequence using the input question and the recalled document, perform key-value filling calculation on the initial text sequence based on a hybrid key-value strategy to obtain a target text sequence, and perform reasoning calculation using the target text sequence to obtain a target answer, wherein the hybrid key-value strategy is used to characterize a key-value confirmation mechanism corresponding to multiple word units in different model layers, and the key-value confirmation mechanism includes one of the following: a key-value reuse mechanism based on the key-value cache, a hybrid mechanism of the key-value reuse mechanism and a key-value calculation mechanism; The client is further configured to output the target answer.

12. An electronic device, characterized in that: include: A memory storing an executable program; A processor is used to run the program, wherein the program, when running, executes the document question and answer processing method described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the document question and answer processing method described in any one of claims 1 to 10.

14. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the document question and answer processing method as described in any one of claims 1 to 10.

Images