Context compression method and device based on query guidance

Abstract

The invention provides a context compression method based on query guidance. The context compression method comprises the following steps: filling query information and a context to be compressed into a dialogue template; respectively calculating first attention numerical values of a plurality of lexical elements in the context of the dialogue template; performing attention reconstruction on the context according to the first attention numerical values corresponding to the plurality of lexical elements; and compressing the context after the attention reconstruction to be within a preset target compression ratio. The invention further provides a context compression system based on query guidance, a storage medium and electronic equipment. Therefore, key information can be more efficiently reserved, a shorter context is input to the generative model, the reasoning efficiency of the generative model in a long context reasoning scene is improved, and the reasoning cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of context compression, and in particular to a query-guided context compression method, device, storage medium and computer equipment. Background Art

[0002] Context compression aims to reduce the reasoning complexity and cost of long-context reasoning scenarios for large language models (LLMs). On the one hand, by removing noise from the context to reduce the length, the reasoning efficiency can be improved. On the other hand, it not only meets the input length limit of open source large language models, but also reduces the cost of accessing proprietary large language models. Existing context compression methods mainly implement sentence-level compression through summarization and word-level compression by estimating information entropy.

[0003] Sentence-level compression usually uses a large language model to directly summarize the recalled documents, or trains a proprietary model to extract the most useful sentence subsets from the documents. There are also works that further rewrite these sentences with a generative model based on the extracted candidate sentences to improve the coherence and naturalness of the summary. However, sentence-level summary generation methods cannot directly control the compression rate of the prompt, so word-level and term-level compression methods have received more attention recently. This type of method uses the self-information in the prompt to screen word-units. This method believes that perplexity can be used to measure the uncertainty of the large language model about the text information. Lexical units with lower perplexity contain less information, so they can be filtered out. Existing work has further proposed a two-stage compression controller from coarse screening to fine screening, so as to achieve different degrees of compression for documents with different amounts of information. On the basis of compression based on the amount of information in the document itself, there is also work that further proposes the need to consider the relationship between the document and the question, and proposes a method for calculating the document conditional perplexity based on the question.

[0004] Of the two methods mentioned above, context compression at the word level by estimating information entropy can usually achieve higher compression rates and better performance. This method uses a small external model to evaluate the perplexity of individual words to identify those words that contribute the least information gain, and then compresses or deletes the words that show low information. However, the problem with this method is that the perplexity of words for the compression model with small parameters and the generation model with large parameters is often inconsistent, which makes it difficult to align their judgments on the importance of words.

[0005] Existing context compression methods based on information entropy reduce the length of context text by scoring the importance of words in the context and removing words with low scores. This type of method uses an external large language model as a compression model to evaluate the amount of information of each word in the context, and inputs words containing more information into the downstream generation model. The score for the amount of information is derived from the perplexity of the compression model when generating the context. However, inconsistencies between the compression model and the generation model may still lead to inconsistent judgments on which words are considered to have low entropy gain. The reason for this difference is that different models may differ in interpreting and processing the same information. As a result, the context filtered by the compression model is difficult to meet the information requirements of the generation model for answering the current query. Moreover, existing technologies all rely on compression models with large parameters, which greatly increases the cost of inference.

[0006] In summary, the prior art obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the invention

[0007] In view of the above-mentioned defects, the purpose of the present invention is to provide a query-guided context compression method, device, storage medium and computer equipment, which can retain key information more efficiently, input shorter context into the generation model, improve the reasoning efficiency of the generation model in long-context reasoning scenarios, and reduce the reasoning cost.

[0008] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0009] In a first aspect, an embodiment of the present invention provides a context compression method based on query guidance, the method comprising:

[0010] Fill the query information and the context to be compressed into the dialogue template;

[0011] Calculating first attention values ​​of a plurality of words in the context of the dialogue template respectively;

[0012] Reconstructing the context according to the first attention values ​​corresponding to the plurality of word-units;

[0013] The context after attention reconstruction is compressed within a predetermined target compression rate.

[0014] According to the method of the present invention, the step of respectively calculating the first attention values ​​of a plurality of first word-units in the context of the dialogue template comprises:

[0015] Setting the last token in the dialogue template as a sentinel token;

[0016] The first attention values ​​of the plurality of words in the context are obtained by calculating the attention distribution of the sentinel word.

[0017] According to the method of the present invention, the step of calculating the attention distribution of the sentinel word-unit comprises:

[0018] The attention distribution of the sentinel word is calculated through a self-attention mechanism.

[0019] According to the method of the present invention, the step of reconstructing the context according to the first attention value corresponding to the first word-gram list includes:

[0020] The first attention value corresponding to the word-gram is converted into a quantitative measure of the importance of each word-gram in the context.

[0021] According to the method of the present invention, before the step of converting the first attention value corresponding to the first word-gram list into a quantitative measure of the importance of each word-gram in the context, the step includes:

[0022] The first attention value corresponding to the first word list of the context is normalized.

[0023] According to the method of the present invention, the step of compressing the context after attention reconstruction to within a predetermined target compression rate comprises:

[0024] Performing Gaussian filtering on the plurality of word-units and the corresponding first attention values ​​to obtain the plurality of word-units and the first attention values ​​after filtering;

[0025] Calculating second attention values ​​corresponding to a plurality of sentences in the context respectively, the second attention value being equal to the sum of the first attention values ​​of a plurality of word-units contained in the plurality of sentences after filtering;

[0026] Filtering target sentences whose second attention value reaches a second threshold to obtain sentence-level filtering results, and the sum of the lengths of the target sentences is not greater than the product of the original length of the context multiplied by the target compression rate;

[0027] Calculating the number of selectable word units that meet the target compression rate, and calculating the number of selectable second word units after removing the first word unit included in the target sentence;

[0028] Filter the second word whose first attention value reaches the first threshold to obtain a word-level filtering result and add it to the selection set;

[0029] The word-level filtering result and the sentence-level filtering result are combined to compress the context.

[0030] According to the method of the present invention, the step of performing Gaussian filtering on the plurality of word-units and the corresponding first attention values ​​comprises:

[0031] Use Gaussian filtering to allocate part of the target word's attention to adjacent words to optimize the attention distribution;

[0032] The step of combining the word-level filtering result and the sentence-level filtering result to compress the context comprises:

[0033] The word-level filtering result and the sentence-level filtering result are concatenated to obtain a final context compression result.

[0034] In a second aspect, an embodiment of the present invention provides a context compression device based on query guidance, the device comprising:

[0035] A template filling module, used to fill the query information and the context to be compressed into the dialogue template;

[0036] An attention calculation module, used to respectively calculate first attention values ​​of a plurality of word-units in the context of the dialogue template;

[0037] an attention reconstruction module, configured to reconstruct the attention of the context according to the first attention values ​​corresponding to the plurality of word-units;

[0038] The context compression module is used to compress the context after attention reconstruction to within a predetermined target compression rate.

[0039] In a third aspect, an embodiment of the present invention provides a storage medium for storing a computer program for executing any of the query-guided context compression methods described above.

[0040] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements any of the query-guided context compression methods described above when executing the computer program.

[0041] The present invention can overcome the problems in the prior art that the compression model and the generation model are difficult to align, the compression model has a large number of parameters, and the additional reasoning cost is high. A context compression technology that uses an attention mechanism to select context to retain important information is proposed, including: first, filling the query information and the context to be compressed into a dialogue template, then calculating the first attention values ​​of several word units in the context respectively, then reconstructing the context according to the first attention values ​​corresponding to the word units, and finally compressing the context based on the target compression rate. In this way, the present invention evaluates the importance of word units by using attention indicators instead of perplexity, evaluates the importance of word units by using the self-attention mechanism of the transformer, and selects contexts related to the current query. Because the attention mechanism provides a method for directly analyzing the relationship between the query and the context, it goes beyond the single reliance on the probability uncertainty of the model, and helps to accurately identify the most critical information of the current task, so that the present invention can more efficiently retain key information, input shorter contexts into the generation model, improve the reasoning efficiency of the generation model in the long context reasoning scenario, and reduce the reasoning cost.

[0042] More importantly, attention-based filtering can be implemented in a small large language model. Compared with the previous compression model that required tens of billions of parameters, the present invention can achieve better results using a small large language model with only 500 million parameters, thereby reducing the inference cost and improving computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of a context compression method based on query guidance provided in the first embodiment of the present invention;

[0044] Figure 2 is a flow chart of a context compression method based on query guidance provided in the second embodiment of the present invention;

[0045] Figure 3 It is a structural diagram of a context compression device based on query guidance provided in Embodiment 1 of the present invention;

[0046] Figure 4 is a schematic diagram of the structure of a context compression device based on query guidance provided in Embodiment 2 of the present invention;

[0047] Figure 5 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] It should be noted that references to "one embodiment", "embodiment", "example embodiment", etc. in this specification refer to the embodiment described, which may include specific features, structures or characteristics, but not every embodiment must include these specific features, structures or characteristics. In addition, such expressions do not refer to the same embodiment. Furthermore, when describing specific features, structures or characteristics in conjunction with an embodiment, whether or not there is an explicit description, it has been shown that it is within the knowledge of those skilled in the art to combine such features, structures or characteristics into other embodiments.

[0050] In addition, certain words are used in the specification and subsequent claims to refer to specific components or parts. Those with ordinary knowledge in the relevant field should understand that manufacturers can use different nouns or terms to refer to the same component or part. This specification and subsequent claims do not use differences in names as a way to distinguish components or parts, but use differences in the functions of components or parts as the criteria for distinction. "Including" and "including" mentioned throughout the specification and subsequent claims are open-ended terms and should be interpreted as "including but not limited to". In addition, the word "connected" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connection through other devices.

[0051] The query-guided context compression method provided by the embodiment of the present invention is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0052] Figure 1 : is a flow chart of a query-guided context compression method provided in Embodiment 1 of the present invention, the method comprising:

[0053] Step S101, filling the query information and the context to be compressed into the dialogue template.

[0054] Step S102, respectively calculating the first attention values ​​of a plurality of words in the context of the dialogue template.

[0055] Preferably, the plurality of tokens can be presented and saved in the form of a token list. The context and query information are embedded into the dialogue template, and finally end with a signal, which is the sentinel token. The sentinel token prompts the model to start generating a response, and a compression model of a multi-head self-attention mechanism is used to process the filled dialogue template, and the attention of the sentinel token to the previous text is calculated.

[0056] Step S103, reconstructing the context according to the first attention values ​​corresponding to the plurality of word-units.

[0057] Step S104, compressing the attention-reconstructed context to within a predetermined target compression rate.

[0058] The query-guided context compression method provided in Example 1 of the present invention can more efficiently retain more important information while compressing the context to meet the target compression rate, input shorter context into the generation model, and improve the reasoning efficiency of the generation model in long-context reasoning scenarios.

[0059] The query-guided context compression method provided by the embodiment of the present invention can overcome the problems of the difficulty in aligning the compression model with the generation model in the prior art, the large number of compression model parameters, and the high additional reasoning cost, and proposes a context compression technology that uses an attention mechanism to select context to retain important information, including: first filling the query information and the context to be compressed into the dialogue template, then calculating the first attention values ​​of several word units in the context respectively, then reconstructing the context according to the first attention values ​​corresponding to the word units, and finally compressing the context based on the target compression rate. In this way, the present invention evaluates the importance of word units by using attention indicators instead of perplexity, evaluates the importance of word units by using the self-attention mechanism of the transformer, and selects the context related to the current query. Because the attention mechanism provides a method for directly analyzing the relationship between the query and the context, it goes beyond the single reliance on the probability uncertainty of the model, and helps to accurately identify the most critical information of the current task, so that the present invention can more efficiently retain key information, input shorter contexts into the generation model, improve the reasoning efficiency of the generation model in the long context reasoning scenario, and reduce the reasoning cost.

[0060] Figure 2 : is a flow chart of a query-guided context compression method provided in Embodiment 2 of the present invention. This embodiment uses a dynamic filtering method as an example, and uses the compression method of the present invention to accelerate long-context reasoning, including:

[0061] Step S201, filling the query information and the context to be compressed into the dialogue template.

[0062] Preferably, after the query information and the context to be compressed are filled into the dialogue template, the last reminder of the dialogue template is to start answering the question, thereby obtaining a text sequence.

[0063] Step S202, setting the last word in the dialogue template as a sentinel word.

[0064] Preferably, a special sentinel word is defined using a dialogue template, and the sentinel word encapsulates the overall understanding of the task by the compression model. A compression model with a multi-head self-attention mechanism is used to generate the text sequence by reasoning.

[0065] Step S203, obtaining the first attention values ​​of several words in the context by calculating the attention distribution of the sentinel word.

[0066] Preferably, the plurality of word-grams may be presented and saved in the form of a word-gram list.

[0067] Preferably, this step can obtain an attention array whose length is the sum of the lengths of the dialogue template, the context, and the query information as the first attention value.

[0068] Preferably, the context and query information are embedded into the dialogue template and finally end with a signal, which is a sentinel word. The sentinel word prompts the model to start generating a response. A compression model of a multi-head self-attention mechanism is used to process the filled dialogue template and calculate the attention of the sentinel word to the previous text.

[0069] Preferably, the attention distribution of the sentinel word is calculated through the self-attention mechanism to obtain the first attention values ​​of several words in the context.

[0070] Step S204, reconstructing the context according to the first attention values ​​corresponding to the plurality of word-units.

[0071] Preferably, under the premise of ensuring that attention is not diluted by the dialogue template and query information, the first attention value corresponding to the word-unit is converted into a quantitative measure of the importance of each word-unit in the context, so as to achieve attention reconstruction of the context.

[0072] Preferably, before reconstructing the attention to the context, the following steps may also be included:

[0073] Normalize the first attention value corresponding to the first word list of the context. The purpose of normalization is to avoid diluting part of the attention array by the template and the question. Better, the attention of the context is normalized using the softmax function.

[0074] Step S205, performing Gaussian filtering on a plurality of word units and corresponding first attention values ​​to obtain a plurality of filtered word units and first attention values.

[0075] Step S205 is phrase-level filtering. Preferably, Gaussian filtering is used to allocate part of the attention of the target word to adjacent words to optimize the attention distribution, create a more continuous and comprehensive attention distribution, improve the model's ability to identify key information, and strengthen the overall expression of context information, ultimately achieving phrase-level context screening.

[0076] Step S206, respectively calculate the second attention values ​​corresponding to several sentences in the context, where the second attention value is equal to the sum of the filtered first attention values ​​of several word units contained in the several sentences.

[0077] Step S207, screening target sentences whose second attention values ​​reach the second threshold to obtain sentence-level filtering results, and the sum of the lengths of the target sentences is not greater than the product of the length of the original context multiplied by the target compression rate.

[0078] Preferably, target sentences with high attention values ​​(for example, the second threshold is the top 30% high scores) are selected, and the sum of the lengths of the selected sentences is not higher than the original context length multiplied by the target compression rate.

[0079] Steps S206 to S207 are sentence-level filtering steps. Preferably, semantic units are extracted at the sentence level to retain more comprehensive semantic information. Sentences with higher attention scores are prioritized for inclusion in the selection set, while ensuring that the total number of words meets the target compression rate.

[0080] Step S208, calculating the number of selectable word-grams that meet the target compression rate, and calculating the number of selectable second word-grams after removing the first word-gram included in the target sentence.

[0081] Step S209 , screening the second word-units whose first attention value reaches the first threshold to obtain word-unit-level filtering results and add them to the selection set.

[0082] The steps S208 to S209 are word-level filtering steps.

[0083] Step S210 , combining the word-level filtering result and the sentence-level filtering result to compress the context.

[0084] Preferably, the word-level filtering result and the sentence-level filtering result are concatenated to obtain a final context compression result.

[0085] The step S210 is a dynamic filtering step. Since sentence-level filtering often results in a compression rate higher than the target compression rate, in order to optimize budget utilization, the present invention combines the sentence-level filtering result with the word-level filtering result to form the final contextual compression output.

[0086] The above steps S205 to S209 are steps in which the context budget controller screens and filters the context.

[0087] The context compression method based on query guidance of the present invention mainly includes the following contents:

[0088] (1) Given the current query, multiple related contexts, and a target compression rate, the original context is compressed to filter out the context that is most useful for answering the current query, and the output context length is controlled within the target compression rate.

[0089] (2) Query-guided self-attention scoring. The importance of the word in the context is evaluated by calculating the attention distribution of the sentinel word, and the dynamic scoring of the importance of the word in the context is achieved. This includes: organizing all inputs by filling in the prompt template; using the trigger attention distribution to calculate the importance of all inputs; and obtaining the importance of the word in the context through context attention reconstruction.

[0090] (3) After obtaining the word list and the corresponding attention weight array, the context compression budget controller is used to filter the word list first, so as to control the length of the filtered context within the predetermined target compression rate while including as many important word lists as possible and retaining the semantic information.

[0091] The purpose of the present invention is to overcome the problems of the prior art that the compression model is difficult to align with the generation model, the large number of compression model parameters, and the high additional reasoning cost. A context compression method that uses an attention mechanism to select context to retain important information is proposed. Based on query-guided context compression, the self-attention mechanism of the transformer is used to evaluate the importance of word units and select contexts related to the current query. The present invention evaluates the importance of word units by using attention indicators rather than perplexity. Because the attention mechanism provides a method for directly analyzing the relationship between queries and contexts, it goes beyond the single reliance on the probability uncertainty of the model and helps to accurately identify the most critical information for the current task. More importantly, attention-based filtering can be implemented in a small large language model. Compared with the previous compression model that required tens of billions of parameters, the present invention can achieve better results using a small large language model with only 500 million parameters, thereby reducing the cost of reasoning and improving computational efficiency.

[0092] The following experiments are conducted in open-domain question answering and long-form question scenarios, and the results are shown in Table 1. The datasets used include NQ (Natural Questions) and ASQA (Answer Selection Question Answering). Accuracy is used as an evaluation metric on the NQ dataset to determine whether the generated response accurately contains the correct answer; for the ASQA dataset, the answers to the questions may be multifaceted because there are many ambiguous questions, and the evaluation indicators include exact matching (EM), question answering scores (DisambigF1) based on RoBERTa (Robustly Optimized BERT Method), and ROUGE (Recall-Oriented Understudy for GistingEvaluation, an auxiliary tool based on recall). Three state-of-the-art compression methods were selected as baselines: for methods that do not consider queries, Selective-Context and LLMLingua (an accelerated large language model opened by Microsoft) were selected, the latter of which implements cross-entropy scoring to remove redundant vocabulary. For methods that consider queries, the present invention is compared with LongLLMLingua. LongLLMLingua implements a two-stage compression approach. It first evaluates and re-ranks multiple retrieved contexts, followed by a token-level compression phase that assigns different compression budgets to these contexts based on their preliminary scores. For a fair comparison, the context re-ranking phase of LongLLMLingua (marked as ), focusing on word-level compression.

[0093] Table 1: Performance of various compression methods at different compression ratios

[0094]

[0095] Furthermore, the performance of the compression method of the present invention at different real context positions in the NQ dataset is analyzed. Each query in the dataset contains 20 context document fragments, only one of which contains the answer and is designated as the real document. The influence of the positioning of the document at the 1st, 5th, 10th, 15th and 20th positions on the effects of various compression strategies is evaluated, and the results are shown in Table 2.

[0096] Table 2: Performance of different ground truth context locations

[0097]

[0098] In addition, the performance of the compression method of the present invention in combination with different generation models is evaluated, including Longchat-13B-16k, Llama3-8b-Instruct, GLM4-9b-chat and Mistral-7b-instruct. These models are tested on the NQ dataset using a context with a 2x compression ratio, and the responses generated by these compressed contexts are compared with those generated by uncompressed contexts. The results are shown in Table 3.

[0099] Table 3: Performance of different generation models

[0100]

[0101] The results in Table 1, Table 2 and Table 3 show that the method adopted by the present invention can effectively extract more useful content for the query in the long context and accelerate the long context reasoning under the premise of using a smaller compression model.

[0102] It should be noted that the query-guided context compression method provided in the embodiment of the present invention can be executed by an electronic device, a query-guided context compression system, or a control module in the query-guided context compression system for executing the query-guided context compression method. In the embodiment of the present invention, the query-guided context compression system executing the query-guided context compression method is taken as an example to illustrate the query-guided context compression system provided in the embodiment of the present invention.

[0103] Figure 3 1 is a schematic diagram of the structure of a query-guided context compression device provided in Embodiment 1 of the present invention. The query-guided context compression device 100 includes a template filling module 10, an attention calculation module 20, an attention reconstruction module 30 and a context compression module 40, wherein:

[0104] The template filling module 10 is used to fill the query information and the context to be compressed into the dialogue template.

[0105] The attention calculation module 20 is used to respectively calculate the first attention values ​​of a plurality of word-units in the context of the dialogue template.

[0106] The attention reconstruction module 30 is used to reconstruct the attention of the context according to the first attention values ​​corresponding to a plurality of word-units.

[0107] The context compression module 40 is used to compress the context after attention reconstruction to within a predetermined target compression rate.

[0108] Figure 4is a structural diagram of a query-guided context compression device provided in Embodiment 2 of the present invention. The query-guided context compression device 100 includes a template filling module 10, an attention calculation module 20, an attention reconstruction module 30 and a context compression module 40, wherein:

[0109] The template filling module 10 is used to fill the query information and the context to be compressed into the dialogue template.

[0110] The attention calculation module 20 is used to respectively calculate the first attention values ​​of a plurality of word-units in the context of the dialogue template. Preferably, the attention calculation module 20 further comprises:

[0111] The sentinel setting submodule 21 is used to set the last word in the dialogue template as a sentinel word.

[0112] The attention calculation submodule 22 is used to obtain the first attention values ​​of several words in the context by calculating the attention distribution of the sentinel word. Preferably, the attention distribution of the sentinel word is calculated by a self-attention mechanism.

[0113] The attention reconstruction module 30 is used to reconstruct the context according to the first attention values ​​corresponding to a plurality of word-grams. Preferably, the first attention values ​​corresponding to the word-grams are converted into a quantitative measure of the importance of each word-gram in the context to reconstruct the context. Preferably, before reconstructing the context, the first attention values ​​corresponding to the first word-gram list of the context may be normalized.

[0114] The context compression module 40 is used to compress the context after attention reconstruction to within a predetermined target compression rate. Preferably, the context compression module 40 further includes:

[0115] The filtering processing submodule 41 is used to perform Gaussian filtering on a plurality of word units and corresponding first attention values ​​to obtain a plurality of filtered word units and first attention values.

[0116] The first calculation submodule 42 is used to calculate the second attention values ​​corresponding to the plurality of sentences in the context, respectively, the second attention value being equal to the sum of the filtered first attention values ​​of the plurality of word-units contained in the plurality of sentences. Preferably, Gaussian filtering is used to allocate part of the attention of the target word-unit to the adjacent word-units to optimize the attention distribution.

[0117] The first screening submodule 43 is used to screen the target sentences whose second attention value reaches the second threshold to obtain sentence-level filtering results, and the sum of the lengths of the target sentences is not greater than the product of the length of the original context multiplied by the target compression rate.

[0118] The second calculation submodule 44 is used to calculate the number of selectable word-grams that meet the target compression rate, and calculate the number of selectable second word-grams after removing the first word-gram included in the target sentence.

[0119] The second screening submodule 45 is used to screen the second word-units whose first attention value reaches the first threshold value to obtain word-unit-level filtering results and add them to the selection set.

[0120] The dynamic filtering submodule 46 is used to combine the word-level filtering result and the sentence-level filtering result to compress the context. Preferably, the word-level filtering result and the sentence-level filtering result are spliced ​​to obtain the final context compression result.

[0121] The query-guided context compression system provided by the embodiment of the present invention can achieve Figures 1-2 To avoid repetition, the various processes implemented in the query-guided context compression method embodiment are not described here in detail.

[0122] The query-guided context compression system provided by the embodiment of the present invention can overcome the problems of the difficulty in aligning the compression model with the generation model in the prior art, the large number of compression model parameters, and the high additional reasoning cost, and proposes a context compression technology that uses an attention mechanism to select context to retain important information, including: first filling the query information and the context to be compressed into the dialogue template, then calculating the first attention values ​​of several word units in the context respectively, then reconstructing the context according to the first attention values ​​corresponding to the word units, and finally compressing the context based on the target compression rate. In this way, the present invention evaluates the importance of word units by using attention indicators instead of perplexity, evaluates the importance of word units by using the self-attention mechanism of the transformer, and selects the context related to the current query. Because the attention mechanism provides a method for directly analyzing the relationship between the query and the context, it goes beyond the single reliance on the probability uncertainty of the model, and helps to accurately identify the most critical information of the current task, so that the present invention can more efficiently retain key information, input shorter contexts into the generation model, improve the reasoning efficiency of the generation model in the long context reasoning scenario, and reduce the reasoning cost.

[0123] The present invention also provides a storage medium for storing Figure 1-2A computer program of any of the query-guided context compression methods. For example, a computer program instruction, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer, and can achieve the same technical effect. To avoid repetition, it will not be repeated here. The program instruction for calling the method of the present invention may be stored in a fixed or removable storage medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium and / or stored in a storage medium of a computer device that runs according to the program instruction.

[0124] According to one embodiment of the present invention, the present invention also provides a Figure 5 The electronic device 400 shown, the electronic device 400 optionally includes a storage medium 200 for storing a computer program and a processor 300 for executing the computer program, wherein when the computer program is executed by the processor 300, any of the above-mentioned context compression methods based on query guidance is implemented, triggering the electronic device 400 to execute the methods and / or technical solutions based on the aforementioned multiple embodiments, and can achieve the same technical effect, to avoid repetition, it is not repeated here. It should be noted that the electronic devices in the embodiments of the present invention include mobile electronic devices and non-mobile electronic devices. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, a super mobile personal computer, a netbook or a personal digital assistant, etc., and the non-mobile electronic device can be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiments of the present invention.

[0125] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present invention (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and the like. In addition, some steps or functions of the present invention can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0126] The present invention can be implemented on a computer as a computer-implemented method, or implemented in dedicated hardware, or implemented in a combination of the two. The executable code or part thereof for the method according to the present invention can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Optionally, the computer program product includes a non-temporary program code component stored on a computer-readable medium so as to perform the method according to the present invention when the program product is executed on a computer.

[0127] In an alternative embodiment, the computer program comprises computer program code means adapted to perform all the steps of the method according to the invention when the computer program is run on a computer. Optionally, the computer program is embodied on a computer readable medium.

[0128] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0129] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A query-guided context compression method, characterized in that: include: Fill the query information and the context to be compressed into the dialogue template; Calculating first attention values ​​of a plurality of words in the context of the dialogue template respectively; Reconstructing the context according to the first attention values ​​corresponding to the plurality of word-units; The context after attention reconstruction is compressed to within a predetermined target compression rate.

2. The method according to claim 1, characterized in that The step of respectively calculating the first attention values ​​of a plurality of first word units in the context of the dialogue template comprises: Setting the last token in the dialogue template as a sentinel token; The first attention values ​​of the plurality of words in the context are obtained by calculating the attention distribution of the sentinel word.

3. The method according to claim 1, characterized in that The step of calculating the attention distribution of the sentinel word comprises: The attention distribution of the sentinel word is calculated through a self-attention mechanism.

4. The method according to claim 1, characterized in that The step of reconstructing the context according to the first attention value corresponding to the first word-gram list comprises: The first attention value corresponding to the word-gram is converted into a quantitative measure of the importance of each word-gram in the context.

5. The method according to claim 4, characterized in that The step of converting the first attention value corresponding to the first word-gram list into a quantitative measure of the importance of each word-gram in the context includes: The first attention value corresponding to the first word list of the context is normalized.

6. The method according to claim 1, characterized in that The step of compressing the context after attention reconstruction to within a predetermined target compression rate comprises: Performing Gaussian filtering on the plurality of word-units and the corresponding first attention values ​​to obtain the plurality of word-units and the first attention values ​​after filtering; Calculating second attention values ​​corresponding to a plurality of sentences in the context respectively, the second attention value being equal to the sum of the first attention values ​​of a plurality of word-units contained in the plurality of sentences after filtering; Filtering target sentences whose second attention value reaches a second threshold to obtain sentence-level filtering results, and the sum of the lengths of the target sentences is not greater than the product of the original length of the context multiplied by the target compression rate; Calculating the number of selectable word units that meet the target compression rate, and calculating the number of selectable second word units after removing the first word unit included in the target sentence; Filter the second word whose first attention value reaches the first threshold to obtain a word-level filtering result and add it to the selection set; The word-level filtering result and the sentence-level filtering result are combined to compress the context.

7. The method according to claim 6, characterized in that The step of performing Gaussian filtering on the plurality of word units and the corresponding first attention values ​​comprises: Gaussian filtering is used to allocate part of the target word's attention to adjacent words to optimize the attention distribution. The step of combining the word-level filtering result and the sentence-level filtering result to compress the context comprises: The word-level filtering result and the sentence-level filtering result are concatenated to obtain a final context compression result.

8. A query-guided context compression device constructed based on the method according to any one of claims 1 to 7, characterized in that: include: A template filling module, used to fill the query information and the context to be compressed into the dialogue template; An attention calculation module, used to respectively calculate first attention values ​​of a plurality of word-units in the context of the dialogue template; an attention reconstruction module, configured to reconstruct the attention of the context according to the first attention values ​​corresponding to the plurality of word-units; The context compression module is used to compress the context after attention reconstruction to within a predetermined target compression rate.

9. A storage medium, characterized in that: Used to store a computer program for executing the query-guided context compression method according to any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the query-guided context compression method according to any one of claims 1 to 7 is implemented.

Images