A text summary generation method and system based on sparse attention acceleration

Abstract

This invention discloses a text summarization method and system based on sparse attention acceleration, comprising: reading long text data, performing word segmentation and embedding encoding processing, extracting sequence feature vectors, and mapping them to a query matrix Q, a key matrix K, and a value matrix V; constructing a summarization generation network, including a sparse attention calculation module, a feedforward calculation module, a prediction head module, and a key-value caching module; inputting the query matrix Q, key matrix K, and value matrix V into the summarization generation network, first passing through a backbone network composed of multiple stacks of the sparse attention calculation module and the feedforward calculation module, and then being processed by the prediction head module, while the key-value caching module caches historical decoding states in real time to obtain an initial text feature vector; based on the initial text feature vector, performing an autoregressive decoding process through the summarization generation network to output the final summary result. This invention can accelerate the decoding process and ensure the semantic integrity and coherence of the generated summary.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and text generation technology, and in particular relates to a text summarization method and system based on sparse attention acceleration. Background Technology

[0002] With the rapid development of the internet, digital office systems, and various information systems, the volume of text data, including news reports, industry reports, academic documents, and legal documents, is growing exponentially. How to quickly and accurately extract core viewpoints and key information from these lengthy texts has become a critical problem that urgently needs to be solved in the field of natural language processing. Early automatic summarization techniques mostly adopted extraction methods, directly filtering sentences from the original text through statistical features or graphical models. While preserving factual accuracy, the generated summaries often lacked semantic coherence. Subsequent generative methods based on recurrent neural networks, although attempting to simulate human inductive abilities, are prone to gradient vanishing and information forgetting problems when processing long sequences, making it difficult to generate high-quality long text summaries.

[0003] In recent years, pre-trained language models based on the Transformer architecture, such as GPT, Llama, and Qwen, have become the mainstream choice for text summarization tasks. These models, relying on powerful self-attention mechanisms, can effectively capture long-distance dependencies in text, resulting in significantly improved fluency, logical coherence, and semantic accuracy in the generated summaries. However, while the Transformer network brings a performance leap, it also introduces huge computational overhead. The computational complexity and memory usage of its core self-attention mechanism are proportional to the square of the input sequence length. When processing long documents of thousands or even tens of thousands of words, the model faces a severe computational bottleneck during the inference phase.

[0004] To alleviate this computational bottleneck in Transformer networks, existing techniques typically introduce sparse attention mechanisms as an optimization method. Currently, the main sparse attention mechanisms can be divided into two categories: static sparse attention mechanisms and dynamic sparse attention mechanisms. Static sparse attention mechanisms employ a predefined sparse connection pattern that remains fixed during both training and inference phases, significantly reducing the computational complexity of attention. Dynamic sparse attention patterns, on the other hand, adaptively determine the connections between tokens based on the input content, better reflecting the actual data distribution and aiming to efficiently approximate full attention capabilities with minimal performance loss.

[0005] However, existing sparse attention mechanisms still face many limitations in practical applications. Static sparse attention mechanisms are limited by fixed connection patterns and cannot flexibly adjust their focus based on the semantic characteristics of the input data. This easily leads to the omission of global information that is outside the sliding window but is crucial for understanding the whole text during the summarization process. While existing dynamic sparse attention mechanisms aim to improve adaptability, they often have limitations: some methods (such as NSA, MoBA, and SeerAttention) rely on expensive re-pre-training or fine-tuning; others that do not require training (such as FlexPrefill and Minference) are insufficient in their ability to capture global semantic relationships. Therefore, how to achieve an efficient and content-adaptive sparse mechanism without re-training, especially how to accurately index key information in long texts during the inference stage, has become a key technical challenge that urgently needs to be solved to successfully apply sparse attention to text summarization tasks and overcome the bottleneck of inference speed. Summary of the Invention

[0006] To overcome the problem of low inference efficiency in long text summarization in existing technologies, this invention provides a text summarization method and system based on sparse attention acceleration, which can accelerate the decoding process without modifying the weights of the pre-trained model, and ensure the semantic integrity and coherence of the generated summary.

[0007] A text summarization method based on sparse attention acceleration includes the following steps:

[0008] (1) Read the long text data to be processed, perform word segmentation and embedding encoding processing, extract the sequence feature vector of the text, and map it into query matrix Q, key matrix K and value matrix V;

[0009] (2) Construct a summary generation network, including: a sparse attention computation module, a feedforward computation module, a prediction head module, and a key-value caching module;

[0010] (3) Input the query matrix Q, key matrix K and value matrix V into the summary generation network. First, it passes through the backbone network composed of multiple stacks of sparse attention calculation module and feedforward calculation module, and is processed by the prediction head module to extract the global and local contextual semantics of the text. At the same time, the historical decoding state is cached in real time through the key value caching module to obtain the initial text feature vector.

[0011] (4) Based on the initial text feature vector, the summarization network performs an autoregressive decoding process. In each decoding step, the summarization network dynamically calculates the predicted distribution of the next word based on the currently generated summary fragment and its cache, and iteratively updates the key value cache module until a complete end marker is generated and the final summary result is output.

[0012] In step (1), the embedding encoding process includes: converting the word segment ID sequence after word segmentation into a dense vector sequence through a word embedding layer, and superimposing position encoding information to obtain a sequence feature vector.

[0013] In step (2), the sparse attention calculation module performs the following steps:

[0014] (2-1) Logically divide the query matrix Q and the key matrix K into blocks, and perform pooling reduction on the blocks to generate a low-dimensional representative query matrix and representative key matrix;

[0015] (2-2) Based on the representative query matrix and the representative key matrix, calculate the block-level attention score, and dynamically generate a combined scheduling mask according to the block-level attention score;

[0016] (2-3) Based on the combined scheduling mask, the control hardware computing unit only loads and computes the key-value data blocks indicated by the mask that are highly relevant to the current query, skips invalid computing blocks, thereby performing sparse attention computation and outputting attention results.

[0017] In step (2-1), the pooling reduction operation uses mean pooling.

[0018] In step (2-2), the specific process of logical combination is as follows:

[0019] First, based on the block-level attention score, the top-p method is used to obtain the block-level adaptive mask, and the top-k method is used to obtain the vertical binary mask;

[0020] Subsequently, the vertical binary mask, the block-level adaptive mask, and the sliding window mask are logically ORed to obtain an initial candidate mask;

[0021] Finally, the initial candidate mask and the causal mask are logically ANDed to ensure that all retained computational blocks conform to the causal time constraints of text generation, thus obtaining the final combinatorial scheduling mask.

[0022] In step (2), the feedforward computation module consists of one or more fully connected layers and corresponding nonlinear activation functions. It is used to perform nonlinear transformation, dimension mapping, and information integration on the feature representation output by the sparse attention computation module to obtain a high-dimensional semantic feature vector. Structurally, this module can adopt a standard fully connected feedforward neural network or a more complex architecture such as a hybrid expert network. The hybrid expert network dynamically activates some expert subnetworks through a routing mechanism, which can significantly improve the model capacity and adaptability while maintaining computational efficiency.

[0023] In step (2), the prediction head module consists of one or more linear transformation layers, which are used to map the high-dimensional semantic feature vector output by the feedforward calculation module to a dimension that matches the size of the target vocabulary and transform it into a probability distribution on the vocabulary space, thereby providing a prediction basis for the selection of lexical units in the autoregressive decoding process.

[0024] In step (2), the key-value caching module is used to cache and reuse the intermediate calculation results of the key matrix K and value matrix V corresponding to the historical lexical in each decoding step of the autoregressive decoding process; in the next decoding step, the sparse attention calculation module directly reads the cached key and value data blocks for calculation to eliminate repeated calculations across time steps.

[0025] Step (3) is the pre-filling stage, where the network processes the entire source text sequence. At this stage, the key-value caching module initializes and stores the K and V vectors corresponding to the source text. The sparse attention computation module primarily performs efficient encoding of long source documents, and the mask it generates acts on the self-attention computation within the source text, significantly reducing computational overhead.

[0026] Step (4) is the autoregressive decoding stage. The network iteratively generates each word in the summary, starting from the start symbol. In each decoding step: First, the sparse attention calculation module dynamically calculates and generates the combined scheduling mask for the current step based on the cached summary history K and V in the key-value cache module and the K and V of the source document; then, it uses this mask to perform sparse attention calculation and aggregate context information; then, the result is passed through the feedforward calculation module and the prediction head module to obtain the predicted probability distribution of the next word; finally, the next word is sampled or selected according to the prediction distribution, and its corresponding new K and V vectors are updated in the key-value cache module to prepare for the next decoding step. This loop continues until the end symbol is predicted, forming a complete summary sequence.

[0027] A text summarization generation system based on sparse attention acceleration includes a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they implement the aforementioned text summarization generation method based on sparse attention acceleration.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. The method of this invention can be directly and seamlessly integrated into the inference stage of existing pre-trained summarization models based on the Transformer architecture, without the cost of expensive fine-tuning or retraining. For long-sequence summarization scenarios such as long news articles, academic documents, or legal documents, dynamic sparse scheduling significantly reduces the time complexity of inference, effectively solving the problems of high latency and slow response of traditional models when generating long text summaries, and greatly improving the real-time performance of the system.

[0030] 2. This invention introduces a representative attention score matrix to accurately locate and select the global key blocks that have the greatest influence on the currently generated content. This is combined with local sliding window logic and an exclusionary Top-P strategy. This hybrid protection mechanism of "key blocks + local context + medium-to-long-range logical connections" ensures that when generating summaries, the model can maintain the syntactic coherence of local sentences using a sliding window, while minimizing the loss of historical key information crucial to the current summary topic (such as core facts in long reports or key arguments in literature) across long distances. Thus, while achieving high sparsity to significantly reduce latency, it effectively captures global semantic relationships, avoiding the common problems of "taking things out of context" or missing key information in long text summarization. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of a text summarization method based on sparse attention acceleration according to an embodiment of the present invention.

[0033] Figure 2 This is a flowchart illustrating the sparse attention calculation module in an embodiment of the present invention.

[0034] Figure 3 The attention mask selected using only the top-p method.

[0035] Figure 4 The attention mask selected for using this method. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0038] like Figure 1As shown, a text summarization generation method based on sparse attention acceleration includes the following steps:

[0039] Step S1: Read the long text data to be processed, perform word segmentation and embedding encoding processing on it, extract the sequence feature vector of the text, and map it into query matrix Q, key matrix K and value matrix V.

[0040] Specifically, a long text to be summarized is read. First, the input text is segmented using a tokenizer (such as the BPE tokenizer) that is identical to the underlying pre-trained model (e.g., Qwen2-7B-Instruct), resulting in a sequence of token IDs. Then, this sequence is input into the model's embedding layer, mapping each token ID to a high-dimensional dense vector and superimposing sinusoidal or rotational positional encodings to form an initial feature representation H0 containing sequence order information. Finally, H0 is passed through the linear projection layer in the first layer of the Transformer decoder to calculate the query matrix Q, key matrix K, and value matrix V required for the attention mechanism of that layer. For subsequent decoding layers, the input is the hidden state output from the previous layer, similarly projected into new Q, K, and V.

[0041] Step S2: Construct a dynamic summary generation network that includes a sparse attention computation module, a feedforward computation module, a prediction head module, and a key-value caching module.

[0042] The network core of this invention inherits from a standard pre-trained Transformer decoder (e.g., the Llama 2 decoder architecture) and contains multiple identical layers. Each layer mainly consists of a sparse attention computation module, a feedforward computation module, residual connections, and layer normalization. The core improvement of this invention lies in replacing the original standard multi-head attention mechanism with the aforementioned sparse attention computation module. Furthermore, the network includes a top prediction head module (linear layer + Softmax) and a key-value caching module that continuously operates during inference. The network employs an autoregressive approach during inference, predicting the next term based on the already generated summary terms.

[0043] like Figure 2 As shown, the sparse attention computation module takes the query matrix Q, key matrix K, and value matrix V as input, and achieves dynamic sparse scheduling and accelerated computation through the following steps:

[0044] (2-1) Logically divide the query matrix Q and the key matrix K into blocks, and perform pooling reduction on the blocks to generate a low-dimensional representative query matrix and representative key matrix. The specific implementation is as follows:

[0045] (1) Set the query block step size Bq and the key block step size Bk aligned with the hardware computing unit, and logically divide the input query matrix Q and key matrix K into blocks according to the step sizes Bq and Bk;

[0046] Specifically, the chunk size Bq is set to an integer multiple of the query chunk size in the underlying improved FlashAttention computation kernel to achieve alignment of computational granularity. Mean reduction is chosen instead of summation or extreme value reduction to avoid amplifying the numerical variance within chunks, ensuring a more balanced distribution of the attention score matrix for subsequent chunks. This allows for a more stable assessment of the importance of different document chunks to the currently generated content in long document summarization tasks, preventing individual highlighted words or sentence fragments from excessively dominating global attention and causing the summary to miss other key paragraphs.

[0047] (2) Perform intra-block pooling reduction operations on the block-based query matrix Q and key matrix K respectively to generate the dimensionality-reduced representative query matrix Q_pool and representative key matrix K_pool;

[0048] (2-2) Based on the representative query matrix and representative key matrix, calculate the block-level attention score, and dynamically generate a combined scheduling mask according to the block-level attention score. The specific implementation is as follows:

[0049] (1) Calculate the product of the representative query matrix Q_pool and the representative key matrix K_pool to obtain the block-level coarse-grained attention score matrix;

[0050] (2) Based on the current context, select the last block from the representative query matrix Q_pool as the anchor block, calculate the correlation between the anchor block and the key matrix K, filter out the key block positions of the K key elements with the highest correlation, and generate the corresponding vertical binary mask;

[0051] Specifically, such as Figure 3 As shown, in the pre-filling stage of existing Transformer model inference, the attention score matrix in some Transformer layers exhibits a clear vertical stripe distribution pattern, meaning that most summary generation models tend to repeatedly focus on the fixed positions of a few core entities, events, or conclusions in the original text. This vertical concentration leads to a few key blocks becoming high-score outliers, thus interfering with the statistical distribution-based filtering process, making it difficult to generate a balanced, accurate, and stable block sparse attention mask in step (2-2). Figure 3 (The part highlighted in red).

[0052] Because this pattern exhibits concentration along the key dimension, this invention only requires calculation using the last query block and all key vectors to identify key vectors that have significantly high attention scores with all query blocks that meet the causal conditions. Specifically, by performing Top-K filtering along the key dimension on the attention scores of the last query block and each key vector, these key vectors with high global influence can be efficiently located. To preserve their important information, the entire key block containing these key vectors needs to be marked as a protected region, and a block-level vertical mask is generated accordingly. This ensures that these key positions are not filtered out during subsequent sparsification, and that the generated summary does not omit the most core facts and arguments of the document, thus guaranteeing the fidelity and completeness of the summary from a mechanistic perspective.

[0053] (3) For the coarse-grained attention score matrix, after excluding the columns already covered by the vertical binary mask and the elements in the local sliding window, normalize the remaining scores of each row to obtain the probability distribution; select the smallest set of attention blocks with cumulative probabilities not exceeding the filtering threshold P for each row to generate a block-level adaptive mask.

[0054] (4) Logically combine the vertical binary mask, block-level adaptive mask, preset causal mask and sliding window mask to obtain the final binary combined scheduling mask;

[0055] Specifically, in step (3) of step (2-2), firstly, two regions are excluded from the block attention score matrix: one is the key block column already covered by the vertical mask generated in step (2) of step (2-2); the other is the adjacent key block column covered by a preset local sliding window (used to preserve local context). The scores of these two regions are set to negative infinity. Then, for the remaining part of the matrix, Softmax normalization is performed independently in each row (each query block) to form a probability distribution. Next, a Top-P filtering operation is performed on each row: retaining the key block positions corresponding to the cumulative probability and all probability values ​​before the first exceeding a preset threshold P (e.g., 0.9). Based on this filtering result, an initial block-level adaptive mask matrix is ​​generated.

[0056] Specifically, steps (2), (3), and (4) in (2-2) aim to adaptively filter out the most critical inter-block connections from the block-level coarse-grained attention score matrix obtained in step (1) to generate the final sparse attention pattern. The key blocks corresponding to the key block column of the vertical mask in step (2) of step (2-2) have high global influence, and their attention scores are usually significantly higher than other regions. If they directly participate in the filtering, they will excessively dominate the Top-P selection result, causing a large number of ordinary but still valuable connections to be masked. The adjacent key block columns covered by the sliding window are also excluded. Specifically, the scores of the above two regions are temporarily set to negative infinity so that they do not enter the subsequent competitive filtering process. The core purpose of this operation is to enable the Top-P mechanism to more fairly and sensitively identify those medium- and long-range dependencies that are still important besides global key blocks and local neighboring blocks in the remaining attention distribution, thereby generating a more comprehensive and accurate block sparse attention mask. Figure 3 and Figure 4 As shown in the red highlighted sections, under the premise of the same Top-P threshold (P value), this method ( Figure 4 The masks selected by this method are better than those obtained by using the Top-P method alone. Figure 3 It covers a more diverse and reasonable contextual connection, thus enabling better integration of global core, local coherence and mid-range relevant information in summary generation.

[0057] (2-3) Based on the combined scheduling mask, the control hardware computing unit only loads and computes the key-value data blocks indicated by the mask that are highly relevant to the current query, skipping invalid computing blocks, thereby performing sparse attention computation and outputting the attention result. The specific implementation is as follows:

[0058] Based on the combined scheduling mask generated in step (2-2), a kernel function based on FlashAttention is invoked to perform precise sparse attention computation: for each query block, only the original K, V data blocks whose mask indications require computation are loaded and computed, completely skipping block pairs with a mask value of 0. Finally, the outputs of all computation blocks are aggregated to obtain the computation result of the current attention layer.

[0059] Specifically, in some embodiments, the feedforward computation module employs a feedforward network structure consistent with the decoder layer of the selected pre-trained model (such as the Llama 3 series). This module performs nonlinear transformations and deep integration on the context-aggregated feature representation output by the sparse attention computation module. Its typical structure includes two fully connected layers and a gated activation function (such as SwiGLU), where the hidden layer dimension is significantly higher than the input / output dimension to enhance the model's expressive power. In text summarization tasks, this module is responsible for deep processing and fusion of focused key information, providing stronger feature representation capabilities for generating fluent and coherent summary text.

[0060] Specifically, in some embodiments, the prediction head module consists of a linear projection layer whose weights are typically shared with the model's input word embedding layer. This module maps the semantically rich high-dimensional feature vector output from the last layer of the decoder to the target vocabulary space and transforms it into a probability distribution for the next word using a softmax function. At each step of the autoregressive summary generation, this module predicts the most likely next summary word based on the currently generated summary prefix and the dynamically focused context from the long original text, thereby driving the progressive generation of the entire summary text.

[0061] Specifically, in some embodiments, the key-value caching module is a continuously maintained dynamic storage structure used to cache the key (K) vectors and value (V) vectors generated in all steps prior to the current decoding step (including the pre-filled source text sequence and the generated summary lexical sequence). Its organization is strictly aligned with the block-based logic of the aforementioned sparse attention computation module; that is, K and V vectors are stored and managed according to their respective block indices. This aligned caching organization supports efficient querying and incremental updates of the required K and V block data in subsequent decoding steps based on dynamically generated block-level scheduling masks, which is crucial for achieving efficient autoregressive decoding. In long text summarization scenarios, this mechanism ensures that when the model generates subsequent parts of the summary, it can backtrack to the identified key information blocks in the preceding text (including the long input original text) with extremely low overhead, effectively maintaining the global consistency and information integrity of the summary.

[0062] Step S3: Autoregressive decoding generates a summary.

[0063] The initial Q, K, and V vectors (for the first decoding layer) obtained in step S1 are input into the network. When generating the first summary term, the model performs "pre-padded" computation on the input long text. During this stage, the key-value caching module stores the K and V vectors for all input positions (organized in blocks). Subsequently, the sparse attention computation module begins its work, outputting an aggregated context vector. After transformation by the feedforward network, the probability distribution of the first term is generated by the prediction head module, and the first term is obtained through sampling (e.g., greedy or bundle search).

[0064] Starting from the second term, the autoregressive decoding stage begins. At each step, only the features of the newly generated term are input into the network. The key-value caching module provides the K and V vectors of all previous steps (including pre-filled source text and generated summary terms). The sparse attention calculation module dynamically calculates the sparse attention based on the current query and historical key-value pairs to obtain the final attention score. The output is used to predict the next term, while the newly generated K and V vectors from the current step are incrementally stored in the cache. This process repeats until an end marker is generated, forming a complete summary.

[0065] Based on the same inventive principle, this embodiment also provides a text summarization generation system based on sparse attention acceleration, including a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-described text summarization generation method based on sparse attention acceleration.

[0066] To verify the technical effectiveness of this invention, a text summarization generation system based on this invention was constructed in a specific experiment. The open-source large model Llama3.1-8B-Instruct weights were loaded, and the English long text summarization task (En.sum) on the InfiniteBench dataset was used for evaluation. This task requires processing book texts with an average length of approximately 171.5K tokens and generating summaries. Under the same experimental environment, compared with the original model using the standard global attention mechanism, the system integrating the sparse attention acceleration mechanism of this invention shows significant advantages under long text input: With the same input, the time consumed in the pre-filling stage of the inference phase decreased from 360.74 seconds in the original method to 187.68 seconds, an improvement of approximately 92% in processing speed; in terms of summary generation quality, the ROUGE-L score of this invention is 32.99, reaching 101.1% of the original method's score (32.63), maintaining comparable generation quality to the original model while significantly improving processing efficiency.

[0067] The embodiments described above provide a detailed explanation of the technical solutions and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A text summarization generation method based on sparse attention acceleration, characterized in that, Includes the following steps: (1) Read the long text data to be processed, perform word segmentation and embedding encoding processing, extract the sequence feature vector of the text, and map it into query matrix Q, key matrix K and value matrix V; (2) Construct a summary generation network, including: a sparse attention computation module, a feedforward computation module, a prediction head module, and a key-value caching module; (3) Input the query matrix Q, key matrix K and value matrix V into the summary generation network. First, it passes through the backbone network composed of multiple stacks of sparse attention calculation module and feedforward calculation module, and is processed by the prediction head module to extract the global and local contextual semantics of the text. At the same time, the historical decoding state is cached in real time through the key value caching module to obtain the initial text feature vector. (4) Based on the initial text feature vector, the summarization network performs an autoregressive decoding process. In each decoding step, the summarization network dynamically calculates the predicted distribution of the next word based on the currently generated summary fragment and its cache, and iteratively updates the key value cache module until a complete end marker is generated and the final summary result is output.

2. The text summarization method based on sparse attention acceleration according to claim 1, characterized in that, In step (1), the embedding encoding process includes: converting the word segment ID sequence after word segmentation into a dense vector sequence through a word embedding layer, and superimposing position encoding information to obtain a sequence feature vector.

3. The text summarization method based on sparse attention acceleration according to claim 1, characterized in that, The sparse attention calculation module performs the following steps: (2-1) Logically divide the query matrix Q and the key matrix K into blocks, and perform pooling reduction on the blocks to generate a low-dimensional representative query matrix and representative key matrix; (2-2) Based on the representative query matrix and the representative key matrix, calculate the block-level attention score, and dynamically generate a combined scheduling mask according to the block-level attention score; (2-3) Based on the combined scheduling mask, the control hardware computing unit only loads and computes the key-value data blocks indicated by the mask that are highly relevant to the current query, skips invalid computing blocks, thereby performing sparse attention computation and outputting attention results.

4. The text summarization method based on sparse attention acceleration according to claim 3, characterized in that, In step (2-1), the pooling reduction operation uses mean pooling.

5. The text summarization method based on sparse attention acceleration according to claim 3, characterized in that, In step (2-2), the specific process of generating the combined scheduling mask is as follows: First, based on the block-level attention score, the top-p method is used to obtain the block-level adaptive mask, and the top-k method is used to create the vertical binary mask; Subsequently, the vertical binary mask, the block-level adaptive mask, and the sliding window mask are logically ORed to obtain an initial candidate mask; Finally, the initial candidate mask and the causal mask are logically ANDed to ensure that all retained computational blocks conform to the causal time constraints of text generation, thus obtaining the final combinatorial scheduling mask.

6. The text summarization method based on sparse attention acceleration according to claim 1, characterized in that, In step (2), the feedforward computation module consists of one or more fully connected layers and corresponding nonlinear activation functions, which are used to perform nonlinear transformation, dimension mapping and information integration on the feature representation output by the sparse attention computation module to obtain a high-dimensional semantic feature vector.

7. The text summarization generation method based on sparse attention acceleration according to claim 1, characterized in that, In step (2), the prediction head module consists of one or more linear transformation layers, which are used to map the high-dimensional semantic feature vector output by the feedforward calculation module to a dimension that matches the size of the target vocabulary and transform it into a probability distribution on the vocabulary space.

8. The text summarization method based on sparse attention acceleration according to claim 1, characterized in that, In step (2), the key-value caching module is used to cache and reuse the intermediate calculation results of the key matrix K and value matrix V corresponding to the historical tokens in each decoding step of the autoregressive decoding process; In the next decoding step, the sparse attention calculation module directly reads the cached key-value data blocks for calculation to eliminate repeated calculations across time steps.

9. A text summarization generation system based on sparse attention acceleration, characterized in that, The method includes a memory and one or more processors, wherein the memory stores executable code, and the one or more processors execute the executable code to implement the text summarization generation method based on sparse attention acceleration as described in any one of claims 1-8.

Images