Multi-round dialogue processing method based on block perception dynamic attention
By dividing multi-turn dialogues into semantic sub-blocks and dynamically allocating attention resources, the computational efficiency and generation quality issues of large language models in long-context multi-turn dialogues are solved, achieving efficient and accurate dialogue processing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-09
- Publication Date
- 2026-04-21
AI Technical Summary
Existing large language models suffer from low computational efficiency and high memory consumption when processing long-context, multi-turn dialogues. Furthermore, traditional attention mechanisms cannot effectively perceive the semantic block structure of the dialogue, leading to a decline in generation quality.
By dividing multi-turn dialogues into semantic sub-blocks and dynamically allocating attention resources based on comprehensive importance scores, hard pruning, sparse attention, or full attention strategies are adopted to optimize the allocation of computational resources.
It significantly improves the reasoning speed and generation accuracy of long-context multi-turn dialogues, reduces computational costs and memory usage, while maintaining the generation quality of short dialogues.
Smart Images
Figure CN121901375A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a multi-turn dialogue processing method based on block-aware dynamic attention, belonging to the fields of natural language processing and artificial intelligence technology. Background Technology
[0002] With the rapid development of large language models, multi-turn dialogue, as a core method of human-computer interaction, is increasingly widely used. However, as the number of dialogue turns increases and the context sequence becomes longer, the model faces significant computational efficiency challenges. The computational complexity of traditional attention mechanisms (such as the full attention mechanism of Transformer) is proportional to the square of the sequence length, resulting in slow inference speed and high memory consumption when processing long dialogues, making it difficult to meet the needs of real-time interaction.
[0003] Existing technologies attempt to address this issue, mainly falling into two categories: First, model-free fine-tuning methods (such as retrieval augmentation and cue compression). These methods rely on external knowledge or engineering skills and lack versatility in diverse dynamic dialogue scenarios. Second, attention-based fine-tuning methods (such as long context fine-tuning and sparse attention). While these methods can directly optimize the model, they suffer from low computational efficiency and require a large amount of fine-tuning data. Specific sparse attention schemes, such as FlashAttention (efficient and accurate attention computation) and InfLLM (long context extrapolation without training), improve the efficiency of long sequence processing, but both treat the dialogue context as a flat sequence of tokens, ignoring the inherent, turn-based semantic block structure of multi-turn dialogues. This "one-size-fits-all" approach fails to perceive dialogue characteristics such as topic shifts and attention jumps, potentially leading to the neglect of key information or the introduction of noise, thus making it difficult to maintain or improve generation quality while improving efficiency. Summary of the Invention
[0004] The purpose of this invention is to provide a multi-turn dialogue processing method based on block-aware dynamic attention. This invention significantly improves the inference speed and generation accuracy of large language models when processing long-context, multi-turn dialogues by recognizing the semantic block structure of the dialogue and dynamically and selectively allocating attention computing resources based on the current query.
[0005] The technical solution of this invention: A multi-turn dialogue processing method based on block-aware dynamic attention, comprising the following steps:
[0006] Step 1: Divide the multi-turn dialogue input sequence into historical records and the current query, and divide the historical records into multiple semantic sub-blocks according to the preset block division rules;
[0007] Step 2: Calculate the overall importance score of each semantic sub-block with the current query. The overall importance score is dynamically generated based on semantic relevance and temporal position weight.
[0008] Step 3: Divide the semantic sub-blocks into three categories based on the comprehensive importance score, and apply hard pruning, sparse attention, or full attention strategies respectively to dynamically allocate computing resources.
[0009] In the above-described multi-turn dialogue processing method based on block-aware dynamic attention, step 1, the block segmentation rules include:
[0010] If the number of dialogue rounds does not exceed 2 or the total number of text units is less than the threshold, the history will be treated as a single sub-block.
[0011] If the number of text units in a single turn of dialogue exceeds a preset threshold, then that turn of dialogue will be treated as an independent sub-block.
[0012] The final round of dialogue was always divided into independent sub-blocks.
[0013] In the aforementioned multi-turn dialogue processing method based on block-aware dynamic attention, step 2, the calculation of the comprehensive importance score includes:
[0014] For each semantic sub-block and the current query, mean pooling is performed to generate the corresponding vector representation;
[0015] The semantic relevance score between the semantic sub-block and the current query is calculated using cosine similarity.
[0016] Combining semantic relevance score and temporal position weight, according to the formula Generate a comprehensive importance score, where Score the semantic relevance. For time-series position weights.
[0017] In the aforementioned multi-turn dialogue processing method based on block-aware dynamic attention, step 3, the classification strategy includes:
[0018] If the overall importance score is lower than the first threshold, hard pruning is applied to the semantic sub-block, and its attention weight is completely ignored;
[0019] If the overall importance score is between the first and second thresholds, then sparse attention is applied, with the sparsity interval being inversely proportional to the importance score.
[0020] If the overall importance score is higher than the second threshold, then the full attention mechanism is applied.
[0021] In the aforementioned multi-turn dialogue processing method based on block-aware dynamic attention, the formula for calculating the sparse interval of the sparse attention is as follows:
[0022] ;
[0023] In the formula, This represents the overall importance score of the semantic sub-blocks.
[0024] In the aforementioned multi-turn dialogue processing method based on block-aware dynamic attention, the number of blocks in step 1 is dynamically determined by the following formula:
[0025] ;
[0026] In the formula, This represents the total number of text units in the history record. This is the preset threshold for text units.
[0027] In the aforementioned multi-turn dialogue processing method based on block-aware dynamic attention, step 2 calculates the temporal position weight based on the average turn index of semantic sub-blocks in the dialogue sequence, with sub-blocks closer to the current query having higher weights.
[0028] The aforementioned multi-turn dialogue processing method based on block-aware dynamic attention is integrated into the decoding process of a large language model based on the Transformer architecture, and is used to optimize the reasoning process of long-context multi-turn dialogues.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] 1. This invention uses a block-aware dynamic attention mechanism to accurately identify the semantic block structure of multi-turn dialogues and the importance difference between each block and the current query. It hard-prunes low-importance blocks, uses sparse attention for medium-importance blocks, and uses full attention for high-importance blocks. This significantly reduces redundant computation while retaining key information, achieving dual optimization of efficiency and quality.
[0031] 2. This invention concentrates limited computing resources on high-value semantic blocks, avoids the ineffective consumption of irrelevant or low-value information, significantly reduces memory usage and computing costs in long-context multi-turn dialogue scenarios, and adapts to the high-efficiency operation requirements under limited computing resources.
[0032] 3. This invention relies on the dual evaluation of semantic relevance and temporal position weights to keenly perceive topic shifts, jumps, and interweaving in dialogues, flexibly adjust the attention focus area, and reduce performance delays and generation degradation caused by long context-irrelevant information.
[0033] 4. This invention, as a modular design integrated into the decoding process of a large language model within the Transformer architecture, requires no modification to the underlying model parameter weights. While optimizing the performance of long-context dialogue processing, it does not affect the model's basic generation capabilities in short dialogues and general tasks, achieving full-scene adaptation for efficient long dialogue processing and stable short dialogue output. Attached Figure Description
[0034] Figure 1This is the block-aware dynamic attention model architecture layer of the present invention. Detailed Implementation
[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention.
[0036] Example: A multi-turn dialogue processing method based on block-aware dynamic attention, integrated into the decoding process of large language models (such as LLaMA and Qwen series) based on the Transformer architecture. Before implementing this method, basic environment configuration and core parameter presets must be completed to ensure the coordinated operation of each module:
[0037] Hardware requirements: GPU supporting CUDA (≥16GB VRAM, A100 or equivalent computing power recommended), CPU ≥16 cores, RAM ≥64GB;
[0038] Software environment: Python 3.8+, PyTorch 2.0+, Transformers library 4.30+, NumPy 1.24+, Scikit-learn 1.2+ (for similarity calculation).
[0039] Preset core parameters:
[0040] Text unit threshold: The threshold for the number of tokens split in a single turn of dialogue, which is set to 400 by default (it can be adjusted according to the maximum context window of the model, such as 500 if the model supports 8192 tokens).
[0041] Attention classification threshold: First threshold =0.5 (hard pruning threshold), second threshold =0.8 (critical value for full attention);
[0042] Vector dimension: consistent with the word embedding dimension of the underlying LLM (e.g., 4096 dimensions in LLaMA-7B) to ensure matching vector dimension after mean pooling;
[0043] Temporal position weight coefficient: calculated based on the average round index of the sub-block in the dialogue sequence, with a weight range of [0.1, 1.0]. The closer the sub-block is to the current query, the higher the weight.
[0044] The method of this invention employs a pluggable three-layer modular architecture, integrated into the decoding process of large language models (such as LLaMA and Qwen series) based on the Transformer architecture. The underlying LLM model retains its original parameter weights unchanged, such as... Figure 1 As shown, the overall structure of the architecture (BDA, or Block Aware Dynamic Attention) is as follows:
[0045] Data chunking layer: As an input processing module, it receives the original multi-turn dialogue sequence, splits it into structured semantic sub-chunks according to the rules in step 1 above, and outputs a regular list of sub-chunks;
[0046] Block-aware attention guidance layer: As the core evaluation module, it receives the output of the data block layer and calculates the comprehensive importance score of each sub-block according to the process in step 2.
[0047] Data chunking layer: As an input processing module, it receives the original multi-turn dialogue sequence, splits it into structured semantic sub-chunks according to the rules in step 1 above, and outputs a regular list of sub-chunks;
[0048] Block-aware attention guidance layer: As the core evaluation module, it receives the output of the data block layer and calculates the comprehensive importance score of each sub-block according to the process in step 2. Output a list of importance scores;
[0049] The three-tiered dynamic sparse attention layer, as the core execution module, receives the importance score from the block-aware attention guidance layer, dynamically allocates attention computing resources according to the three strategies in step 3, outputs the fused context vector, and inputs it into the feedforward network of the LLM to complete the decoding.
[0050] The core advantage of the BDA architecture lies in the deep integration of dialogue structure awareness and dynamic sparse computation. By dynamically optimizing the attention computation graph during online inference, it achieves a dual improvement in efficiency and generation quality for long-context, multi-turn dialogue inference without modifying the underlying LLM parameters.
[0051] The implementation process of this method is as follows:
[0052] Step 1: Divide the multi-turn dialogue input sequence into historical records and the current query, and divide the historical records into multiple semantic sub-blocks according to the preset block division rules.
[0053] The core of this step is to break down the flat, multi-turn dialogue sequence into semantically related structured sub-blocks, providing regularized input for subsequent fine-grained attention allocation. The specific implementation is as follows:
[0054] 1.1 Input sequence of multi-turn dialogues Split into history and the current query ,in, Including the front Round of dialogue content, The latest query initiated by the user (i.e., the first) (round-robin dialogue), satisfying .
[0055] 1.2. Record history according to preset rules Perform semantic sub-block partitioning and generate semantic sub-blocks The specific rules for partitioning are as follows:
[0056] Rule 1: If Total rounds ≤2, or Total number of text units (tokens) < ( (The default text unit threshold is 400). The whole is treated as a single semantic sub-block and is not further divided.
[0057] Rule 2: If The number of tokens in a single-turn dialogue > If so, then this round of dialogue is treated as a separate semantic sub-block;
[0058] Rule 3: The last round of dialogue (i.e., immediately following the current query) (In the previous round), regardless of the number of its tokens, it is always treated as a single independent semantic sub-block;
[0059] Rule 4: For the remaining undivided rounds, the total number of tokens after merging consecutive rounds shall be less than or equal to the total number of tokens in each round. Grouping according to the principle of "", each group forms a semantic sub-block.
[0060] 1.3 Number of semantic sub-blocks The following formula is used to dynamically determine the chunk granularity to ensure it matches the dialog length:
[0061] ;
[0062] In the formula, This represents the total number of text units in the history record. The preset text unit threshold, This is the rounding function.
[0063] Step 2: Calculate the overall importance score of each semantic sub-block with the current query. The overall importance score is dynamically generated based on semantic relevance and temporal position weight.
[0064] This step quantifies the impact of each semantic sub-block on the current query by fusing semantic relevance and temporal position weights. The degree of importance provides a basis for attention strategy allocation; the specific calculation process is as follows:
[0065] 2.1 Vector Representation Generation (Mean Pooling):
[0066] Pair blocks The text content is tokenized (using the same tokenizer as the underlying large language model) to obtain a token sequence:
[0067] ,
[0068] In the formula: for The number of tokens;
[0069] The word embedding vector of each token is extracted through the embedding layer of the underlying large language model. ( The embedding dimension is consistent with the word embedding dimension of the underlying model, such as LLaMA-7B. =4096);
[0070] According to the following formula The word embedding vectors are averaged and pooled to generate sub-block vectors. :
[0071] ;
[0072] In the formula: semantic sub-blocks The number of tokens;
[0073] Apply the following formula to the current query Perform the same mean pooling operation to generate the query vector. :
[0074] ;
[0075] In the formula: For the current query The number of tokens.
[0076] 2.2 Semantic Relevance Score Calculation: Sub-block vectors are calculated using cosine similarity. With query vector semantic relevance score The formula is as follows:
[0077] ;
[0078] In the formula: This represents the vector dot product operation; The L2 norm of a vector; The larger the value, the stronger the semantic relationship between the sub-block and the current query.
[0079] 2.3 Calculation of temporal position weights: Temporal position weights Based on semantic sub-blocks In historical records The average round index calculation in the middle is closer to the current query. The higher the weight of a sub-block, the better, as shown in the formula below:
[0080] ;
[0081] In the formula: For sub-blocks Average round index, For historical records Total number of rounds, Ensure that the earliest sub-blocks retain their basic weights to avoid completely omitting potentially critical information.
[0082] 2.4 Generation of Comprehensive Importance Score: Integrating Semantic Relevance Score With temporal position weights Generate sub-blocks Overall importance score The formula is as follows:
[0083] ;
[0084] in, The larger the value, the more important the sub-block is to the current query.
[0085] Step 3: Divide the semantic sub-blocks into three categories based on the comprehensive importance score, and apply hard pruning, sparse attention, or full attention strategies respectively to dynamically allocate computing resources.
[0086] This step is based on the overall importance score. Differentiated attention strategies are dynamically assigned to different semantic sub-blocks to achieve precise allocation of computing resources. The specific strategies are as follows:
[0087] 3.1 Threshold Setting: Two attention classification thresholds are preset, the first threshold... =0.5 (hard pruning threshold), second threshold =0.8 (full attention threshold), which can be adjusted according to the actual needs of the scenario.
[0088] 3.2 Implementation of Three Types of Attention Strategies:
[0089] Strategy 1: Hard pruning (low importance sub-blocks): If < If the attention weight of the sub-block is completely ignored, the key and value vectors of the sub-block are directly hidden in the Transformer self-attention calculation and do not participate in the attention weight calculation.
[0090] Strategy 2: Sparse Attention (Medium Importance Sub-blocks): If ≤ ≤ Then, an importance-sensitive sparse attention strategy is applied, with sparse intervals. and Inversely proportional, that is .For example When =0.67, ≈1.5, rounded down to 2, meaning one out of every two tokens is retained; during execution, the key and value vectors of this type of sub-block are... After performing interval sampling to generate a sparse key-value matrix, it is used for attention calculation;
[0091] Strategy 3: Full Attention (High Importance Sub-blocks): If > If so, the complete key and value vectors of the sub-block are retained, and the attention weights are calculated according to the Transformer standard self-attention formula.
[0092] 3.3 Attention Fusion Output: The attention calculation results of the three types of sub-blocks are merged to generate the final context vector, which is then input into the Transformer's feed-forward network to complete the decoding process and obtain the current query. The response results.
[0093] The following two concrete examples demonstrate how the BDA architecture operates in real-world multi-turn dialogue scenarios. Each embodiment shows the data segmentation, importance scoring, and attention allocation results step-by-step according to the processing flow. Among these, the threshold... and The default values are 0.5 and 0.8.
[0094] Example 1: Customer Service Dialogue Analysis
[0095] Dialogue Background: The user engages in 12 rounds of dialogue with the intelligent customer service, discussing product troubleshooting, warranty policies, and technical parameters.
[0096] Data is processed in blocks (a total of 8 sub-blocks), as shown in Table 1:
[0097] Table 1
[0098]
[0099] Block-aware attention-guided computation is shown in Table 2:
[0100] Table 2
[0101] History sub-block Relevance score to the current query Temporal position weights Overall Importance Score Calculation process description C1 0.42 0.15 0.57 Early fault description, moderate relevance to warranty policy C2 0.38 0.18 0.56 The device information is somewhat related to the current query. C3 0.45 0.22 0.67 Purchase receipt information is important for warranty determination. C4 0.32 0.25 0.57 The troubleshooting process has a low correlation with policy. C5 0.51 0.28 0.79 The unresolved issue is important for warranty claims. C6 0.35 0.32 0.67 Information on usage environment, moderate importance. C7 0.48 0.35 0.83 Error codes directly indicate the nature of the fault. C8 (Query) 1.00 1.00 1.00 The current query itself
[0102] The three-tiered dynamic sparse attention processing is shown in Table 3:
[0103] Table 3
[0104] Sub-block classification Contains sub-blocks Importance score range Processing strategy Sparse parameter (k) Saving computing resources Hard pruning C1,C2,C4 0.56-0.57 Completely ignore attention weights k=100% Saves 37.5% of attention in computation Sparse attention C3,C6 0.67 Importance, sensitivity, sparse attention k=33% Medium attention density Full attention C5, C7, C8 0.79-1.00 Complete attention mechanism k=0% Complete processing
[0105] Implementation results:
[0106] (1) Original computational complexity: O(8) 2 = 64 attention units.
[0107] (2) Computational complexity after implementation: 3 full attention + 2 sparse attention ≈ 15 effective attention units.
[0108] (3) Calculated savings: approximately 76.6%.
[0109] (4) Output quality: accurately focus on C5 (unresolved issues), C7 (erroneous evidence), and C8 (current query), ignoring irrelevant early investigation details.
[0110] Example 2: Long medical consultation conversation;
[0111] Dialogue background: The patient had 15 rounds of dialogue with the AI medical assistant, involving symptom description, medical history inquiry, examination result analysis and treatment suggestions.
[0112] Data is processed in blocks (9 sub-blocks in total), as shown in Table 4:
[0113] Table 4
[0114] Sub-block ID Dialogue rounds range Content Summary Text length Blocking rules are based on C1 Rounds 1-3 The patient describes symptoms of headache and fever, and their duration. [Summary, approximately 350 characters omitted] 780token If the number of tokens is greater than 400, merge rounds. C2 Round 4 AI asks for the specific location and nature of the pain. [Full details retained] 220 tokens Single-wheel independent sub-block C3 Rounds 5-6 The patient provided additional information about accompanying symptoms (nausea, photophobia) [Summary, approximately 200 characters omitted] 460token Token count > 400, independent sub-block C4 Round 7 AI asks about past medical history and drug allergies [Full details retained] 190token Single-wheel independent sub-block C5 Rounds 8-9 The patient provides a history of hypertension and current medications [Summary, approximately 300 characters omitted]. 550 tokens If the number of tokens is greater than 400, merge rounds. C6 Rounds 10-11 AI suggests performing body temperature monitoring and initial treatment. [Summary, approximately 250 characters omitted] 520token If the number of tokens is greater than 400, merge rounds. C7 Round 12 Patient reports body temperature measurement result (38.5°C) [Keep complete] 150 tokens Single-wheel independent sub-block C8 Rounds 13-14 The AI asked if there was neck stiffness or changes in consciousness. [Summary, approximately 180 characters omitted] 430 tokens Token count > 400, independent sub-block C9 Round 15 Latest query: A patient asked, "Do I need to see a doctor immediately, or can I observe at home first?" 280 tokens Final round of dialogue, independent sub-blocks
[0115] Block-aware attention-guided computation is shown in Table 5:
[0116] Table 5
[0117] History sub-block Relevance score to the current query Temporal position weights Overall Importance Score Calculation process description C1 0.65 0.10 0.75 The core symptom description is highly relevant to medical treatment decisions. C2 0.42 0.15 0.57 Symptom details, moderate importance C3 0.58 0.20 0.78 Accompanying symptoms affect the assessment of urgency. C4 0.31 0.22 0.53 Medical history and allergy information are generally not relevant. C5 0.68 0.25 0.93 A history of hypertension increases the urgency of seeking medical attention. C6 0.35 0.30 0.65 The preliminary treatment suggestions are of some reference value. C7 0.72 0.35 1.07 High fever is a key danger signal C8 0.45 0.40 0.85 Inquiry about neurological symptoms is important C9 (Query) 1.00 1.00 1.00 Current medical treatment decision inquiry
[0118] The three-tiered dynamic sparse attention processing is shown in Table 6:
[0119] Table 6
[0120] Sub-block classification Contains sub-blocks Importance score range Processing strategy Sparse parameter (k) Note the mechanism explanation Hard pruning C2,C4 0.53-0.57 Completely ignore attention weights k=100% Ignoring relatively minor symptom details Sparse attention C1,C6 0.65-0.75 Importance, sensitivity, sparse attention k=25-35% Some focus on core symptoms and initial recommendations Full attention C3, C5, C7, C8, C9 0.78-1.07 Complete attention mechanism k=0% Focus on key medical information
[0121] Implementation results:
[0122] (1) Original computational complexity: O(9 2 = 81 attention units.
[0123] (2) Computational complexity after implementation: 5 full attention + 2 sparse attention ≈ 28 effective attention units.
[0124] (3) Calculated savings: approximately 65.4%.
[0125] (4) Output quality: accurately focus on C1 (core symptoms), C3 (accompanying symptoms), C5 (history of hypertension), C7 (key indicators of high fever), C8 (questioning of neurological symptoms) and C9 (current query), ignore secondary symptom details, and correctly identify the urgency of medical treatment.
[0126] As can be seen from the two examples above, the BDA architecture, through structured segmentation, dynamic importance assessment, and differentiated attention allocation, not only significantly improves the efficiency of long-context, multi-turn dialogue reasoning but also effectively ensures the quality of generation, demonstrating clear feasibility and application value.
Claims
1. A multi-turn dialogue processing method based on block-aware dynamic attention, characterized in that: Includes the following steps: Step 1: Divide the multi-turn dialogue input sequence into historical records and the current query, and divide the historical records into multiple semantic sub-blocks according to the preset block division rules; Step 2: Calculate the overall importance score of each semantic sub-block with the current query. The overall importance score is dynamically generated based on semantic relevance and temporal position weight. Step 3: Divide the semantic sub-blocks into three categories based on the comprehensive importance score, and apply hard pruning, sparse attention, or full attention strategies respectively to dynamically allocate computing resources.
2. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 1, characterized in that, In step 1, the segmentation rules include: If the number of dialogue rounds does not exceed 2 or the total number of text units is less than the threshold, the history will be treated as a single sub-block. If the number of text units in a single turn of dialogue exceeds a preset threshold, then that turn of dialogue will be treated as an independent sub-block. The final round of dialogue was always divided into independent sub-blocks.
3. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 1, characterized in that, In step 2, the calculation of the overall importance score includes: For each semantic sub-block and the current query, mean pooling is performed to generate the corresponding vector representation; The semantic relevance score between the semantic sub-block and the current query is calculated using cosine similarity. Combining semantic relevance score and temporal position weight, according to the formula Generate a comprehensive importance score, where Score the semantic relevance. For time-series position weights.
4. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 1, characterized in that, In step 3, the classification strategy includes: If the overall importance score is lower than the first threshold, hard pruning is applied to the semantic sub-block, and its attention weight is completely ignored; If the overall importance score is between the first and second thresholds, then sparse attention is applied, with the sparsity interval being inversely proportional to the importance score. If the overall importance score is higher than the second threshold, then the full attention mechanism is applied.
5. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 4, characterized in that, The formula for calculating the sparse interval of the sparse attention is: ； In the formula, This represents the overall importance score of the semantic sub-blocks.
6. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 1, characterized in that, The number of blocks in step 1 is dynamically determined by the following formula: ； In the formula, This represents the total number of text units in the history record. This is the preset threshold for text units.
7. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 1, characterized in that, In step 2, the temporal position weight is calculated based on the average round index of the semantic sub-block in the dialogue sequence, and the sub-block closer to the current query has a higher weight.
8. The multi-turn dialogue processing method based on block-aware dynamic attention according to claim 1, characterized in that, The method is integrated into the decoding process of a large language model based on the Transformer architecture to optimize the reasoning process of long-context, multi-turn dialogues.