Algorithm method for large-model long-context reasoning

Abstract

The invention discloses an algorithm method for large-model long context reasoning, which relates to the technical field of large language models and comprises the following steps of: dividing an input long text sequence into a plurality of initial text blocks; a semantic abstract is generated based on the initial text blocks, clustering analysis is conducted on the semantic abstract, the initial text blocks with similar semantics are combined into semantic hyperblocks, and a context organization structure with semantic representativeness is formed; the method comprises the following steps: generating key value cache data of each token in a large model preprocessing stage, grouping the key value cache data by taking semantic hyperblocks as logic boundaries, and establishing a mapping table for recording storage positions and states of the semantic hyperblocks; on the basis of semantic hyperblocks, semantic representative vectors of the semantic hyperblocks are combined, correlation scores between query vectors and the semantic hyperblocks are calculated, a block importance prediction model learns on the basis of context dependency features, high-order semantic prior is provided for subsequent attention screening, and key information omission caused by position offset is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model, and particularly relates to an algorithm method for long context reasoning of large model. BACKGROUND

[0002] The large language model refers to a deep learning model trained using a large amount of text data, so that the model can generate natural language text or understand the meaning of language text, and these models can provide in-depth knowledge and language production about various topics by training on a large data set, and the core idea is to learn the patterns and structures of natural language through large-scale unsupervised training, and to simulate the language cognition and generation process of human beings to a certain extent.

[0003] However, the prior art still has essential defects in the long context management mechanism, and it is difficult to meet the comprehensive needs of efficiency, accuracy and scalability of large models in ultra-long sequence reasoning, and the organization mode of the existing KVCache usually takes token or fixed length text block as the basic unit, and stores continuously according to the physical order of the input sequence, and lacks the modeling ability of the context semantic structure, and this position-driven organization mode cannot effectively identify the context fragments that are semantically key when executing sparse attention or cache replacement, and often causes generation distortion due to misdeletion or failure to load core semantic content. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an algorithm method for long context reasoning of large model, which solves the problem that the prior art still has essential defects in the long context management mechanism, and it is difficult to meet the comprehensive needs of efficiency, accuracy and scalability of large models in ultra-long sequence reasoning, and the organization mode of the existing KVCache usually takes token or fixed length text block as the basic unit, and stores continuously according to the physical order of the input sequence, and lacks the modeling ability of the context semantic structure, and this position-driven organization mode cannot effectively identify the context fragments that are semantically key when executing sparse attention or cache replacement, and often causes generation distortion due to misdeletion or failure to load core semantic content.

[0006] To solve the above technical problems, the present application provides the following technical scheme: In a first aspect, the present application provides an algorithm method for long context reasoning of large model, which includes the following steps: dividing the input long text sequence into a plurality of initial text blocks; generating a semantic summary based on the initial text blocks, and performing clustering analysis on the semantic summary, and merging the initial text blocks with similar semantics into semantic super blocks to form a context organization structure with semantic representation; generating key-value cache data of each token in the large model preprocessing stage, grouping the key-value cache data with semantic superblocks as logical boundaries, and establishing a mapping table recording the storage location and state of each semantic superblock; Based on the semantic superblock, the correlation score between the query vector and each semantic superblock is calculated combined with the semantic representative vector of the semantic superblock, and the block importance prediction model learns based on context-dependent features; According to the correlation score, all semantic superblocks are sorted, and the target semantic superblock participating in the current attention calculation is determined, the physical storage location of the target semantic superblock is obtained by querying the mapping table, and the key-value cache data is loaded into the central processor main memory according to the storage location; Performing attention calculation on the loaded key-value cache data of the target semantic superblock, generating a context-weighted vector, and transmitting the context-weighted vector to the graphics processor to output a new text unit; During the generation process, the access frequency of each semantic superblock is recorded, the hotness state in the mapping table is updated according to the change of the access frequency, and the key-value cache data is migrated between the solid state disk and the central processor according to the hotness state.

[0007] As a preferred scheme of the algorithm method for large model long context reasoning, wherein: the input long text sequence is divided into multiple initial text blocks, and the specific steps are: The input long text sequence is divided by the block strategy to obtain multiple initial text blocks; The long text sequence is provided by user request and represented as a continuous token sequence; The long text sequence is cut according to a fixed length, each cut segment contains the same number of continuous tokens, forming a preliminary data processing unit; Each cut segment is assigned a unique identifier, and the start and end positions in the original sequence are recorded to obtain an initial text block set.

[0008] As a preferred scheme of the algorithm method for large model long context reasoning, wherein: based on the initial text block, a semantic summary is generated, and a clustering analysis is performed on the semantic summary, and the initial text blocks with similar semantics are merged into semantic superblocks to form a context organization structure with semantic representation, and the specific steps are: Each token in each initial text block is vectorized by embedding transformation to obtain a vector representation of each token; Performing mean aggregation operation on all token vectors in each initial text block to obtain a semantic summary vector; A clustering algorithm is used to analyze the semantic relationship of all semantic summary vectors, and the initial text blocks are merged into the same cluster according to the similarity between vectors; Each cluster is defined as a semantic superblock, and each semantic superblock contains a representative vector and an initial set of text blocks, forming a contextual organization structure.

[0009] As a preferred embodiment of the algorithm method for large model long context reasoning described in this invention, the following steps are taken: During the model preprocessing stage, key-value cache data for each token is generated; the key-value cache data is grouped using semantic superblocks as logical boundaries; and a mapping table recording the storage location and state of each semantic superblock is established. A large model forward computation is used to perform prefill processing on the tokens in the input sequence, generating corresponding key vectors and value vectors layer by layer to form key-value cache data; Using the token range covered by the semantic superblock as the boundary, key-value cache data is logically encapsulated to form cache data groups; A storage allocation strategy is adopted to assign physical locations to each cache data group. Based on the location attributes and access characteristics of the semantic superblock in the context, it will be mapped to the graphics processor memory, the central processing unit main memory or the solid-state drive respectively. The storage information of each semantic superblock is registered and a mapping table is established. The mapping table records the storage location, data offset, data size, data format and access popularity status of the semantic superblock.

[0010] As a preferred embodiment of the algorithm method for large-model long-context reasoning described in this invention, the following steps are taken: Based on semantic superblocks, and combined with the semantic representative vectors of the semantic superblocks, a relevance score between the query vector and each semantic superblock is calculated; the block importance prediction model is learned based on context-dependent features. A block importance prediction model is constructed using the semantic representation vector and attention query vector of the semantic superblock as input. The supervised training method optimizes the parameters of the block importance prediction model, using context-dependent features as supervision signals; During the generation phase, the attention query vector is obtained, and the attention query vector is concatenated with the semantic representation vector and then input into the block importance prediction model. Nonlinear transformation is used to extract features from the concatenated vector. After calculation by the hidden layer, a scalar value is output, representing the semantic superblock and relevance score. The expression for the relevance score is as follows: ; in, For the first Relevance score of each semantic superblock This is the attention query vector for the current decoding step. For the first The semantic representation vector of a semantic superblock. This represents the concatenation operation between the attention query vector and the semantic representation vector. , For learnable weight matrices and vectors, For vector bias terms, It is a scalar bias term. This is the Sigmoid activation function.

[0011] As a preferred embodiment of the algorithm method for large-model long-context reasoning described in this invention, the following steps are taken: All semantic superblocks are sorted according to relevance scores to determine the target semantic superblocks participating in the current attention calculation; the physical storage location of the target semantic superblocks is obtained by querying the mapping table; and key-value cache data is loaded into the central processing unit's main memory according to the storage location. The relevance scores are sorted in descending order using a sorting operation to obtain the score sequence and the corresponding semantic superblock index; The threshold screening method selects semantic superblocks from the scoring sequence to form a candidate set; Select a semantic superblock from the candidate set as the target semantic superblock and participate in the attention calculation of the current decoding step; Query the mapping table to obtain the storage location information for each target semantic superblock; The storage medium where the target semantic superblock is currently located is determined based on the storage location information: if it is located in the main memory of the central processing unit, it is directly marked as pending processing. If it is located on a solid-state drive, an asynchronous loading task is started to read from the solid-state drive, decompress it and load it into the CPU main memory cache. The expression for the selection process of the target semantic superblock is as follows: ; in, For a defined set of target semantic superblocks, For the first A semantic superblock, For the corresponding relevance score, It is a descending sorting function. To take the front Operations on items.

[0012] As a preferred embodiment of the algorithm method for large model long context reasoning described in this invention, the following steps are taken: attention calculation is performed on the key-value cache data of the loaded target semantic superblock to generate a context weighted vector, the context weighted vector is transmitted to the graphics processor, and a new text unit is output. The key-value cache data of the target semantic superblock is calculated using a sparse attention mechanism. The specific process is as follows: The query vector generated by the top-layer attention layer of the model is acquired, each original token contained in each semantic super block in the target semantic super block set is traversed, the Key vector pre-cached in the corresponding Transformer layer is extracted, the query vector is matched with the Key vector of each token in terms of similarity, the correlation is measured through dot product operation, and an original attention score is obtained; All attention scores from the target semantic super block are normalized, the Softmax function is used to calculate the normalization weight along the token sequence dimension, and the Value vector corresponding to the token is subjected to weighted summation operation; Each Value vector is multiplied by the attention weight corresponding thereto, and all weighted results are accumulated to generate a context weighted vector of a fixed dimension, which integrates key context information in all target semantic super blocks participating in the calculation; The attention query vector of the current decoding step is matched with the Key vector of all target semantic super blocks in terms of similarity, and the attention weight is calculated; In the current decoding step generated by the autoregressive generation, the attention query vector output by the top layer of the model decoder is acquired, all determined target semantic super blocks are traversed, and the Key vector pre-cached in the corresponding Transformer layer of each token contained in each semantic super block is extracted; For each target token, the Key vector is subjected to dot product operation with the attention query vector of the current decoding step to obtain an original similarity score; All original similarity scores are scaled, that is, each score is divided by the square root value of the Key vector dimension, all scaled scores from the target semantic super block are combined into a unified score sequence, and the Softmax function is applied to the sequence along the token dimension to perform normalization processing, thereby generating a set of normalized attention weights; The Value vectors are subjected to weighted summation according to the weights to generate a context weighted vector.

[0013] The context weighted vector is transmitted to the graphics processor through a high-speed interconnection channel; The residual connection, layer normalization and feedforward network calculation are completed on the graphics processor to generate a new text unit.

[0014] As a preferred scheme of the algorithm method for long context reasoning of a large model, in the generation process, the access frequency of each semantic super block is recorded, the hotness state in the mapping table is updated according to the change of the access frequency, and the key-value cache data is migrated between the solid state disk and the central processing unit according to the hotness state, and the specific steps are as follows: The access frequency of the semantic super block of the target semantic super block is accumulated by using a counting mechanism; Write the updated access frequency to the mapping table and update the popularity status synchronously. The heat status is evaluated using migration judgment logic; When the access frequency of a certain semantic superblock exceeds the rising threshold, the migration of key-value cache data from the solid-state drive to the central processing unit's main memory is triggered. When the access frequency falls below the threshold, a compression migration from main memory to solid-state drive is triggered.

[0015] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the algorithmic method for large model long context reasoning as described in the first aspect of the present invention.

[0016] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the algorithmic method for long-context reasoning of large models as described in the first aspect of the present invention.

[0017] The beneficial effects of this invention are as follows: By dividing the input long text sequence into multiple initial text blocks and clustering and merging them into semantic superblocks based on semantic summarization, a hierarchical semantic organization of the context is achieved. Its function is to break the rigid structure of traditional positional processing and to construct semantically cohesive contextual units, ultimately improving the accuracy of key contextual information recognition. It breaks through the mechanical division method of simply cutting long texts into fixed windows or sliding segments in the existing technology. By first generating the semantic summary vector of each initial text block and then clustering based on vector similarity, semantically coherent content is automatically merged into a semantic superblock, even if they are physically far apart in the original sequence. This semantically driven reorganization mechanism enables the model to understand subsequent reasoning in terms of topic paragraphs rather than isolated fragments, significantly enhancing the ability to capture cross-paragraph logical relationships. Especially when processing complex texts such as legal documents and scientific research papers, semantic superblocks can naturally correspond to functional modules such as background, methods, and conclusions, providing high-order semantic priors for subsequent attention filtering and avoiding the omission of key information due to positional offset. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1A flowchart of an algorithmic method for long-context reasoning of large models. DETAILED DESCRIPTION

[0020] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0021] In the following description, a large number of specific details are set forth in order to facilitate a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the concept of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0022] Secondly, the "one embodiment" or "embodiment" referred to herein means that the specific features, structures or characteristics can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor is it an embodiment that is separate or alternative to other embodiments.

[0023] Reference Figure 1 For one embodiment of the present application, the embodiment provides an algorithmic method for long-context reasoning of large models, comprising the following steps: S1, dividing the input long text sequence into multiple initial text blocks.

[0024] Further, the input long text sequence is divided by using a block strategy to obtain multiple initial text blocks; The long text sequence is provided by a user request and is represented as a continuous token sequence; The long text sequence is cut according to a fixed length, each cut segment contains the same number of continuous tokens, forming a preliminary data processing unit; A unique identifier is assigned to each cut segment, and the starting and ending positions in the original sequence are recorded to obtain an initial text block set.

[0025] It should be noted that the block strategy adopts a fixed-length non-overlapping cutting method, which aims to convert the long text sequence into a structured basic processing unit. This division method avoids the problem of cross-block semantic breakage, and provides a unified data granularity for subsequent semantic summary generation. The identifier of each initial text block is bound with its position information in the original sequence, which ensures that the context source can be accurately traced during reasoning, and provides spatial positioning basis for subsequent cache management and attention calculation.

[0026] S2, generating a semantic summary based on the initial text blocks, and performing clustering analysis on the semantic summary, merging the initial text blocks with similar semantics into semantic super blocks to form a context organization structure with semantic representation.

[0027] Further, the tokens in each initial text block are vectorized using embedding transformation to obtain vector representations of the tokens; A mean aggregation operation is performed on all token vectors in each initial text block to obtain a semantic summary vector; A clustering algorithm is used to analyze the semantic relationships of all semantic summary vectors, and the initial text blocks are merged into the same cluster according to the similarity between vectors; Each cluster is defined as a semantic super block, and each semantic super block contains a representative vector and a set of initial text blocks, forming a context organization structure.

[0028] It should be noted that the semantic summary vector is generated by the mean aggregation operation, which can effectively preserve the overall semantic features of the initial text block while suppressing local noise interference. The clustering algorithm is used to group the semantic summary vectors, which realizes the context reorganization based on semantic similarity, breaks through the limitations of traditional context organization by position order, and the formed semantic super block not only contains text content with similar semantics, but also abstracts its core semantics through the representative vector, providing structured support for subsequent semantic-based relevance scoring and sparse attention mechanism.

[0029] S3, in the large model preprocessing stage, generate key-value cache data for each token, group the key-value cache data according to the logical boundary of the semantic super block, and establish a mapping table recording the storage location and state of each semantic super block.

[0030] Further, the tokens in each initial text block are vectorized using embedding transformation to obtain vector representations of the tokens; The token range covered by the semantic super block is taken as the boundary to logically encapsulate the key-value cache data to form a cache data group; A storage allocation strategy is used to assign physical locations to each cache data group, and according to the position attribute and access characteristics of the semantic super block in the context, it will be mapped to the graphics processor memory, central processor main memory or solid state disk respectively; The storage information of each semantic super block is registered to establish a mapping table, which records the storage location, data offset, data size, data format and access heat state of the semantic super block.

[0031] It should be noted that the key-value cache data is generated layer by layer during the model prefill stage, covering the key and value vectors of all input tokens in each Transformer layer. It is grouped with semantic superblocks as logical boundaries, which changes the organization of KVCache from position-driven to semantic-driven, supporting subsequent scheduling and computation at the semantic unit level. The mapping table, as a metadata management structure, records the physical storage status of each semantic superblock, realizing the decoupling of the logical semantic structure from the underlying storage system, and providing a control basis for efficient collaboration across storage levels.

[0032] S4. Based on semantic superblocks, and combined with the semantic representative vectors of the semantic superblocks, calculate the relevance score between the query vector and each semantic superblock. The block importance prediction model learns based on context-dependent features.

[0033] Furthermore, a block importance prediction model is constructed using the semantic representation vector and attention query vector of the semantic superblock as input. The supervised training method optimizes the parameters of the block importance prediction model, using context-dependent features as supervision signals; During the generation phase, the attention query vector is obtained, and the attention query vector is concatenated with the semantic representation vector and then input into the block importance prediction model. Nonlinear transformation is used to extract features from the concatenated vector. After calculation by the hidden layer, a scalar value is output, representing the semantic superblock and relevance score. The expression for the relevance score is as follows: ; in, For the first Relevance score of each semantic superblock This is the attention query vector for the current decoding step. For the first The semantic representation vector of a semantic superblock. This represents the concatenation operation between the attention query vector and the semantic representation vector. , For learnable weight matrices and vectors, For vector bias terms, It is a scalar bias term. This is the Sigmoid activation function.

[0034] It should be noted that the block importance prediction model is a lightweight learnable network, which integrates the current decoding state and the context semantic unit as input, can dynamically evaluate the contribution of different semantic superblocks to the current generation task, and the large model learns the context-dependent features through supervised training, avoiding the limitations of relying on fixed similarity measures in traditional methods. The relevance score output reflects the context relevance strength of the semantic superblock, providing a learnable and optimized decision basis for target selection in subsequent sparse attention.

[0035] S5, according to the relevance score, sort all semantic superblocks, determine the target semantic superblock participating in the current attention calculation, query the mapping table to obtain the physical storage location of the target semantic superblock, and load the key value cache data to the central processor main memory according to the storage location.

[0036] Further, the relevance score is arranged in descending order by using the sorting operation to obtain a score sequence and a corresponding semantic superblock index; The threshold screening method selects semantic superblocks from the score sequence to form a candidate set; Select semantic superblocks from the candidate set as target semantic superblocks to participate in attention calculation of the current decoding step; Query the mapping table to obtain the storage location information of each target semantic superblock; According to the storage location information, determine the storage medium where the target semantic superblock is currently located: if it is located in the central processor main memory, it is directly marked as a to-be-processed state; If it is located in the solid state disk, start an asynchronous loading task, read and decompress the solid state disk and load it into the central processor main memory cache area; Wherein, the expression of the selection process of the target semantic superblock is: ; Wherein, is a set of determined target semantic superblocks, is the th semantic superblock, is the corresponding relevance score, is a descending order sorting function, is an operation of taking the first items.

[0037] It should be noted that the sorting and filtering mechanism arranges the relevance scores in descending order, preferentially retains the semantic superblocks with the highest semantic relevance for attention calculation, significantly reduces the context scale involved in the calculation, and the selection process of the target semantic superblock realizes the sparse focusing from the full context to the key semantic fragments, effectively alleviating the calculation and memory pressure brought by long context, and through the query mapping table, the physical storage location is obtained, the dynamic mapping of semantic logic and storage physical address is realized, the on-demand loading across storage media is supported, and the continuity and efficiency of the reasoning process are guaranteed.

[0038] S6, attention calculation is performed on the key-value cache data of the loaded target semantic superblock, a context weighted vector is generated, the context weighted vector is transmitted to the graphics processor, and a new text unit is output.

[0039] Furthermore, the key-value cache data of the target semantic superblock is calculated by the sparse attention mechanism, and the specific process is as follows: The query vector generated by the top attention layer of the model is obtained, and for the target semantic superblock set, each original token contained in each semantic superblock is traversed, and the Key vector pre-cached in the corresponding Transformer layer is extracted. The query vector and the Key vector of each token are matched in similarity, the correlation is measured by dot product operation, and the original attention score is obtained; All attention scores from the target semantic superblock are normalized, the Softmax function is used to calculate the normalized weight along the token sequence dimension, and the Value vector corresponding to the token is weighted and summed; Each Value vector is multiplied by its corresponding attention weight, and all weighted results are accumulated to generate a context weighted vector of fixed dimension, which integrates the key context information in all target semantic superblocks involved in the calculation; The attention query vector of the current decoding step is matched with the Key vector of all target semantic superblocks in similarity, and the attention weight is calculated; In the current decoding step generated by autoregression, the attention query vector output by the top layer of the model decoder is obtained, all determined target semantic superblocks are traversed, and the Key vector pre-cached in the corresponding Transformer layer for each token contained in each semantic superblock is extracted; For each target token, the Key vector and the attention query vector of the current decoding step are dot multiplied to obtain the original similarity score; The original similarity scores are scaled, i.e., each score is divided by the square root of the dimension of the Key vector, the scaled scores from the target semantic superblocks are combined into a unified score sequence, and the sequence is normalized along the token dimension by applying the Softmax function to generate a set of normalized attention weights; The Value vectors are weighted and summed to generate a context weighted vector.

[0040] The context weighted vector is transmitted to the graphics processor through a high-speed interconnection channel; The residual connection, layer normalization, and feedforward network calculation are completed on the graphics processor to generate a new text unit.

[0041] It should be noted that the sparse attention calculation process is only performed on the target semantic superblocks loaded into the central processor main memory, avoiding redundant calculation of irrelevant contexts. The attention query vector of the current decoding step is multiplied by the Key vectors of all tokens in each target semantic superblock to calculate the original similarity scores, which are then normalized by Softmax to obtain the normalized attention weight distribution. Finally, the Value vectors are weighted and summed according to the corresponding weights to generate a context weighted vector. This process significantly reduces the computational complexity while maintaining the quality of the model output, achieving efficient inference under long context.

[0042] S7, record the access frequency of each semantic superblock during the generation process, update the hotness state in the mapping table according to the change of the access frequency, and trigger the migration operation of the key-value cache data between the solid state disk and the central processor according to the hotness state.

[0043] Further, a counting mechanism is used to accumulate the access frequency of the target semantic superblock; The updated access frequency is written into the mapping table, and the hotness state is updated synchronously; The hotness state is evaluated using migration judgment logic; When the access frequency of a semantic superblock exceeds the rising threshold, the migration of key-value cache data from the solid state disk to the central processor main memory is triggered; When the access frequency is lower than the falling threshold, the compression migration from the main memory to the solid state disk is triggered.

[0044] It should be noted that the record of access frequency and the update of hotness state constitute the core feedback loop of the dynamic cache management mechanism. Each time a semantic superblock is selected as the target unit, its access frequency is incremented, reflecting its activity level in the generation process. The hotness state in the mapping table synchronizes the frequency changes in real time, providing a basis for decision-making for storage migration. The upper threshold and the lower threshold constitute a hysteresis control logic, preventing frequent migration due to short-term fluctuations and improving system stability. The asynchronous migration mechanism ensures that data flow between the solid state disk and the central processing unit main memory does not block the main reasoning process, achieving parallelization and coordination of computation and I / O.

[0045] The embodiment also provides a computer device suitable for the algorithm method for large model long context reasoning, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the algorithm method for large model long context reasoning proposed in the above embodiment.

[0046] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. In addition, the input device can also be an external keyboard, touchpad or mouse, etc.

[0047] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the algorithm method for long-context reasoning of a large model as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, a magnetic disk or an optical disk.

[0048] To sum up, the application achieves semantic hierarchical organization of context by dividing the input long text sequence into multiple initial text blocks and clustering and merging the initial text blocks into semantic super blocks based on semantic summaries, which breaks the rigid structure of traditional position sequence processing and constructs context units with semantic cohesion, thereby improving the accuracy of key information recognition in context and breaking through the mechanical division method of simply dividing long text into fixed windows or sliding segments in the prior art. The semantic summary vectors of each initial text block are generated first, and then clustering is performed according to the vector similarity, so that semantically coherent content is automatically merged into a semantic super block even if they are far apart in physical position in the original sequence. This semantic-driven reorganization mechanism enables the model to understand in units of thematic paragraphs rather than isolated fragments in subsequent reasoning, significantly enhancing the ability to capture cross-paragraph logical relationships. Especially when dealing with complex structured texts such as legal documents and scientific papers, semantic super blocks can naturally correspond to functional modules such as background, method and conclusion, providing high-level semantic priori for subsequent attention selection and avoiding missing key information due to position deviation.

[0049] It should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit the application. Although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the application, and all should be covered in the scope of the claims of the application.

Claims

1. An algorithmic method for large model long context reasoning, characterized in that: The method comprises the following steps: dividing an input long text sequence into a plurality of initial text blocks; generating semantic summaries based on the initial text blocks, and performing clustering analysis on the semantic summaries to merge initial text blocks with similar semantics into semantic super blocks to form a context organization structure with semantic representation; generating key-value cache data of each token in a model preprocessing stage, grouping the key-value cache data according to the semantic super blocks as logical boundaries, and establishing a mapping table recording the storage location and state of each semantic super block; based on the semantic super blocks, combining the semantic representative vectors of the semantic super blocks to calculate the relevance scores between the query vector and each semantic super block, and learning a block importance prediction model based on context dependency features; sorting all semantic super blocks according to the relevance scores to determine target semantic super blocks participating in current attention calculation, querying the mapping table to obtain the physical storage location of the target semantic super blocks, and loading the key-value cache data to the central processor main memory according to the storage location; performing attention calculation on the loaded key-value cache data of the target semantic super blocks to generate a context weighted vector, transmitting the context weighted vector to a graphics processor, and outputting a new text unit; recording the access frequency of each semantic super block during the generation process, updating the hotness state in the mapping table according to the change of the access frequency, and triggering the migration operation of the key-value cache data between the solid state disk and the central processor according to the hotness state.

2. The algorithmic method for large model long context reasoning of claim 1, wherein: The input long text sequence is divided into a plurality of initial text blocks, and the specific steps are as follows: The input long text sequence is divided into a plurality of initial text blocks, and the specific steps are as follows: The long text sequence is provided by a user request and is represented as a continuous token sequence; The long text sequence is cut into segments of a fixed length, each segment containing the same number of continuous tokens, forming a preliminary data processing unit; Each segment is assigned a unique identifier, and the start and end positions in the original sequence are recorded to obtain a set of initial text blocks.

3. The algorithmic method for large model long context reasoning of claim 2, wherein: The specific steps of generating semantic summaries based on the initial text blocks and performing clustering analysis on the semantic summaries to merge initial text blocks with similar semantics into semantic super blocks to form a context organization structure with semantic representation are as follows: Each token in each initial text block is vectorized using embedding transformation to obtain a vector representation of each token; Perform mean aggregation on all token vectors within each initial text block to obtain a semantic summary vector; Perform semantic relationship analysis on all semantic summary vectors using a clustering algorithm, and merge initial text blocks into the same cluster based on vector similarity; Each cluster is defined as a semantic super block, each semantic super block contains a representative vector and a set of initial text blocks, forming a context organization structure.

4. The algorithmic method for large model long context reasoning of claim 3, wherein: The specific steps of generating key-value cache data of each token in a model preprocessing stage, grouping the key-value cache data according to the semantic super blocks as logical boundaries, and establishing a mapping table recording the storage location and state of each semantic super block are as follows: Perform Prefill processing on tokens in the input sequence using large model forward calculation to generate corresponding Key vectors and Value vectors layer by layer to form key-value cache data; The token range covered by the semantic super block is taken as a boundary to logically encapsulate the key-value cache data to form a cache data group; A storage allocation strategy is adopted to assign a physical location to each cache data group, and according to the position attribute and access characteristic of the semantic super block in the context, the semantic super block is respectively mapped to the graphics processor memory, the central processor main memory or the solid state disk; The storage information of each semantic super block is registered to establish a mapping table, which records the storage location, data offset, data size, data format and access heat state of the semantic super block.

5. The algorithmic method for large model long context reasoning of claim 4, wherein: Based on the semantic super block, the correlation score between the query vector and each semantic super block is calculated based on the semantic representative vector of the semantic super block, and the block importance prediction model is learned based on the context-dependent feature, and the specific steps are as follows: The semantic representative vector of the semantic super block and the attention query vector are taken as inputs to construct a block importance prediction model; The block importance prediction model is optimized by a supervised training method, and the context-dependent feature is used as a supervision signal; In the generation stage, the attention query vector is obtained, the attention query vector and the semantic representative vector are spliced and input into the block importance prediction model; The spliced vector is feature extracted by a nonlinear transformation, and a scalar value is output after calculation by a hidden layer, representing the semantic super block and the correlation score; The expression of the correlation score is: ; wherein, is the relevance score of the th semantic superblock, is the attention query vector of the current decoding step, is the semantic representative vector of the th semantic superblock, denotes the concatenation operation of the attention query vector and the semantic representative vector, , is the learnable weight matrix and vector, is the vector-type bias term, is the scalar-type bias term, is the Sigmoid activation function.

6. The algorithmic method for large model long context reasoning of claim 5, wherein: The correlation score is sorted in descending order by a sorting operation to obtain a score sequence and a corresponding semantic super block index; A threshold screening method is used to select semantic super blocks from the score sequence to form a candidate set; The semantic super blocks are selected from the candidate set as target semantic super blocks to participate in the attention calculation of the current decoding step; The storage location information of each target semantic super block is obtained by querying the mapping table; According to the storage location information, it is judged that the target semantic super block is currently located in the storage medium: if it is located in the central processor main memory, it is directly marked as a to-be-processed state; If it is located in the solid state disk, an asynchronous loading task is started to read and decompress the target semantic super block from the solid state disk and load it into the central processor main memory buffer area; The expression of the target semantic super block selection process is: The key-value cache data of the loaded target semantic super block is calculated by a sparse attention mechanism, and the specific process is as follows: ; wherein, is a set of determined target semantic superblocks, is a first semantic superblock, is a corresponding relevance score, is a descending order sorting function, is a take the first operation.

7. The algorithmic method for large model long context reasoning of claim 6, wherein: The query vector generated by the top attention layer of the model is obtained, and for the target semantic super block set, the original token contained in each semantic super block is traversed, and the Key vector pre-cached in the corresponding Transformer layer is extracted. The query vector and the Key vector of each token are matched in similarity, and the correlation is measured by dot product operation to obtain the original attention score. ​ ​ All scores are scaled, and all attention scores from target semantic superblocks are normalized. The Softmax function is used to calculate the normalized weight along the token sequence dimension, and the Value vector corresponding to the token is weighted and summed. Each Value vector is multiplied by its corresponding attention weight, and all weighted results are accumulated to generate a fixed-dimensional context weighted vector that integrates the key context information in all target semantic superblocks involved in the calculation. The attention query vector of the current decoding step is matched with the Key vector of all target semantic superblocks to calculate the attention weight. In the current decoding step of autoregressive generation, the attention query vector output by the model decoder top layer is obtained, and all determined target semantic superblocks are traversed to extract the Key vector of each semantic superblock containing each token in the corresponding Transformer layer. For each target token, the Key vector is dot multiplied with the attention query vector of the current decoding step to obtain the original similarity score. All original similarity scores are scaled by dividing each score by the square root of the Key vector dimension. All scaled scores from target semantic superblocks are combined into a unified score sequence, and the Softmax function is applied along the token dimension to normalize the sequence and generate a set of normalized attention weights. The context weighted vector is transmitted to the graphics processor through a high-speed interconnection channel. Residual connection, layer normalization, and feedforward network calculation are completed on the graphics processor to generate a new text unit.

8. The algorithmic method for large model long context reasoning of claim 7, wherein, During generation, the access frequency of each semantic superblock is recorded, the hotness state in the mapping table is updated according to the change of the access frequency, and the key-value cache data is migrated between the solid state disk and the central processing unit according to the hotness state. The specific steps are as follows: The counting mechanism is used to accumulate the access frequency of the target semantic superblock. The updated access frequency is written into the mapping table to update the hotness state synchronously. The migration judgment logic is used to evaluate the hotness state. When the access frequency of a semantic superblock exceeds the rising threshold, the key-value cache data is migrated from the solid state disk to the central processing unit main memory. When the access frequency is lower than the falling threshold, the compression migration from the main memory to the solid state disk is triggered. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: The processor executes the computer program to realize the steps of the algorithm method for large model long context reasoning according to any one of claims 1-8.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the steps of the algorithm method for large model long context reasoning according to any one of claims 1-8.

Images