An algorithm method for large model long context reasoning

Abstract

The application discloses an algorithm method for large model long context reasoning, and relates to the technical field of large language models, and comprises the following steps: dividing an input long text sequence into multiple initial text blocks; generating a semantic abstract based on the initial text blocks, and performing clustering analysis on the semantic abstract, merging initial text blocks with similar semantics into semantic super blocks to form a context organization structure with semantic representation; generating key-value cache data of each token in the large model preprocessing stage, grouping the key-value cache data with the semantic super blocks as logical boundaries, and establishing a mapping table recording the storage positions and states of each semantic super block; based on the semantic super blocks, combining the semantic representative vectors of the semantic super blocks, calculating the correlation scores between the query vector and each semantic super block, and learning a block importance prediction model based on context dependency features to provide high-order semantic prior for subsequent attention screening and avoid missing of key information caused by position offset.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model, and particularly relates to an algorithm method for long context reasoning of large model. BACKGROUND

[0002] Large language model refers to a deep learning model trained using a large amount of text data, so that the model can generate natural language text or understand the meaning of language text. These models can provide in-depth knowledge and language production on various topics by training on a large data set. The core idea is to learn the patterns and structures of natural language through large-scale unsupervised training, and to simulate the language cognition and generation process of human beings to a certain extent.

[0003] However, the existing technology still has essential defects in the long context management mechanism, and it is difficult to meet the comprehensive needs of efficiency, accuracy and scalability of large models in ultra-long sequence reasoning. The organization mode of the existing KVCache usually takes token or fixed length text block as the basic unit, and stores continuously according to the physical order of the input sequence, which lacks the modeling ability of the context semantic structure. This position-driven organization mode leads to the inability to effectively identify the context fragments that are semantically important when performing sparse attention or cache replacement, often causing generation distortion due to misdeletion or failure to load core semantic content. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides an algorithm method for long context reasoning of large model to solve the problem that the existing technology still has essential defects in the long context management mechanism, and it is difficult to meet the comprehensive needs of efficiency, accuracy and scalability of large models in ultra-long sequence reasoning. The organization mode of the existing KVCache usually takes token or fixed length text block as the basic unit, and stores continuously according to the physical order of the input sequence, which lacks the modeling ability of the context semantic structure. This position-driven organization mode leads to the inability to effectively identify the context fragments that are semantically important when performing sparse attention or cache replacement, often causing generation distortion due to misdeletion or failure to load core semantic content.

[0006] To solve the above technical problems, the present application provides the following technical scheme:

[0007] In a first aspect, the present application provides an algorithm method for long context reasoning of large model, which includes the following steps:

[0008] Divide the input long text sequence into multiple initial text blocks;

[0009] Generate a semantic summary based on the initial text block, and perform cluster analysis on the semantic summary to merge initial text blocks with similar semantics into semantic super blocks, forming a context organization structure with semantic representation;

[0010] Generate key-value cache data for each token in the large model preprocessing stage, group the key-value cache data according to the logical boundary of the semantic super block, and establish a mapping table recording the storage location and state of each semantic super block;

[0011] Based on the semantic super block, combined with the semantic representative vector of the semantic super block, calculate the relevance score between the query vector and each semantic super block, and the block importance prediction model learns based on the context dependence feature;

[0012] According to the relevance score, sort all semantic super blocks to determine the target semantic super block participating in the current attention calculation, query the mapping table to obtain the physical storage location of the target semantic super block, and load the key-value cache data to the central processor main memory according to the storage location;

[0013] Perform attention calculation on the loaded key-value cache data of the target semantic super block to generate a context weighted vector, transfer the context weighted vector to the graphics processor, and output a new text unit;

[0014] Record the access frequency of each semantic super block during generation, update the hotness state in the mapping table according to the change of the access frequency, and trigger the migration operation of the key-value cache data between the solid state disk and the central processor according to the hotness state.

[0015] As a preferred scheme of the algorithm method for large model long context reasoning, wherein: the input long text sequence is divided into multiple initial text blocks, and the specific steps are:

[0016] The input long text sequence is divided by the block strategy to obtain multiple initial text blocks;

[0017] The long text sequence is provided by user request and represented as a continuous token sequence;

[0018] The long text sequence is cut according to a fixed length, and each cut segment contains the same number of continuous tokens to form a preliminary data processing unit;

[0019] Each cut segment is assigned a unique identifier, and the start and end positions in the original sequence are recorded to obtain an initial text block set.

[0020] As a preferred scheme of the algorithm method for long context reasoning of large models, wherein: a semantic summary is generated based on the initial text blocks, and a cluster analysis is performed on the semantic summary, and the initial text blocks with similar semantics are merged into semantic super blocks to form a context organization structure with semantic representation, and the specific steps are:

[0021] The token in each initial text block is vectorized by using embedding transformation to obtain the vector representation of each token.

[0022] The mean aggregation operation is performed on all token vectors in each initial text block to obtain a semantic summary vector.

[0023] A clustering algorithm is used to analyze the semantic relationship of all semantic summary vectors, and the initial text blocks are merged into the same cluster according to the similarity between vectors.

[0024] Each cluster is defined as a semantic super block, each semantic super block contains a representative vector and a set of initial text blocks, and a context organization structure is formed.

[0025] As a preferred scheme of the algorithm method for long context reasoning of large models, wherein: key-value cache data of each token is generated in the model preprocessing stage, the key-value cache data is grouped with semantic super blocks as the logical boundary, and a mapping table is established to record the storage location and state of each semantic super block, and the specific steps are:

[0026] The Prefill processing is performed on the token in the input sequence by using the large model forward calculation, and the corresponding Key vector and Value vector are generated layer by layer to form the key-value cache data.

[0027] The token range covered by the semantic super block is taken as the boundary to logically encapsulate the key-value cache data to form a cache data group.

[0028] The storage allocation strategy is used to assign the physical location of each cache data group, and the semantic super block is mapped to the graphics processor video memory, the central processor main memory or the solid state disk according to the position attribute and access characteristic of the semantic super block in the context.

[0029] The storage information of each semantic super block is registered to establish a mapping table, and the mapping table records the storage location, data offset, data size, data format and access heat state of the semantic super block.

[0030] As a preferred scheme of the algorithm method for long context reasoning of large models, wherein: based on the semantic super block, the correlation score between the query vector and each semantic super block is calculated combined with the semantic representative vector of the semantic super block, and the block importance prediction model is learned based on the context dependence feature, and the specific steps are:

[0031] A block importance prediction model is constructed using the semantic representation vector and attention query vector of the semantic superblock as input.

[0032] The supervised training method optimizes the parameters of the block importance prediction model, using context-dependent features as supervision signals;

[0033] During the generation phase, the attention query vector is obtained, and the attention query vector is concatenated with the semantic representation vector and then input into the block importance prediction model.

[0034] Nonlinear transformation is used to extract features from the concatenated vector. After calculation by the hidden layer, a scalar value is output, representing the semantic superblock and relevance score.

[0035] The expression for the relevance score is as follows:

[0036] ;

[0037] in, For the first Relevance score of each semantic superblock This is the attention query vector for the current decoding step. For the first The semantic representation vector of a semantic superblock. This represents the concatenation operation between the attention query vector and the semantic representation vector. , For learnable weight matrices and vectors, For vector bias terms, It is a scalar bias term. This is the Sigmoid activation function.

[0038] As a preferred embodiment of the algorithm method for large-model long-context reasoning described in this invention, the following steps are taken: All semantic superblocks are sorted according to relevance scores to determine the target semantic superblocks participating in the current attention calculation; the physical storage location of the target semantic superblocks is obtained by querying the mapping table; and key-value cache data is loaded into the central processing unit's main memory according to the storage location.

[0039] The relevance scores are sorted in descending order using a sorting operation to obtain the score sequence and the corresponding semantic superblock index;

[0040] The threshold screening method selects semantic superblocks from the scoring sequence to form a candidate set;

[0041] Select a semantic superblock from the candidate set as the target semantic superblock and participate in the attention calculation of the current decoding step;

[0042] Query the mapping table to obtain the storage location information for each target semantic superblock;

[0043] According to the storage location information, it is judged that the target semantic super block is currently located in which storage medium: if it is located in the central processor main memory, it is directly marked as a to-be-processed state;

[0044] If it is located in the solid state disk, an asynchronous loading task is started, the solid state disk is read and decompressed, and then loaded into the central processor main memory buffer area;

[0045] Wherein, the expression of the selection process of the target semantic super block is:

[0046]

[0047] Wherein, is a determined target semantic super block set, is the i-th semantic super block, is the corresponding relevance score, is a descending order sorting function, is a front k operation. As a preferred scheme of the algorithm method for long context reasoning of large models according to the application, wherein: the key value cache data of the loaded target semantic super block is executed attention calculation, a context weighted vector is generated, the context weighted vector is transmitted to the graphics processor, and a new text unit is output, and the specific steps are:

[0048] The key value cache data of the target semantic super block is calculated through the sparse attention mechanism, and the specific process is as follows:

[0049] The query vector generated by the model top attention layer is obtained, each semantic super block in the target semantic super block set is traversed, the original token contained in each semantic super block is extracted, the Key vector pre-cached in the corresponding Transformer layer is extracted, the query vector and the Key vector of each token are matched in similarity, the relevance is measured through dot product operation, and the original attention score is obtained;

[0050] The attention scores from all target semantic super blocks are normalized, the Softmax function is used to calculate the normalized weight along the token sequence dimension, and the Value vector corresponding to the token is weighted and summed;

[0051] Each Value vector is multiplied by its corresponding attention weight, and all weighted results are accumulated to generate a fixed-dimensional context weighted vector, which integrates the key context information in all target semantic super blocks participating in the calculation;

[0052]

[0053] ​​​The attention query vector of the current decoding step is matched with the Key vectors of all target semantic superblocks to calculate the attention weights;

[0054] In the current decoding step of autoregressive generation, the attention query vector output by the top layer of the model decoder is obtained, and all determined target semantic superblocks are traversed to extract the Key vectors of each token contained in each semantic superblock in the corresponding Transformer layer;

[0055] For each target token, the Key vector is dot multiplied with the attention query vector of the current decoding step to obtain the original similarity score;

[0056] The original similarity scores are scaled, i.e., each score is divided by the square root value of the Key vector dimension, and the scaled scores from all target semantic superblocks are combined into a unified score sequence, and the sequence is normalized along the token dimension by applying the Softmax function to generate a set of normalized attention weights;

[0057] The Value vectors are weighted and summed to generate a context weighted vector.

[0058] The context weighted vector is transmitted to the graphics processor through a high-speed interconnection channel;

[0059] The residual connection, layer normalization and feedforward network calculation are completed on the graphics processor to generate a new text unit.

[0060] As a preferred scheme of the algorithm method for long context reasoning of large models, the access frequency of each semantic superblock is recorded during the generation process, the hotness state in the mapping table is updated according to the change of the access frequency, and the key-value cache data is migrated between the solid state disk and the central processing unit according to the hotness state, and the specific steps are as follows:

[0061] The access frequency of the target semantic superblock is accumulated by using a counting mechanism;

[0062] The updated access frequency is written into the mapping table to update the hotness state synchronously;

[0063] The hotness state is evaluated by using a migration judgment logic;

[0064] When the access frequency of a certain semantic superblock exceeds the rising threshold, the key-value cache data is migrated from the solid state disk to the central processing unit main memory;

[0065] When the access frequency is lower than the falling threshold, the compression migration from the main memory to the solid state disk is triggered.

[0066] In a second aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and wherein the computer program, when executed by the processor, implements any step of the algorithmic method for long-context reasoning of large models according to the first aspect of the present application.

[0067] In a third aspect, the present application provides a computer-readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements any step of the algorithmic method for long-context reasoning of large models according to the first aspect of the present application.

[0068] The present application has the following beneficial effects: by dividing the input long text sequence into multiple initial text blocks and clustering and merging them into semantic super blocks based on semantic summaries, the semantic hierarchical organization of the context is realized, which breaks the rigid structure of traditional position-order processing and constructs context units with semantic cohesion, ultimately improving the accuracy of key information recognition in the context and breaking through the mechanical division method of simply dividing long texts into fixed windows or sliding fragments in the prior art. By generating semantic summary vectors of each initial text block first and then clustering according to vector similarity, semantically coherent content is automatically merged into a semantic super block even if they are physically far apart in the original sequence. This semantic-driven reorganization mechanism enables the model to understand in topic paragraphs rather than isolated fragments in subsequent reasoning, significantly enhancing the ability to capture cross-paragraph logical relationships, especially when dealing with complex structured texts such as legal documents and research papers. Semantic super blocks can naturally correspond to functional modules such as background, method, and conclusion, providing high-level semantic prior for subsequent attention selection and avoiding key information omission due to position deviation. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0070] Figure 1 The flowchart of the algorithmic method for long-context reasoning of large models. DETAILED DESCRIPTION

[0071] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0072] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details set forth in this description. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present application.

[0073] Secondly, the "one embodiment" or "an embodiment" referred to herein means a specific feature, structure, or characteristic under discussion. Each of the various embodiments presented in this specification are not necessarily mutually exclusive, but can be combined with each other in various ways. Moreover, each of the embodiments presented in this specification can be combined with each other in various ways.

[0074] Reference Figure 1 For one embodiment of the present application, the embodiment provides an algorithmic method for large model long context reasoning, comprising the following steps:

[0075] S1, divide the input long text sequence into multiple initial text blocks.

[0076] Further, the input long text sequence is divided by a block strategy to obtain multiple initial text blocks;

[0077] The long text sequence is provided by user request and is represented as a continuous token sequence;

[0078] The long text sequence is cut according to a fixed length, and each cut segment contains the same number of continuous tokens to form a preliminary data processing unit;

[0079] A unique identifier is assigned to each cut segment, and the starting and ending positions in the original sequence are recorded to obtain an initial text block set.

[0080] It should be noted that the block strategy adopts a fixed-length non-overlapping cutting method, which aims to convert the long text sequence into a structured basic processing unit. This division method avoids the problem of cross-block semantic breakage and provides a unified data granularity for subsequent semantic summary generation. The identifier of each initial text block is bound to its position information in the original sequence, ensuring that its context source can be accurately traced during reasoning, and providing spatial positioning basis for subsequent cache management and attention calculation.

[0081] S2, generate semantic summaries based on the initial text blocks, and perform clustering analysis on the semantic summaries to merge the initial text blocks with similar semantics into semantic super blocks to form a context organization structure with semantic representation.

[0082] Further, an embedding transformation is used to vectorize the tokens in each initial text block to obtain the vector representation of each token.

[0083] performing a mean aggregation operation on all token vectors within each initial text block to obtain a semantic summary vector;

[0084] performing semantic relationship analysis on all semantic summary vectors by using a clustering algorithm, and merging the initial text blocks into the same cluster according to the similarity between vectors;

[0085] defining each cluster as a semantic super block, each semantic super block containing a representative vector and a set of initial text blocks, and forming a context organization structure.

[0086] It should be noted that the semantic summary vector is generated by the mean aggregation operation, which can effectively retain the overall semantic features of the initial text block while suppressing local noise interference. The semantic summary vector is grouped by using a clustering algorithm, which realizes the context reorganization based on semantic similarity, breaks through the limitations of traditional context organization by position order, and forms a semantic super block that not only contains text content with similar semantics, but also abstracts its core semantics through a representative vector, providing structured support for subsequent semantic-based relevance scoring and sparse attention mechanism.

[0087] S3, generating key-value cache data for each token in the large model preprocessing stage, grouping the key-value cache data according to the logical boundary of the semantic super block, and establishing a mapping table recording the storage location and state of each semantic super block.

[0088] Further, the Prefill process is performed on the tokens in the input sequence by using the forward calculation of the large model, and the corresponding Key vector and Value vector are generated layer by layer to form the key-value cache data;

[0089] Taking the token range covered by the semantic super block as the boundary, the key-value cache data is logically encapsulated to form a cache data group;

[0090] Using a storage allocation strategy to assign a physical location to each cache data group, and according to the position attribute and access characteristics of the semantic super block in the context, it will be mapped to the graphics processor memory, central processor main memory or solid state disk respectively;

[0091] Registering the storage information of each semantic super block, establishing a mapping table, and the mapping table records the storage location, data offset, data size, data format and access heat state of the semantic super block.

[0092] It should be noted that the key-value cache data is generated layer by layer during the model prefill stage, covering the key and value vectors of all input tokens in each Transformer layer. It is grouped with semantic superblocks as logical boundaries, which changes the organization of KVCache from position-driven to semantic-driven, supporting subsequent scheduling and computation at the semantic unit level. The mapping table, as a metadata management structure, records the physical storage status of each semantic superblock, realizing the decoupling of the logical semantic structure from the underlying storage system, and providing a control basis for efficient collaboration across storage levels.

[0093] S4. Based on semantic superblocks, and combined with the semantic representative vectors of the semantic superblocks, calculate the relevance score between the query vector and each semantic superblock. The block importance prediction model learns based on context-dependent features.

[0094] Furthermore, a block importance prediction model is constructed using the semantic representation vector and attention query vector of the semantic superblock as input.

[0095] The supervised training method optimizes the parameters of the block importance prediction model, using context-dependent features as supervision signals;

[0096] During the generation phase, the attention query vector is obtained, and the attention query vector is concatenated with the semantic representation vector and then input into the block importance prediction model.

[0097] Nonlinear transformation is used to extract features from the concatenated vector. After calculation by the hidden layer, a scalar value is output, representing the semantic superblock and relevance score.

[0098] The expression for the relevance score is as follows:

[0099] ;

[0100] in, For the first Relevance score of each semantic superblock This is the attention query vector for the current decoding step. For the first The semantic representation vector of a semantic superblock. This represents the concatenation operation between the attention query vector and the semantic representation vector. , For learnable weight matrices and vectors, For vector bias terms, It is a scalar bias term. This is the Sigmoid activation function.

[0101] It should be noted that the block importance prediction model is a lightweight learnable network, which fuses the current decoding state and the context semantic unit as input, can dynamically evaluate the contribution of different semantic superblocks to the current generation task, and the large model learns the context-dependent features through supervised training, avoiding the limitations of relying on fixed similarity measures in traditional methods. The relevance score output reflects the context relevance strength of the semantic superblock, providing a learnable and optimized decision basis for target selection in subsequent sparse attention.

[0102] S5, according to the relevance score, all semantic superblocks are sorted to determine the target semantic superblock participating in the current attention calculation, and the mapping table is queried to obtain the physical storage location of the target semantic superblock. The key value cache data is loaded into the central processor main memory according to the storage location.

[0103] Further, the relevance score is arranged in descending order by using the sorting operation to obtain the score sequence and the corresponding semantic superblock index;

[0104] The threshold screening method selects semantic superblocks from the score sequence to form a candidate set;

[0105] The semantic superblocks are selected from the candidate set as target semantic superblocks to participate in the attention calculation of the current decoding step;

[0106] The mapping table is queried to obtain the storage location information of each target semantic superblock;

[0107] According to the storage location information, it is judged that the target semantic superblock is currently located in the storage medium: if it is located in the central processor main memory, it is directly marked as a to-be-processed state;

[0108] If it is located in the solid state disk, an asynchronous loading task is started to read and decompress the solid state disk and load it into the central processor main memory cache area;

[0109] Wherein, the expression of the selection process of the target semantic superblock is:

[0110] ;

[0111] Wherein, is the determined target semantic superblock set, is the th semantic superblock, is the corresponding relevance score, is the descending order sorting function, is the operation of taking the first items.

[0112] It should be noted that the sorting and filtering mechanism arranges the relevance scores in descending order, preferentially retains the semantic superblocks with the highest semantic relevance for attention calculation, significantly reduces the context scale involved in the calculation, and the selection process of the target semantic superblock realizes the sparse focusing from the full context to the key semantic fragment, effectively alleviating the calculation and memory pressure brought by long context, and through the query mapping table, the physical storage location is obtained, the dynamic mapping of semantic logic and storage physical address is realized, the on-demand loading across storage media is supported, and the continuity and efficiency of the reasoning process are guaranteed.

[0113] S6, attention calculation is performed on the key-value cache data of the loaded target semantic superblock, a context weighted vector is generated, the context weighted vector is transmitted to the graphics processor, and a new text unit is output.

[0114] Further, the key-value cache data of the target semantic superblock is calculated by the sparse attention mechanism, and the specific process is as follows:

[0115] The query vector generated by the top attention layer of the model is obtained, and for the target semantic superblock set, each original token contained in each semantic superblock is traversed, and the Key vector pre-cached in the corresponding Transformer layer is extracted. The query vector and the Key vector of each token are matched in similarity, the relevance is measured by dot product operation, and the original attention score is obtained;

[0116] All attention scores from the target semantic superblock are normalized, and the Softmax function is used to calculate the normalized weight along the token sequence dimension, and the Value vector corresponding to the token is weighted and summed;

[0117] Each Value vector is multiplied by its corresponding attention weight, and all weighted results are accumulated to generate a fixed-dimensional context weighted vector, which integrates the key context information in all target semantic superblocks involved in the calculation;

[0118] The attention query vector of the current decoding step is matched with the Key vector of all target semantic superblocks in similarity, and the attention weight is calculated;

[0119] In the current decoding step of autoregressive generation, the attention query vector output by the top layer of the model decoder is obtained, all determined target semantic superblocks are traversed, and the Key vector pre-cached in the corresponding Transformer layer for each token contained in each semantic superblock is extracted;

[0120] For each target token, the Key vector and the attention query vector of the current decoding step are dot multiplied to obtain the original similarity score;

[0121] The original similarity scores are scaled, i.e. each score is divided by the square root of the dimension of the Key vector, the scaled scores from all target semantic superblocks are combined into a unified score sequence, and the sequence is normalized by applying a Softmax function along the token dimension to generate a set of normalized attention weights;

[0122] The Value vectors are weighted and summed to generate a context weighted vector.

[0123] The context weighted vector is transmitted to the graphics processor through a high-speed interconnection channel;

[0124] The residual connection, layer normalization and feedforward network calculation are completed on the graphics processor to generate a new text unit.

[0125] It should be noted that the sparse attention calculation process is only performed on the target semantic superblocks loaded into the central processor main memory, avoiding redundant calculation of irrelevant contexts. The attention query vector of the current decoding step is multiplied by the Key vectors of all tokens in each target semantic superblock to calculate the original similarity scores, which are then normalized by Softmax to obtain the normalized attention weight distribution. Finally, the Value vectors are weighted and summed according to the corresponding weights to generate a context weighted vector. This process significantly reduces the computational complexity while maintaining the quality of the model output, achieving efficient inference under long context.

[0126] S7, record the access frequency of each semantic superblock during the generation process, update the hotness state in the mapping table according to the change of the access frequency, and trigger the migration operation of the key-value cache data between the solid state disk and the central processor according to the hotness state.

[0127] Further, a counting mechanism is used to accumulate the access frequency of the target semantic superblock;

[0128] The updated access frequency is written into the mapping table, and the hotness state is updated synchronously;

[0129] The hotness state is evaluated using a migration judgment logic;

[0130] When the access frequency of a semantic superblock exceeds the rising threshold, the key-value cache data is migrated from the solid state disk to the central processor main memory;

[0131] When the access frequency is lower than the falling threshold, the compression migration from the main memory to the solid state disk is triggered.

[0132] It should be noted that the record of access frequency and the update of hotness state constitute the core feedback loop of the dynamic cache management mechanism. Each time a semantic superblock is selected as the target unit, its access frequency is incremented, reflecting its activity level in the generation process. The hotness state in the mapping table synchronizes the frequency changes in real time, providing a basis for decision-making for storage migration. The upper threshold and the lower threshold constitute a hysteresis control logic, preventing frequent migration due to short-term fluctuations and improving system stability. The asynchronous migration mechanism ensures that data flow between the solid state disk and the central processing unit main memory does not block the main reasoning process, achieving parallelization and coordination of computation and I / O.

[0133] The embodiment also provides a computer device suitable for the algorithm method for large model long context reasoning, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to realize the algorithm method for large model long context reasoning proposed in the above embodiment.

[0134] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, operator network, NFC (near field communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device. In addition, the input device can also be an external keyboard, touchpad or mouse, etc.

[0135] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to implement the algorithm method for long-context reasoning of a large model as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, a magnetic disk or an optical disk.

[0136] To sum up, the application achieves semantic hierarchical organization of context by dividing the input long text sequence into multiple initial text blocks and clustering and merging the initial text blocks into semantic super blocks based on semantic summaries, which breaks the rigid structure of traditional position sequence processing and constructs context units with semantic cohesion, thereby improving the accuracy of key information recognition in context and breaking through the mechanical division method of simply dividing long text into fixed windows or sliding segments in the prior art. The semantic summary vectors of each initial text block are generated first, and then clustering is performed according to the vector similarity, so that semantically coherent content is automatically merged into a semantic super block even if they are far apart in physical position in the original sequence. This semantic-driven reorganization mechanism enables the model to understand in topic paragraphs rather than isolated fragments in subsequent reasoning, significantly enhancing the ability to capture cross-paragraph logical relationships. Especially when dealing with complex structured texts such as legal documents and scientific papers, semantic super blocks can naturally correspond to functional modules such as background, method and conclusion, providing high-level semantic priori for subsequent attention selection and avoiding missing key information due to position deviation.

[0137] It should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit the application. Although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the application, which should be covered by the scope of the claims of the application.

Claims

1. An algorithmic method for long-context reasoning in large models, characterized in that: Includes the following steps: Divide the input long text sequence into multiple initial text blocks; A semantic summary is generated based on the initial text block, and cluster analysis is performed on the semantic summary to merge semantically similar initial text blocks into semantic superblocks, forming a semantically representative context organization structure. In the large model preprocessing stage, key-value cache data for each token is generated. The key-value cache data is grouped with semantic superblocks as logical boundaries, and a mapping table is established to record the storage location and status of each semantic superblock. Based on semantic superblocks, and combined with the semantic representative vectors of the semantic superblocks, the relevance score between the query vector and each semantic superblock is calculated. The block importance prediction model is learned based on context-dependent features. All semantic superblocks are sorted according to the relevance score to determine the target semantic superblocks participating in the current attention calculation. The mapping table is queried to obtain the physical storage location of the target semantic superblocks. The key-value cache data is loaded into the main memory of the central processing unit according to the storage location. Attention calculations are performed on the key-value cached data of the loaded target semantic superblock to generate a context-weighted vector. The context-weighted vector is then transmitted to the graphics processor to output a new text unit. During the generation process, the access frequency of each semantic superblock is recorded, and the popularity status in the mapping table is updated according to the changes in access frequency. Based on the popularity status, the key-value cache data is triggered to migrate between the solid-state drive and the central processing unit.

2. The algorithm method for long-context reasoning in large models as described in claim 1, characterized in that: The specific steps for dividing the input long text sequence into multiple initial text blocks are as follows: A block-splitting strategy is used to divide the input long text sequence into multiple initial text blocks; The long text sequence is provided by the user and is represented as a continuous sequence of tokens; The long text sequence is divided into segments of fixed length, each segment containing the same number of consecutive tokens, forming a preliminary data processing unit; Assign a unique identifier to each segment and record its start and end positions in the original sequence to obtain an initial set of text blocks.

3. The algorithm method for long-context reasoning in large models as described in claim 2, characterized in that: The steps for generating a semantic summary based on the initial text block, performing cluster analysis on the semantic summary, merging semantically similar initial text blocks into semantic superblocks, and forming a semantically representative contextual organization structure are as follows: The tokens in each initial text block are vectorized using an embedding transformation to obtain vector representations of each token; Perform mean aggregation on all token vectors within each initial text block to obtain a semantic summary vector; A clustering algorithm is used to perform semantic relationship analysis on all semantic summary vectors, and the initial text blocks are grouped into the same cluster based on the similarity between vectors; Each cluster is defined as a semantic superblock, and each semantic superblock contains a representative vector and an initial set of text blocks, forming a contextual organization structure.

4. The algorithm method for long-context reasoning in large models as described in claim 3, characterized in that: The steps for generating key-value cache data for each token during the model preprocessing stage, grouping the key-value cache data using semantic superblocks as logical boundaries, and establishing a mapping table recording the storage location and state of each semantic superblock are as follows: A large model forward computation is used to perform prefill processing on the tokens in the input sequence, generating corresponding key vectors and value vectors layer by layer to form key-value cache data; Using the token range covered by the semantic superblock as the boundary, key-value cache data is logically encapsulated to form cache data groups; A storage allocation strategy is adopted to assign physical locations to each cache data group. Based on the location attributes and access characteristics of the semantic superblock in the context, it will be mapped to the graphics processor memory, the central processing unit main memory or the solid-state drive respectively. The storage information of each semantic superblock is registered and a mapping table is established. The mapping table records the storage location, data offset, data size, data format and access popularity status of the semantic superblock.

5. The algorithm method for long-context reasoning in large models as described in claim 4, characterized in that: The process involves calculating the relevance score between the query vector and each semantic superblock based on the semantic superblock's semantic representative vector. The block importance prediction model is learned based on context-dependent features, and the specific steps are as follows: A block importance prediction model is constructed using the semantic representation vector and attention query vector of the semantic superblock as input. The supervised training method optimizes the parameters of the block importance prediction model, using context-dependent features as supervision signals; During the generation phase, the attention query vector is obtained, and the attention query vector is concatenated with the semantic representation vector and then input into the block importance prediction model. Nonlinear transformation is used to extract features from the concatenated vector. After calculation by the hidden layer, a scalar value is output, representing the semantic superblock and relevance score. The expression for the relevance score is as follows: ; in, For the first Relevance score of each semantic superblock This is the attention query vector for the current decoding step. For the first The semantic representation vector of a semantic superblock. This represents the concatenation operation between the attention query vector and the semantic representation vector. , For learnable weight matrices and vectors, For vector bias terms, It is a scalar bias term. This is the Sigmoid activation function.

6. The algorithm method for long-context reasoning in large models as described in claim 5, characterized in that: The steps are as follows: Sort all semantic superblocks according to relevance scores to determine the target semantic superblocks participating in the current attention calculation; query the mapping table to obtain the physical storage location of the target semantic superblocks; and load the key-value cache data into the central processing unit's main memory according to the storage location. The relevance scores are sorted in descending order using a sorting operation to obtain the score sequence and the corresponding semantic superblock index; The threshold screening method selects semantic superblocks from the scoring sequence to form a candidate set; Select a semantic superblock from the candidate set as the target semantic superblock and participate in the attention calculation of the current decoding step; Query the mapping table to obtain the storage location information for each target semantic superblock; The storage medium where the target semantic superblock is currently located is determined based on the storage location information: if it is located in the main memory of the central processing unit, it is directly marked as pending processing. If it is located on a solid-state drive, an asynchronous loading task is started to read from the solid-state drive, decompress it and load it into the CPU main memory cache. The expression for the target semantic superblock selection process is as follows: ; in, For a defined set of target semantic superblocks, For the first A semantic superblock, For the corresponding relevance score, It is a descending sorting function. To take the front Operations on items.

7. The algorithm method for long-context reasoning in large models as described in claim 6, characterized in that: The specific steps are as follows: Attention calculation is performed on the key-value cache data of the loaded target semantic superblock to generate a context-weighted vector. This context-weighted vector is then transmitted to the graphics processor to output a new text unit. The key-value cache data of the target semantic superblock is calculated using a sparse attention mechanism. The specific process is as follows: Obtain the query vector generated by the top attention layer of the model. For the target semantic superblock set, traverse the original tokens contained in each semantic superblock, extract the key vectors that are pre-cached in the corresponding Transformer layer, match the query vector with the key vectors of each token for similarity, measure their relevance through dot product operation, and obtain the original attention score. All scores are scaled, all attention scores from the target semantic superblock are normalized, the Softmax function is used to calculate the normalized weights along the token sequence dimension, and the value vectors of the corresponding tokens are weighted and summed. Each Value vector is multiplied by its corresponding attention weight, and all weighted results are summed to generate a fixed-dimensional context-weighted vector. The context-weighted vector integrates key contextual information from all target semantic superblocks involved in the computation. The attention query vector of the current decoding step is matched with the key vectors of all target semantic superblocks for similarity, and the attention weights are calculated. In the current decoding step generated by autoregression, the attention query vector output by the top layer of the model decoder is obtained, and the system traverses all the determined target semantic superblocks to extract the key vectors of each token contained in each semantic superblock that are pre-cached in the corresponding Transformer layer. For each target token, perform a dot product operation between the Key vector and the attention query vector of the current decoding step to obtain the original similarity score; All original similarity scores are scaled by dividing each score by the square root of the Key vector dimension. All scaled scores from the target semantic superblock are merged into a unified score sequence. The sequence is then normalized by applying the Softmax function along the token dimension to generate a set of normalized attention weights. The context-weighted vector is transmitted to the graphics processor via a high-speed interconnect channel; Residual connections, layer normalization, and feedforward network calculations are performed on a graphics processor to generate new text units.

8. The algorithm method for long-context reasoning in large models as described in claim 7, characterized in that, The process of recording the access frequency of each semantic superblock during generation, updating the heat status in the mapping table based on changes in access frequency, and triggering the migration operation of key-value cache data between the solid-state drive and the central processing unit based on the heat status are as follows: A counting mechanism is used to accumulate the access frequency of the semantic superblock of the target semantic superblock; Write the updated access frequency to the mapping table and update the popularity status synchronously. The heat status is evaluated using migration judgment logic; When the access frequency of a certain semantic superblock exceeds the rising threshold, the migration of key-value cache data from the solid-state drive to the central processing unit's main memory is triggered. When the access frequency falls below the threshold, a compression migration from main memory to solid-state drive is triggered.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the algorithmic method for long context reasoning of large models as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the algorithmic method for long context reasoning of large models as described in any one of claims 1 to 8.

Images