Data processing method and device, computer device, storage medium and computer program product

Abstract

The application relates to a data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: obtaining a word sequence corresponding to query content; performing block processing on the words in the word sequence to obtain a block sequence; determining target sampling information corresponding to a current word block in the block sequence based on initial sampling information and position information of the current word block in the block sequence; determining a corresponding associated word block from a forward word block based on the relevance between the forward word block and the current word block and the target sampling information; determining an updated word corresponding to the current word based on the relevance between a reference word and the current word; and determining reply content based on the updated words corresponding to the words in the word sequence. The application can improve the efficiency of data processing and the accuracy of reply content.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of artificial intelligence technology, Large Language Model (LLM) technology has emerged and is widely used in document analysis, code generation, and complex agent workflows. However, as a crucial component of LLM, the computational complexity of self-attention mechanisms increases quadratically with the length of the input sequence, posing a significant challenge to handling the ever-growing volume of input sequences. Related techniques address this by selecting K highly relevant words for attention processing instead of updating all words preceding the current word, aiming to reduce processing load. However, while pursuing high efficiency, these methods struggle to maintain the model's inference accuracy. Summary of the Invention

[0003] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve data processing efficiency and accuracy in response to the above-mentioned technical problems.

[0004] Firstly, this application provides a data processing method. The method includes:

[0005] Retrieve the word sequence corresponding to the query content;

[0006] The lexical sequences are divided into blocks to obtain block sequences;

[0007] For the current word block in the block sequence, based on the initial sampling information and the position information of the current word block in the block sequence, the target sampling information corresponding to the current word block is determined; wherein, the target sampling information is used to characterize the sampling quantity of the associated word block;

[0008] Based on the relevance between the preceding word block and the current word block and the target sampling information, a corresponding associated word block is determined from the preceding word block; wherein, the preceding word block is a word block in the block sequence that is earlier than the current word block;

[0009] For the current word element in the current word element block, based on the relevance between the reference word element and the current word element, the updated word element corresponding to the current word element is determined; wherein, the reference word element includes the related word element block and word elements in the current word element block that are in a position earlier than the current word element;

[0010] The response content is determined based on the updated tokens corresponding to each token in the token sequence.

[0011] Secondly, this application also provides a data processing apparatus. The apparatus includes:

[0012] The acquisition module is used to obtain the word sequence corresponding to the query content;

[0013] The block segmentation module is used to segment the lexical units in the lexical sequence into blocks to obtain a block sequence.

[0014] The first determining module is used to determine the target sampling information corresponding to the current word block in the block sequence based on the initial sampling information and the position information of the current word block in the block sequence; wherein, the target sampling information is used to characterize the sampling quantity of the associated word block;

[0015] The second determining module is used to determine the corresponding associated word block from the preceding word block based on the relevance between the preceding word block and the current word block and the target sampling information; wherein, the preceding word block is a word block in the block sequence that is earlier than the current word block;

[0016] The third determining module is used to determine the updated word element corresponding to the current word element in the current word element block based on the relevance between the reference word element and the current word element; wherein, the reference word element includes the associated word element block and word elements in the current word element block that are in a position earlier than the current word element;

[0017] The generation module determines the response content based on the updated tokens corresponding to each token in the token sequence.

[0018] In one embodiment, the first determining module is further configured to:

[0019] Obtain the number of word blocks between the position of the current word block in the block sequence and the position of the initial word block;

[0020] If the number of word blocks is greater than the initial sampling information, the sampling attenuation magnitude is determined based on the sampling attenuation factor and the position information of the current word block in the block sequence;

[0021] Based on the initial sampling information and the sampling attenuation magnitude, the target sampling information corresponding to the current word block is determined.

[0022] In one embodiment, the first determining module is further configured to:

[0023] If the number of word blocks is less than or equal to the initial sampling information, the target sampling information corresponding to the current word block is determined based on the number of word blocks.

[0024] In one embodiment, the second determining module is further configured to:

[0025] Obtain the magnitude information of the feature values ​​corresponding to the preceding word block, and the correlation between the preceding word block and the current word block;

[0026] Based on the amplitude information and the relevance, the information contribution of the forward lexical block to the current lexical block is determined;

[0027] The word block with the highest information contribution that satisfies the target sampling information is selected from the forward word blocks and designated as the associated word block.

[0028] In one embodiment, the second determining module is further configured to:

[0029] Norm calculation is performed on the feature values ​​corresponding to the forward lexical block to obtain the amplitude information of the feature values ​​corresponding to the forward lexical block;

[0030] The feature query corresponding to the current lexical block and the feature key corresponding to the preceding lexical block are fused to obtain the relevance between the preceding lexical block and the current lexical block.

[0031] In one embodiment, the second determining module is further configured to:

[0032] Obtain the feature query matrix and feature key matrix corresponding to the word sequence;

[0033] The query matrix and the feature key matrix are fused to obtain the word relevance matrix;

[0034] For each target region of the lexical relevance matrix, a first downsampling process is performed to obtain an intermediate relevance matrix;

[0035] A second downsampling process is performed on each target region of the intermediate relevance matrix to obtain the word block relevance matrix;

[0036] Based on the word block relevance matrix, the relevance between the preceding word block and the current word block is determined.

[0037] In one embodiment, the second determining module is further configured to:

[0038] For each target region of the lexical relevance matrix, determine the cumulative information of each element on the anti-diagonal line of the target region;

[0039] Based on the cumulative information corresponding to each target region in the lexical relevance matrix, the intermediate relevance matrix is ​​determined.

[0040] In one embodiment, the second determining module is further configured to:

[0041] For each target region of the intermediate relevance matrix, determine the pooling information of each element in the target region;

[0042] Based on the pooling information corresponding to each target region in the intermediate relevance matrix, the word block relevance matrix is ​​determined.

[0043] In one embodiment, the first determining module is further configured to:

[0044] The total number of word blocks is determined based on the total number of words in the word block sequence and the number of words corresponding to each word block;

[0045] The initial sampling information is determined based on the scaling factor corresponding to the length of the lexical sequence and the total number of lexical blocks.

[0046] In one embodiment, the second determining module is further configured to:

[0047] The first lexical block is determined from the forward lexical block based on the first number of lexical blocks closest to the initial lexical block;

[0048] A second lexical block is determined from the preceding lexical blocks based on the second number of lexical blocks closest to the current lexical block;

[0049] From the remaining word blocks, determine the word block with the highest relevance to the target sampling information, and determine the third word block; wherein, the remaining word blocks include word blocks other than the first word block and the second word block in the forward word blocks;

[0050] Based on the first word block, the second word block, and the third word block, a related word block is determined.

[0051] In one embodiment, the third determining module is further configured to:

[0052] Obtain the feature keys and feature values ​​corresponding to the reference lexical units;

[0053] Retrieve the feature query, feature key, and feature value corresponding to the current word element;

[0054] Based on the relevance between the feature query of the current word element and the target feature key, the feature values ​​corresponding to the target feature key are fused to obtain the updated word element corresponding to the current word element; wherein the target feature key includes the feature key of the reference word element.

[0055] In one embodiment, the generation module is further configured to:

[0056] The updated tokens in the token sequence are divided into blocks to obtain the updated block sequence;

[0057] For the current updated word block in the update block sequence, the target sampling information corresponding to the current updated word block is determined based on the initial sampling information and the position information of the current updated word block in the update block sequence;

[0058] Based on the relevance between the forward-updated lexical block and the currently updated lexical block and the target sampling information, the associated updated lexical block is determined from the forward-updated lexical block;

[0059] For the currently updated word element in the currently updated word element block, the target word element corresponding to the currently updated word element is determined based on the relevance between the reference updated word element and the currently updated word element;

[0060] The response content is determined based on the target word corresponding to each updated word in the word sequence.

[0061] In one embodiment, the generation module is further configured to:

[0062] Based on the updated tokens corresponding to each token in the token sequence, the initial tokens of the response content are determined;

[0063] Based on the relevance between the updated lexical units corresponding to each lexical unit and the initial lexical unit, the backward lexical units of the initial lexical unit are determined; wherein, the backward lexical units include lexical units that are located after the initial lexical unit;

[0064] Based on the initial lexical units and the subsequent lexical units, the response content is determined.

[0065] In one embodiment, the acquisition module is further configured to:

[0066] The text content and the analysis prompt are segmented into words to obtain the corresponding word sequence;

[0067] The generation module is also used for:

[0068] Based on the updated tokens corresponding to each token in the token sequence, the analysis and response content for the text content is determined.

[0069] In one embodiment, the acquisition module is further configured to:

[0070] The prompt text of the generated code content is segmented into words to obtain the corresponding word sequence;

[0071] The generation module is also used for:

[0072] The generated code content is determined based on the updated tokens corresponding to each token in the token sequence.

[0073] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement an embodiment of the method described in any of the embodiments of this disclosure.

[0074] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements an embodiment of the method described in any of the embodiments of this disclosure.

[0075] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements an embodiment of the method described in any of the embodiments of this disclosure.

[0076] The aforementioned data processing methods, apparatus, computer equipment, storage media, and computer program products, in this application, for the current lexical block in a block sequence, determine the target sampling information corresponding to the current lexical block based on initial sampling information and the position information of the current lexical block in the block sequence. Based on the relevance between the preceding lexical block and the current lexical block and the target sampling information, corresponding associated lexical blocks are determined from the preceding lexical blocks. That is, the earlier the position information of the current lexical block is, the greater its influence on subsequent layers, requiring sampling of more associated lexical blocks; correspondingly, the later the position information of the current lexical block is, the greater its influence on subsequent layers, allowing sampling of fewer associated lexical blocks to improve computational efficiency. The embodiments of this application employ a linearly decaying sparsity strategy, improving fidelity and inference efficiency. Furthermore, this application first divides the lexical sequence into blocks, and then quickly filters the associated lexical blocks corresponding to the current lexical block using a block-level sparsity strategy. Subsequently, for the current lexical block, based on the relevance between the reference lexical and the current lexical, the updated lexical corresponding to the current lexical is determined. This coarse-to-fine approach, compared to traditional fine-grained lexical-level calculations, can reduce the computation of attention logarithms, significantly shorten the inference time in the pre-filling stage, and improve the output efficiency of the response content. In addition, this application can be used as a training-free plugin, applicable to both dense models and integrated into sparse models to further compress the number of associated lexical blocks. Combined with sparse operators, it significantly reduces response content latency and memory overhead while maintaining model accuracy. Attached Figure Description

[0077] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.

[0078] Figure 2 This is a flowchart illustrating a data processing method in one embodiment;

[0079] Figure 3 This is a schematic diagram of the large language model network structure in one embodiment;

[0080] Figure 4 This is a schematic diagram of recursive error propagation in the attention mechanism network layer in one embodiment;

[0081] Figure 5 This is a schematic diagram illustrating how target sampling information decays as the position information of word blocks increases in one embodiment;

[0082] Figure 6 This is a flowchart illustrating the word block relevance data processing in another embodiment;

[0083] Figure 7 This is a flowchart illustrating a data processing method in one embodiment;

[0084] Figure 8 This is a schematic diagram of the target area for the first downsampling in one embodiment;

[0085] Figure 9 This is a schematic diagram of the filtering of related word blocks in one embodiment;

[0086] Figure 10 This is a flowchart illustrating a data processing method in one embodiment;

[0087] Figure 11 This is a data processing architecture diagram from one embodiment;

[0088] Figure 12 This is implementation effect data for one embodiment;

[0089] Figure 13 This is implementation effect data for another embodiment;

[0090] Figure 14 This is implementation effect data for yet another embodiment;

[0091] Figure 15 This is a schematic block diagram of the data processing device in one embodiment;

[0092] Figure 16 This is an internal structural diagram of a computer device in one embodiment;

[0093] Figure 17 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0094] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0095] The data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on the cloud or other servers. For example, terminal 102 obtains query content and sends it to server 104. Server 104 divides the lexical sequences into blocks to obtain a block sequence. For the current lexical block in the block sequence, based on initial sampling information and the position information of the current lexical block in the block sequence, it determines the target sampling information corresponding to the current lexical block; based on the relevance between the preceding lexical block and the current lexical block and the target sampling information, it determines the associated lexical block from the preceding lexical block; for the current lexical in the current lexical block, based on the relevance between the reference lexical and the current lexical, it determines the updated lexical corresponding to the current lexical; based on the updated lexical corresponding to each lexical in the lexical sequence, it determines the response content. The response content is then returned to terminal 102. The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0096] In one embodiment, such as Figure 2 As shown, a data processing method is provided, which can be applied to... Figure 1 Taking terminal 102 or server 104 as an example, the following steps are included:

[0097] Step S201: Obtain the word sequence corresponding to the query content.

[0098] Among them, the query content can include content in any field that expects to be processed by the large language model. The fields can include multiple types, such as the field of resource interaction, the medical field, the education field, the office field, the tourism field, etc. The query content can include the input question or uploaded files, pictures, etc. Taking the input question as an example, for example: Please analyze the technical differences between two files. It can be understood that the query content can be input into the large language model together with the context information, and the query content can also be input into the large language model together with the prompt. Therefore, in the embodiments of the present disclosure, the query content can generally refer to any content to be processed input into the large language model.

[0099] Among them, a token is the basic unit processed by the large language model, not strictly equivalent to a character or a word. A token in Chinese can correspond to one character or multiple characters, or can also correspond to punctuation marks. For example, "you" can be processed as one token, and "today" can be processed as one token. In English, it can also correspond to a complete word or sub-word. For example, the word "unbelievable" may be split into three tokens: "un", "believe", and "able". Punctuation marks such as "。" or "," can also be considered as one token. Exemplarily, the query content can be split into tokens one by one through a tokenization algorithm.

[0100] Exemplarily, for the tokenized tokens, through the embedding layer in the large language model, the tokens are mapped into high-dimensional vectors of a fixed length, which can be called token embeddings. For example: "天" corresponds to [0.23, -0.56, 0.89, …, 0.12]. The tokens processed in this application can be in the form of token embeddings of the tokens.

[0101] In the embodiments of the present disclosure, the token sequence can be tokens arranged in sequence based on the order of appearance of the tokens in the query content. For example: today (token1) weather (token2) is (token3) good (token4), and the corresponding token sequence can be expressed as: token1, token2, token3, and token4.

[0102] Step S203: Perform chunking processing on the tokens in the token sequence to obtain a chunk sequence.

[0103] In the specific implementation process, multiple consecutive tokens in the token sequence can be processed as one token chunk. Exemplarily, there are M tokens in the token sequence. The 1st token to the Bth token are used as token chunk 1, the (B + 1)th token to the 2Bth token are used as token chunk 2, the (2B + 1)th token to the 3Bth token are used as token chunk 3, and so on. In this way, token chunks 1, token chunks 2, …, token chunks n are obtained, and token chunks 1, token chunks 2 to token chunks n are used as the chunk sequence.

[0104] Step S205: For the current word block in the block sequence, based on the initial sampling information and the position information of the current word block in the block sequence, determine the target sampling information corresponding to the current word block; wherein, the target sampling information is used to characterize the sampling quantity of the associated word block.

[0105] In the specific implementation, the position information of the current word block in the block sequence is represented by i, meaning that the current word block is the i-th word block in the block sequence. The initial sampling information can be a preset value, which can be used as a hyperparameter of the model.

[0106] In this embodiment of the disclosure, the target sampling information can be along with the current word block position information. The decay decreases as the value increases. For example, the decay function is expressed as... ,in, along with If the value decreases as the value increases, then the target sampling information can be represented as: .in, This represents the initial sampling information. In the specific implementation, optionally, it includes the position information of the current lexical block. The corresponding preceding lexical blocks can all be retained, that is, all preceding lexical blocks are used as associated lexical blocks, and the target sampling information is obtained. Optionally, the position information of the current word block. Target sampling information with It decreases as it increases, for example: .

[0107] Step S207: Based on the relevance between the forward lexical block and the current lexical block and the target sampling information, determine the associated lexical block from the forward lexical block; wherein, the forward lexical block is the lexical block in the block sequence that is earlier than the current lexical block.

[0108] The relevance is used to characterize the degree of similarity between two word blocks. In the specific implementation, the relevance between the forward word block and the current word block can be obtained by calculating the cosine similarity, Euclidean distance, or dot product between the two word blocks. In an exemplary embodiment, the current word block is embedded in the form of a word embedding. Multiply by a trainable weight matrix Get the current word block Corresponding feature query . Forward lexical block ( Multiply by a trainable weight matrix , thus obtaining the forward word block Corresponding feature keys Current word block With forward word blocks The relevance can be expressed as: .

[0109] In this embodiment of the disclosure, the preceding word block is a word block in the block sequence that precedes the current word block. For example, if the current word block in the block sequence is the 3rd block, the preceding word blocks are the 1st and 2nd blocks. As another example, if the current word block in the block sequence is the 4th block, the preceding word blocks are the 1st, 2nd, and 3rd blocks.

[0110] In this embodiment of the disclosure, the term block with the highest relevance to the current term block can be selected from the forward term block. Each word block is used as a related word block.

[0111] Step S209: For the current word element in the current word element block, based on the relevance between the reference word element and the current word element, determine the updated word element corresponding to the current word element; wherein, the reference word element includes the related word element block and the word elements in the current word element block that are in a position earlier than the current word element.

[0112] The current word block contains multiple words, and each word in the current word block can be updated separately. For example, the current word block... Corresponding related word blocks ( Get the current word block. Each word element ( Feature query corresponding to ) Feature keys and eigenvalues Targeting related word blocks Each word in the current lexical block, as well as the word in the current lexical block that precedes the current word, are collectively referred to as reference words. Obtain reference terms Feature query Feature keys and eigenvalues . The updated lexical units can be obtained using the following formula:

[0113] (1)

[0114] in, express The dimension of the corresponding word embedding (or feature vector).

[0115] Step S210: Determine the response content based on the updated tokens corresponding to each token in the token sequence.

[0116] Steps S201 to S210 above belong to the pre-filling stage of the large language model inference process. In this stage, the attention mechanism network layer can contain multiple layers, as shown in the reference. Figure 3 The attention mechanism network layer 301 is used. Therefore, the updated words corresponding to each word in the word sequence output by the current layer l of the attention mechanism network are used as the input of the backward layer (l+1). In the backward layer (l+1) of the attention mechanism network, the process can also go through steps S201 to S210 as described above. It can be understood that the output of the backward layer of the attention mechanism network can be used as the input of the backward layer (l+2), and so on, until the output of the last layer of the attention mechanism network is obtained. Then, the autoregressive decoding stage is entered. Based on the stored updated words, the first content of the response is predicted. The first content and the updated words are input again into the large language model to output the next content. This process is repeated to obtain the complete response content.

[0117] Due to the causal constraint, in any of the attention mechanism networks... In the layer, the first Output of each word It is obtained strictly by weighted summation of the eigenvalue vectors of the preceding positions:

[0118] (2)

[0119] This formula reveals the fundamental asymmetry of lexical participation: the first eigenvalue As a component, it appears in each In the calculation of (i=1..N, where N represents the length of the word sequence), the feature value of the last word is compared to... Only participate in the last output The calculation of this intra-layer asymmetry is further amplified during cross-layer propagation. In deep large language models, the feature values ​​of the next layer... The transition of the (l+1)th layer is derived from the output of the current layer via a composite mapping T (including FFN (feedforward network), residual connections, and normalization 303).

[0120] (3)

[0121] Figure 4 This demonstrates the impact of inter-layer recursive error propagation. (Reference) Figure 4 As shown, the dark circles represent the lexical units that have been sparsified (removed). If the initial lexical units are pruned... Error 401 will cause global distortion in all lexical units at the next level, as shown in the figure. Affected lexical units include... , , etc. If the last word is trimmed. 403 only causes localized tail errors; affected lexical units include... Therefore, the lexical units at the initial positions act as "recursive anchors," if sparsification is performed at the l-th layer. The error propagates recursively and amplifies to all N words in the (l+1)th layer. Therefore, maintaining high fidelity of words in the initial positions is crucial for the accuracy of the global model. Conversely, words in later positions are more tolerant of sparsity.

[0122] Based on this, this application, for the current lexical block in the block sequence, determines the target sampling information corresponding to the current lexical block based on the initial sampling information and the position information of the current lexical block in the block sequence. Based on the relevance between the preceding lexical block and the current lexical block and the target sampling information, the corresponding associated lexical blocks are determined from the preceding lexical blocks. That is, the earlier the position information of the current lexical block is, the greater its influence on subsequent layers, and the more associated lexical blocks need to be sampled. Conversely, the later the position information of the current lexical block is, the greater its influence on subsequent layers, and the fewer associated lexical blocks can be sampled to improve computational efficiency. Compared with the Top-k algorithm in related technologies, which applies a constant computational upper limit to all query positions, the embodiments of this application adopt a linearly decaying sparsity strategy, which improves fidelity and inference efficiency. Furthermore, this application first divides the lexical sequence into blocks, and then quickly filters the related lexical blocks corresponding to the current lexical block using a block-level sparsity strategy. Subsequently, for the current lexical block, based on the relevance between the reference lexical and the current lexical, the updated lexical corresponding to the current lexical is determined. This coarse-to-fine approach, compared to traditional fine-grained lexical-level calculations, can reduce the computational attention logarithm, significantly shorten the inference time in the pre-filling stage, and improve the output efficiency of the response content.

[0123] In one embodiment, determining the target sampling information corresponding to the current lexical block based on the initial sampling information and the position information of the current lexical block in the block sequence includes:

[0124] Obtain the number of lexical blocks between the position of the current lexical block in the block sequence and the position of the initial lexical block.

[0125] If the number of word blocks is greater than the initial sampling information, the sampling attenuation magnitude is determined based on the sampling attenuation factor and the position information of the current word block in the block sequence.

[0126] Based on the initial sampling information and the sampling attenuation magnitude, the target sampling information corresponding to the current word block is determined.

[0127] Here, the initial lexical block refers to the first lexical block in the block sequence, or simply the first lexical block. In this embodiment of the disclosure, the number of lexical blocks between the position of the current lexical block in the block sequence and the position of the initial lexical block is used to characterize the position information of the current lexical block in the block sequence. It can be understood that the more lexical blocks there are, the farther away from the initial lexical block; the fewer lexical blocks there are, the greater the distance from the initial lexical block.

[0128] As shown in the above embodiments, the initial sampling information can be a preset value, which can be set as a hyperparameter of the model. Optionally, the initial sampling information is represented as... =4. For example, the current word block If it is the initial lexical block, the preceding lexical block it can utilize does not exist, the target sampling information is 1, and it has not been achieved. That is, the initial lexical block itself. For example, the current lexical block... If it's the second word block, the forward word block it can utilize is the initial word block, and the target sampling information is 2, which also fails to achieve the target. .

[0129] In an exemplary embodiment, the current word block It is the 4th word block, the target sampling information is 4, reaching... The available lexical blocks include the first to fourth lexical blocks, and the target sampling information is not attenuated. Current lexical block It is the 5th word block, reaching The available lexical blocks include the 1st to 5th lexical blocks, and the target sampling information can be attenuated.

[0130] In one exemplary embodiment, we linearly reduce the initial sampling information to . refer to Figure 5 As shown, when = When (represented as k_start in the diagram), the target sampling information is equal to... The corresponding coordinates of point B are 501. = The target sampling information is represented as This corresponds to the coordinates of point A, 503. For example, based on the coordinates of points A and B, it can be determined that... The linear relationship with the target sampling information is used to determine the current word block. In the interval Corresponding target sampling information .

[0131] (4)

[0132] in, Indicates the number of word blocks. Indicates initial sampling information, ∈(0,1] is the attenuation factor. This indicates a round-down operation.

[0133] In the above embodiments, when the number of word blocks is greater than the initial sampling information, based on the sampling attenuation factor... The position information of the current lexical block in the block sequence Determine the sampling attenuation amplitude Based on the initial sampling information and the sampling attenuation amplitude, the target sampling information corresponding to the current word block is determined. This ensures... When the target sampling information is less than or equal to the initial sampling information, the preceding word block is completely preserved, and the word block with the highest position is given the maximum fidelity. The target sampling information of subsequent blocks is linearly decayed, and the last word block is set to... This allows for adjustment of the total number of screening blocks by controlling the attenuation factor.

[0134] In one embodiment, after obtaining the number of lexical blocks between the position of the current lexical block in the block sequence and the position of the initial lexical block, the method further includes:

[0135] If the number of word blocks is less than or equal to the initial sampling information, the target sampling information corresponding to the current word block is determined based on the number of word blocks.

[0136] In this embodiment of the disclosure, for example, the initial sampling information is represented as: =5. For example, the current word block... If it is the initial lexical block (the first lexical block), there are no preceding lexical blocks it can utilize, and the distance between the current lexical block and the initial lexical block is equal to 0. Then, the target sampling information corresponding to the associated lexical block is... Equals 0. For example, the current word block... If it is the second lexical block, and the distance between the current lexical block and the initial lexical block is equal to 1, then the target sampling information corresponding to the associated lexical block is 1. For example, the current lexical block... If it is the 3rd word block, and the distance between the current word and the initial word is 2, then the target sampling information corresponding to the associated word block is 2.

[0137] In the above embodiments, when the number of word blocks is less than or equal to the initial sampling information, the target sampling information corresponding to the current word block is determined based on the number of word blocks. That is, within a preset range between the current word block and the initial word block. Within the current lexical block, the associated lexical block is the preceding lexical block of the current lexical block. There is no need to filter the preceding lexical blocks, which ensures the fidelity of important lexical blocks.

[0138] In one embodiment, determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information includes:

[0139] Obtain the magnitude information of the feature values ​​corresponding to the preceding word block, and the correlation between the preceding word block and the current word block;

[0140] Based on the amplitude information and the relevance, the information contribution of the forward lexical block to the current lexical block is determined;

[0141] The word block with the highest information contribution that satisfies the target sampling information is selected from the forward word blocks and designated as the associated word block.

[0142] In this embodiment of the disclosure, the magnitude information of the eigenvalue is used to describe the size of the eigenvalue. Referring to equation (2), the attention mechanism calculates the eigenvalue. The weighted sum, the magnitude of the eigenvalues ​​and the correlation. Similarly, it affects the outcome.

[0143] In an exemplary embodiment, various non-negative scalars can be used to describe the amplitude information, such as L1 norm, RMS (Root Mean Square), maximum absolute value, or average absolute value. As long as they can stably characterize the energy of the eigenvalue, they should be covered within the scope of protection of this application.

[0144] In the above embodiments, the information contribution of the preceding word block to the current word block is determined by the amplitude information of the feature values ​​corresponding to the preceding word block and the relevance. The word block with the largest information contribution satisfying the target sampling information is then selected as the associated word block from the preceding word blocks. Therefore, the aforementioned information contribution can simulate the actual output contribution, ensuring that words with moderate attention scores but carrying high-energy information are retained, thus reducing information loss.

[0145] In one embodiment, obtaining the magnitude information of the feature values ​​corresponding to the preceding lexical block, and the correlation between the preceding lexical block and the current lexical block, includes:

[0146] Norm calculation is performed on the feature values ​​corresponding to the forward lexical block to obtain the amplitude information of the feature values ​​corresponding to the forward lexical block.

[0147] The feature query corresponding to the current lexical block and the feature key corresponding to the preceding lexical block are fused to obtain the relevance between the preceding lexical block and the current lexical block.

[0148] In an exemplary embodiment, the feature values ​​of the forward lexical block are represented as The magnitude information of the feature values ​​of the forward word block can be represented as In one exemplary embodiment, the feature query of the current lexical block is represented as follows: The feature keys corresponding to the forward lexical blocks are represented as Output It is expressed as follows:

[0149] (5)

[0150] The relevance between the forward lexical block and the current lexical block is as follows: As shown in Equation (5), the attention probability is proportional to the dot product of the feature query and the feature key, i.e. ∝ Therefore, the information contribution of the forward lexical block is related to Proportional. To construct an efficient metric, embodiments of this disclosure perform a logarithmic transformation on the information contribution, converting multiplication into addition, resulting in:

[0151] (6)

[0152] In one exemplary embodiment, the output perception metric is represented as follows:

[0153] (7)

[0154] In equation (7), the routing term aligns with the standard output-aware metric, measuring the matching degree between the feature query and the feature key. The magnitude term captures the information density of the feature value vector. Optionally, this can be achieved by introducing a nonnegativity constraint. and balance coefficient This adjusts the numerical scale of both to ensure that the magnitude term functions as a distinguishing indicator without obscuring the semantics of the routing term. Optionally, Set it to 0.2.

[0155] In the above embodiments, the amplitude information of the feature values ​​corresponding to the forward lexical blocks is obtained by calculating the norm of the feature values ​​corresponding to the forward lexical blocks. This is consistent with the output formula (5). It directly corresponds to the eigenvalue, is mathematically more natural, and is stable in implementation. It can accurately represent the energy information of the eigenvalue.

[0156] In one embodiment, reference Figure 7 As shown, before determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information, the method further includes:

[0157] Step S701: Obtain the feature query matrix and feature key matrix corresponding to the word sequence.

[0158] In the specific implementation, each lexical unit is a 1×d vector. The lexical unit sequence contains N lexical units, labeled Token1, Token2…TokenN. Therefore, the lexical unit sequence can be represented as an N×d feature matrix. (Reference) Figure 6 As shown, the N×d feature matrix is ​​multiplied by the trainable weight matrix. This yields the feature query matrix 601 corresponding to the word sequence. This N×d feature matrix is ​​then multiplied by a trainable weight matrix. We obtain the feature key matrix 603 corresponding to the word sequence.

[0159] Step S703: The query matrix and the feature key matrix are fused to obtain the word relevance matrix.

[0160] In the specific implementation process, the feature query matrix 601 and the feature key matrix are multiplied to obtain the lexical relevance matrix 605. The elements 1 to N in the first row of the lexical relevance matrix 605 correspond to the relevance scores of Token1 with Token1, Token1 with Token2, Token1 with Token3, ..., Token1 with TokenN, respectively. Similarly, the elements 1 to N in the second row of the lexical relevance matrix 605 correspond to the relevance scores of Token2 with Token1, Token2 with Token2, Token2 with Token3, ..., Token2 with TokenN, respectively. And so on, the elements 1 to N in the Nth row of the lexical relevance matrix 605 correspond to the relevance scores of TokenN with Token1, TokenN with Token2, TokenN with Token3, ..., TokenN with TokenN, respectively.

[0161] Step S705: Perform a first downsampling process on each target region of the word relevance matrix to obtain an intermediate relevance matrix.

[0162] Here, target region 611 can represent a region of a preset size, such as an 8×8 region or a 16×16 region. The first downsampling process is performed on target region a1 to obtain a corresponding element, the first downsampling process is performed on target region a2 to obtain a corresponding element, and so on, until the first downsampling process is performed on target region am to obtain a corresponding element. Here, m equals (size of the lexical relevance matrix) / (size of the target region). After the first downsampling process, the size of the intermediate relevance matrix 607 is obtained, for example: (N / 16) × (N / 16).

[0163] Step S707: Perform a second downsampling process on each target region of the intermediate relevance matrix to obtain the word block relevance matrix.

[0164] The target region 613 can represent a region of a preset size, such as an 8×8 region or a 16×16 region. A second downsampling process is performed on target region b1 to obtain a corresponding element, the same process is performed on target region b2, and so on, until a second downsampling process is performed on target region bm to obtain a corresponding element. Here, m equals (the size of the intermediate relevance matrix) / (the size of the target region). After the second downsampling process, the size of the resulting word block relevance matrix 609 is, for example, (N / 128) × (N / 128).

[0165] Step S709: Based on the word block relevance matrix, determine the relevance between the forward word block and the current word block.

[0166] In the lexical block relevance matrix, each element This represents the relevance between two word groups. For example, This indicates the relevance between word block 1 and word block 3. This indicates the relevance between word block 2 and word block 5. Therefore, based on the positional information of the current word block, the relevance between the current word block and the preceding word block can be indexed.

[0167] In the above embodiments, the feature query matrix and feature key matrix corresponding to the word sequence are obtained; the query matrix and the feature key matrix are fused to obtain the word relevance matrix; the word relevance matrix is ​​downsampled to obtain the word block relevance matrix, thereby facilitating the indexing of the corresponding word block relevance and improving computational efficiency.

[0168] In one embodiment, the first downsampling process for each target region of the relevance matrix to obtain an intermediate relevance matrix includes:

[0169] For each target region in the lexical relevance matrix, the cumulative information of each element on the anti-diagonal line of the target region is determined.

[0170] Based on the cumulative information corresponding to each target region in the lexical relevance matrix, the intermediate relevance matrix is ​​determined.

[0171] In one exemplary embodiment, reference is made to Figure 8 As shown, for each target region 611 in the lexical relevance matrix, the cumulative information of each element on the anti-diagonal line 801 of the target region is determined. For example, for a 3×3 target region, the cumulative information of each element on its anti-diagonal line is processed, including... + + This accumulated information is used as an element of the target region. As a result, the number of elements in the intermediate relevance matrix is ​​reduced compared to the number of elements in the word relevance matrix, which is equivalent to downsampling the word relevance matrix.

[0172] In the above embodiments, in natural language processing, important attention patterns are usually vertical (focusing on specific lexical units) or diagonal (focusing on adjacent or related sequences). The geometric features of the anti-diagonal line enable it to naturally intersect with these vertical and diagonal patterns. Therefore, by determining the cumulative information of each element on the anti-diagonal line of the target region as the relevance score of the entire lexical block, it is possible to efficiently determine whether the block contains key information.

[0173] In one embodiment, the second downsampling process for each target region of the intermediate relevance matrix to obtain the word block relevance matrix includes:

[0174] For each target region of the intermediate relevance matrix, determine the pooling information of each element in the target region.

[0175] Based on the pooling information corresponding to each target region in the intermediate relevance matrix, the word block relevance matrix is ​​determined.

[0176] In the specific implementation process, for each target region 613 of the intermediate relevance matrix 607, pooling processing is performed on each element within the target region, for example, by calculating the average value, to obtain a corresponding pooling information. This pooling information is used as the relevance score representation of the corresponding target region. It can be understood that since the pooling processing of the elements in the target region 613 results in only one pooling information, it is equivalent to downsampling the intermediate relevance matrix. In an exemplary embodiment, the word block relevance matrix is ​​determined based on the pooling information corresponding to each target region in the intermediate relevance matrix.

[0177] It is understood that the embodiments of this disclosure can combine the first downsampling based on the cumulative information of each element on the anti-diagonal line of the target region in the above embodiments, and the second downsampling based on the pooling information of each element in the target region. This can both retain important relevance information and ensure that the size of the word block after downsampling meets the requirements.

[0178] In one embodiment, before determining the target sampling information corresponding to the current word block in the block sequence based on initial sampling information and the position information of the current word block in the block sequence, the method further includes:

[0179] The total number of word blocks is determined based on the total number of words in the word block sequence and the number of words corresponding to each word block.

[0180] The initial sampling information is determined based on the scaling factor corresponding to the length of the lexical sequence and the total number of lexical blocks.

[0181] In the specific implementation, the total number of lexical units in the lexical sequence is represented by N, and the number of lexical units corresponding to each lexical block is represented by B. Therefore, the total number of lexical blocks can be represented as... In one exemplary embodiment, the scaling factor decreases as the length of the lexical sequence increases. For example, when the lexical sequence length is 8k-16k, the corresponding scaling factor is 0.2. When the lexical sequence length is 16k or more, the corresponding scaling factor is 0.1. In one exemplary embodiment, the initial sampling information can be equal to the product of the scaling factor and the total number of lexical blocks. For example, when the lexical sequence length is 8k-16k, the corresponding initial sampling information is represented as 0.2 × 10^6. When the word sequence length is 16k or more, the corresponding initial sampling information is represented as 0.1× .

[0182] In the above embodiments, the total number of lexical blocks is determined based on the total number of lexical units in the lexical sequence and the number of lexical units corresponding to each lexical block. Initial sampling information is determined based on the scaling factor corresponding to the length of the lexical sequence and the total number of lexical blocks. This limitation on the size of the initial sampling information limits the total size of the associated data blocks, ensuring the controllability of the computation.

[0183] In one embodiment, determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information includes:

[0184] The first lexical block is determined from the forward lexical blocks based on the first number of lexical blocks closest to the initial lexical block.

[0185] A second lexical block is determined from the preceding lexical block based on the second number of lexical blocks closest to the current lexical block.

[0186] From the remaining word blocks, determine the word block with the highest relevance to the target sampling information, and determine the third word block; wherein, the remaining word blocks include word blocks other than the first word block and the second word block in the forward word blocks.

[0187] Based on the first word block, the second word block, and the third word block, a related word block is determined.

[0188] In the specific implementation process, the first quantity and the second quantity can be set according to requirements; for example, the first quantity is 4, and the second quantity is 4. In an exemplary embodiment, refer to... Figure 9 As shown, the current lexical block is represented as Bi. Due to the limitations of the causal mask, the current lexical block Bi can only select associated lexical blocks from the preceding lexical blocks. Optionally, the first number is 2, and the first number of lexical blocks closest to the initial lexical block B1 are B1 and B2. Optionally, the second number is 2, and the second number of lexical blocks closest to the current lexical block are B(i-1) and B(i). In an exemplary embodiment, for example, if the target sampling information is 8, then 8 third lexical blocks are determined from the remaining lexical blocks 901 (i.e., B3-B(i-2)). Then the associated lexical block consists of the first lexical block, the second lexical block, and the third lexical block.

[0189] In the above embodiments, by fixing and retaining the first number of word blocks closest to the initial word block and the second number of word blocks closest to the current word block, the first number of word blocks are related to the beginning of the query content, and the second number of word blocks are related to the current word block, which are relatively critical information. Fixing and retaining them is beneficial to the accuracy of reasoning.

[0190] In one embodiment, for a current word element in the current word element block, determining the updated word element corresponding to the current word element based on the relevance between the reference word element and the current word element includes:

[0191] Obtain the feature keys and feature values ​​corresponding to the reference lexical.

[0192] Retrieve the feature query, feature key, and feature value corresponding to the current word.

[0193] Based on the relevance between the feature query of the current word element and the target feature key, the feature values ​​corresponding to the target feature key are fused to obtain the updated word element corresponding to the current word element; wherein the target feature key includes the feature key of the reference word element.

[0194] In the specific implementation process, referencing lexical units Corresponding feature keys and eigenvalues You can refer to the lexical units Multiply by the trainable weight matrix respectively and trainable weight matrix Obtained. Current word element. Corresponding feature query Feature keys and eigenvalues This can be achieved by multiplying the current word by the trained weight matrix. , , Obtained. Current word element. The updated lexical can be obtained through equation (1).

[0195] The above embodiments improve computational efficiency by performing block-level filtering on related word blocks of the current word block and then performing precise word-level attention calculation on each word in the current word block.

[0196] In one embodiment, reference Figure 10 As shown, the step of determining the response content based on the updated tokens corresponding to each token in the token sequence includes:

[0197] Step S1001: The updated tokens in the token sequence are divided into blocks to obtain the updated block sequence.

[0198] In the specific implementation process, multiple consecutive updated words in the word sequence can be processed as an updated word block. For example, if there are M updated words in the word sequence, the first to the Bth updated words are treated as an updated word block 1, the (B+1)th to the 2Bth updated words are treated as an updated word block 2, the (2B+1)th to the 3Bth updated words are treated as an updated word block 3, and so on, to obtain updated word block 1, updated word block 2, ..., updated word block n. The updated word block 1, updated word block 2 to updated word block n are then treated as a block sequence.

[0199] Step S1003: For the current updated lexical block in the update block sequence, based on the initial sampling information and the position information of the current updated lexical block in the update block sequence, determine the target sampling information corresponding to the current updated lexical block.

[0200] In the specific implementation, the position information of the current updated word block in the block sequence is represented by i, meaning that the current updated word block is the i-th word block in the block sequence. One appeared.

[0201] In this embodiment of the disclosure, the target sampling information can be updated along with the current word block position information. The decay decreases as the value increases. For example, the decay function is expressed as... ,in, along with If the value decreases as the value increases, then the target sampling information can be represented as: .in, This represents the initial sampling information. In the specific implementation, optionally, the position information of the current update lexical block can be used. The corresponding forward update lexical blocks can all be retained, that is, all forward update lexical blocks are used as associated update lexical blocks, target sampling information. Optionally, update the position information of the current word block. Target sampling information with It decreases as it increases, for example: .

[0202] Step S1005: Based on the relevance between the forward-updated lexical block and the currently updated lexical block and the target sampling information, determine the associated updated lexical block from the forward-updated lexical block.

[0203] In the specific implementation, the relevance between the forward-updated lexical block and the currently updated lexical block can be obtained by calculating the cosine similarity, Euclidean distance, or dot product between the two updated lexical blocks. In an exemplary embodiment, the currently updated lexical block is in the form of word embeddings. Multiply by a trainable weight matrix Get the currently updated word block Corresponding feature query Update the word block forward. ( Multiply by a trainable weight matrix , thus obtaining the forward update lexical block Corresponding feature keys Currently updating the term block. With forward update lexical blocks The relevance can be expressed as: .

[0204] In this embodiment of the disclosure, the forward-updating lexical block is the updated lexical block in the update block sequence that is preceding the current lexical block. For example, if the current updated lexical block in the block sequence is the 3rd block, then the forward-updating lexical blocks of the current updated lexical block are the 1st and 2nd blocks. As another example, if the current updated lexical block in the block sequence is the 4th block, then the forward-updating lexical blocks of the current updated lexical block are the 1st, 2nd, and 3rd blocks.

[0205] In this embodiment of the disclosure, the term block with the highest relevance to the currently updated term block can be selected from the forward-updated term block. Each updated term block is used as an associated updated term block.

[0206] Step S1007: For the currently updated word element in the currently updated word element block, determine the target word element corresponding to the currently updated word element based on the relevance between the reference updated word element and the currently updated word element.

[0207] The reference updated lexical includes the associated updated lexical block and the updated lexical in the current updated lexical block whose position precedes the current updated lexical.

[0208] The current update term block contains multiple update terms, and each update term in the current update term block can be updated separately. For example, the current update term block... Corresponding related update term blocks ( Get the currently updated word block. Each updated terminology ( Feature query corresponding to ) Feature keys and eigenvalues Update related term blocks. Each updated term in the current block, as well as the updated terms that precede the current updated term in the current updated term block, are collectively referred to as reference updated terms. Get reference updated lexicons Feature query Feature keys and eigenvalues . The updated lexical can be obtained through equation (1).

[0209] Step S1009: Determine the response content based on the target word corresponding to each updated word in the word sequence.

[0210] Steps S1001 to S1009 described above belong to the pre-filling stage of the large language model's inference process. In this stage, the attention mechanism network layer can contain multiple layers. Therefore, the updated lexical units output from steps S201 to S210 in the current layer l of the attention network are used as inputs to the backward layer (l+1). The backward layer (l+1) of the attention mechanism network also undergoes the processing described in steps S1001 to S1009. It can be understood that the output of the backward layer of the attention mechanism network can then be used as input to the backward layer (l+2), and so on, until the output of the last layer of the attention mechanism network is obtained. Next, the autoregressive decoding stage begins. Based on the stored updated lexical units, the first content of the response is predicted. This first content, along with the updated lexical units, is then input back into the large language model to output the next content. This process is repeated until the complete response content is obtained.

[0211] In one embodiment, determining the response content based on the updated lexical units corresponding to each lexical unit in the lexical unit sequence includes:

[0212] The initial content of the response is determined based on the updated tokens corresponding to each token in the token sequence.

[0213] Based on the relevance between the updated lexical units corresponding to each lexical unit and the initial lexical units corresponding to the initial content, the backward lexical units of the initial lexical units are determined; wherein, the backward lexical units include lexical units that are located after the initial lexical units.

[0214] Based on the initial lexical units and the subsequent lexical units, the response content is determined.

[0215] In its implementation, the large language model stores the updated lexical units corresponding to each lexical unit in the lexical sequence. Using these updated lexical units, it calculates the initial content of the response, such as "artificial". Next, it calculates the feature query, feature key, and feature value of the initial lexical unit corresponding to the initial content, and stores these values. This storage area holds both the original query content and the initial content of the generated response. The initial content is then used to perform attention calculations with the feature key and feature value of the query content, focusing on all information from the beginning of the query content to the previous answer, generating the next content of the response. This process is continuously iterated; each time new content is generated, it is used as input to calculate the corresponding feature query, feature key, and feature value.

[0216] In the above embodiments, the initial content of the response is determined based on the updated lexical units corresponding to each lexical unit in the lexical unit sequence. The backward lexical units of the initial lexical units are determined based on the relevance between the updated lexical units corresponding to each lexical unit and the initial lexical units corresponding to the initial content. This achieves automatic generation of the response content.

[0217] In one embodiment, the query content includes text content and document analysis prompts for the text content, and obtaining the lexical sequence corresponding to the query content includes:

[0218] The text content and the analysis prompts are segmented into words to obtain the corresponding word sequence.

[0219] The step of determining the response content based on the updated tokens corresponding to each token in the token sequence includes:

[0220] Based on the updated tokens corresponding to each token in the token sequence, the analysis and response content for the text content is determined.

[0221] In a specific implementation process, the query content may include a speech draft T1, for example: "Dear teachers...". For this query content, the input analysis prompt P1 is, for example: "Role setting: You are a senior text analyst who is good at deconstructing the logical context of an article and capturing the implied meaning of the author; Task objective: Please read the text I provided and generate a detailed analysis report according to the following framework; Analysis framework: Core theme and summary, etc.". Exemplarily, perform word segmentation on T1, for example: "dear", "teachers", etc. Then, extract features from the word tokens after word segmentation to obtain the vector form corresponding to the word tokens. Next, perform chunking on the word tokens in the word token sequence to obtain a chunk sequence. For the current word token chunk in the chunk sequence, based on the initial sampling information and the position information of the current word token chunk in the chunk sequence, determine the target sampling information corresponding to the current word token chunk; wherein, the target sampling information is used to represent the sampling quantity of associated word token chunks. Based on the relevance between the forward word token chunk and the current word token chunk and the target sampling information, determine the associated word token chunks from the forward word token chunks. For the current word token in the current word token chunk, based on the relevance between the reference word token and the current word token, determine the updated word token corresponding to the current word token. Based on the updated word tokens respectively corresponding to each word token in the word token sequence, determine the reply content. For example: "This speech draft has high technical value and application scenarios, providing a reliable solution for technical improvement, etc.".

[0222] In one embodiment, the query content includes a prompt for generating code content, and the obtaining of the word token sequence corresponding to the query content includes:

[0223] Perform word segmentation on the prompt for generating code content to obtain the corresponding word token sequence.

[0224] The determining of the reply content based on the updated word tokens respectively corresponding to each word token in the word token sequence includes:

[0225] Based on the updated word tokens respectively corresponding to each word token in the word token sequence, determine the generated code content.

[0226] In a specific implementation process, the query content may include a prompt P2 for generating code content, such as: "Role setting: You are a senior software engineer with more than 10 years of experience; Task objective: Generate a high-quality and directly executable code based on the requirement description I provide; Code generation requirements: 1. Function implementation; 2. Code quality; 3. Robustness and error handling, etc.". Exemplarily, perform word segmentation on P2, such as: "role", "setting", etc. And extract features from the segmented tokens to obtain the vector form corresponding to the tokens. Then, perform chunking on the tokens in the token sequence to obtain a chunk sequence. For the current token chunk in the chunk sequence, based on the initial sampling information and the position information of the current token chunk in the chunk sequence, determine the target sampling information corresponding to the current token chunk; wherein, the target sampling information is used to represent the sampling quantity of associated token chunks. Based on the relevance between the forward token chunks and the current token chunk and the target sampling information, determine the associated token chunks from the forward token chunks. For the current token in the current token chunk, based on the relevance between the reference token and the current token, determine the updated token corresponding to the current token. Based on the updated tokens corresponding to each token in the token sequence, determine the reply content. For example: "class FileReadError(Exception): / / Custom exception: Used to handle errors related to file reading / / pass".

[0227] In a specific embodiment, the method of the present application can be applied to the scenario of large language model data processing for query content. In the related art, for the attention processing of tokens, K tokens with a relatively high relevance to the token are selected for update, rather than using all the tokens before the appearance of the token for update, in order to reduce the processing amount. However, while pursuing efficient processing, the above method is difficult to maintain the inference accuracy of the model. The present application provides a data processing method that can improve the efficiency and accuracy of large language model data processing.

[0228] Refer to Figure 11 As shown, the data processing method of the present application includes: a target sampling information determination stage 1101, a perception metric determination stage 1103, an associated token chunk screening stage 1105, a token update stage, and a reply content generation stage.

[0229] In the target sampling information determination stage 1101, the method includes: step S201, obtaining a lexical sequence corresponding to the query content. Step S203, dividing the lexical sequences into blocks to obtain a block sequence. Step S205, for the current lexical block in the block sequence, determining the target sampling information corresponding to the current lexical block based on initial sampling information and the position information of the current lexical block in the block sequence; wherein, the target sampling information is used to characterize the sampling quantity of associated lexical blocks. In an exemplary embodiment, determining the target sampling information corresponding to the current lexical block based on initial sampling information and the position information of the current lexical block in the block sequence includes: obtaining the number of lexical blocks between the position of the current lexical block in the block sequence and the position of the initial lexical block; if the number of lexical blocks is greater than the initial sampling information, determining the sampling attenuation amplitude based on a sampling attenuation factor and the position information of the current lexical block in the block sequence; and determining the target sampling information corresponding to the current lexical block based on the initial sampling information and the sampling attenuation amplitude. In one exemplary embodiment, after obtaining the number of word blocks between the position of the current word block in the block sequence and the position of the initial word block, the method further includes: if the number of word blocks is less than or equal to the initial sampling information, determining target sampling information corresponding to the current word block based on the number of word blocks. In another exemplary embodiment, before determining the target sampling information corresponding to the current word block in the block sequence based on the initial sampling information and the position information of the current word block in the block sequence, the method further includes: determining the total number of word blocks based on the total number of words in the word sequence and the number of words corresponding to each word block; and determining the initial sampling information based on the scaling factor corresponding to the length of the word sequence and the total number of word blocks.

[0230] In the perceptual metric determination stage 1103, the method includes: determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information, including: obtaining amplitude information of the feature value corresponding to the preceding lexical block, and the relevance between the preceding lexical block and the current lexical block; determining the information contribution of the preceding lexical block to the current lexical block based on the amplitude information and the relevance; and determining the lexical block with the largest information contribution satisfying the target sampling information from the preceding lexical blocks as the associated lexical block. In an exemplary embodiment, obtaining the amplitude information of the feature value corresponding to the preceding lexical block, and the relevance between the preceding lexical block and the current lexical block includes: performing norm calculation on the feature value corresponding to the preceding lexical block to obtain the amplitude information of the feature value corresponding to the preceding lexical block; and performing fusion processing on the feature query corresponding to the current lexical block and the feature key corresponding to the preceding lexical block to obtain the relevance between the preceding lexical block and the current lexical block. In an exemplary embodiment, before determining the associated lexical block from the forward lexical block based on the relevance between the forward lexical block and the current lexical block and the target sampling information, the method further includes: Step S701, obtaining the feature query matrix and feature key matrix corresponding to the lexical sequence. Step S703, fusing the query matrix and the feature key matrix to obtain a lexical relevance matrix. Step S705, performing a first downsampling process on each target region of the lexical relevance matrix to obtain an intermediate relevance matrix. Step S707, performing a second downsampling process on each target region of the intermediate relevance matrix to obtain a lexical block relevance matrix. Step S709, determining the relevance between the forward lexical block and the current lexical block based on the lexical block relevance matrix.

[0231] In one exemplary embodiment, the first downsampling process for each target region of the lexical relevance matrix to obtain an intermediate relevance matrix includes: determining the cumulative information of each element on the anti-diagonal of each target region in the lexical relevance matrix; and determining the intermediate relevance matrix based on the cumulative information corresponding to each target region in the lexical relevance matrix. In another exemplary embodiment, the second downsampling process for each target region of the intermediate relevance matrix to obtain a lexical block relevance matrix includes: determining the pooling information of each element in each target region of the intermediate relevance matrix; and determining the lexical block relevance matrix based on the pooling information corresponding to each target region in the intermediate relevance matrix.

[0232] In the screening stage 1105 of the associated word block, the method includes: determining the associated word block from the forward word block based on the relevance between the forward word block and the current word block and the target sampling information, including: determining a first word block from the forward word block based on a first number of word blocks closest to the initial word block; determining a second word block from the forward word block based on a second number of word blocks closest to the current word block; determining a third word block from the remaining word blocks based on the word block with the highest relevance to the target sampling information; wherein, the remaining word blocks include word blocks from the forward word block other than the first word block and the second word block; and determining the associated word block based on the first word block, the second word block, and the third word block.

[0233] In the lexical update stage, the method includes: for the current lexical in the current lexical block, determining the updated lexical corresponding to the current lexical based on the relevance between the reference lexical and the current lexical includes: obtaining the feature key and feature value corresponding to the reference lexical; obtaining the feature query, feature key, and feature value corresponding to the current lexical; and performing fusion processing on the feature value corresponding to the target feature key based on the relevance between the feature query of the current lexical and the target feature key to obtain the updated lexical corresponding to the current lexical; wherein the target feature key includes the feature key of the reference lexical.

[0234] In the response content generation stage, the method includes: determining the response content based on the updated lexical units corresponding to each lexical unit in the lexical unit sequence, including: dividing the updated lexical units in the lexical unit sequence into blocks to obtain an updated block sequence; for the current updated lexical unit block in the updated block sequence, determining the target sampling information corresponding to the current updated lexical unit block based on the initial sampling information and the position information of the current updated lexical unit block in the updated block sequence; determining the associated updated lexical unit block from the forward updated lexical unit block based on the relevance between the forward updated lexical unit block and the current updated lexical unit block and the target sampling information; for the current updated lexical unit in the current updated lexical unit block, determining the target lexical unit corresponding to the current updated lexical unit based on the relevance between the reference updated lexical unit and the current updated lexical unit; and determining the response content based on the target lexical unit corresponding to each updated lexical unit in the lexical unit sequence. In an exemplary embodiment, determining the response content based on the updated lexical units corresponding to each lexical unit in the lexical unit sequence includes: determining the initial lexical unit of the response content based on the updated lexical units corresponding to each lexical unit in the lexical unit sequence; determining the backward lexical unit of the initial lexical unit based on the relevance between the updated lexical units corresponding to each lexical unit and the initial lexical unit; wherein the backward lexical unit includes lexical units that are located after the initial lexical unit; and determining the response content based on the initial lexical unit and the backward lexical unit.

[0235] In a specific application scenario, during the specific implementation process, the query content may include a speech script T1, for example: "Dear teachers...". For this query content, the input analysis prompt P1 is, for example: "Role setting: You are a senior text analyst who is good at deconstructing the logical context of an article and capturing the implied meaning of the author; Task objective: Please read the text I provided and generate a detailed analysis report according to the following framework; Analysis framework: Core theme and summary, etc.". Exemplarily, perform word segmentation on T1, for example: "dear", "teachers", etc. Based on the updated tokens corresponding to each token in the token sequence, determine the reply content. For example: "This speech script has high technical value and application scenarios, providing a reliable solution for technical improvement, etc.".

[0236] In a specific application scenario, during the specific implementation process, the query content may include a prompt P2 for generating code content, for example: "Role setting: You are a senior software engineer with more than 10 years of experience; Task objective: Generate a high-quality, directly executable piece of code based on the requirements description I provided; Code generation requirements: 1. Function implementation; 2. Code quality; 3. Robustness and error handling, etc.". Exemplarily, perform word segmentation on P2, for example: "role", "setting", etc. And perform feature extraction on the segmented tokens to obtain the vector form corresponding to the tokens. Then, perform chunking on the tokens in the token sequence to obtain a chunk sequence. Based on the updated tokens corresponding to each token in the token sequence, determine the reply content. For example: "class FileReadError(Exception): / / Custom exception: Used to handle errors related to file reading / / pass".

[0237] This application is applied to large language models and achieves higher accuracy than existing zero-shot methods with a lower computational budget (reduced by about 15% - 20%), and even approaches the lossless effect of full attention (Dense) in some tasks. In addition, this solution has extremely high generality. When integrated as a plug-in into large language model C and based on the trained sparse model, it can still further compress the budget and significantly reduce the latency in the prefill stage. This feature proves that this invention can not only optimize general models but also explore the redundant space of dedicated sparse models, providing a robust and practical solution for the efficient deployment of large models on resource-constrained devices.

[0238] The total computational cost of the position decay strategy of this application is expressed as follows:

[0239] (8)

[0240] where, : Total computational cost after adopting the linear decay strategy. : The total length of the input sequence. Initial sampling information, corresponding to the unified baseline . Attenuation ratio To control the rate of budget decay. In the experiment, [the following was taken]: Uniform Baseline: That is, when all locations use Total calculations during budgeting. Savings due to attenuation: This refers to the computational savings of the attenuation strategy compared to a uniform baseline.

[0241] This formula shows that: when The smaller (the more aggressive the decay) or The larger the value (the longer the sequence), the greater the decay savings. This strategy ensures that sufficient budget is reserved at the initial position (recursive anchor point). To maintain the integrity of the recursive information flow, and at the same time, to determine the subsequent positions based on... Gradual budget cuts significantly reduce computing overhead.

[0242] The sparse inference method (Stem) for long-context large language models proposed in this invention achieves significant benefits in terms of model accuracy, computational efficiency, and generality through the Token Position Decay (TPD) strategy and Output Awareness Metric (OAM).

[0243] Achieve superior model accuracy with extremely low sparse budget. Compared to existing training-free sparse methods, this application maintains higher model accuracy with a significantly lower computational budget. (References) Figure 12 As shown, in LongBench benchmark tests, this invention achieves the highest average accuracy (AVG) among all sparse baselines on both the Llama-3.1-8B-Instruct and Qwen3-8B models while compressing the sparse budget to only 25%-31%. On Qwen3-8B, the average accuracy of this invention (STEM) (31.64%) is 1.37% and 1.18% higher than the Minference algorithm (30.27%) and XAttention algorithm (30.46%), respectively, which use higher budgets. On the Llama-3.1-8B model, the accuracy of this invention (41.48%) is almost on par with the baseline accuracy (42.02%) of full attention (Dense), achieving near-lossless sparsity. This demonstrates that protecting the "recursive anchor" and introducing "amplitude awareness" can effectively avoid the loss of long contextual information. Figure 12In the table, rows represent commonly used evaluation criteria in the model field, and columns represent different attention algorithms.

[0244] This application significantly reduces the latency of long context pre-filling with minimal additional overhead. By drastically reducing the number of lexical units involved in attention computation and employing efficient block sparse operators, this invention significantly reduces the latency of long sequence inference and exhibits superior short sequence adaptability compared to competing products. References Figure 13 As shown, in the practical tests of the Llama-3.1-8B-Instruct model (BatchSize=1, BF16), this invention demonstrates superior acceleration capabilities. Particularly in the 128K ultra-long context scenario, this invention significantly reduces the total latency from 1540ms of the standard FlashAttention-2 baseline to 420ms, achieving a 3.7x end-to-end speedup. Thanks to the efficient block-level downsampling design, the additional overhead for metric computation in the 128K length is only approximately 90ms. Furthermore, thanks to the position decay strategy, this application effectively reduces the average computational budget, maintaining the lowest latency across all test lengths from 16K to 128K, verifying the robustness and efficiency of this invention in different length scenarios.

[0245] This application also possesses excellent versatility and orthogonality. This invention is not only applicable to dense models but also has a "plug-in" characteristic, allowing seamless integration into already trained sparse models to exploit their remaining redundancy. References Figure 14 As shown, DeepSeek-V3.2, by applying this invention to the native Dynamic Sparse Attention (DSA) mechanism, further reduces the sparse budget by 15%, while even slightly improving the model's average accuracy (from 42.84% to 43.16%). MiniCPM-4.1, by applying this invention to the native InfLLMv2 mechanism, further reduces the sparse budget by 18%, while maintaining stable average accuracy (from 42.75% to 42.76%). This demonstrates that the "causal information flow" principle revealed in this application has universal applicability, providing additional efficiency gains for various large models without retraining model parameters, thus significantly reducing deployment costs.

[0246] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0247] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0248] In one embodiment, such as Figure 15 As shown, a data processing apparatus 1500 is provided, comprising:

[0249] The acquisition module 1501 is used to acquire the word sequence corresponding to the query content;

[0250] Blocking module 1503 is used to divide the lexical units in the lexical sequence into blocks to obtain a block sequence;

[0251] The first determining module 1505 is used to determine the target sampling information corresponding to the current word block in the block sequence based on the initial sampling information and the position information of the current word block in the block sequence; wherein, the target sampling information is used to characterize the sampling quantity of the associated word block;

[0252] The second determining module 1507 is used to determine the corresponding associated word block from the preceding word block based on the relevance between the preceding word block and the current word block and the target sampling information; wherein, the preceding word block is a word block in the block sequence that is earlier than the current word block;

[0253] The third determining module 1509 is used to determine the updated word element corresponding to the current word element based on the relevance between the reference word element and the current word element in the current word element block; wherein, the reference word element includes the associated word element block and word elements in the current word element block that are positioned earlier than the current word element;

[0254] The generation module 1511 determines the response content based on the updated word units corresponding to each word unit in the word unit sequence.

[0255] In one embodiment, the first determining module is further configured to:

[0256] Obtain the number of word blocks between the position of the current word block in the block sequence and the position of the initial word block;

[0257] If the number of word blocks is greater than the initial sampling information, the sampling attenuation magnitude is determined based on the sampling attenuation factor and the position information of the current word block in the block sequence;

[0258] Based on the initial sampling information and the sampling attenuation magnitude, the target sampling information corresponding to the current word block is determined.

[0259] In one embodiment, the first determining module is further configured to:

[0260] If the number of word blocks is less than or equal to the initial sampling information, the target sampling information corresponding to the current word block is determined based on the number of word blocks.

[0261] In one embodiment, the second determining module is further configured to:

[0262] Obtain the magnitude information of the feature values ​​corresponding to the preceding word block, and the correlation between the preceding word block and the current word block;

[0263] Based on the amplitude information and the relevance, the information contribution of the forward lexical block to the current lexical block is determined;

[0264] The word block with the highest information contribution that satisfies the target sampling information is selected from the forward word blocks and designated as the associated word block.

[0265] In one embodiment, the second determining module is further configured to:

[0266] Norm calculation is performed on the feature values ​​corresponding to the forward lexical block to obtain the amplitude information of the feature values ​​corresponding to the forward lexical block;

[0267] The feature query corresponding to the current lexical block and the feature key corresponding to the preceding lexical block are fused to obtain the relevance between the preceding lexical block and the current lexical block.

[0268] In one embodiment, the second determining module is further configured to:

[0269] Obtain the feature query matrix and feature key matrix corresponding to the word sequence;

[0270] The query matrix and the feature key matrix are fused to obtain the word relevance matrix;

[0271] For each target region of the lexical relevance matrix, a first downsampling process is performed to obtain an intermediate relevance matrix;

[0272] A second downsampling process is performed on each target region of the intermediate relevance matrix to obtain the word block relevance matrix;

[0273] Based on the word block relevance matrix, the relevance between the preceding word block and the current word block is determined.

[0274] In one embodiment, the second determining module is further configured to:

[0275] For each target region of the lexical relevance matrix, determine the cumulative information of each element on the anti-diagonal line of the target region;

[0276] Based on the cumulative information corresponding to each target region in the lexical relevance matrix, the intermediate relevance matrix is ​​determined.

[0277] In one embodiment, the second determining module is further configured to:

[0278] For each target region of the intermediate relevance matrix, determine the pooling information of each element in the target region;

[0279] Based on the pooling information corresponding to each target region in the intermediate relevance matrix, the word block relevance matrix is ​​determined.

[0280] In one embodiment, the first determining module is further configured to:

[0281] The total number of word blocks is determined based on the total number of words in the word block sequence and the number of words corresponding to each word block;

[0282] The initial sampling information is determined based on the scaling factor corresponding to the length of the lexical sequence and the total number of lexical blocks.

[0283] In one embodiment, the second determining module is further configured to:

[0284] The first lexical block is determined from the forward lexical block based on the first number of lexical blocks closest to the initial lexical block;

[0285] A second lexical block is determined from the preceding lexical blocks based on the second number of lexical blocks closest to the current lexical block;

[0286] From the remaining word blocks, determine the word block with the highest relevance to the target sampling information, and determine the third word block; wherein, the remaining word blocks include word blocks other than the first word block and the second word block in the forward word blocks;

[0287] Based on the first word block, the second word block, and the third word block, a related word block is determined.

[0288] In one embodiment, the third determining module is further configured to:

[0289] Obtain the feature keys and feature values ​​corresponding to the reference lexical units;

[0290] Retrieve the feature query, feature key, and feature value corresponding to the current word element;

[0291] Based on the relevance between the feature query of the current word element and the target feature key, the feature values ​​corresponding to the target feature key are fused to obtain the updated word element corresponding to the current word element; wherein the target feature key includes the feature key of the reference word element.

[0292] In one embodiment, the generation module is further configured to:

[0293] The updated tokens in the token sequence are divided into blocks to obtain the updated block sequence;

[0294] For the current updated word block in the update block sequence, the target sampling information corresponding to the current updated word block is determined based on the initial sampling information and the position information of the current updated word block in the update block sequence;

[0295] Based on the relevance between the forward-updated lexical block and the currently updated lexical block and the target sampling information, the associated updated lexical block is determined from the forward-updated lexical block;

[0296] For the currently updated word element in the currently updated word element block, the target word element corresponding to the currently updated word element is determined based on the relevance between the reference updated word element and the currently updated word element;

[0297] The response content is determined based on the target word corresponding to each updated word in the word sequence.

[0298] In one embodiment, the generation module is further configured to:

[0299] Based on the updated tokens corresponding to each token in the token sequence, the initial tokens of the response content are determined;

[0300] Based on the relevance between the updated lexical units corresponding to each lexical unit and the initial lexical unit, the backward lexical units of the initial lexical unit are determined; wherein, the backward lexical units include lexical units that are located after the initial lexical unit;

[0301] Based on the initial lexical units and the subsequent lexical units, the response content is determined.

[0302] In one embodiment, the acquisition module is further configured to:

[0303] The text content and the analysis prompt are segmented into words to obtain the corresponding word sequence;

[0304] The generation module is also used for:

[0305] Based on the updated tokens corresponding to each token in the token sequence, the analysis and response content for the text content is determined.

[0306] In one embodiment, the acquisition module is further configured to:

[0307] The prompt text of the generated code content is segmented into words to obtain the corresponding word sequence;

[0308] The generation module is also used for:

[0309] The generated code content is determined based on the updated tokens corresponding to each token in the token sequence.

[0310] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0311] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 16As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data for data processing, such as feature key data, feature value data, and feature query data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0312] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 17 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data processing method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0313] Those skilled in the art will understand that Figure 17The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0314] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0315] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0316] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0317] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: Retrieve the word sequence corresponding to the query content; The lexical sequences are divided into blocks to obtain block sequences; For the current word block in the block sequence, based on the initial sampling information and the position information of the current word block in the block sequence, the target sampling information corresponding to the current word block is determined; wherein, the target sampling information is used to characterize the sampling quantity of the associated word block; Based on the relevance between the preceding word block and the current word block and the target sampling information, a corresponding associated word block is determined from the preceding word block; wherein, the preceding word block is a word block in the block sequence that is earlier than the current word block; For the current word element in the current word element block, based on the relevance between the reference word element and the current word element, the updated word element corresponding to the current word element is determined; wherein, the reference word element includes the related word element block and word elements in the current word element block that are in a position earlier than the current word element; The response content is determined based on the updated tokens corresponding to each token in the token sequence.

2. The method according to claim 1, characterized in that, The step of determining the target sampling information corresponding to the current lexical block based on the initial sampling information and the position information of the current lexical block in the block sequence includes: Obtain the number of word blocks between the position of the current word block in the block sequence and the position of the initial word block; If the number of word blocks is greater than the initial sampling information, the sampling attenuation magnitude is determined based on the sampling attenuation factor and the position information of the current word block in the block sequence; Based on the initial sampling information and the sampling attenuation magnitude, the target sampling information corresponding to the current word block is determined.

3. The method according to claim 2, characterized in that, After obtaining the number of lexical blocks between the position of the current lexical block in the block sequence and the position of the initial lexical block, the method further includes: If the number of word blocks is less than or equal to the initial sampling information, the target sampling information corresponding to the current word block is determined based on the number of word blocks.

4. The method according to claim 1, characterized in that, The step of determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information includes: Obtain the magnitude information of the feature values ​​corresponding to the preceding word block, and the correlation between the preceding word block and the current word block; Based on the amplitude information and the relevance, the information contribution of the forward lexical block to the current lexical block is determined; The word block with the highest information contribution that satisfies the target sampling information is selected from the forward word blocks and designated as the associated word block.

5. The method according to claim 4, characterized in that, The step of obtaining the magnitude information of the feature values ​​corresponding to the preceding lexical block, and the correlation between the preceding lexical block and the current lexical block, includes: Norm calculation is performed on the feature values ​​corresponding to the forward lexical block to obtain the amplitude information of the feature values ​​corresponding to the forward lexical block; The feature query corresponding to the current lexical block and the feature key corresponding to the preceding lexical block are fused to obtain the relevance between the preceding lexical block and the current lexical block.

6. The method according to claim 1, characterized in that, Before determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information, the method further includes: Obtain the feature query matrix and feature key matrix corresponding to the word sequence; The query matrix and the feature key matrix are fused to obtain the word relevance matrix; For each target region of the lexical relevance matrix, a first downsampling process is performed to obtain an intermediate relevance matrix; A second downsampling process is performed on each target region of the intermediate relevance matrix to obtain the word block relevance matrix; Based on the word block relevance matrix, the relevance between the preceding word block and the current word block is determined.

7. The method according to claim 6, characterized in that, The first downsampling process is performed on each target region of the word relevance matrix to obtain an intermediate relevance matrix, including: For each target region of the lexical relevance matrix, determine the cumulative information of each element on the anti-diagonal line of the target region; Based on the cumulative information corresponding to each target region in the lexical relevance matrix, the intermediate relevance matrix is ​​determined.

8. The method according to claim 6, characterized in that, The second downsampling process is performed on each target region of the intermediate relevance matrix to obtain the word block relevance matrix, including: For each target region of the intermediate relevance matrix, determine the pooling information of each element in the target region; Based on the pooling information corresponding to each target region in the intermediate relevance matrix, the word block relevance matrix is ​​determined.

9. The method according to claim 1, characterized in that, Before determining the target sampling information corresponding to the current word block in the block sequence based on the initial sampling information and the position information of the current word block in the block sequence, the method further includes: The total number of word blocks is determined based on the total number of words in the word block sequence and the number of words corresponding to each word block; The initial sampling information is determined based on the scaling factor corresponding to the length of the lexical sequence and the total number of lexical blocks.

10. The method according to claim 1, characterized in that, The step of determining the associated lexical block from the preceding lexical block based on the relevance between the preceding lexical block and the current lexical block and the target sampling information includes: The first lexical block is determined from the forward lexical block based on the first number of lexical blocks closest to the initial lexical block; A second lexical block is determined from the preceding lexical blocks based on the second number of lexical blocks closest to the current lexical block; From the remaining word blocks, determine the word block with the highest relevance to the target sampling information, and determine the third word block; wherein, the remaining word blocks include word blocks other than the first word block and the second word block in the forward word blocks; Based on the first word block, the second word block, and the third word block, a related word block is determined.

11. The method according to claim 1, characterized in that, For the current word element in the current word element block, determining the updated word element corresponding to the current word element based on the relevance between the reference word element and the current word element includes: Obtain the feature keys and feature values ​​corresponding to the reference lexical units; Retrieve the feature query, feature key, and feature value corresponding to the current word element; Based on the relevance between the feature query of the current word element and the target feature key, the feature values ​​corresponding to the target feature key are fused to obtain the updated word element corresponding to the current word element; wherein the target feature key includes the feature key of the reference word element.

12. The method according to claim 1, characterized in that, The step of determining the response content based on the updated tokens corresponding to each token in the token sequence includes: The updated tokens in the token sequence are divided into blocks to obtain the updated block sequence; For the current updated word block in the update block sequence, the target sampling information corresponding to the current updated word block is determined based on the initial sampling information and the position information of the current updated word block in the update block sequence; Based on the relevance between the forward-updated lexical block and the currently updated lexical block and the target sampling information, the associated updated lexical block is determined from the forward-updated lexical block; For the currently updated word element in the currently updated word element block, the target word element corresponding to the currently updated word element is determined based on the relevance between the reference updated word element and the currently updated word element; The response content is determined based on the target word corresponding to each updated word in the word sequence.

13. The method according to claim 1, characterized in that, The step of determining the response content based on the updated tokens corresponding to each token in the token sequence includes: Based on the updated tokens corresponding to each token in the token sequence, the initial tokens of the response content are determined; Based on the relevance between the updated lexical units corresponding to each lexical unit and the initial lexical unit, the backward lexical units of the initial lexical unit are determined; wherein, the backward lexical units include lexical units that are located after the initial lexical unit; Based on the initial lexical units and the subsequent lexical units, the response content is determined.

14. The method according to claim 1, characterized in that, The query content includes text content and document analysis prompts for the text content. Obtaining the lexical sequence corresponding to the query content includes: The text content and the analysis prompt are segmented into words to obtain the corresponding word sequence; The step of determining the response content based on the updated tokens corresponding to each token in the token sequence includes: Based on the updated tokens corresponding to each token in the token sequence, the analysis and response content for the text content is determined.

15. The method according to claim 1, characterized in that, The query content includes a prompt to generate code content, and obtaining the lexical sequence corresponding to the query content includes: The prompt text of the generated code content is segmented into words to obtain the corresponding word sequence; The step of determining the response content based on the updated tokens corresponding to each token in the token sequence includes: The generated code content is determined based on the updated tokens corresponding to each token in the token sequence.

16. A data processing apparatus, characterized in that, The device includes: The acquisition module is used to obtain the word sequence corresponding to the query content; The block segmentation module is used to segment the lexical units in the lexical sequence into blocks to obtain a block sequence. The first determining module is used to determine the target sampling information corresponding to the current word block in the block sequence based on the initial sampling information and the position information of the current word block in the block sequence; wherein, the target sampling information is used to characterize the sampling quantity of the associated word block; The second determining module is used to determine the corresponding associated word block from the preceding word block based on the relevance between the preceding word block and the current word block and the target sampling information; wherein, the preceding word block is a word block in the block sequence that is earlier than the current word block; The third determining module is used to determine the updated word element corresponding to the current word element in the current word element block based on the relevance between the reference word element and the current word element; wherein, the reference word element includes the associated word element block and word elements in the current word element block that are in a position earlier than the current word element; The generation module determines the response content based on the updated tokens corresponding to each token in the token sequence.

17. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 15.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 15.

19. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 15.

Images