Processing method, artificial intelligence model reasoning method and electronic equipment

Abstract

The invention discloses a processing method, an artificial intelligence model reasoning method and electronic equipment, and the processing method comprises the steps: dividing a first vector set into a plurality of continuous vector blocks based on a sampling unit according to the generation sequence of vector pairs in the first vector set under the condition that the first vector set in a cache meets a preset condition, based on the corresponding global time sequence position of each vector block in the input data sequence, sampling each vector block to obtain a second vector set; the first vector set comprises a plurality of vector pairs consisting of first vectors and second vectors; the closer the time sequence distance between the global time sequence position corresponding to the vector block and the current generation time sequence is, the more vector pairs sampled from the vector block are; and determining an attention weight according to the first vector in the second vector set based on the attention calculation unit, and generating a new semantic unit based on the attention weight and the second vector in the second vector set.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a processing method, an artificial intelligence model reasoning method, and an electronic device. Background Art

[0002] Currently, in the field of generative AI, many models use dynamic analysis of the relationships between sequence elements to achieve content generation. However, this generation method puts severe pressure on the hardware system's memory and graphics card usage, making it prone to resource constraints. Summary of the Invention

[0003] The technical solution of this application is as follows:

[0004] The first aspect of the present application provides a processing method, comprising:

[0005] When the first vector set in the cache meets the preset conditions, the first vector set is divided into multiple continuous vector blocks based on the sampling unit according to the generation order of each vector pair in the first vector set, and each vector block is sampled based on the global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the first vector set includes multiple vector pairs consisting of a first vector and a second vector; the first vector is used to identify a semantic unit in the input data sequence of the target model; the second vector is used to represent the semantics of the semantic unit; the closer the time series distance between the global time series position corresponding to the vector block and the current generated time series, the more vector pairs are sampled from the vector block;

[0006] Based on the attention calculation unit, an attention weight is determined according to the first vector in the second vector set, and a new semantic unit is generated based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

[0007] The sampling of each of the vector blocks based on a global temporal position corresponding to each of the vector blocks in the input data sequence includes:

[0008] Allocating an initial sampling ratio to the target vector block in each of the vector blocks whose timing distance between the global timing position and the currently generated timing is the shortest;

[0009] As the difference between the global temporal position of each other vector block and the target vector block increases, the sampling ratio of the other vector blocks is gradually reduced from the initial sampling ratio to obtain the sampling ratio of the other vector blocks;

[0010] Each of the vector blocks is sampled based on the sampling ratio of each of the vector blocks.

[0011] The step of decreasing the sampling ratio from the initial sampling ratio as the difference between the global timing positions of the target vector block and the other vector blocks increases includes:

[0012] As the difference between the global timing positions of the target vector block and the other vector blocks increases, the sampling ratio is linearly decreased starting from the initial sampling ratio.

[0013] The step of decreasing the sampling ratio from the initial sampling ratio as the difference between the global timing positions of the target vector block and the other vector blocks increases includes:

[0014] As the difference between the global temporal positions of the target vector block and the other vector blocks increases, the sampling ratio is nonlinearly decreased starting from the initial sampling ratio.

[0015] The sampling of each of the vector blocks based on the sampling ratio of each of the vector blocks includes:

[0016] Determining an intra-block timing distance between a global timing position corresponding to each vector pair in each of the vector blocks and the currently generated timing;

[0017] Within the same vector block, by comparing the size of the intra-block timing distance of each vector pair, sampling is performed according to the set sampling interval starting from the vector pair with the smallest intra-block timing distance until the number of sampled vector pairs reaches the sampling ratio of the vector block.

[0018] The sampling of each of the vector blocks based on the sampling ratio of each of the vector blocks includes:

[0019] Based on the sampling ratio of each vector block, the vector pairs in each vector block are randomly sampled.

[0020] After sampling each of the vector blocks based on the global time sequence position corresponding to each of the vector blocks in the input data sequence to obtain a second vector set, the method further includes:

[0021] Overwrite the first vector set in the cache with the second vector set.

[0022] The case where the first vector set in the cache satisfies a preset condition includes: a case where the number of vector pairs in the first vector set in the cache satisfies a set number threshold;

[0023] After generating a new semantic unit based on the attention weight and the second vector in the second vector set, the method further includes:

[0024] updating the vector set in the cache based on the new semantic unit to obtain a new first vector set;

[0025] If the number of vector pairs in the new first vector set meets the set number threshold, continuing to perform the steps of dividing the first vector set into a plurality of continuous vector blocks based on the generation order of each vector pair in the first vector set by the sampling unit, and sampling each vector block based on a global time series position corresponding to each vector block in the input data sequence to obtain a second vector set;

[0026] When the number of vector pairs in the new first vector set does not meet the set number threshold, based on the attention calculation unit, continue to generate new semantic units according to the new first vector set, and continue to execute the step of updating the vector set in the cache based on the new semantic units to obtain a new first vector set.

[0027] Another aspect of the present application provides an artificial intelligence model reasoning method, comprising:

[0028] Get the input data sequence;

[0029] Processing the input data sequence based on an artificial intelligence model to obtain a first vector set;

[0030] Dividing the first vector set into a plurality of continuous vector blocks according to a generation order of each vector pair in the first vector set, and sampling each vector block based on a global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the closer the time series distance between the global time series position corresponding to the vector block and the currently generated time series, the greater the number of vector pairs sampled from the vector block;

[0031] Determining, based on the artificial intelligence model, an attention weight according to the first vector in the second vector set, and generating a new semantic unit based on the attention weight and the second vector in the second vector set, wherein the attention weight represents a degree of association between the first vector and the generation task of the current generation time sequence;

[0032] Based on the artificial intelligence model and the new semantic unit, a new input data sequence is obtained.

[0033] A third aspect of the present application provides an electronic device, including:

[0034] A sampling unit is used to divide the first vector set into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set when the first vector set in the cache meets the preset conditions, and sample each vector block based on the global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the first vector set includes multiple vector pairs consisting of a first vector and a second vector; the first vector is used to identify a semantic unit in the input data sequence of the target model; the second vector is used to represent the semantics of the semantic unit; the closer the time series distance between the global time series position corresponding to the vector block and the current generated time series, the greater the number of vector pairs sampled from the vector block;

[0035] An attention calculation unit is used to determine an attention weight based on the first vector in the second vector set, and generate a new semantic unit based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0037] Figure 1 A schematic flow chart of a treatment method provided in Example 1 of the present application;

[0038] Figure 2 A schematic diagram of the interaction between an attention calculation unit and a sampling unit provided in this application;

[0039] Figure 3 A schematic diagram of an implementation scenario of a processing method provided in Example 3 of the present application;

[0040] Figure 4 A schematic diagram of a second vector set provided in this application;

[0041] Figure 5 A schematic diagram of changes in computing power or power consumption provided by this application;

[0042] Figure 6 A schematic diagram of an implementation scenario of a processing method provided in Example 4 of the present application;

[0043] Figure 7 A schematic diagram of an implementation scenario of a processing method provided in Example 5 of the present application;

[0044] Figure 8 A schematic diagram of another implementation scenario of a processing method provided in Example 5 of the present application;

[0045] Figure 9 A schematic diagram of another implementation scenario of a processing method provided in Example 6 of the present application;

[0046] Figure 10 A schematic flow chart of a treatment method provided in Example 7 of the present application;

[0047] Figure 11 A schematic flow chart of a treatment method provided in Example 8 of the present application;

[0048] Figure 12 A schematic diagram of the time dimension of a processing method provided in this application. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0050] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0051] The terms "first", "second" etc. in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of identical properties when describing them in the embodiments of the present application. In addition, the terms "comprise" and "have" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] In the field of artificial intelligence, models are core components for achieving various intelligent tasks. Generative models, as an important branch of this field, are dedicated to learning the underlying distribution of data and generating new samples that are similar to the training data.

[0054] Common generative models include but are not limited to:

[0055] Generative Language Models (GLMs): Large Language Models (LLMs), such as the GPT series and LLaMA, generate coherent text through autoregression.

[0056] Text-to-image models: such as Stable Diffusion and the DALL-E series, which convert text descriptions into high-quality images through a diffusion process.

[0057] Image-text generation models: such as Florence and KOSMOS, can achieve cross-modal content generation and understanding.

[0058] Expert model: A vertical domain model formed by fine-tuning domain data based on a general large model, such as a medical diagnosis model, a legal document generation model, etc.

[0059] A common feature of these models is their use of the attention mechanism within the Transformer architecture. During attention calculation, context modeling is implemented using QKV (Query-Key-Value) triplets. This computational complexity increases significantly with sequence length. GPU memory usage and computational latency are particularly problematic when generating long text (such as document creation) or high-resolution images.

[0060] Reference Figure 1 , is a flow chart of a processing method provided in Example 1 of the present application, such as Figure 1 As shown, the method may include but is not limited to the following steps:

[0061] Step S101: When a first vector set in the cache meets a preset condition, the sampling unit divides the first vector set into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set, and samples each vector block based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set.

[0062] The first vector set may include a plurality of vector pairs consisting of a first vector and a second vector. The first vector may be used to identify a semantic unit (e.g., a character, a word, a phrase, a syntactic fragment, etc.) in an input data sequence of a target model. The second vector may be used to represent the semantics of the semantic unit.

[0063] In this embodiment, each semantic unit in the input data sequence can be processed based on the artificial intelligence model to generate a vector pair corresponding to each semantic unit, where the vector pair includes a first vector and a second vector. The vector pairs are generated in the same order as the order in which the semantic units appear in the input data sequence (i.e., their global temporal positions).

[0064] When actually stored in the cache, the vector pairs may be stored contiguously in the order in which they are generated in the form of a single vector matrix.

[0065] For example, the input data sequence may include "I love natural language processing", and the corresponding semantic units may include: ["I", "love", "nature", "nature", "language", "speech", "processing", "processing"], and the temporal position t of each semantic unit ranges from 0 to 7.

[0066] When the AI model processes "I" (t=0), it generates k_0 and v_0 and stores them in cache[0];

[0067] When the AI model processes "love" (t=1), it generates k_1 and v_1 and stores them in cache[1], and so on. Here, cache[0] can represent the 0th column in cache; cache[1] can represent the 1st column in cache.

[0068] The column index in the cache matrix can represent the order in which the vector pairs are generated, which is consistent with the temporal position of the corresponding semantic units. The row index in the cache matrix can represent the dimension of the vector pairs.

[0069] Of course, when actually stored in the cache, all first vectors can also be stored continuously in an independent first vector matrix (which can be expressed as K_cache), and the matrix dimensions can be expressed as [M, D_K], where M can represent the number of first vectors and D_K can represent the dimension of the first vector. All second vectors can also be stored synchronously and continuously in another independent second vector matrix (which can be expressed as V_cache), and the matrix dimensions can be expressed as [M, D_V], where M can represent the number of second vectors and D_V can represent the dimension of the second vector. The number of first vectors and second vectors can remain consistent, and the dimensions of the first vector and the dimension of the second vector can remain consistent.

[0070] For example, the input data sequence may include "I love natural language processing", and the corresponding semantic units may include: ["I", "love", "nature", "nature", "language", "speech", "processing", "processing"], and the temporal position t of each semantic unit ranges from 0 to 7.

[0071] When the AI model processes "I" (t=0), it generates k_0 and v_0 and stores them in K_cache[0] and V_cache[0];

[0072] When the AI model processes "love" (t=1), it generates k_1 and v_1 and stores them in K_cache[1] and V_cache[1], and so on. K_cache[0] can represent the 0th column in K_cache, V_cache[0] can represent the 0th column in V_cache; K_cache[1] can represent the 1st column in K_cache, and V_cache[1] can represent the 1st column in V_cache.

[0073] The column indices in the K_cache matrix can represent the order in which the first vector is generated, consistent with the temporal position of the corresponding semantic unit. The row indices in the K_cache matrix can represent the dimension of the first vector. Correspondingly, the column indices in the V_cache matrix can represent the order in which the second vector is generated, consistent with the temporal position of the corresponding semantic unit. The row indices in the V_cache matrix can represent the dimension of the second vector.

[0074] As the user enters more prompt words, the number of vector pairs increases rapidly. Directly processing all vector pairs would result in a significant increase in computational effort and complexity, potentially exceeding the hardware's computational capacity, causing computational delays or even system crashes. By using a sampling unit to divide the first vector set into vector blocks and sample them based on global temporal positions, a second vector set, much smaller than the first, is generated. This effectively reduces the number of vector pairs required for subsequent processing, significantly reducing the computational effort and enabling the attention calculation unit to complete the task within a reasonable timeframe, improving computational efficiency.

[0075] Furthermore, storing a large number of vector pairs in the cache consumes significant memory and video memory resources. As the number of vector pairs increases, memory and video memory usage continues to climb, impacting the normal operation of other tasks. The sampling operation significantly reduces the number of vector pairs stored in the cache by filtering the vector pairs, retaining only a subset to form a second vector set. This effectively reduces memory and video memory usage, freeing up space for other tasks and improving the overall performance and stability of attention calculations.

[0076] In this embodiment, the minimum global temporal position of each vector pair within each vector block can be used as the global temporal position corresponding to the vector block, but is not limited to it. For example, in the above-mentioned "I love natural language processing" example, the first vector set can be divided into two consecutive vector blocks, namely vector block 1 and vector block 2. Vector block 1 can include a vector pair consisting of k_0 and v_0, a vector pair consisting of k_1 and v_1, a vector pair consisting of k_2 and v_2, and a vector pair consisting of k_3 and v_3. Vector block 2 can include a vector pair consisting of k_4 and v_4, a vector pair consisting of k_5 and v_5, a vector pair consisting of k_6 and v_6, and a vector pair consisting of k_7 and v_7. The generation order of the vector pairs consisting of k_0 and v_0 in vector block 1 (i.e., the minimum global timing position in vector block 1) can be used as the global timing position corresponding to vector block 1, and the generation order of the vector pairs consisting of k_4 and v_4 in vector block 2 (i.e., the minimum global timing position in vector block 2) can be used as the global timing position corresponding to vector block 2.

[0077] In this embodiment, the AI model can predict new semantic units at the current generation time. For example, in the "I love natural language processing" example above, the time sequence positions t of the semantic units ["I", "love", "nature", "nature", "language", "language", "process", "process"] range from 0 to 7. When the current generation time is t = 8, it means that the AI model is predicting the 9th semantic unit.

[0078] The closer the timing distance between the global timing position corresponding to the vector block and the current generated timing, the greater the number of vector pairs sampled from the vector block. For example, in the above example of vector block 1 and vector block 2, the timing distance between the global timing position corresponding to vector block 2 (i.e., t=4) and the current generated timing (i.e., t=8) is closer than the timing distance between the global timing position corresponding to vector block 1 (i.e., t=0) and the current generated timing (i.e., t=8). All vector pairs can be sampled from vector block 2, and 1 or 2 vector pairs can be sampled from vector block 1.

[0079] The preset conditions can be set as needed and are not limited in this application.

[0080] Step S102: Based on the attention calculation unit, determine the attention weight according to the first vector in the second vector set, and generate a new semantic unit based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

[0081] In this embodiment, if Figure 2As shown, the sampling unit can pass the second vector set to the attention calculation unit. The attention calculation unit determines the attention weight according to the first vector (which can be expressed as K) in the second vector set, and generates a new semantic unit based on the attention weight and the second vector (which can be expressed as V) in the second vector set. The process may include but is not limited to:

[0082] matmul1: Perform a dot product operation on the query vector (which can be expressed as Q) and each of the first vectors in the set of second vectors to obtain the initial attention weight.

[0083] scale: Scale the initial attention weight to obtain the scaled attention weight.

[0084] mask: Mask the scaled attention weights to obtain the masked attention weights.

[0085] Softmax: Normalize the masked attention weights to obtain the attention weights.

[0086] matmul2: Performs a dot product operation on the attention weights and each second vector in the second vector set to obtain a weighted fused vector.

[0087] The weighted fused vector can be used to predict new semantic units.

[0088] like Figure 2 As shown, in this embodiment, no modifications are made to the attention calculation unit. The sampling unit and the attention calculation unit can run independently and can be matched and connected. Since the internal logic of the attention calculation unit is not modified, the execution logic of the sampling unit can be flexibly adapted to the execution logic framework of the attention calculation unit.

[0089] It should be noted that the sampling unit is not always in a continuous running state, but is started based on a specific trigger condition. In this embodiment, the trigger condition may include but is not limited to: the first vector set in the cache meets a preset condition.

[0090] In this embodiment, when the first vector set in the cache meets the preset conditions, the sampling unit divides the first vector set into multiple continuous vector blocks based on the generation order of each vector pair in the first vector set, and samples each vector block based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set. During the sampling process, the closer the timing distance between the global timing position corresponding to the vector block and the current generation timing, the greater the number of vector pairs sampled from the vector block. Through this sampling method, the number of vector pairs in the second vector set can be less than that in the first vector set, but the complete loss of local information can be effectively avoided. At the same time, it can also ensure that more vector pairs in the vector block with a timing distance close to the current generation timing are retained.

[0091] On this basis, the attention calculation unit determines an attention weight based on the first vector in the second vector set, and generates a new semantic unit based on the attention weight and the second vector in the second vector set. This can reduce the computational complexity of attention calculation, thereby reducing memory and video memory usage and computation time. Furthermore, because the complete loss of local information is avoided during the sampling phase, the attention calculation process can effectively prevent the occurrence of semantic fragmentation.

[0092] In addition, since more vector pairs are retained for vector blocks with a shorter temporal distance to the current generation sequence in the sampling phase, the attention calculation process can fully capture the local contextual information that has a greater impact on the generation task of the current generation sequence, thereby ensuring that the new semantic units generated by the current generation sequence have higher accuracy and the logical coherence between the semantic units, thereby maintaining the accuracy of reasoning and effectively avoiding the loss of precision that may be caused by sampling operations.

[0093] As another optional embodiment of the present application, a processing method is provided in Embodiment 2 of the present application. This embodiment is mainly an implementation method of sampling each vector block based on the global time sequence position corresponding to each vector block in the input data sequence. Specifically, it may include but is not limited to the following steps:

[0094] Step S11: allocating an initial sampling ratio to a target vector block having the shortest timing distance between its global timing position and the currently generated timing in each of the vector blocks.

[0095] The initial sampling ratio can be set as needed and is not limited in this application. For example, the initial sampling ratio can include 100%, that is, all vector pairs in the target vector block are sampled.

[0096] Step S12: As the difference between the global temporal position of each other vector block and the target vector block increases, the sampling ratio is gradually reduced from the initial sampling ratio to obtain the sampling ratio of the other vector blocks.

[0097] The decreasing method can be set as needed and is not limited in this application.

[0098] Step S13: sampling each of the vector blocks based on the sampling ratio of each of the vector blocks.

[0099] Sampling each of the vector blocks based on the sampling ratio of each of the vector blocks can ensure that the number of vector pairs sampled in each vector block meets its sampling ratio.

[0100] In this embodiment, the target vector block is more critical to the current task of generating a time series. By allocating the initial sampling ratio of the target vector block, the integrity of key information during sampling is ensured, allowing the attention calculation process to fully capture the local context information that directly affects the current reasoning, thereby improving reasoning accuracy. Although the other vector blocks are less relevant to the current task than the target vector block, they may still provide auxiliary information. By gradually reducing the sampling ratio, some historical information is retained to maintain semantic coherence while reducing redundant computation.

[0101] As another optional embodiment of the present application, a processing method is provided in Example 3 of the present application. This embodiment is mainly an implementation of the above step S12. Step S12 may include but is not limited to the following steps:

[0102] Step S121 : As the difference between the global temporal positions of the target vector block and the other vector blocks increases, the sampling ratio is linearly decreased starting from the initial sampling ratio.

[0103] In this embodiment, the linear decrease process can be implemented using a fixed decrease amount strategy. That is, in each decrease operation, the sampling ratio of the current vector block is reduced by a fixed decrease amount compared to the sampling ratio of its adjacent vector blocks at the global timing position. For example, the current generated timing t=400, the first vector set includes 400 vector pairs, and the first vector set can be divided into 4 consecutive vector blocks, represented as vector block 1, vector block 2, vector block 3 and vector block 4 respectively. The global timing position corresponding to vector block 1 is T0=0, the global timing position corresponding to vector block 2 is T1=100, the global timing position corresponding to vector block 3 is T2=200, and the global timing position corresponding to vector block 4 is T3=300. The global timing position corresponding to vector block 4 has the shortest timing distance with the current generated timing, that is, the target vector block can be vector block 4.

[0104] like Figure 3As shown, vector block 1 includes vector pairs from global timing position t=0 to global timing position t=99 (100 in total), vector block 2 includes vector pairs from global timing position t=100 to global timing position t=199 (100 in total), vector block 3 includes vector pairs from global timing position t=200 to global timing position t=299 (100 in total), and vector block 4 includes vector pairs from global timing position t=300 to global timing position t=399 (100 in total).

[0105] The initial sampling ratio of the vector block 4 may include 100%, that is, all 100 vector pairs in the vector block 4 are sampled.

[0106] The fixed decrement amount may include 20%, and for vector block 3, the sampling ratio is linearly decreased from 100% to 80%, that is, 80 vector pairs are actually sampled.

[0107] For vector block 2, the sampling ratio is linearly reduced from 80% to 60%, that is, 60 vector pairs are actually sampled.

[0108] For vector block 1, the sampling ratio is linearly reduced from 60% to 40%, that is, 40 vector pairs are actually sampled.

[0109] like Figure 4 As shown, the number of vector pairs in the second vector set is 280. Compared with the first vector set, the number of vector pairs is reduced, which can reduce the amount of subsequent calculations.

[0110] It should be noted that Figure 3 and Figure 4 It is only a visual example diagram of the first vector set and the second vector set, used to intuitively present the division of vector blocks, the timing distance between the global timing position and the current generated timing, and the logical relationship of the decreasing sampling ratio. It does not serve as any restrictive constraints on the storage architecture, physical layout, and data organization form of the first vector set in the actual cache system.

[0111] In this embodiment, as the difference between the global temporal position of each other vector block and the target vector block increases, the sampling ratio of the target vector block is linearly decreased, which can maintain the smoothness of the change in the data amount of the sampled vector pairs. While reducing the amount of calculation, as many vector pairs as possible are retained to ensure the accuracy and coherence of the generated new semantic units.

[0112] For example, Figure 5As shown, from the perspective of the generation timing dimension, as the input data sequence is continuously processed, the number of vector pairs continues to increase. When the number of vector pairs reaches 2000 (i.e., the generation timing is t=1999), the computational complexity or power consumption will reach a peak, and sampling will be triggered. The sampling unit can sample 660 vector pairs from the 2000 vector pairs. When the generation timing advances to t=2000, the attention calculation unit can perform attention calculation based on the sampled 660 vector pairs. Compared with the attention calculation directly based on the 2000 vector pairs, the computational complexity is significantly reduced. After the sampling is triggered, as new semantic units are continuously generated, the amount of vector pair data will begin to accumulate again. Although the computational complexity growth rate has slowed down compared to the computational complexity under the unsampling method, the computational complexity will still gradually increase as the generation timing advances, but the power consumption will not exceed the peak value mentioned above during the entire process.

[0113] It should be noted that Figure 5 The changing trends of the computing amount or power consumption shown are presented only as an example and do not constitute any limitation or constraint on the actual changes in the computing amount or power consumption.

[0114] As another optional embodiment of the present application, a processing method is provided in Example 4 of the present application. This embodiment is mainly an implementation of the above step S12. Step S12 may include but is not limited to the following steps:

[0115] Step S122 : As the difference between the global temporal positions of the target vector block and the other vector blocks increases, the sampling ratio is nonlinearly decreased starting from the initial sampling ratio.

[0116] In this embodiment, the nonlinear decrease may include but is not limited to: exponential decrease.

[0117] Nonlinear decrement may also include, but is not limited to: decrementing with different decrement amounts according to the difference between the global timing positions of other vector blocks and the target vector block. For example, the current generated timing is t=2000, and the first vector set includes 2000 vector pairs. The first vector set can be divided into 4 consecutive vector blocks, represented as vector block 1, vector block 2, vector block 3 and vector block 4, respectively. The global timing position corresponding to vector block 1 is T0=0, the global timing position corresponding to vector block 2 is T1=500, the global timing position corresponding to vector block 3 is T2=1000, and the global timing position corresponding to vector block 4 is T3=1500. The global timing position corresponding to vector block 4 has the shortest timing distance with the current generated timing, that is, the target vector block can be vector block 4.

[0118] like Figure 6As shown, vector block 1 includes vector pairs from global timing position t=0 to vector pairs from global timing position t=499 (a total of 500), vector block 2 includes vector pairs from global timing position t=500 to vector pairs from global timing position t=999 (a total of 500), vector block 3 includes vector pairs from global timing position t=1000 to vector pairs from global timing position t=1499 (a total of 500), and vector block 4 includes vector pairs from global timing position t=1500 to vector pairs from global timing position t=1999 (a total of 500).

[0119] The initial sampling ratio of the vector block 4 may include 100%, that is, all 500 vector pairs in the vector block 4 are sampled.

[0120] For vector block 3, the sampling ratio is reduced from 100% to 20%, that is, 100 vector pairs are actually sampled.

[0121] For vector block 2, the sampling ratio is reduced from 20% to 10%, that is, 50 vector pairs are actually sampled.

[0122] For vector block 1, the sampling ratio is reduced from 10% to 2%, that is, 10 vector pairs are actually sampled.

[0123] The second vector set may include 660 vector pairs, which is a significant reduction in the number of vector pairs compared to the first vector set.

[0124] It should be noted that Figure 6 It is only a visual example diagram of the first vector set, used to intuitively present the division of vector blocks, the timing distance between the global timing position and the current generated timing, and the logical relationship of the decreasing sampling ratio. It does not serve as any restrictive constraints on the storage architecture, physical layout, and data organization of the first vector set in the actual cache system.

[0125] In this embodiment, as the difference between the global timing position of each other vector block and the target vector block increases, the nonlinear decrease starting from the initial sampling ratio can significantly reduce the amount of calculation when the first vector set is relatively large, while maintaining the accuracy of reasoning, and effectively avoiding the loss of accuracy that may be caused by the sampling operation.

[0126] As another optional embodiment of the present application, a processing method is provided in Example 5 of the present application. This embodiment is mainly an implementation of the above step S13. Step S13 may include but is not limited to the following steps:

[0127] Step S131: Determine the intra-block timing distance between the global timing position corresponding to each vector pair in each vector block and the currently generated timing.

[0128] For each of the vector blocks, the absolute value of the difference between the global timing position corresponding to each vector pair in the vector block and the currently generated timing may be used as the intra-block timing distance of the vector pair.

[0129] Step S132: Within the same vector block, by comparing the size of the intra-block timing distance of each vector pair, sampling is performed starting from the vector pair with the smallest intra-block timing distance according to the set sampling interval until the number of sampled vector pairs reaches the sampling ratio of the vector block.

[0130] The sampling interval can be set as needed and is not limited in this application.

[0131] Next, we will take the example of setting the sampling interval to include 1, that is, continuous sampling within the vector block. For example, the current generated timing is t=400, and the first vector set includes 400 vector pairs. The first vector set can be divided into 4 consecutive vector blocks, represented as vector block 1, vector block 2, vector block 3 and vector block 4. The global timing position corresponding to vector block 1 is T0=0, the global timing position corresponding to vector block 2 is T1=100, the global timing position corresponding to vector block 3 is T2=200, and the global timing position corresponding to vector block 4 is T3=300. The global timing position corresponding to vector block 4 has the shortest timing distance with the current generated timing, that is, the target vector block can be vector block 4.

[0132] like Figure 7 As shown, vector block 1 includes vector pairs from global timing position t=0 to global timing position t=99 (100 in total), vector block 2 includes vector pairs from global timing position t=100 to global timing position t=199 (100 in total), vector block 3 includes vector pairs from global timing position t=200 to global timing position t=299 (100 in total), and vector block 4 includes vector pairs from global timing position t=300 to global timing position t=399 (100 in total).

[0133] The initial sampling ratio of vector block 4 may include 100%, that is, the number of vector pairs required to be sampled in vector block 4 may include 100. In vector block 4, starting from the vector pair at t=399, sampling is continued until the vector pair at t=300 is sampled.

[0134] For vector block 3, the sampling ratio decreases linearly from 100% to 80%, that is, the number of vector pairs that need to be sampled in vector block 3 can include 80. In vector block 3, starting from the vector pair at t=299, sampling continues until the vector pair at t=220 is sampled.

[0135] For vector block 2, the sampling ratio decreases linearly from 80% to 60%, that is, the number of vector pairs that need to be sampled in vector block 2 can include 60. In vector block 2, starting from the vector pair at t=199, sampling continues until the vector pair at t=140 is sampled.

[0136] For vector block 1, the sampling ratio decreases linearly from 60% to 40%, that is, the number of vector pairs that need to be sampled in vector block 1 can include 40. In vector block 1, starting from the vector pair at t=99, sampling continues until the vector pair at t=60 is sampled.

[0137] For another example, if the current generated time sequence is t = 2000 and the first vector set includes 2000 vector pairs, the first vector set can be divided into four consecutive vector blocks, denoted as vector block 1, vector block 2, vector block 3, and vector block 4. The global timing position corresponding to vector block 1 is T0 = 0, the global timing position corresponding to vector block 2 is T1 = 500, the global timing position corresponding to vector block 3 is T2 = 1000, and the global timing position corresponding to vector block 4 is T3 = 1500. The global timing position corresponding to vector block 4 has the shortest timing distance to the current generated time sequence, that is, the target vector block can be vector block 4.

[0138] like Figure 8 As shown, vector block 1 includes vector pairs from global timing position t=0 to global timing position t=499 (500 in total), vector block 2 includes vector pairs from global timing position t=500 to global timing position t=999 (500 in total), vector block 3 includes vector pairs from global timing position t=1000 to global timing position t=1499 (500 in total), and vector block 4 includes vector pairs from global timing position t=1500 to global timing position t=1999 (500 in total).

[0139] The initial sampling ratio of vector block 4 may include 100%, that is, the number of vector pairs to be sampled in vector block 4 may include 500. In vector block 4, sampling starts from the vector pair at t=1999 and continues until the vector pair at t=1500 is sampled.

[0140] For vector block 3, the sampling ratio is reduced from 100% to 20%, that is, the number of vector pairs that need to be sampled in vector block 3 may include 100. In vector block 3, sampling starts from the vector pair at t=1499 and continues until the vector pair at t=1400 is sampled.

[0141] For vector block 2, the sampling ratio is reduced from 20% to 10%, that is, the number of vector pairs that need to be sampled in vector block 2 can include 50. In vector block 2, starting from the vector pair at t=999, sampling is continued until the vector pair at t=950 is sampled.

[0142] For vector block 1, the sampling ratio is gradually reduced from 10% to 2%, that is, the number of vector pairs that need to be sampled in vector block 1 can include 10. In vector block 1, starting from the vector pair at t=499, sampling is continued until the vector pair at t=490 is sampled.

[0143] In this embodiment, the vector pair with a smaller intra-block timing distance usually contains context information that is more critical to the generation task of the current generation timing. Therefore, by determining the intra-block timing distance between the global timing position corresponding to each vector pair in each of the vector blocks and the current generation timing, within the same vector block, by comparing the size of the intra-block timing distance of each vector pair, sampling is performed according to the set sampling interval starting from the vector pair with the smallest intra-block timing distance until the number of sampled vector pairs reaches the sampling ratio of the vector block. On the basis of ensuring that more vector pairs in the vector block with a shorter timing distance to the current generation timing are retained, it is further ensured that the sampling within the vector block also gives priority to retaining vector pairs with a higher correlation with the generation task of the current generation timing, thereby further improving the accuracy of reasoning while reducing the amount of calculation.

[0144] As another optional embodiment of the present application, a processing method is provided in Example 6 of the present application. This embodiment is mainly an implementation of the above step S13. Step S13 may include but is not limited to the following steps:

[0145] Step S133: Based on the sampling ratio of each vector block, randomly sample the vector pairs in each vector block.

[0146] For example, Figure 9 As shown, the number of vector pairs in vector block 1 is 500, and the sampling ratio is 40%. 40% (200 vector pairs) need to be sampled from vector block 1. 200 vector pairs can be randomly selected from the 500 vector pairs in vector block 1 (e.g., vector pairs at global timing position t=10, ..., vector pairs at global timing position t=400, vector pairs at global timing position t=460, vector pairs at global timing position t=469, vector pairs at global timing position t=490, and vector pairs at global timing position t=499), instead of selecting only 200 pairs with the closest timing distance in the vector block.

[0147] In this embodiment, on the basis of retaining more vector pairs in vector blocks with a shorter temporal distance to the currently generated temporal sequence, the vector pairs in each vector block are randomly sampled based on the sampling ratio of each vector block. This can ensure that in the process of attention calculation, excessive reliance on vector pairs with local high correlation is avoided, the risk of new semantic units deviating from the topic is reduced, and the accuracy and logical coherence of the generated new semantic units are guaranteed.

[0148] As another optional embodiment of the present application, refer to Figure 10 , is a flow chart of a treatment method provided in Example 7 of the present application, such as Figure 10 As shown, the method may include but is not limited to the following steps:

[0149] Step S201: When a first vector set in the cache meets a preset condition, based on the sampling unit, the first vector set is divided into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set, and each vector block is sampled based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set; the first vector set includes multiple vector pairs consisting of a first vector and a second vector.

[0150] The first vector is used to identify a semantic unit in an input data sequence of a target model.

[0151] The second vector is used to represent the semantics of the semantic unit.

[0152] The closer the timing distance between the global timing position corresponding to the vector block and the currently generated timing is, the more vector pairs are sampled from the vector block.

[0153] The detailed process of step S201 can be found in the relevant introduction of step S101 in embodiment 1, and will not be repeated here.

[0154] Step S202: Overwrite the first vector set in the cache with the second vector set.

[0155] After overwriting the first vector set in the cache, only the second vector set is retained in the cache, and the first vector set is deleted, thereby releasing a portion of cache space and reducing cache space overhead.

[0156] Step S203: Based on the attention calculation unit, determine the attention weight according to the first vector in the second vector set, and generate a new semantic unit based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

[0157] The detailed process of step S203 can be found in the relevant introduction of step S102 in embodiment 1, and will not be repeated here.

[0158] In this embodiment, the first vector set is sampled through step S201 to obtain the second vector set. In the subsequent attention calculation link, only the sampled second vector set is used as the basis for calculation, which greatly reduces the scale of vector pairs involved in the calculation, directly and effectively reduces the computational burden of attention calculation, and thus significantly reduces the consumption of key resources such as memory and video memory, alleviating the system operation pressure.

[0159] At the same time, step S202 overwrites the first vector set in the cache with the second vector set, fundamentally avoiding the continuous occupation of video memory and main memory by invalid data, ensuring the effective use of system resources, and providing strong support for the efficient operation of artificial intelligence models.

[0160] As another optional embodiment of the present application, refer to Figure 11 , is a flow chart of a treatment method provided in Example 8 of the present application, such as Figure 11 As shown, the method may include but is not limited to the following steps:

[0161] Step S301: When the number of vector pairs in a first vector set in the cache meets a set number threshold, based on the sampling unit, the first vector set is divided into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set, and each vector block is sampled based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set.

[0162] The first vector set includes a plurality of vector pairs consisting of a first vector and a second vector.

[0163] When the number of vector pairs in the first vector set in the cache meets a set number threshold, this is an implementation method when the first vector set in the cache meets a preset condition. The set number threshold can be set as needed and is not limited in this application.

[0164] The first vector is used to identify a semantic unit in an input data sequence of a target model.

[0165] The second vector is used to represent the semantics of the semantic unit.

[0166] The closer the timing distance between the global timing position corresponding to the vector block and the currently generated timing is, the more vector pairs are sampled from the vector block.

[0167] Step S302: Overwrite the first vector set in the cache with the second vector set.

[0168] After overwriting the first vector set in the cache, only the second vector set is retained in the cache, and the first vector set is deleted, so that the video memory occupancy is reduced.

[0169] Step S303: Based on the attention calculation unit, determine the attention weight according to the first vector in the second vector set, and generate a new semantic unit based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

[0170] The detailed process of steps S301-S303 can be found in the relevant introduction of steps S201-S203 in Example 7, and will not be repeated here.

[0171] Step S304: Update the vector set in the cache based on the new semantic unit to obtain a new first vector set.

[0172] In this embodiment, the new semantic unit can be processed based on the artificial intelligence model to generate a vector pair corresponding to the new semantic unit, and the vector pair corresponding to the new semantic unit can be added to the cached vector set. For example, if the cached vector set includes 660 vector pairs, after adding the vector pair corresponding to the new semantic unit to the cached vector set, the new first vector set may include 661 vector pairs, that is, 661 vector pairs are stored in the cache.

[0173] Step S305: Determine whether the number of vector pairs in the new first vector set meets the set number threshold.

[0174] The quantity threshold can be set as needed and is not limited in this application. For example, the quantity threshold can include 2000.

[0175] If yes, continue to execute step S301; if no, execute step S306.

[0176] Step S306: Based on the attention calculation unit, continue to generate new semantic units according to the new first vector set, and continue to execute step S304.

[0177] The detailed process of continuing to generate new semantic units based on the attention calculation unit and the new first vector set is similar to the method of generating new semantic units based on the second vector set in S303, and will not be repeated here.

[0178] In this embodiment, steps S301-S306 are described in detail by taking an example. For example, the set quantity threshold may include 2000. The current generated timing t=2000, the first vector set includes 2000 vector pairs, and the first vector set can be divided into 4 consecutive vector blocks, respectively represented as vector block 1, vector block 2, vector block 3 and vector block 4. The global timing position corresponding to vector block 1 is T0=0, the global timing position corresponding to vector block 2 is T1=500, the global timing position corresponding to vector block 3 is T2=1000, and the global timing position corresponding to vector block 4 is T3=1500. The global timing position corresponding to vector block 4 has the shortest timing distance with the current generated timing, that is, the target vector block can be vector block 4.

[0179] Vector block 1 includes vector pairs from global timing position t=0 to vector pairs from global timing position t=499 (a total of 500), vector block 2 includes vector pairs from global timing position t=500 to vector pairs from global timing position t=999 (a total of 500), vector block 3 includes vector pairs from global timing position t=1000 to vector pairs from global timing position t=1499 (a total of 500), and vector block 4 includes vector pairs from global timing position t=1500 to vector pairs from global timing position t=1999 (a total of 500).

[0180] All 500 vector pairs in vector block 4 are sampled; for vector block 3, 100 vector pairs are actually sampled; for vector block 2, 50 vector pairs are actually sampled; and for vector block 1, 10 vector pairs are actually sampled. Accordingly, the second vector set may include 660 vector pairs.

[0181] The 660 vector pairs, ie, the second vector set, overwrite the 2000 vector pairs, ie, the first vector set, in the cache.

[0182] Based on the attention calculation unit, a new semantic unit is generated according to the 660 vector pairs, and the vector set in the cache is updated based on the new semantic unit to obtain a new first vector set, namely 661 vector pairs.

[0183] The number of 661 vector pairs does not meet 2000. Based on the attention calculation unit, new semantic units are continued to be generated according to the 661 vector pairs, and the vector set in the cache (i.e., 661 vector pairs) is continued to be updated based on the new semantic units to obtain a new first vector set, i.e., 662 vector pairs.

[0184] The number of 662 vector pairs does not meet 2000. Based on the attention calculation unit, new semantic units are continued to be generated according to the 662 vector pairs, and the vector set in the cache (i.e., 662 vector pairs) is continued to be updated based on the new semantic units to obtain a new first vector set, i.e., 663 vector pairs.

[0185] The number of 663 vector pairs does not meet 2000. Based on the attention calculation unit, new semantic units are continued to be generated according to the 663 vector pairs, and the vector set in the cache (i.e., 663 vector pairs) is continued to be updated based on the new semantic units to obtain a new first vector set, i.e., 664 vector pairs.

[0186] This cycle is repeated until the number of vector pairs in the cache reaches 2000, and the sampling operation is performed again. That is, the sampling unit divides the new first vector set into multiple continuous vector blocks based on the generation order of each vector pair in the new first vector set, and samples each vector block based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set.

[0187] From a temporal perspective, the entire processing flow can be divided into a data prefill phase and a decoding generation phase. In each phase, as vector pairs are continuously generated, the number of vector pairs in the first vector set in the cache accumulates, and sampling operations are dynamically triggered based on a set threshold.

[0188] like Figure 12 As shown in the figure, in the prefill phase, the input data sequence is processed based on the artificial intelligence model to obtain vector pairs corresponding to each semantic unit. During this process, time nodes T0, T1, and T2 are successively experienced. As the data corresponding to each time node is processed, vector pairs are continuously generated and stored in the cache to obtain the first vector set. However, after experiencing the three time nodes T0, T1, and T2, the number of vector pairs contained in the first vector set in the cache has not yet reached the set number threshold (for example, 2000), and the sampling operation will not be triggered.

[0189] Entering the decode phase, the AI model continues to generate new semantic units based on the first vector set in the cache and updates the vector set in the cache. This phase goes through time nodes T3 and T4 in sequence. At time nodes T3 and T4, the sampling operation will not be performed because the number of vector pairs in the first vector set in the cache has not yet reached the set threshold. This means that throughout the process from T0 to T4, although the number of vector pairs in the cache has gradually increased, all vector pairs generated from T0 to the current time node are always retained, without any sampling or culling operations, and all vector pairs in the cache participate in the attention calculation.

[0190] As new semantic units are continuously generated, the number of vector pairs in the cache continues to accumulate. When time advances to the stage from T5 to T6, the number of vector pairs in the first vector set in the cache finally reaches the set threshold of 2000. At this point, the sampling mechanism is triggered. Based on the sampling unit, the vector pairs in the new first vector set are divided into multiple continuous vector blocks according to the generation order of each vector pair. The vector pairs generated in the T6 stage can be divided into vector block 4, the vector pairs generated in the T4-T5 stage can be divided into vector block 3, the vector pairs generated in the T3 stage can be divided into vector block 2, and the vector pairs generated in the T0-T2 stage can be divided into vector block 1. The sampling ratio of vector block 4 can include 100%. However, since the vector pairs generated in the T4-T5 stage, the T3 stage, and the T0-T2 stage are farther away in time sequence than the current generation time sequence, they can each be partially retained during the sampling process.

[0191] It should be noted that Figure 10 The deletion symbol “\” for the T4-T5 stage, T3 stage, and T0-T2 stage only indicates sampling of the vector pairs generated in the stage, rather than deleting all of them.

[0192] In this embodiment, if the number of vector pairs in the new first vector set does not meet the set threshold, sampling is not triggered. Instead, new semantic units are generated directly based on the new first vector set. This improves the efficiency of generating new semantic units. Furthermore, the new first vector set can provide richer recent contextual information for the generation task, ensuring the accuracy and logical coherence of the new semantic units.

[0193] As another optional embodiment of the present application, an artificial intelligence model reasoning method provided in Example 8 of the present application may include but is not limited to the following steps:

[0194] Step S401: Obtain an input data sequence.

[0195] The input data sequence can be a serialized representation of data in various forms such as text, speech, and images after preprocessing. For example, in natural language processing tasks, the input data sequence is usually a text sequence consisting of characters, words, phrases, etc.

[0196] Suppose in a natural language processing scenario, the input data sequence is "I love natural language processing". The subsequent artificial intelligence model can reason based on this sequence and gradually generate new semantic units to expand or explain the sequence.

[0197] Step S402: Process the input data sequence based on the artificial intelligence model to obtain a first vector set.

[0198] Step S403: Divide the first vector set into a plurality of continuous vector blocks according to the generation order of each vector pair in the first vector set, and sample each vector block based on the global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the closer the time series distance between the global time series position corresponding to the vector block and the currently generated time series, the more vector pairs sampled from the vector block;

[0199] Step S404: Based on the artificial intelligence model, an attention weight is determined for the first vector in the second vector set, and a new semantic unit is generated based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation time series.

[0200] The detailed process of steps S402-S404 can be found in the relevant introduction in Example 1 and will not be repeated here.

[0201] Step S405: Based on the artificial intelligence model and the new semantic unit, a new input data sequence is obtained.

[0202] In this embodiment, the generated new semantic unit can be added to the input data sequence to obtain a new input data sequence. For example, the input data sequence is "I love natural language processing". Assume that in the previous reasoning step, based on a series of processing (such as steps S402-S403) based on this sequence, the artificial intelligence model generates a new semantic unit "technology" according to the current generation task and attention calculation results.

[0203] At this time, after the generated new semantic unit "technology" is added to the input data sequence "I love natural language processing", the new input data sequence becomes "I love natural language processing technology".

[0204] In this embodiment, the first vector set is divided into a plurality of continuous vector blocks based on the generation order of each vector pair in the first vector set, and each vector block is sampled based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set. During the sampling process, the closer the timing distance between the global timing position corresponding to the vector block and the current generated timing, the greater the number of vector pairs sampled from the vector block. Through this sampling method, the number of vector pairs in the second vector set can be less than that in the first vector set, but the complete loss of local information can be effectively avoided. At the same time, it can also ensure that more vector pairs in vector blocks with a closer timing distance to the current generated timing are retained.

[0205] On this basis, an artificial intelligence model is used to determine an attention weight based on the first vector in the second vector set, and a new semantic unit is generated based on the attention weight and the second vector in the second vector set. This can reduce the computational complexity of attention calculations, thereby reducing memory and video memory usage. Furthermore, because the complete loss of local information is avoided during the sampling phase, the attention calculation process can effectively prevent the occurrence of semantic fragmentation.

[0206] In addition, since more vector pairs are retained for vector blocks with a closer temporal distance to the current generation time sequence in the sampling stage, the local contextual information that has a greater impact on the generation task of the current generation time sequence can be fully captured during the reasoning process based on the artificial intelligence model, thereby ensuring that the new semantic units generated by the current generation time sequence have higher accuracy and the logical coherence between the semantic units, thereby maintaining the accuracy of the new input data sequence and effectively avoiding the loss of accuracy that may be caused by sampling operations.

[0207] Next, the electronic device provided by the present application is introduced. The electronic device introduced below and the processing method introduced above can be referenced to each other.

[0208] Electronic devices may include:

[0209] A sampling unit is used to divide the first vector set into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set when the first vector set in the cache meets the preset conditions, and sample each vector block based on the global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the first vector set includes multiple vector pairs consisting of a first vector and a second vector; the first vector is used to identify a semantic unit in the input data sequence of the target model; the second vector is used to represent the semantics of the semantic unit; the closer the time series distance between the global time series position corresponding to the vector block and the current generated time series, the greater the number of vector pairs sampled from the vector block.

[0210] An attention calculation unit is used to determine an attention weight based on the first vector in the second vector set, and generate a new semantic unit based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

[0211] In this embodiment, the sampling unit samples each vector block based on the global temporal position corresponding to each vector block in the input data sequence, which may specifically include:

[0212] Allocating an initial sampling ratio to the target vector block in each of the vector blocks whose timing distance between the global timing position and the currently generated timing is the shortest;

[0213] As the difference between the global temporal position of each other vector block and the target vector block increases, the sampling ratio of the other vector blocks is gradually reduced from the initial sampling ratio to obtain the sampling ratio of the other vector blocks;

[0214] Each of the vector blocks is sampled based on the sampling ratio of each of the vector blocks.

[0215] In this embodiment, the sampling unit decreases the sampling ratio starting from the initial sampling ratio as the difference between the global timing positions of the other vector blocks and the target vector block increases, which may specifically include:

[0216] As the difference between the global timing positions of the target vector block and the other vector blocks increases, the sampling ratio is linearly decreased starting from the initial sampling ratio.

[0217] In this embodiment, the sampling unit decreases the sampling ratio starting from the initial sampling ratio as the difference between the global timing positions of the other vector blocks and the target vector block increases, which may specifically include:

[0218] As the difference between the global temporal positions of the target vector block and the other vector blocks increases, the sampling ratio is nonlinearly decreased starting from the initial sampling ratio.

[0219] In this embodiment, the sampling unit samples each of the vector blocks based on the sampling ratio of each of the vector blocks, which may specifically include:

[0220] Determining an intra-block timing distance between a global timing position corresponding to each vector pair in each of the vector blocks and the currently generated timing;

[0221] Within the same vector block, by comparing the size of the intra-block timing distance of each vector pair, sampling is performed according to the set sampling interval starting from the vector pair with the smallest intra-block timing distance until the number of sampled vector pairs reaches the sampling ratio of the vector block.

[0222] In this embodiment, the sampling unit samples each of the vector blocks based on the sampling ratio of each of the vector blocks, which may specifically include:

[0223] Based on the sampling ratio of each vector block, the vector pairs in each vector block are randomly sampled.

[0224] In this embodiment, the sampling unit may further include: overwriting the first vector set in the cache with the second vector set.

[0225] The case where the first vector set in the cache meets a preset condition may include: a case where the number of vector pairs in the first vector set in the cache meets a set number threshold.

[0226] After generating a new semantic unit based on the attention weight and the second vector in the second vector set, the method further includes:

[0227] The sampling unit may also be configured to update the vector set in the cache based on the new semantic unit to obtain a new first vector set.

[0228] When the number of vector pairs in the new first vector set meets the set number threshold, the sampling unit continues to divide the first vector set into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set, and samples each vector block based on the global timing position corresponding to each vector block in the input data sequence to obtain a second vector set.

[0229] When the number of vector pairs in the new first vector set does not meet the set number threshold, the attention calculation unit continues to generate new semantic units based on the new first vector set, and triggers the sampling unit to continue to execute the step of updating the vector set in the cache based on the new semantic units to obtain a new first vector set.

[0230] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0231] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0232] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0233] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state drive (SSD)).

Claims

1. A processing method comprising: When the first vector set in the cache meets the preset conditions, the first vector set is divided into multiple continuous vector blocks based on the sampling unit according to the generation order of each vector pair in the first vector set, and each vector block is sampled based on the global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the first vector set includes multiple vector pairs consisting of a first vector and a second vector; the first vector is used to identify a semantic unit in the input data sequence of the target model; the second vector is used to represent the semantics of the semantic unit; the closer the time series distance between the global time series position corresponding to the vector block and the current generated time series, the more vector pairs are sampled from the vector block; Based on the attention calculation unit, an attention weight is determined according to the first vector in the second vector set, and a new semantic unit is generated based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

2. The processing method according to claim 1, wherein sampling each of the vector blocks based on a global temporal position corresponding to each of the vector blocks in the input data sequence comprises: Allocating an initial sampling ratio to the target vector block in each of the vector blocks whose timing distance between the global timing position and the currently generated timing is the shortest; As the difference between the global temporal position of each other vector block and the target vector block increases, the sampling ratio of the other vector blocks is gradually reduced from the initial sampling ratio to obtain the sampling ratio of the other vector blocks; Each of the vector blocks is sampled based on the sampling ratio of each of the vector blocks.

3. The processing method according to claim 2, wherein as the difference between the global timing positions of each other vector block and the target vector block increases, the initial sampling ratio is gradually reduced, comprising: As the difference between the global timing positions of the target vector block and the other vector blocks increases, the sampling ratio is linearly decreased starting from the initial sampling ratio.

4. The processing method according to claim 2, wherein as the difference between the global timing positions of each other vector block and the target vector block increases, the initial sampling ratio is gradually reduced, comprising: As the difference between the global temporal positions of the target vector block and the other vector blocks increases, the sampling ratio is nonlinearly decreased starting from the initial sampling ratio.

5. The processing method according to claim 2, wherein sampling each of the vector blocks based on the sampling ratio of each of the vector blocks comprises: Determining an intra-block timing distance between a global timing position corresponding to each vector pair in each of the vector blocks and the currently generated timing; Within the same vector block, by comparing the size of the intra-block timing distance of each vector pair, sampling is performed according to the set sampling interval starting from the vector pair with the smallest intra-block timing distance until the number of sampled vector pairs reaches the sampling ratio of the vector block.

6. The processing method according to claim 2, wherein sampling each of the vector blocks based on the sampling ratio of each of the vector blocks comprises: Based on the sampling ratio of each vector block, the vector pairs in each vector block are randomly sampled.

7. The processing method according to claim 1, after sampling each of the vector blocks based on the global time sequence position corresponding to each of the vector blocks in the input data sequence to obtain the second vector set, further comprising: Overwrite the first vector set in the cache with the second vector set.

8. The processing method according to claim 7, wherein when the first vector set in the cache meets a preset condition, the method comprises: In a case where the number of vector pairs in the first vector set in the cache meets a set number threshold; After generating a new semantic unit based on the attention weight and the second vector in the second vector set, the method further includes: updating the vector set in the cache based on the new semantic unit to obtain a new first vector set; If the number of vector pairs in the new first vector set meets the set number threshold, continuing to perform the steps of dividing the first vector set into a plurality of continuous vector blocks based on the generation order of each vector pair in the first vector set by the sampling unit, and sampling each vector block based on a global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; When the number of vector pairs in the new first vector set does not meet the set number threshold, based on the attention calculation unit, continue to generate new semantic units according to the new first vector set, and continue to execute the step of updating the vector set in the cache based on the new semantic units to obtain a new first vector set.

9. An artificial intelligence model reasoning method, comprising: Get the input data sequence; Processing the input data sequence based on an artificial intelligence model to obtain a first vector set; Dividing the first vector set into a plurality of continuous vector blocks according to a generation order of each vector pair in the first vector set, and sampling each vector block based on a global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the closer the time series distance between the global time series position corresponding to the vector block and the currently generated time series, the greater the number of vector pairs sampled from the vector block; Determining, based on the artificial intelligence model, an attention weight according to the first vector in the second vector set, and generating a new semantic unit based on the attention weight and the second vector in the second vector set, wherein the attention weight represents a degree of association between the first vector and the generation task of the current generation time sequence; Based on the artificial intelligence model and the new semantic unit, a new input data sequence is obtained.

10. An electronic device comprising: A sampling unit is used to divide the first vector set into multiple continuous vector blocks according to the generation order of each vector pair in the first vector set when the first vector set in the cache meets the preset conditions, and sample each vector block based on the global time series position corresponding to each vector block in the input data sequence to obtain a second vector set; the first vector set includes multiple vector pairs consisting of a first vector and a second vector; the first vector is used to identify a semantic unit in the input data sequence of the target model; the second vector is used to represent the semantics of the semantic unit; the closer the time series distance between the global time series position corresponding to the vector block and the current generated time series, the greater the number of vector pairs sampled from the vector block; An attention calculation unit is used to determine an attention weight based on the first vector in the second vector set, and generate a new semantic unit based on the attention weight and the second vector in the second vector set; the attention weight represents the degree of association between the first vector and the generation task of the current generation sequence.

Images