Data processing method and device, computer equipment, readable storage medium and program product
By dividing the attention matrix of the Transformer model into blocks and loading it into the cache in batches for computation, the problem of high memory bandwidth consumption in traditional methods is solved, thus improving the model training efficiency.
Patent Information
- Application Number
- CN202511080295.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-05-20
- Filing Date
- 2025-08-01
- Publication Date
- 2025-11-14
AI Technical Summary
During the training process, traditional Transformer models consume a lot of memory bandwidth due to the frequent reading of intermediate attention and probability matrices from GPU memory, which affects the training efficiency of the model.
The initial attention matrix is divided into query matrix blocks, key matrix blocks, and value matrix blocks, and loaded into the target cache in batches for calculation. The normalization factor and attention score are calculated through a preset loading strategy and combination relationship. The target normalization factor and attention score are stored only during backpropagation.
This reduces memory bandwidth requirements, decreases the number of accesses to HBM, avoids repeated readings of intermediate matrices, and improves the efficiency of model training.
Smart Images
Figure CN120950253A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence hardware technology, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology
[0002] With the development of artificial intelligence technology, the Transformer model has been widely used in the field of natural language processing.
[0003] In traditional technology, when the terminal trains the Transformer model, the Transformer model extracts features from the training samples to obtain the attention matrix. The processor loads the entire attention matrix into the high-bandwidth memory (HBM). In the calculation of the attention score, the intermediate attention matrix and probability matrix are stored as the target result. During the gradient calculation process of backpropagation, the intermediate results and probability matrix are read from HBM multiple times.
[0004] However, in current traditional techniques, the calculation of attention scores and gradients requires frequent reading of intermediate attention matrices and probability matrices from GPU memory, resulting in high memory bandwidth consumption and consequently low model training efficiency. Summary of the Invention
[0005] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product to address the aforementioned technical problems.
[0006] In a first aspect, this application provides a data processing method, including:
[0007] The initial attention matrix is divided into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0008] Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches;
[0009] According to the preset batch combination relationship, the key matrix block and the value matrix block loaded in the current batch are combined and calculated with each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0010] The target normalization factor and target attention score are determined based on the first normalization factor and the first attention score for each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0011] In one embodiment, the step of dividing the initial attention matrix into blocks to obtain target attention matrix blocks includes:
[0012] The block partitioning strategy is determined based on the available storage space of the target cache;
[0013] Based on the block partitioning strategy, the query matrix in the initial attention matrix is divided into rows to obtain query matrix blocks, and the key matrix and value matrix in the initial attention matrix are divided into columns to obtain key matrix blocks and value matrix blocks.
[0014] In one embodiment, loading the target attention matrix blocks into the target cache in batches based on a preset loading strategy includes:
[0015] The key matrix block and the value matrix block are loaded into the target cache in batches;
[0016] For each loading batch, based on the association between the key matrix block, the value matrix block, and the query matrix block, each query matrix block is loaded into the target cache.
[0017] In one embodiment, the step of combining and calculating the key matrix block and the value matrix block loaded in the current batch with each query matrix block according to a preset batch combination relationship to obtain the first normalization factor and the first attention score of the current batch includes:
[0018] Based on a preset batch combination relationship, the key matrix block, the value matrix block, and the query matrix block loaded in the current batch are determined; the preset batch combination relationship is obtained by using the key matrix block and the value matrix block as the outer loop matrix block and the query matrix block as the inner loop matrix block;
[0019] Based on the key matrix block and the value matrix block loaded in the current batch, intra-block calculations are performed on each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0020] In one embodiment, the method further includes:
[0021] Construct a sparse mask matrix based on preset rules;
[0022] During the normalization process, the target attention matrix block is filtered element by element based on the sparse mask matrix to obtain a sparse attention matrix block; the sparse attention matrix block includes the query matrix block, the key matrix block, and the value matrix block after sparse processing.
[0023] The step of performing intra-block calculations on each query matrix block based on the key matrix block and the value matrix block loaded in the current batch to obtain the first normalization factor and the first attention score for the current batch includes:
[0024] Based on the sparsely processed key matrix block and value matrix block, intra-block calculations are performed on each of the sparsely processed query matrix blocks to obtain the first normalization factor and the first attention score for the current batch.
[0025] In one embodiment, after determining the target normalization factor and target attention score based on the first normalization factor and the first attention score for each batch, the method further includes:
[0026] In response to the backpropagation requirement of the target model, the target normalization factor and the target attention score are read from the video memory;
[0027] Based on the target normalization factor, recalculate the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block;
[0028] The gradient result is determined based on the intermediate attention matrix block, the probability matrix block, and the target attention score.
[0029] Secondly, this application also provides a data processing apparatus, comprising:
[0030] The block segmentation module is used to segment the initial attention matrix into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block;
[0031] The loading module is used to load the target attention matrix blocks into the target cache in batches based on a preset loading strategy;
[0032] The first calculation module is used to perform combined calculations on the key matrix block and the value matrix block loaded in the current batch and each query matrix block according to the preset batch combination relationship, so as to obtain the first normalization factor and the first attention score of the current batch.
[0033] The second calculation module is used to determine the target normalization factor and target attention score based on the first normalization factor and the first attention score of each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0034] In one embodiment, the segmentation module is specifically used to determine a segmentation strategy based on the available storage space of the target cache;
[0035] Based on the block partitioning strategy, the query matrix in the initial attention matrix is divided into rows to obtain query matrix blocks, and the key matrix and value matrix in the initial attention matrix are divided into columns to obtain key matrix blocks and value matrix blocks.
[0036] In one embodiment, the loading module is specifically used to load the key matrix block and the value matrix block into the target cache in batches;
[0037] For each loading batch, based on the association between the key matrix block, the value matrix block, and the query matrix block, each query matrix block is loaded into the target cache.
[0038] In one embodiment, the first calculation module is specifically used to determine the key matrix block, the value matrix block, and the query matrix block loaded in the current batch according to a preset batch combination relationship; the preset batch combination relationship is obtained by using the key matrix block and the value matrix block as the outer loop matrix block and the query matrix block as the inner loop matrix block;
[0039] Based on the key matrix block and the value matrix block loaded in the current batch, intra-block calculations are performed on each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0040] In one embodiment, the device further includes:
[0041] The building module is used to construct a sparse mask matrix based on preset rules;
[0042] The filtering module is used to perform element-wise filtering on the target attention matrix block based on the sparse mask matrix during the normalization process to obtain a sparse attention matrix block; the sparse attention matrix block includes the query matrix block, the key matrix block, and the value matrix block after sparse processing.
[0043] The first calculation module is specifically used to perform intra-block calculations on each of the sparsely processed query matrix blocks based on the sparsely processed key matrix block and the value matrix block, to obtain the first normalization factor and the first attention score of the current batch.
[0044] In one embodiment, the device further includes:
[0045] In response to the backpropagation requirement of the target model, the target normalization factor and the target attention score are read from the video memory;
[0046] Based on the target normalization factor, recalculate the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block;
[0047] The gradient result is determined based on the intermediate attention matrix block, the probability matrix block, and the target attention score.
[0048] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0049] The initial attention matrix is divided into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0050] Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches;
[0051] According to the preset batch combination relationship, the key matrix block and the value matrix block loaded in the current batch are combined and calculated with each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0052] The target normalization factor and target attention score are determined based on the first normalization factor and the first attention score for each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0053] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0054] The initial attention matrix is divided into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0055] Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches;
[0056] According to the preset batch combination relationship, the key matrix block and the value matrix block loaded in the current batch are combined and calculated with each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0057] The target normalization factor and target attention score are determined based on the first normalization factor and the first attention score for each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0058] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0059] The initial attention matrix is divided into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0060] Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches;
[0061] According to the preset batch combination relationship, the key matrix block and the value matrix block loaded in the current batch are combined and calculated with each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0062] The target normalization factor and target attention score are determined based on the first normalization factor and the first attention score for each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0063] The aforementioned data processing methods, apparatus, computer devices, computer-readable storage media, and computer program products, by dividing the large-scale initial attention matrix into smaller target attention matrix blocks, can avoid the memory bandwidth pressure caused by loading the entire initial attention matrix at once, thus reducing memory bandwidth requirements; by loading the target attention matrix blocks into the target cache for computation, the number of accesses to the HBM is reduced; by loading the key matrix blocks and value matrix blocks in batches, the target attention matrix blocks are loaded on demand, reducing the bandwidth-constrained memory access requirements; finally, by storing only the target normalization factor and target attention score in memory, repeated readings of the intermediate attention matrix and probability matrix are avoided in the gradient calculation during backpropagation, thereby improving the efficiency of training the target model. Attached Figure Description
[0064] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0065] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.
[0066] Figure 2 This is a schematic diagram of the block processing of the initial attention matrix in one embodiment;
[0067] Figure 3 This is a schematic diagram illustrating the calculation of attention scores based on the segmented target attention matrix in one embodiment;
[0068] Figure 4 This is a schematic diagram of the process of loading the target attention matrix in one embodiment;
[0069] Figure 5 This is a flowchart illustrating the process of determining the first normalization factor and the first attention score corresponding to the current batch based on a preset batch combination relationship in one embodiment.
[0070] Figure 6 This is a flowchart illustrating the process of constructing an approximate attention algorithm by filtering the target attention matrix based on a sparse mask matrix in one embodiment.
[0071] Figure 7 This is a flowchart illustrating the process of determining gradient results based on the target normalization factor and the target attention score in one embodiment.
[0072] Figure 8 This is a structural block diagram of a data processing device in one embodiment;
[0073] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0074] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0075] In traditional matrix computation techniques, the attention matrix includes a K (Key) matrix, a Q (Query) matrix, and a V (Value) matrix. First, the processor reads the Q and K matrices from GPU memory, calculates the product of the Q matrix and the transpose of the K matrix to obtain an intermediate attention matrix S, and then normalizes the intermediate attention matrix S to obtain a probability matrix P. Finally, the attention score O is calculated based on the probability matrix P and the V matrix. The intermediate attention matrix S and the probability matrix P are immediately written to GPU memory after computation. The calculation of the probability matrix P requires reading the intermediate attention matrix S from GPU memory; the calculation of the attention score O also requires further reading of the intermediate attention matrix S and the probability matrix P from GPU memory. Furthermore, for gradient calculation during backpropagation, the processor also needs to repeatedly read the calculated intermediate attention matrix S and the probability matrix P from GPU memory to perform gradient calculation.
[0076] Throughout the training process of a Transformer model, a large-scale attention matrix needs to be loaded into the high-bandwidth memory of the GPU for computation at once, and intermediate results need to be read multiple times until the computation is complete. When processing extremely long texts, traditional techniques require reading large-scale attention matrices and intermediate results multiple times, which increases memory bandwidth pressure, leads to excessive GPU memory usage, affects inference speed, and results in poor model training efficiency.
[0077] In one embodiment, such as Figure 1 As shown, to address the aforementioned technical problems, a data processing method is provided. This embodiment illustrates the method's application to a terminal, but it is understood that the method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:
[0078] Step 102: Divide the initial attention matrix into blocks to obtain the target attention matrix blocks.
[0079] The target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0080] In this embodiment, the initial attention matrix is obtained by encoding and processing the input sequence during model training or model inference, resulting in a feature representation of the input, and then performing a linear transformation based on that feature representation. The target model can be a Transformer model (a deep learning model based on an attention mechanism), and the input sequence can be long text data.
[0081] Before calculating the attention score based on the initial attention matrix, the terminal first divides the large-scale matrix into blocks according to preset block rules. Specifically, the terminal divides the query matrix in the initial attention matrix into multiple smaller query matrix blocks, the key matrix into multiple smaller key matrix blocks, and the value matrix into multiple smaller value matrix blocks. These block-based blocks together constitute the target attention matrix block. When processing very long texts, this avoids the problems of increased memory bandwidth pressure and excessive GPU memory usage caused by loading a large-scale attention matrix all at once. By dividing the large-scale initial attention matrix into multiple smaller matrix blocks, the terminal can reduce the data size processed each time, reduce the pressure on GPU memory bandwidth, and allow for more flexible management and use of memory resources during computation.
[0082] Step 104: Based on the preset loading strategy, load the target attention matrix blocks into the target cache in batches.
[0083] The target cache is the fast on-chip SRAM (Static Random-Access Memory) of the GPU (Graphics Processing Unit).
[0084] In this embodiment, since the read / write speed of video memory is relatively slow, and loading all target attention matrix blocks at once would cause significant bandwidth pressure, the target cache SRAM has a higher read / write speed. By loading the target attention matrix blocks into the target cache in batches, frequent read / write operations on video memory can be avoided, improving data reading and processing efficiency, while also preventing excessive video memory usage.
[0085] The preset loading strategy is a pre-defined rule that determines how to load the segmented target attention matrix blocks into the target cache in batches. Then, the terminal loads the query matrix block, key matrix block, and value matrix block into the target cache sequentially according to the preset batch and order.
[0086] Specifically, the terminal can pre-determine the available space size of the target cache to determine the number of matrix blocks that can be loaded each time. Then, according to a preset loading strategy, the terminal divides all target attention matrix blocks into multiple batches, each batch containing a certain number of query matrix blocks, key matrix blocks, and value matrix blocks. The terminal loads the target attention matrix blocks of each batch from video memory into the target cache in batch order. Simultaneously, during the loading process, the terminal monitors the available space of the target cache in real time; that is, after each batch of target attention matrix blocks is loaded, the terminal updates the usage and remaining available space information of the target cache to ensure that the target cache has sufficient space to accommodate the target attention matrix blocks of the current batch.
[0087] In an optional embodiment, the terminal determines a fixed batch size based on the capacity of the target cache. The number of target attention matrix blocks loaded each time remains constant until all target attention matrix blocks are loaded. For example, if the target cache capacity is 100MB and the average size of each target attention matrix block is 10MB, the terminal sets the batch size to 10 target attention matrix blocks. Ten target attention matrix blocks are loaded into the target cache each time for calculation. After calculation, the cache is cleared, and the next batch of target attention matrix blocks is loaded. Optionally, the preset loading strategy can also determine the number of blocks loaded in the current batch according to a dynamic batch adjustment strategy. During the loading process, the terminal dynamically adjusts the batch size based on the real-time usage of the target cache. If the remaining cache space is large, the batch size can be increased appropriately; if the remaining space is small, the batch size can be decreased.
[0088] Step 106: Based on the preset batch combination relationship, perform combination calculations on each query matrix block for the key matrix block and value matrix block loaded in the current batch to obtain the first normalization factor and the first attention score of the current batch.
[0089] In this embodiment, the preset batch combination relationship is a predefined combination method for the key matrix block, value matrix block, and each query matrix block loaded in the current batch. After the terminal determines the combination relationship of the key matrix block, value matrix block, and query matrix block loaded in the current batch, it first calculates the product of the query matrix block and the transpose of the key matrix block based on the query matrix block and key matrix block of the current batch to obtain the intermediate attention matrix block. Then, the terminal normalizes the intermediate attention matrix block using the softmax function. Specifically, to avoid numerical overflow or underflow issues when calculating the softmax function, the maximum value of each row of the intermediate attention matrix can be subtracted first, and then the softmax function can be used to calculate the probability matrix block. Finally, the terminal calculates the first attention score of the current batch based on the probability matrix block and the value matrix block, as the local attention result. At the same time, during the normalization calculation process, the terminal also obtains a normalization factor, which serves as the first normalization factor.
[0090] Step 108: Determine the target normalization factor and target attention score based on the first normalization factor and first attention score of each batch.
[0091] Among them, the target normalization factor and the target attention score are used to recalculate the target result.
[0092] In this embodiment, after completing the calculation of all batches, the terminal summarizes and integrates the first normalization factor and the first attention score obtained from each batch. The terminal can then concatenate the first attention scores calculated based on each loaded batch to obtain the complete target attention score. In the calculation of the target attention matrix block for each batch, the terminal updates the old first normalization factor according to the first normalization factor of the current batch. That is, it updates the shared target normalization factor through the target attention matrix block of the newly loaded batch, until the calculation of all target attention matrix blocks is completed, thus obtaining the target normalization factor.
[0093] The target normalization factor and target attention score are stored in the graphical memory (HBM) and related identification information is recorded. During backpropagation of the target model, the terminal can recalculate the intermediate attention matrix and probability matrix required for gradient calculation using the target normalization factor and target attention score. This avoids storing the intermediate attention matrix and probability matrix and avoids reading them multiple times during gradient calculation, thereby improving model inference speed and training efficiency.
[0094] In the above data processing method, by dividing the large-scale initial attention matrix into smaller target attention matrix blocks, the memory bandwidth pressure caused by loading the entire initial attention matrix at once can be avoided, thus reducing the memory bandwidth requirement. Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches for computation, reducing the number of accesses to HBM. By loading the key matrix blocks and value matrix blocks in batches, the target attention matrix blocks are loaded on demand, reducing the memory access requirements that are limited by bandwidth. Finally, only the target normalization factor and the target attention score are stored in memory, avoiding repeated reading of the intermediate attention matrix and probability matrix in the gradient calculation during backpropagation, thereby improving the efficiency of training the target model.
[0095] In one exemplary embodiment, such as Figure 2 As shown, step 102 includes steps 202 to 204. Wherein:
[0096] Step 202: Determine the block partitioning strategy based on the available storage space of the target cache.
[0097] In this embodiment, the target cache is a memory area used to temporarily store data required for matrix calculations, and its available storage space determines the amount of data that can be processed at once. When processing large-scale matrix operations, such as matrix operations in attention mechanisms, due to the limitations of the target cache SRAM's high speed and small capacity, it is impossible to load all initial attention matrices into the target cache SRAM for calculation at once.
[0098] First, the terminal obtains the available storage space of the target cache through interfaces provided by the operating system or memory management tools, and determines the final block allocation strategy based on a preset scaling factor and the available storage space. For example, Figure 3 As shown in the first row, the dimensions of the query matrix, key matrix, and value matrix are... Taking this as an example, the query matrix, key matrix, and value matrix are initially stored in the GPU's high-bandwidth memory (HBM). Simultaneously, the GPU includes an on-chip static random-access memory (SRAM) with an available storage space of M for temporary storage and computation. The preset scaling factor can be... Therefore, the terminal should reserve at least four times the matrix column dimension. The space is used to store the initial attention matrix after block division, i.e. and This refers to the block size.
[0099] Step 204: Based on the block strategy, the query matrix in the initial attention matrix is divided into rows to obtain query matrix blocks, and the key matrix and value matrix in the initial attention matrix are divided into columns to obtain key matrix blocks and value matrix blocks.
[0100] In this embodiment, after determining the segmentation strategy, the terminal performs segmentation operations on the initial attention matrix according to the segmentation strategy. Specifically, for the query matrix, the terminal divides it equally by row, dividing it into multiple query matrix blocks of the same size, so as to separate the data of different rows of the query matrix for batch calculation. For the key matrix and value matrix, the terminal divides them equally by column, dividing the key matrix and value matrix into multiple key matrix blocks and value matrix blocks of the same size, so that the key matrix blocks and value matrix blocks can process the key and value information column by column when calculating the attention score.
[0101] In this embodiment, the block partitioning strategy is determined by the available storage space of the target cache, and the query matrix, key matrix and value matrix in the initial attention matrix are partitioned according to the block partitioning strategy. This reduces the GPU memory required to load all data at once and serves as the basis for subsequent batch calculation of attention scores. This can effectively utilize the storage space of the target cache and improve the model inference efficiency.
[0102] In one exemplary embodiment, such as Figure 4 As shown, step 104 includes steps 402 to 404. Wherein:
[0103] Step 402: Load the key matrix block and value matrix block into the target cache in batches.
[0104] In this embodiment, the terminal calculates the target attention score using a method combining intra-block computation and inter-block concatenation. After processing the key matrix and value matrix into blocks to obtain key matrix blocks and value matrix blocks, the terminal loads these blocks into the target cache in batches for subsequent computation. For example... Figure 3 In lines 5 and 6, the terminal iterates through all the divided key matrix blocks and value matrix blocks. In each loop, the key matrix blocks and value matrix blocks of the current batch are loaded from high bandwidth memory (HBM) into on-chip SRAM (target cache).
[0105] Because the target cache has limited capacity, it cannot hold the entire key and value matrices at once. By loading key and value matrix blocks in batches, it can be ensured that the target cache only stores the portion of matrix data that needs to be processed at the moment, avoiding the risk of memory overflow. Simultaneously, performing efficient computations on key and value matrix blocks within the target cache reduces frequent transfers between different storage media, i.e., HBM and SRAM, thereby improving overall computational efficiency.
[0106] Step 404: For each loading batch, based on the association between the key matrix block, value matrix block, and query matrix block, load each query matrix block into the target cache.
[0107] In this embodiment, after the terminal completes the loading of the key matrix block and value matrix block for the current batch, the terminal loads each query matrix block into the target cache according to the association relationship between the key matrix block, value matrix block and query matrix block for each batch of loaded key matrix blocks, so as to perform subsequent matrix operations, such as the calculation of intermediate attention matrix block and first attention score.
[0108] like Figure 3 As shown, after each batch of key matrix blocks and value matrix blocks are loaded, the terminal loads the query matrix blocks from high-bandwidth memory (HBM) to the target cache (SRAM) in batches, providing the data foundation for the subsequent calculation of the first attention score and the first normalization factor. For example, taking a query matrix divided into 64 query matrix blocks as an example, after each batch of key matrix blocks and value matrix blocks are loaded into the target cache, the terminal sequentially loads the 64 query matrix blocks... to Load into the target cache.
[0109] In this embodiment, by first loading the key matrix block and value matrix block into the target cache in batches, and then loading each query matrix block based on the association between the key matrix block, value matrix block and query matrix block, the number of repeated loading of key matrix blocks and value matrix blocks can be reduced, thereby reducing the access pressure of high bandwidth memory HBM. This can improve the performance of model data processing and the efficiency of model inference and model training during the model inference stage of model training and model application.
[0110] In one exemplary embodiment, such as Figure 5 As shown, step 106 includes steps 502 to 504. Wherein:
[0111] Step 502: Determine the key matrix block, value matrix block, and query matrix block loaded in the current batch according to the preset batch combination relationship.
[0112] The preset batch combination relationship is obtained by using the key matrix block and the value matrix block as the outer loop matrix block and the query matrix block as the inner loop matrix block.
[0113] In this embodiment of the application, in order to ensure that after each batch of key-value matrix blocks is loaded into the cache, it can be calculated sequentially with all query matrix blocks, the terminal uses the key matrix blocks and value matrix blocks as outer loop matrix blocks and the query matrix blocks as inner loop matrix blocks. Based on the preset batch combination relationship, the terminal determines the key matrix blocks, value matrix blocks and query matrix blocks that need to be loaded in the current batch, and then determines the query matrix blocks that need to be loaded based on the key matrix blocks and value matrix blocks of the current loop batch.
[0114] The key and value matrices are divided into blocks by column. In each outer loop iteration, the terminal sequentially determines the key and value matrix blocks to be loaded, while the query matrix is divided by row. For each batch of key and value matrix blocks loaded into the cache, the terminal uses an inner loop to sequentially determine the corresponding query matrix blocks to be loaded. That is, after each batch of key-value matrix blocks enters the cache, the terminal combines each query matrix block with the corresponding key and value matrix block for subsequent calculations. During the calculation process, by using a pre-defined batch combination relationship, the terminal fully utilizes the key-value matrix blocks already loaded into the target cache and performs calculations with each query matrix block, thus laying the foundation for the initial attention matrix block calculation.
[0115] Step 504: Perform intra-block calculations on each query matrix block based on the key matrix block and value matrix block loaded in the current batch to obtain the first normalization factor and the first attention score of the current batch.
[0116] In this embodiment, the terminal performs intra-block calculations on each query matrix block based on the key matrix block and value matrix block loaded in the current batch, thereby obtaining the first normalization factor and the first attention score for the current batch. Specifically, after loading the key matrix block, value matrix block, and query matrix block into the cache, the terminal first calculates the product of the query matrix block and the transpose of the key matrix block to obtain the intermediate attention matrix block for the current batch. This intermediate attention matrix is used to measure the similarity between each query vector in the query matrix block and each key vector in the key matrix block. In the attention mechanism, the similarity between the query vector and each key vector in the key matrix block reflects the attention that the query vector should give to different key-value pairs. For example, in the text understanding task of natural language processing using the Transformer model, a query term may be more closely associated with certain keywords in the text. By calculating the attention score, the degree of association between the query term and each keyword can be clearly identified, and attention can be reasonably allocated when generating output.
[0117] After obtaining the intermediate attention matrix block, as follows Figure 3 As shown in line 10, the terminal normalizes the intermediate attention matrix block to obtain the probability matrix block for the currently loaded batch. Specifically, the terminal first determines the intermediate attention matrix block based on the key matrix block and the query matrix block, and then determines the first normalization factor based on the intermediate attention matrix block. The terminal further calculates the first attention score based on the normalization result of the intermediate attention matrix block and the value matrix block. This first attention score is used to update the global attention score to obtain the target attention score. In a specific embodiment, after the terminal completes the calculation of the normalization factor and the first attention score in each batch, as shown... Figure 3As shown in lines 11 to 13, the terminal updates the current global attention score based on the contribution of the current first attention score to the global attention score, and updates the current global normalization factor based on the first normalization factor of the current batch, thus obtaining the target attention score and the target normalization factor.
[0118] In this embodiment, by performing intra-block calculations on the query matrix block, key matrix block, and value matrix block, the first calculation factor and the first attention score are obtained, laying the foundation for the accurate generation of the target attention score and the target normalization factor. This realizes the calculation of the target attention score through the block-based target attention matrix block and the recalculation of the data required for backpropagation, avoiding repeated reading of the intermediate attention matrix and probability matrix, thereby improving the efficiency of training the target model.
[0119] In one exemplary embodiment, such as Figure 6 As shown, the method further includes steps 602 to 604. Wherein:
[0120] Step 602: Construct a sparse mask matrix based on preset rules.
[0121] In this embodiment, the terminal can also implement block sparsity functionality to construct an approximate attention algorithm. First, before training the target model, the terminal constructs a sparse mask matrix based on preset rules. These preset rules are predefined rules for generating the sparse mask matrix, including random sparsity, sliding window attention, and local attention. Different preset rules generate different types of sparse mask matrices. For example, the random sparsity rule randomly selects a portion of blocks with a certain probability and sets their corresponding values to 1, while setting the remaining blocks to 0. The random sparsity rule introduces randomness, which can make the model more exploratory in certain scenarios. For the sliding window attention rule, only blocks near the main diagonal are activated; that is, when calculating attention, the focus is mainly on the element relationships around the main diagonal, which can reduce computational load to some extent and focus on key local information. For the local attention rule, each position only focuses on its nearby blocks, making the target model more focused on the local information around the elements, which meets the requirements of some tasks that require local correlation. The sparse mask matrix generated by the preset rules provides a foundation for subsequent sparsification processing of the attention matrix blocks.
[0122] Step 604: During the normalization process, the target attention matrix block is filtered element by element based on the sparse mask matrix to obtain the sparse attention matrix block.
[0123] The sparse attention matrix block includes a sparsely processed query matrix block, a key matrix block, and a value matrix block; the sparse attention matrix block is used to calculate the first normalization factor and the first attention score for the current batch.
[0124] Step 504 specifically includes:
[0125] Step 606: Based on the sparsely processed key matrix block and value matrix block, perform intra-block calculations on each sparsely processed query matrix block to obtain the first normalization factor and the first attention score for the current batch.
[0126] In this embodiment, after constructing the sparse mask matrix, the terminal performs element-wise filtering of the target attention matrix block based on the sparse mask matrix during normalization. The sparse mask matrix consists of 0s and 1s. When filtering the target attention matrix block, elements in the target attention matrix block corresponding to positions with a value of 1 in the mask matrix are retained, while elements corresponding to positions with a value of 0 are discarded. By implementing element-wise filtering through the sparse mask matrix, the target attention matrix block can be sparsified, resulting in a sparse attention matrix block with less data. This reduces computational complexity based on the calculation of the first normalization factor and the first attention score of the sparse attention matrix block.
[0127] In this embodiment, the target attention matrix block is filtered element by element by a sparse mask matrix to obtain a sparse attention matrix block. This can avoid unnecessary calculations during computation, thereby reducing the complexity of the algorithm. The sparsity processing enables the approximate attention algorithm to improve the efficiency of model application and model inference calculation during model training while ensuring a certain level of accuracy.
[0128] In one exemplary embodiment, such as Figure 7 As shown, step 108 is followed by steps 702 to 706. Wherein:
[0129] Step 702: In response to the backpropagation requirement of the target model, read the target normalization factor and target attention score from the video memory.
[0130] In this embodiment, the purpose of backpropagation is to calculate the gradient of each parameter based on the output error of the target model, and then update and optimize the parameters of the target model. When the target model generates a backpropagation requirement, the terminal first retrieves the target normalization factor and target attention score from the video memory.
[0131] The target normalization factor is used to convert the attention score into a probability distribution, thus obtaining the weight of each element. The target attention score, on the other hand, is obtained during the calculation of the attention mechanism by multiplying the transposes of the lookup matrix block and the key matrix block, reflecting the correlation between the input elements. During the backpropagation phase, the target normalization factor and the target attention score are used to recalculate the target attention matrix. Therefore, in response to the backpropagation request, the target normalization factor and the target attention score are read from GPU memory, providing the data basis for subsequent recalculation.
[0132] Step 704: Recalculate the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block based on the target normalization factor.
[0133] In this embodiment, after reading the target normalization factor and target attention score, the terminal recalculates the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block based on the target normalization factor. The target attention matrix block is a set of multiple matrix blocks into which the complete attention matrix is divided according to certain rules. This can improve computational efficiency in large-scale data processing. Through the target normalization factor, the calculation of the intermediate attention matrix block and probability matrix block can be achieved based on the target attention matrix block in matrix block form.
[0134] The intermediate attention matrix block is an intermediate result in the process of calculating the attention score; it is the product of the transpose of the query matrix block and the key matrix block. The probability matrix block is obtained by normalizing the intermediate attention matrix block and represents the weight distribution of each element in the attention mechanism.
[0135] Step 706: Determine the gradient result based on the intermediate attention matrix block, the probability matrix block, and the target attention score.
[0136] In this embodiment, after obtaining the intermediate attention matrix block, probability matrix block, and attention score, the terminal calculates the gradient result based on these components. In deep learning, the gradient reflects the degree of influence of small changes in model parameters on the loss function.
[0137] In the backpropagation of the attention mechanism, the terminal needs to calculate the gradients of each parameter based on the intermediate attention matrix block, probability matrix block, and attention score. Specifically, the terminal uses a chain rule to progressively backpropagate the gradient of the loss function with respect to the final output to each parameter of the attention mechanism. The intermediate attention matrix block and probability matrix block provide intermediate result information for each step in the forward propagation process, while the attention score is the basis for calculating these results. By reading the target attention score and combining it with the recalculated intermediate attention matrix block and probability matrix block, the terminal can obtain the gradient result for each parameter. This gradient result will be used for subsequent parameter updates, enabling the target model to optimize in the direction of reducing the loss function.
[0138] In this embodiment, recalculating the intermediate attention matrix block and probability matrix block during backpropagation avoids the situation of reading the intermediate attention matrix and probability matrix multiple times in HBM. Instead, it recalculates based on the target normalization factor, which improves the overall performance. The performance improvement often outweighs the loss caused by the increased computation, thereby improving the efficiency of model inference.
[0139] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0140] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.
[0141] In one exemplary embodiment, such as Figure 8 As shown, a data processing device 800 is provided, including: a block module 801, a loading module 802, a first calculation module 803, and a second calculation module 804, wherein:
[0142] The block segmentation module 801 is used to segment the initial attention matrix into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0143] The loading module 802 is used to load the target attention matrix blocks into the target cache in batches based on a preset loading strategy;
[0144] The first calculation module 803 is used to perform combined calculations on the key matrix block and value matrix block loaded in the current batch and each query matrix block according to the preset batch combination relationship, so as to obtain the first normalization factor and the first attention score of the current batch.
[0145] The second calculation module 804 is used to determine the target normalization factor and target attention score based on the first normalization factor and first attention score of each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0146] In one embodiment, the block segmentation module 801 is specifically configured to determine a block segmentation strategy based on the available storage space of the target cache;
[0147] Based on the block-based strategy, the query matrix in the initial attention matrix is divided into rows to obtain query matrix blocks, and the key matrix and value matrix in the initial attention matrix are divided into columns to obtain key matrix blocks and value matrix blocks.
[0148] In one embodiment, the loading module 802 is specifically used to load the key matrix block and the value matrix block into the target cache in batches;
[0149] For each loading batch, based on the association between the key matrix block, value matrix block, and query matrix block, each query matrix block is loaded into the target cache.
[0150] In one embodiment, the first calculation module 803 is specifically used to determine the key matrix block, value matrix block and query matrix block loaded in the current batch according to a preset batch combination relationship; the preset batch combination relationship is obtained by using the key matrix block and value matrix block as the outer loop matrix block and the query matrix block as the inner loop matrix block;
[0151] Based on the key matrix block and value matrix block loaded in the current batch, perform intra-block calculations on each query matrix block to obtain the first normalization factor and the first attention score for the current batch.
[0152] In one embodiment, the device 800 further includes:
[0153] The building module is used to construct a sparse mask matrix based on preset rules;
[0154] The filtering module is used to filter the target attention matrix block element by element based on the sparse mask matrix during the normalization process to obtain the sparse attention matrix block; the sparse attention matrix block includes the sparsely processed query matrix block, key matrix block and value matrix block.
[0155] The first calculation module 803 is specifically used to perform intra-block calculations on each query matrix block after sparse processing based on the key matrix block and value matrix block after sparse processing, so as to obtain the first normalization factor and the first attention score of the current batch.
[0156] In one embodiment, the device 800 further includes:
[0157] In response to the backpropagation requirement of the target model, the target normalization factor and target attention score are read from the video memory.
[0158] Recalculate the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block based on the target normalization factor;
[0159] The gradient result is determined based on the intermediate attention matrix block, the probability matrix block, and the target attention score.
[0160] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0161] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a data processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.
[0162] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0163] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0164] The initial attention matrix is divided into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block.
[0165] Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches;
[0166] Based on the preset batch combination relationship, the key matrix block and value matrix block loaded in the current batch are combined and calculated for each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
[0167] The target normalization factor and target attention score are determined based on the first normalization factor and first attention score of each batch; the target normalization factor and target attention score are used to recalculate the target result.
[0168] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0169] The block partitioning strategy is determined based on the available storage space of the target cache;
[0170] Based on the block-based strategy, the query matrix in the initial attention matrix is divided into rows to obtain query matrix blocks, and the key matrix and value matrix in the initial attention matrix are divided into columns to obtain key matrix blocks and value matrix blocks.
[0171] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0172] Load the key matrix blocks and value matrix blocks into the target cache in batches;
[0173] For each loading batch, based on the association between the key matrix block, value matrix block, and query matrix block, each query matrix block is loaded into the target cache.
[0174] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0175] Based on the preset batch combination relationship, determine the key matrix block, value matrix block, and query matrix block loaded in the current batch; the preset batch combination relationship is obtained by using the key matrix block and value matrix block as the outer loop matrix block and the query matrix block as the inner loop matrix block;
[0176] Based on the key matrix block and value matrix block loaded in the current batch, perform intra-block calculations on each query matrix block to obtain the first normalization factor and the first attention score for the current batch.
[0177] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0178] Construct a sparse mask matrix based on preset rules;
[0179] During the normalization process, the target attention matrix block is filtered element by element based on the sparse mask matrix to obtain the sparse attention matrix block; the sparse attention matrix block includes the sparsely processed query matrix block, key matrix block and value matrix block.
[0180] Based on the key matrix block and value matrix block loaded in the current batch, intra-block calculations are performed on each query matrix block to obtain the first normalization factor and first attention score for the current batch, including:
[0181] Based on the sparsely processed key matrix blocks and value matrix blocks, intra-block calculations are performed on each sparsely processed query matrix block to obtain the first normalization factor and the first attention score for the current batch.
[0182] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0183] In response to the backpropagation requirement of the target model, the target normalization factor and target attention score are read from the video memory.
[0184] Recalculate the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block based on the target normalization factor;
[0185] The gradient result is determined based on the intermediate attention matrix block, the probability matrix block, and the target attention score.
[0186] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.
[0187] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0189] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0190] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0191] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A data processing method, characterized in that, The method includes: The initial attention matrix is divided into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block. Based on a preset loading strategy, the target attention matrix blocks are loaded into the target cache in batches; According to the preset batch combination relationship, the key matrix block and the value matrix block loaded in the current batch are combined and calculated with each query matrix block to obtain the first normalization factor and the first attention score of the current batch. The target normalization factor and target attention score are determined based on the first normalization factor and the first attention score for each batch; the target normalization factor and target attention score are used to recalculate the target result.
2. The method according to claim 1, characterized in that, The step of dividing the initial attention matrix into blocks to obtain the target attention matrix blocks includes: The block partitioning strategy is determined based on the available storage space of the target cache; Based on the block partitioning strategy, the query matrix in the initial attention matrix is divided into rows to obtain query matrix blocks, and the key matrix and value matrix in the initial attention matrix are divided into columns to obtain key matrix blocks and value matrix blocks.
3. The method according to claim 1, characterized in that, The step of loading the target attention matrix blocks into the target cache in batches based on a preset loading strategy includes: The key matrix block and the value matrix block are loaded into the target cache in batches; For each loading batch, based on the association between the key matrix block, the value matrix block, and the query matrix block, each query matrix block is loaded into the target cache.
4. The method according to claim 1, characterized in that, The step of combining and calculating the key matrix block and the value matrix block loaded in the current batch with each query matrix block according to the preset batch combination relationship to obtain the first normalization factor and the first attention score of the current batch includes: Based on a preset batch combination relationship, the key matrix block, the value matrix block, and the query matrix block loaded in the current batch are determined; the preset batch combination relationship is obtained by using the key matrix block and the value matrix block as the outer loop matrix block and the query matrix block as the inner loop matrix block; Based on the key matrix block and the value matrix block loaded in the current batch, intra-block calculations are performed on each query matrix block to obtain the first normalization factor and the first attention score of the current batch.
5. The method according to claim 1, characterized in that, The method further includes: Construct a sparse mask matrix based on preset rules; During the normalization process, the target attention matrix block is filtered element by element based on the sparse mask matrix to obtain a sparse attention matrix block; the sparse attention matrix block includes the query matrix block, the key matrix block, and the value matrix block after sparse processing. The step of performing intra-block calculations on each query matrix block based on the key matrix block and the value matrix block loaded in the current batch to obtain the first normalization factor and the first attention score for the current batch includes: Based on the sparsely processed key matrix block and value matrix block, intra-block calculations are performed on each of the sparsely processed query matrix blocks to obtain the first normalization factor and the first attention score for the current batch.
6. The method according to claim 1, characterized in that, After determining the target normalization factor and target attention score based on the first normalization factor and the first attention score for each batch, the method further includes: In response to the backpropagation requirement of the target model, the target normalization factor and the target attention score are read from the video memory; Based on the target normalization factor, recalculate the intermediate attention matrix block and probability matrix block corresponding to each target attention matrix block; The gradient result is determined based on the intermediate attention matrix block, the probability matrix block, and the target attention score.
7. A data processing apparatus, characterized in that, The device includes: The block segmentation module is used to segment the initial attention matrix into blocks to obtain the target attention matrix block; the target attention matrix block includes a query matrix block, a key matrix block, and a value matrix block; The loading module is used to load the target attention matrix blocks into the target cache in batches based on a preset loading strategy; The first calculation module is used to perform combined calculations on the key matrix block and the value matrix block loaded in the current batch and each query matrix block according to the preset batch combination relationship, so as to obtain the first normalization factor and the first attention score of the current batch. The second calculation module is used to determine the target normalization factor and target attention score based on the first normalization factor and the first attention score of each batch; the target normalization factor and target attention score are used to recalculate the target result.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.