Data processing method, device and equipment and computer readable storage medium

Abstract

The invention discloses a data processing method, device and equipment and a computer readable storage medium, and belongs to the technical field of computers. The method comprises the steps of obtaining an auxiliary mask matrix of a neural network; obtaining an attention matrix of the input data, wherein the attention matrix comprises a plurality of data blocks; and for any data block, under the condition that it is determined that any data block has a mask calculation requirement according to the auxiliary mask matrix, obtaining an attention mask matrix corresponding to any data block according to the auxiliary mask matrix, and obtaining a result of processing the input data through the neural network based on the attention mask matrix corresponding to any data block. According to the invention, the attention mask matrix used in the mask calculation process can be obtained only when the data block is determined to participate in the mask calculation according to the auxiliary mask matrix, so that the flexibility is high. The attention mask matrix which does not need to be used is effectively prevented from being obtained under the condition that the data block does not have the mask calculation requirement, and the calculation amount is reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to data processing methods, apparatus, devices and computer-readable storage media. Background Technology

[0002] In the field of computer technology, neural networks can be used to process input data. For example, neural networks can be used for text recognition, speech recognition, and image synthesis. In some cases, neural networks employ attention masks during input data processing to specify whether the network focuses on the correlations between features in the input data.

[0003] In related technologies, for the input data fed into the neural network, an attention mask matrix is ​​generated based on the sequence length of the input data. The number of rows and columns of the generated attention mask matrix is ​​consistent with the sequence length. Then, the input data is processed based on the generated attention mask matrix.

[0004] The methods described above all require generating an attention mask matrix based on the sequence length of the input data during the input data processing process. This results in low flexibility, low processing efficiency, and high computational resource consumption. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, and computer-readable storage medium to solve the problems existing in the related technologies. The technical solution is as follows:

[0006] In a first aspect, a data processing method is provided, the method comprising: obtaining an auxiliary mask matrix of a neural network, the auxiliary mask matrix being used for auxiliary mask calculation; obtaining an attention matrix of input data, the attention matrix comprising multiple data blocks; for any data block, if it is determined from the auxiliary mask matrix that any data block has a mask calculation requirement, obtaining the attention mask matrix corresponding to any data block from the auxiliary mask matrix, and obtaining the result of processing the input data by the neural network based on the attention mask matrix corresponding to any data block.

[0007] The data processing method provided in this application first determines whether a data block will participate in subsequent mask calculations based on an auxiliary mask matrix. Only when a data block participates in mask calculations, i.e., when mask calculation is required, is the attention mask matrix used in the mask calculation process obtained, offering high flexibility. This effectively avoids obtaining unnecessary attention mask matrices when data blocks do not require mask calculations, reducing unnecessary acquisition operations, saving computational resources, and improving the efficiency of input data processing. The attention mask matrix can be obtained directly from the auxiliary mask matrix, making the acquisition process simple and consuming low computational resources.

[0008] In one possible implementation, obtaining the auxiliary mask matrix of the neural network includes: obtaining the size of the basic computational blocks used to divide the data blocks included in the attention matrix; determining the matrix shape based on the size of the basic computational blocks, the matrix shape indicating the number of rows and columns of the auxiliary mask matrix to be generated; and generating the auxiliary mask matrix according to the matrix shape. The matrix shape of the auxiliary mask matrix is ​​determined based on the basic computational blocks, without referencing the attention matrix of the input data. The generated auxiliary mask matrix is ​​applicable to attention matrices of arbitrary shapes, exhibiting high versatility.

[0009] In one possible implementation, each data block includes multiple correlations, and each data block has a corresponding submatrix in the auxiliary mask matrix. The submatrix includes multiple elements, and the values ​​of these elements indicate whether the correlation corresponding to the element is relevant. After obtaining the attention matrix of the input data, the process further includes: if the values ​​of the multiple elements in the submatrix are the same, determining that no data block requires mask calculation; or, if the values ​​of the multiple elements in the submatrix are different, determining that any data block requires mask calculation. By comparing whether the values ​​of the elements in the submatrix are the same, it is possible to determine whether a data block requires mask calculation. This process is simple and efficient.

[0010] In one possible implementation, multiple elements having the same value includes multiple elements having a first value or multiple elements having a second value. The first element having a first value indicates that the correlation corresponding to the first element is not being considered, and the second element having a second value indicates that the correlation corresponding to the second element is being considered. The first and second elements can be any one of the multiple elements. This application does not limit the data block to those without mask calculation requirements; the multiple correlations included in the data block can be all unconsidered or all considered, demonstrating high versatility.

[0011] In one possible implementation, the method further includes: determining the position of any data block in the attention matrix; mapping any data block to an auxiliary mask matrix based on its position in the attention matrix; and determining a submatrix based on the mapping result. The submatrix corresponding to the data block can be determined in the auxiliary mask matrix through mapping, which is a simple process with low operational complexity.

[0012] In one possible implementation, after obtaining the attention matrix of the input data, the process further includes: if, based on the auxiliary mask matrix, it is determined that no mask calculation is required for any data block, and the multiple correlations included in any data block are not of concern, then the processing operations following mask calculation for that data block are stopped; or, if, based on the auxiliary mask matrix, it is determined that no mask calculation is required for any data block, and the multiple correlations included in any data block are of concern, then the processing operations for that data block are continued. Besides determining whether mask calculation is required based on the auxiliary mask matrix, if no mask calculation is required, it further determines whether the correlations included in the data block are of concern. If none of the correlations included in the data block are of concern, the processing operations for that data block are stopped promptly, controlling the number of correlations involved in the processing operations and reducing the computational load.

[0013] In one possible implementation, obtaining the attention mask matrix corresponding to any data block based on the auxiliary mask matrix includes: processing the sub-matrix corresponding to any data block in the auxiliary mask matrix according to the application scenario of the input data, and using the processed sub-matrix as the attention mask matrix corresponding to any data block; or, determining the sub-matrix corresponding to any data block as the attention mask matrix corresponding to any data block. This application does not limit the method of obtaining the attention mask matrix based on the auxiliary mask matrix; it can directly use the sub-matrix in the auxiliary mask matrix as the attention mask matrix corresponding to the data block, or it can process the sub-matrix and use the processed sub-matrix as the attention mask matrix corresponding to the data block, offering high flexibility.

[0014] In one possible implementation, the neural network is applied to at least one of augmented reality (AR), virtual reality (VR), artificial intelligence (AI), or application software. This application does not limit the application scenarios of the neural network; it is applicable to a variety of scenarios and has high versatility.

[0015] In a second aspect, a data processing apparatus is provided, comprising: an acquisition module for acquiring an auxiliary mask matrix of a neural network, the auxiliary mask matrix being used for auxiliary mask calculation; the acquisition module further for acquiring an attention matrix of input data, the attention matrix including multiple data blocks; and a processing module for, for any data block, if it is determined from the auxiliary mask matrix that any data block requires mask calculation, acquiring the attention mask matrix corresponding to any data block based on the auxiliary mask matrix, and acquiring the result of processing the input data through the neural network based on the attention mask matrix corresponding to any data block.

[0016] In one possible implementation, an acquisition module is used to acquire the size of a basic computation block, which is used to divide the data blocks included in the attention matrix; determine the matrix shape based on the size of the basic computation block, the matrix shape indicating the number of rows and columns of the auxiliary mask matrix to be generated; and generate the auxiliary mask matrix according to the matrix shape.

[0017] In one possible implementation, any data block includes multiple correlation degrees, and any data block has a corresponding submatrix in the auxiliary mask matrix. The submatrix includes multiple elements, and the value of each element indicates whether the correlation degree corresponding to the element is relevant. The processing module is also used to determine that any data block does not have a mask calculation requirement if the values ​​of the multiple elements included in the submatrix are the same; or, if the values ​​of the multiple elements included in the submatrix are different, determine that any data block has a mask calculation requirement.

[0018] In one possible implementation, having multiple elements with the same value includes having multiple elements with the first value or having multiple elements with the second value. The first element is set to the first value to indicate the degree of relevance for which the first element is not being followed, and the second element is set to the second value to indicate the degree of relevance for which the second element is being followed. The first element and the second element can be any one of the multiple elements.

[0019] In one possible implementation, the processing module is further configured to determine the position of any data block in the attention matrix; map any data block to an auxiliary mask matrix based on its position in the attention matrix; and determine a submatrix based on the mapping result.

[0020] In one possible implementation, the processing module is further configured to stop performing subsequent processing operations on any data block after mask calculation if it is determined from the auxiliary mask matrix that no mask calculation is required for any data block and the multiple correlations included in any data block are not of concern; or, if it is determined from the auxiliary mask matrix that no mask calculation is required for any data block and the multiple correlations included in any data block are of concern, then continue performing processing operations on any data block.

[0021] In one possible implementation, the processing module is used to process the sub-matrix corresponding to any data block in the auxiliary mask matrix according to the application scenario of the input data, and use the processed sub-matrix as the attention mask matrix corresponding to any data block; or, the sub-matrix corresponding to any data block is determined as the attention mask matrix corresponding to any data block.

[0022] In one possible implementation, the neural network is applied to at least one of AR, VR, AI, or application software.

[0023] Thirdly, a data processing apparatus is provided, the apparatus including a processor for loading and executing at least one instruction to cause the data processing apparatus to perform the method of the first aspect or any possible implementation thereof.

[0024] In one possible implementation, the device includes a memory coupled to a processor, the memory storing at least one instruction.

[0025] Fourthly, a computer-readable storage medium is provided, wherein at least one instruction is stored in the computer-readable storage medium, the instruction being loaded and executed by a processor to implement the data processing method in the first aspect or any possible implementation of the first aspect.

[0026] Fifthly, a computer program (product) is provided, comprising computer program / instructions, which are executed by a processor to enable a computer to implement the data processing method in the first aspect or any possible implementation thereof.

[0027] A sixth aspect provides a communication device comprising: a transceiver, a memory, and a processor. The transceiver, the memory, and the processor communicate with each other via an internal connection path. The memory stores instructions, and the processor executes the instructions stored in the memory to control the transceiver to receive and transmit signals. When the processor executes the instructions stored in the memory, it causes the processor to perform the method of the first aspect or any possible implementation thereof.

[0028] Optionally, there may be one or more processors and one or more memories.

[0029] Alternatively, the memory can be integrated with the processor, or the memory can be set up separately from the processor.

[0030] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not limit the type of memory or the way the memory and processor are set.

[0031] In a seventh aspect, a chip is provided, including a processor for calling and running program instructions or code stored in a memory, causing a communication device equipped with the chip to perform the methods in the foregoing aspects.

[0032] Eighthly, another chip is provided, comprising: an input interface, an output interface, a processor, and a memory, wherein the input interface, the output interface, the processor, and the memory are connected via an internal connection path, and the processor is used to execute code in the memory, and when the code is executed, the processor is used to perform the methods in the above aspects.

[0033] It should be understood that the beneficial effects of the technical solutions and corresponding possible implementations of the second to eighth aspects of this application can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the structure of a transformer provided in an embodiment of this application;

[0035] Figure 2 A flowchart illustrating the processing of a Decoder Block is provided in this embodiment of the application.

[0036] Figure 3 A flowchart illustrating the processing of MultiHead Attention as provided in an embodiment of this application;

[0037] Figure 4 A schematic diagram of an implementation environment provided for an embodiment of this application;

[0038] Figure 5 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0039] Figure 6 A schematic diagram of an attention matrix provided in an embodiment of this application;

[0040] Figure 7 A schematic diagram of another attention matrix provided in an embodiment of this application;

[0041] Figure 8A schematic diagram of an attention mask matrix provided in an embodiment of this application;

[0042] Figure 9 A schematic diagram illustrating the correspondence between a submatrix and a data block provided in an embodiment of this application;

[0043] Figure 10 A schematic diagram of a submatrix processing method provided in an embodiment of this application;

[0044] Figure 11 A flowchart illustrating another data processing method provided in this application embodiment;

[0045] Figure 12 A flowchart illustrating another data processing method provided in this application embodiment;

[0046] Figure 13 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0047] Figure 14 This application provides a schematic diagram of the structure of a network device according to an embodiment of the present application.

[0048] Figure 15 This is a schematic diagram of the structure of another network device provided in an embodiment of this application. Detailed Implementation

[0049] The terminology used in the embodiments section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0050] In the field of computer science, transformer architectures are increasingly used in applications, such as in large models. A transformer is a deep learning model that employs an attention mechanism to assign different weights based on the importance of different parts of the input data. Large models typically refer to neural networks with millions to billions of parameters. These large models are capable of processing input data and performing tasks including, but not limited to, natural language processing, computer vision, and speech recognition. In some cases, large models are also called foundation models.

[0051] As the amount of data processed by the Transformer architecture increases and the data sequences become longer, the memory consumption problem of the Transformer architecture becomes more and more serious. Therefore, an attention mechanism can be introduced. During the training or inference of the attention part of the Transformer architecture, the correlation between tokens can be specified as to control the number of correlations that are focused on, thereby reducing the computational cost of the Transformer architecture.

[0052] Figure 1 This is a schematic diagram of a transformer structure provided in an embodiment of this application. The transformer structure includes an embedding layer, a linear (LN) layer, a decoder block, an LN layer, and a transposed embedding layer. T Layers such as the Softmax layer and the normalization layer. Figure 1 The Decoder Block in [the code] introduces an attention mechanism, see [link / reference]. Figure 2 The flowchart shown is for the processing of the decoding block. Figure 2 The Decoder Block in the model comprises an LN layer, a multi-head attention layer, and a multilayer perceptron (MLP). The multi-head attention layer uses an attention mechanism to focus on the correlation between tokens. In some cases, during the process of focusing on the correlation between tokens using the attention mechanism, an attention mask matrix can be used to indicate whether to focus on the correlation between tokens. For example... Figure 3 As shown, Figure 3 It shows Figure 2 The processing flow of MultiHead Attention in [the context of the text] Figure 3 In the `atten_mask` directive, the attention mask matrix is ​​defined, and `(S, S)` indicates the number of rows and columns of the attention mask matrix. After obtaining the attention matrix including the correlation degree through matrix multiplication (MatMul), vector multiplication (Muls), etc., a causal relationship (is_causal) determination process can be performed on the attention matrix. If no causal relationship exists, no mask calculation is performed on the attention matrix; if a causal relationship is determined, the attention mask matrix is ​​used to perform mask calculation. Regarding... Figure 3 For other steps and the detailed process of obtaining the attention matrix, please refer to the relevant descriptions in the embodiments below, which will not be repeated here.

[0053] In related technique one, after acquiring the input data to be processed, an attention mask matrix is ​​generated outside the operator based on the sequence length of the input data. The number of rows and columns of this attention mask matrix is ​​consistent with the sequence length of the input data, which can be represented as seq*seq. The generated attention mask matrix is ​​then passed into the operator for masking. However, as the neural network grows, the sequence length increases, and the memory consumption of the attention mask matrix generated based on the sequence length increases quadratically. Furthermore, since the memory occupied by the attention mask matrix is ​​limited, and the size of the attention mask matrix is ​​also limited, the sequence length and batch size of input data that can be processed are also limited, resulting in low versatility. Moreover, generating a different attention mask matrix each time based on the sequence length leads to low flexibility and high computational cost.

[0054] In related technique two, during the input data processing, the attention mask matrix is ​​generated online within the operator via parameter instructions, or each correlation is judged point-by-point online. Online generation of the attention mask matrix introduces additional computational resources, leading to a decrease in efficiency.

[0055] This application provides a data processing method, please refer to the embodiments therein. Figure 4 The illustration shows a schematic diagram of the implementation environment of the data processing method provided in the embodiments of this application. The implementation environment includes a computing device 04, which can run a neural network. The computing device 04 also provides an auxiliary mask matrix to the neural network so as to process the attention matrix of the input data using the auxiliary mask matrix during the process of processing the input data through the neural network.

[0056] Optionally, the computing device 04 can be any device with data processing capabilities, and can be any terminal device, including but not limited to desktop computers, laptops, tablets, or smartphones. Optionally, the computing device 04 can also be a server, such as a central server, an edge server, or a local server in a local data center. The server can be a physical server or a cloud server providing cloud computing services. The entity executing the data processing method can be the entire computing device, or a component of the computing device, such as a single board, line card, or functional module on the computing device, or a chip used to implement the method. This application embodiment does not specifically limit the entity executing the method.

[0057] This application provides a data processing method that can be applied to the above-mentioned... Figure 4 The implementation environment shown is illustrated using the method applied to a computing device as an example. The flowchart of the method is as follows: Figure 5 As shown, it includes S501-S503.

[0058] S501, obtain the auxiliary mask matrix of the neural network. The auxiliary mask matrix is ​​used for auxiliary mask calculation.

[0059] For example, a neural network, also known as an artificial neural network (ANN), is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks to perform distributed parallel information processing. In one possible scenario, the neural network running on a computing device includes a transformer structure. For instance, a neural network can be an algorithmic mathematical model obtained by combining a transformer structure with other structures, including but not limited to convolutional neural networks (CNN)-transformers, long short-term memory networks (LSTM)-transformers, etc. Optionally, the neural network running on the computing device can be a neural network in the training phase or a neural network in the inference phase; the inference phase can sometimes be referred to as the usage phase.

[0060] This application does not limit the application scenario of the neural network. The neural network can be applied to at least one of AR, VR, AI, or application software. AI includes, but is not limited to, multiple fields such as generative intelligence-generated content (AIGC), deep learning, and machine learning (ML). Application software can be any type of software, including but not limited to game software, social software, logistics software, etc. In one possible scenario, neural networks applied to different application scenarios can process different types of input data, including but not limited to text data, voice data, image data, and video data. In some cases, the input data can also be called media data, which can be data transmitted using any information transmission medium, such as tools for disseminating text, sound, images, etc. This application also does not limit the processing operations of the neural network on the input data. Taking text data as an example, the neural network can process text data through text recognition, text translation, summary generation, or poster generation. Taking voice data as an example, the neural network can process voice data through voice recognition, subtitle generation, etc.

[0061] Regardless of the type of neural network, when an attention mechanism is used during the processing of input data, the computing device can obtain an auxiliary mask matrix to perform mask calculations using the attention mechanism. This auxiliary mask matrix is ​​an auxiliary matrix used in the process of obtaining the attention mask matrix. Obtaining the attention mask matrix can sometimes be referred to as mask calculation. Examples of auxiliary mask matrices can be found in the following embodiments describing the process of obtaining the auxiliary mask matrix; they will not be repeated here.

[0062] In one possible scenario, the computing device can offline generate an auxiliary mask matrix outside the attention operator of the neural network before processing the input data using the neural network—that is, during the preprocessing stage or the model initialization stage. This generated auxiliary mask matrix can then be directly used during subsequent processing of the input data using the neural network. Here, "operator" refers to a computational operation, including but not limited to addition, multiplication, matrix multiplication, and matrix transpose. "Attention operator" refers to the operator used by the attention part of the neural network, such as... Figure 2The MultiHead Attention operator shown is used in some cases. In some cases, the attention operator is a fusion operator of a neural network processing unit (NPU). A fusion operator is a large operator obtained by fusing multiple basic operators, such as addition, multiplication, and matrix multiplication as described in the above embodiments. Optionally, the NPU's fusion operator includes, but is not limited to, a prefilled flashattention (PFA) or a reordered flashattention (FA).

[0063] In one possible implementation, the process of the computing device generating the auxiliary mask matrix includes: obtaining the size of a basic computational block used to divide the data blocks included in the attention matrix; determining the matrix shape based on the size of the basic computational block, the matrix shape indicating the number of rows and columns of the auxiliary mask matrix to be generated; and generating the auxiliary mask matrix according to the matrix shape.

[0064] In one possible scenario, during matrix computation, the size of the matrix processed by a computing device is limited by the hardware chip. If the size of the matrix exceeds the chip's computational capabilities, the matrix is ​​divided into blocks. For example, the computing device can divide the matrix into blocks based on basic computational blocks. Taking the attention matrix obtained during input data processing as an example, the attention matrix has 10,000 rows and 10,000 columns. However, the chip in the computing device cannot compute a 10,000x10,000 attention matrix at once. Therefore, the computing device first determines the size of the basic computational block, divides the attention matrix according to the size of the basic computational block, obtains multiple data blocks, and then computes the data blocks sequentially. Here, the basic computational block indicates the matrix that the chip can process, and the size of the basic computational block indicates the number of rows and columns; in some cases, this can be referred to as the shape of the basic computational block. For example, the size of the basic computational block can be determined based on the computing device's chip and the application scenario. For instance, the size of the basic computational block might be determined to be 512*1024 based on the chip. This size does not exceed the chip's computational limit and can support attention matrices of input data processed in any application scenario.

[0065] Since the size of the basic computation block indicates the shape of the matrix that the computing device supports for operations, meaning that the computing device processes arbitrary matrices using the chip at the basic computation block level, the size of the processed matrix will not exceed the size of the basic computation block. Therefore, the computing device can determine the number of rows and columns of the auxiliary mask matrix to be generated based on the size of the basic computation block. The generated auxiliary mask matrix can be reused throughout the lifetime of the neural network, exhibiting high versatility.

[0066] Optionally, the process by which the computing device determines the number of rows and columns is, for example, to select the largest value from the number of rows and columns of the basic computing block, and multiply the largest value by a first value to obtain the number of rows and columns of the auxiliary mask matrix. The first value can be a positive integer set based on experience, for example, the first value is set to 2 based on experience. Taking a basic computing block size of 3*5 as an example, the basic computing block has 3 rows and 5 columns. The largest value among the number of rows and columns is that the number of columns is 5. The computing device multiplies 5 by 2 to get 10, thereby determining that the number of rows and columns of the auxiliary mask matrix is ​​10. Alternatively, taking a basic computing block size of 512*1024 in the above embodiment as an example, the number of rows and columns of the auxiliary mask matrix is ​​1024*2 = 2048, that is, the matrix shape of the auxiliary mask matrix is ​​2048*2048.

[0067] Since the number of rows and columns of the auxiliary mask matrix acquired by the computing device is related to the size of the computation block (e.g., fixed at 2048*2048), it is independent of the sequence length of the input data processed by the neural network. This makes it applicable to input data of any sequence length, offering high versatility. The sequence length of the input data indicates the number of tokens obtained by processing the input data. Furthermore, because the size of the auxiliary mask matrix is ​​fixed, computational bottlenecks (bounds) can occur in scenarios where the sequence length of FA-related fusion operators in the Faster Transformer acceleration library increases. Even with increasing sequence lengths, the size of the auxiliary mask matrix remains unchanged, improving the performance of FA-related fusion operators, controlling computational load, and thus controlling the bounds issue.

[0068] After determining the number of rows and columns of the auxiliary mask matrix, the computing device determines the value of each element in the auxiliary mask matrix according to the matrix type of the auxiliary mask matrix. Matrix types include, but are not limited to, dense matrices, sparse matrices, triangular matrices, striped matrices, diagonal matrices, and tridiagonal matrices. The matrix type of the auxiliary mask matrix can be selected based on experience. For example, the computing device acquires multiple matrix types of attention mask matrices that may be involved in running the neural network. Based on the transformation relationships and experience of multiple matrix types, a target type is selected from the multiple matrix types. This target type can be transformed to obtain other matrix types. In some cases, the target type can be a lower triangular matrix. In this case, the values ​​of the elements above the diagonal in the auxiliary mask matrix are all second values, and the values ​​of the elements below the diagonal are all first values. The first and second values ​​can be different values ​​set based on experience; for example, the first value is 1 and the second value is 0, or the first value is 0 and the second value is 1. This application embodiment does not limit this.

[0069] In one possible implementation, the computing device can also generate the auxiliary mask matrix online within the operator, i.e., the kernel. For example, if the computing device has not obtained the auxiliary mask matrix in advance, it can generate it online. Alternatively, even if the computing device has already generated the auxiliary mask matrix in advance, but the size of the basic computation block changes, the computing device can generate the auxiliary mask matrix online using the above method based on the changed size of the basic computation block. This change in the size of the basic computation block could be due to changes in the neural network running on the computing device, or an upgrade to the computing device's chip.

[0070] S502, obtain the attention matrix of the input data, the attention matrix includes multiple data blocks.

[0071] In one possible implementation, during the processing of input data using a neural network, the computing device obtains an attention matrix of the input data. The attention matrix is ​​a matrix obtained based on the Q-vector and K-vector. The attention matrix includes multiple correlation degrees, one of which indicates the degree of matching between two feature vectors. The degree of matching is used to indicate the correlation between the two feature vectors. The higher the degree of matching, the stronger the correlation between the two feature vectors. In some cases, the degree of matching can also be called the correlation. The correlation degree can be called a score indicating the degree of matching or correlation.

[0072] In one possible scenario, the position of the correlation degree within the attention matrix can reflect the two feature vectors corresponding to that correlation degree. For example, the two feature vectors corresponding to the correlation degree located in the first row and second column of the attention matrix might be the first and second feature vectors, respectively. For example, the correlation between the first feature vector and the second feature vector is 7.5.

[0073] Optionally, the feature vector is a vector obtained by processing the input data. For example, the input data includes multiple data segments, and each data segment is embedded into a high-dimensional space through an embedding layer to obtain a feature vector. Here, a data segment indicates the smallest unit of the input data, and the type of the data segment is related to the type of input data. In the case of text data, the data segment can be a character, word, or sentence. In the case of image data, the data segment can be a pixel or a patch obtained by segmenting the image data. In the case of audio data, the data segment can be an audio frame. In the case of video data, the data segment can be a single frame image from the video. Since the feature vector is a vector obtained by processing the smallest unit of input data, in some cases, the feature vector can also be called a token.

[0074] Figure 3 The process of obtaining the attention matrix of the input data is shown. Figure 3 In this context, the query (Q-vector) is used to calculate the correlation between a feature vector and its neighboring feature vectors. S indicates length, B indicates batch size, and H indicates dimension. The key (K-vector) describes the features of neighboring feature vectors, and the value (V-vector) contains the feature information of neighboring feature vectors. The Q, K, and V vectors are obtained by processing the feature vectors. For example, each feature vector is positionally encoded, and a fully connected computation is performed on the positionally encoded feature vectors to obtain the Q, K, and V vectors for each feature vector. After processing the input data to obtain the Q, K, and V vectors, the K-vector can be transposed, and the transposed K and Q vectors can be multiplied using a matrix multiplication (MatMul). Finally, a vector multiplication (Muls) is performed on the result of the matrix multiplication, and the resulting matrix is ​​the attention matrix for the input data.

[0075] In one possible scenario, after acquiring the attention matrix, the computing device, due to the limited matrix size supported by the chip, will divide the attention matrix according to the size of the basic computation block, resulting in multiple data blocks. The size of these data blocks will not exceed the size of the basic computation block. The size of a data block can be equal to or smaller than the size of the basic computation block.

[0076] Optionally, the size of the multiple data blocks included in the attention matrix can be equal to the size of the basic computation block, for example... Figure 6 As shown, Figure 6 In this approach, an attention matrix comprises 15 data blocks, each with a size identical to the basic computation block (3x5). Alternatively, the attention matrix may contain data blocks of equal or smaller size than the basic computation block. For instance, if the attention matrix is ​​not divisible by the basic computation block size, it may include data blocks smaller than the basic computation block size. Figure 7 For example, Figure 7 The attention matrix size is 14*14. With the basic computation block size being 3*5, the resulting data blocks are 8 data blocks of size 3*5, 4 data blocks of size 3*4, 2 data blocks of size 2*5, and 1 data block of size 2*4.

[0077] S503, for any data block, if it is determined from the auxiliary mask matrix that any data block has a mask calculation requirement, obtain the attention mask matrix corresponding to any data block from the auxiliary mask matrix, and obtain the result of processing the input data by the neural network based on the attention mask matrix corresponding to any data block.

[0078] In one possible scenario, the attention matrix may include correlations that do not require attention. See also Figure 8 , Figure 8 This shows an attention mask matrix corresponding to an attention matrix. Figure 8 The black boxes indicate the relevance levels that require attention, while the white boxes indicate the relevance levels that do not require attention. Figure 8 The diagram also shows the data blocks included in the two attention matrices, namely data block 1 and data block 2. In data block 1, the multiple correlations correspond to white boxes, meaning these correlations do not require attention. In contrast, in data block 2, the multiple correlations correspond to black boxes, meaning these correlations require attention.

[0079] In the above embodiments, where the attention matrix includes both attention-bearing and non-attention-bearing correlations, the computing device can use masking calculations to mask the non-attention-bearing correlations in the attention matrix while retaining the attention-bearing correlations. In some cases, the computing device will first determine whether a masking calculation is required for a data block during processing. Since the process of processing multiple data blocks is similar, the following will use any single data block as an example to illustrate the data block processing process.

[0080] For example, the computing device may first determine the submatrix corresponding to any data block in the auxiliary mask matrix; and then determine whether any data block requires mask calculation based on the submatrix. In one possible case, the process of determining the submatrix includes: determining the position of any data block in the attention matrix; mapping any data block to the auxiliary mask matrix based on its position in the attention matrix; and determining the submatrix based on the mapping result.

[0081] In one possible implementation, the position of any data block in the attention matrix can be indicated by an index. The index distinguishes data blocks at different positions. Optionally, the index of a data block includes its coordinates within the attention matrix. These coordinates could be, for example, the coordinates of the top-left corner, top-right corner, bottom-left corner, or bottom-right corner of the data block. The top-left corner coordinates refer to the row and column coordinates of the correlation degree located at the top-left corner of the data block within the attention matrix. Figure 9Taking data block 1 as an example, the coordinates of the top-left corner of data block 1 are (0, 10). The meanings of the coordinates of the top-right, bottom-left, and bottom-right corners are similar to those of the top-left corner. Continuing with... Figure 9 Taking data block 1 as an example, the coordinates of the top right corner are (0, 14), the coordinates of the bottom left corner are (2, 10), and the coordinates of the bottom right corner are (2, 14). In addition, the index of a data block can include a single coordinate, such as only the coordinates of the top left corner, or it can include multiple coordinates, such as the coordinates of the top left corner and the coordinates of the bottom right corner.

[0082] Regardless of the method used to determine the position of the data block in the attention matrix, the computing device can calculate the position of the data block mapped to the auxiliary mask matrix based on the index. Taking the index as the coordinate (i, j) of the top-left corner of the attention matrix, the computing device calculates the offset (delta) of the data block based on the top-left corner coordinates, and then determines the position of the data block in the auxiliary mask matrix based on delta. In one possible case, delta = southerIdx * southerSize - sinnerIdx * sinnerSize, where southerIdx indicates the block index of the data block in the row dimension, sinnerIdx indicates the block index of the data block in the column dimension, southerSize indicates the size of the data block in the row dimension (i.e., the number of correlations included in a row of the data block), and sinnerSize indicates the size of the data block in the column dimension (i.e., the number of correlations included in a column of the data block). Where outerIdx*outerSize equals i, sinnerIdx*sinnerSize equals j, and the above delta = outerIdx*outerSize - sinnerIdx*sinnerSize can also be expressed as delta = ij.

[0083] In one possible scenario, after calculating delta, the computing device can compare delta with a reference value. The reference value can be set based on experience and the implementation environment. Taking a reference value of 0 as an example, if delta < 0, the computing device determines the coordinates of the top-left corner of the auxiliary mask matrix mapped to the data block as (0, min(-delta, sinnerSize)); if delta ≥ 0, it determines the coordinates as (min(delta, sinnerSize), 0). After determining the coordinates of the top-left corner of the auxiliary mask matrix mapped to the data block, the computing device can select elements included in the auxiliary mask matrix to the right and downwards according to the shape of the data block to obtain the submatrix corresponding to the data block. Continuing with... Figure 9 For example, Figure 9The coordinates of the top-left corner of data block 1 mapped to the auxiliary mask matrix are (0, 5). Since the shape of the data block is 3*5, starting from (0, 5), four columns are selected to the right and two rows are selected downwards to obtain... Figure 9 The left figure shows submatrix 1. The process of calculating the position of the submatrix by including other coordinates in the index is similar to the process of calculating the position of the submatrix by including the coordinates of the top left corner in the index. Please refer to the relevant descriptions, which will not be repeated here.

[0084] Furthermore, the above embodiments are intended to illustrate the process of determining the submatrix corresponding to a data block, and are not intended to limit the determined submatrix. Submatrices corresponding to different data blocks may have overlapping elements, for example... Figure 9 The data block 1 corresponds to submatrix 1, and the data block 3 corresponds to submatrix 3. Furthermore, a data block can correspond to one submatrix or multiple submatrixes; this embodiment does not limit this.

[0085] After determining the submatrix corresponding to the data block in the auxiliary mask matrix, the computing device can determine whether the data block has a mask calculation requirement based on the submatrix. For example, the process of determining the mask calculation requirement includes: if the values ​​of multiple elements in the submatrix are the same, determining that any data block does not have a mask calculation requirement; or, if the values ​​of multiple elements in the submatrix are different, determining that any data block has a mask calculation requirement.

[0086] The value of an element indicates whether to focus on the correlation degree corresponding to that element. The correspondence between an element and its correlation degree means that the position of the correlation degree in the data block matches the position of the element in the submatrix. Continuing with... Figure 9 For example, Figure 9 Both the submatrix and the data block are 3*5 matrix shapes. The elements represented by the first row and first column of submatrix 1 correspond to the degree of association represented by the first row and first column of data block 1.

[0087] In one possible scenario, an element can take either a first value or a second value. Taking any element as either the first or second element as an example, the first element taking the first value indicates the degree of relevance for elements not being followed, while the second element taking the second value indicates the degree of relevance for elements being followed.

[0088] For example, multiple elements having the same value could mean that all elements have the first value, and all elements in the submatrix have the first value, indicating that the various correlations included in the data block do not need to be considered in subsequent calculations. Optionally, multiple elements having the same value could also mean that all elements have the second value, and all elements in the submatrix have the second value, indicating that the various correlations included in the data block are considered in subsequent calculations. Since the consideration result for the multiple correlations in the data block is consistent regardless of whether all correlations in the data block are considered or not, there is no need to filter out the correlations to be considered and the correlations that do not need to be considered through mask calculation. Therefore, there is no need for mask calculation for the data block.

[0089] Optionally, if multiple elements have different values, for example, Figure 9 Submatrix 3, corresponding to data block 3, contains elements that are either above the diagonal (taking the second value) or below the diagonal (taking the first value). Therefore, some correlations in data block 3 require attention, while others do not. The attention results for these correlations are inconsistent. Masking is used to filter out the correlations of interest from the data block. Determining whether a data block participates in masking calculation based on its position within the auxiliary mask matrix significantly reduces computational cost and improves the training and inference performance of the neural network.

[0090] After determining whether a data block requires mask calculation based on the auxiliary mask matrix, the computing device can obtain the attention mask matrix of the data block if such a requirement exists, and then perform mask calculation using the attention mask matrix. The attention mask matrix is ​​a matrix containing a third and a fourth value, which indicate whether the correlation at the corresponding position is being considered. Figure 8 This application provides an attention mask matrix corresponding to an attention matrix. The shape of the attention mask matrix is ​​the same as that of the attention matrix, with each square representing a number. Black squares indicate the third value 1, and white squares indicate the fourth value 0. The attention mask matrix is ​​used to determine the position of the relevance to be focused on; this relevance may also be referred to as the effective relevance in some cases. This application does not limit the method of obtaining the attention mask matrix; it can be obtained through methods including, but not limited to, the following two methods.

[0091] Method 1: Determine the submatrix corresponding to any data block as the attention mask matrix corresponding to any data block.

[0092] In one possible scenario, the computing device determines the matrix type of the attention mask matrix to be acquired based on the application scenario of the input data. The application scenario of the input data corresponds to at least one of AR, VR, AI, or application software used in the neural network application described in the above embodiments. Since the algorithms used in the attention part of the neural network differ in different application scenarios, the matrix type of the attention mask matrix required by the attention part of the neural network also differs. The computing device can determine the matrix type of the attention mask matrix based on the neural network; for example, it can determine that the matrix type of the attention mask matrix used by the neural network is a band matrix, a causal matrix, or a prefix matrix, etc. Here, the attention mask matrix used by the neural network refers to the attention mask matrix corresponding to the entire attention matrix, and the size of this attention mask matrix is ​​the same as the size of the attention matrix.

[0093] After determining the matrix type of the attention mask matrix, the computing device can compare whether the matrix types of the attention mask matrix and the auxiliary mask matrix are consistent. If the matrix types of the attention mask matrix and the auxiliary mask matrix are consistent, the submatrix corresponding to the data block in the auxiliary mask matrix is ​​determined as the attention matrix corresponding to that data block. Continuing with... Figure 9 Taking the auxiliary mask matrix, which is a lower triangular matrix, as an example, since the attention mask matrix is ​​also a lower triangular matrix, the computing device can... Figure 9 Submatrix 3 in the matrix serves as the attention mask matrix corresponding to data block 3. The attention mask matrix corresponding to the data block is a matrix containing the third and fourth values, and its shape corresponds to the shape of the data block.

[0094] Method 2: Process the sub-matrix corresponding to any data block in the auxiliary mask matrix according to the application scenario of the input data, and use the processed sub-matrix as the attention mask matrix corresponding to any data block.

[0095] In one possible scenario, where the matrix type of the auxiliary mask matrix differs from the matrix type of the attention mask matrix determined based on the application scenario, the computing device will further process the sub-matrix corresponding to the data block and obtain the attention mask matrix corresponding to the data block based on the processing result. Assuming the auxiliary mask matrix is ​​a lower triangular matrix and the attention mask matrix is ​​of type [missing information], the computing device will further process the sub-matrix corresponding to the data block and obtain the attention mask matrix corresponding to the data block. Figure 8 Taking the striped matrix as an example, a striped matrix refers to a matrix in which all non-zero elements are concentrated in a strip region centered on the main diagonal. Because striped matrices differ from lower triangular matrices, computing devices process the submatrices to obtain the attention mask matrix corresponding to the data blocks.

[0096] Figure 10This application provides a process for processing a submatrix. Figure 10 The data block size is 10*10. Mapping the same data block to the auxiliary mask matrix yields two sub-matrices, namely... Figure 10 The left image shows the front mask (MaskPre) and back mask (MaskNext). Figure 10 In the top left image, unlabeled cells have a value of 1, and in the bottom left image, unlabeled cells have a value of 0. The computing device inverts MaskPre, and then ORs the inverted MaskPre and MaskNext to obtain... Figure 10 The attention mask matrix is ​​shown in the right figure. The above processing can be represented by the formula RstMask=(~MaskPre)|(MaskNext), where ~ indicates negation and | indicates OR.

[0097] In this embodiment, even if the matrix types of the attention mask matrix required by the neural network and the auxiliary mask matrix are different, the sub-matrices can be processed to obtain the attention mask matrix actually used for mask calculation. The auxiliary mask matrix is ​​applicable to various matrix types of attention mask matrices and has high versatility.

[0098] After determining the attention mask matrix corresponding to a data block, the computing device can perform mask calculations on the data block using the attention mask matrix. The attention mask matrix is ​​then passed into the operator, where it is used to mask the irrelevant correlations within the data block. For example, if a row of data blocks has correlations of 9.125, 7.5, 3, 5.65, and 6.25, the last three correlations are irrelevant and can be masked to -∞ using the attention mask matrix. The resulting row is 9.125, 7.5, -∞, -∞, -∞. The attention mask matrix passed into the operator of the neural network is the attention mask matrix corresponding to the filtered data blocks. Even if the neural network uses a correlation fusion operator, the size and number of transmitted attention mask matrices can be controlled, effectively improving throughput.

[0099] Optionally, after performing mask calculation on a data block and obtaining the mask result, the computing device can perform the processing operations included in the neural network following the mask calculation on the mask result to obtain the result of the neural network processing the input data. In one possible case, since the data blocks that do not require mask calculation include all data blocks that do not require attention and all data blocks that require attention, for all data blocks that require attention, the correlation degree included in the data block is the effective correlation degree, corresponding to the correlation degree that was not transformed by the mask during the above mask calculation process. Data blocks including effective correlation degrees also participate in subsequent processing operations. Therefore, if it is determined from the auxiliary mask matrix that any data block does not require mask calculation, the computing device will also determine whether to continue performing processing operations on any data block based on the auxiliary mask matrix.

[0100] For example, if the computing device determines, based on the auxiliary mask matrix, that it does not care about the multiple correlations included in any data block, it decides to stop processing any data block; or, if it determines, based on the auxiliary mask matrix, that it cares about the multiple correlations included in any data block, it decides to continue processing any data block. The determination of whether to care about correlations can be made through a submatrix corresponding to the data block. For example, if all the elements in the submatrix corresponding to any currently processed data block have a first value, it is determined that the multiple correlations included in any data block are not being cared about. If all the elements in the submatrix corresponding to any currently processed data block have a second value, it is determined that the multiple correlations included in any data block are being cared about.

[0101] For data blocks where processing operations are halted, the computing device can directly exit the operation on that data block and proceed to the computation of the next data block. For data blocks where processing operations can continue, the computing device directly applies the neural network processing operations to the data block and other data blocks after mask computation. For example, the computing device... Figure 3 The diagram shows the application of the Softmax function to normalize the data blocks involved in the processing operation, followed by matrix multiplication (MatMul) of the normalized result and the V vector. The product of these two normalized results is then used as the input vector. Figure 2 The output data of the MultiHead Attention module shown is added to the input data of the Decoder Block. The result of the addition is transformed using LN transformation, and then processed by a multilayer perceptron. The sum of the processed and added results is used as the output data of the Decoder Block. Then, the output data of the Decoder Block is processed... Figure 1 The subsequent operations of the Decoder Block shown are used to process the input data through a neural network and obtain the processing result of the neural network, that is... Figure 1 The output of Softmax is shown.

[0102] This application does not limit the processing results obtained by the neural network processing the input data. Taking text data as an example, the processing results of the neural network processing the text data include, but are not limited to: recognition results obtained by recognizing the text data, translation results obtained by translating the text data, and summaries or posters corresponding to the text data. Taking speech data as an example, the processing results include, but are not limited to, the human voice information or subtitles corresponding to the speech data.

[0103] In addition, the above Figures 1-3 The transformer structure shown is intended to illustrate the attention mechanism included in the transformer structure of a neural network, rather than to limit the layers in the neural network to which the auxiliary mask matrix is ​​applied. The auxiliary mask matrix provided in this application embodiment can be applied to different layers of the neural network, that is, different layers can share the same auxiliary mask matrix, which has high versatility.

[0104] In summary, the data processing method provided in this application first determines whether a data block participates in subsequent mask calculations based on an auxiliary mask matrix. Only when a data block participates in mask calculations is the attention mask matrix used in the mask calculation process obtained, offering high flexibility. This effectively avoids obtaining unnecessary attention mask matrices when data blocks do not require mask calculations, reducing unnecessary operations, computational load, and improving input data processing efficiency. Obtaining the attention mask matrix based on the auxiliary mask matrix is ​​simple and, compared to directly generating the attention mask matrix, effectively reduces the computational load and online generation costs. Pre-generating a fixed-shape auxiliary mask matrix to replace attention mask matrices of different shapes generated in related technologies reduces memory consumption. For example, with a sequence length of 64 kilobytes (K), a 2048*2048 auxiliary mask matrix can reduce memory consumption by 4 gigabytes (G).

[0105] Figure 11 The flowchart below illustrates a data processing method provided in this application embodiment. The following example uses classification as an example of the processing operation performed by a neural network on the input data.

[0106] S1101 generates an auxiliary attention mask matrix of fixed shape offline outside the large operator.

[0107] For example, the auxiliary attention mask matrix corresponds to the auxiliary mask matrix in the above embodiments, and the process of generating the auxiliary mask matrix can be found in the above description. Figure 5The process of generating the auxiliary mask matrix in embodiment S501 shown will not be repeated here. In one possible case, the auxiliary mask matrix generated by the computing device has a shape of 2048×2048.

[0108] S1102, the algorithm obtains a sub-matrix from the auxiliary attention mask matrix based on the position of the basic block; and determines whether the basic block should participate in the calculation based on the sub-matrix.

[0109] For example, the basic block corresponds to the data block included in the attention matrix in the above embodiments, and the position of the basic block corresponds to the position of the data block in the attention matrix in the above embodiments. The description of obtaining the sub-matrix from the auxiliary attention mask matrix according to the position of the basic block can be found above. Figure 5 In the illustrated embodiment S503, the description of determining the submatrix corresponding to the data block in the auxiliary mask matrix, and the description of determining whether the basic block participates in the calculation based on the submatrix, can be found above. Figure 5 The description of determining the mask calculation requirements based on the submatrix in the illustrated embodiment S503 will not be repeated here.

[0110] S1103, For the basic blocks involved in the calculation, perform mask calculation on the basic blocks according to the submatrix corresponding to the basic blocks.

[0111] For a description of masking basic blocks, please refer to the above. Figure 5 The description of mask calculation for data blocks in embodiment S503 shown will not be repeated here.

[0112] S1104, Obtain the output of the neural network based on the mask calculation result.

[0113] The process of further processing based on the mask calculation results can refer to the above. Figure 5 The process of performing the processing operation in the illustrated embodiment S503 will not be described again here. When a neural network is used for classification, the output can be the classification result of the input data.

[0114] Figure 12 A flowchart illustrating the processing of input data using related technologies is shown. Figure 12(1) Corresponding to the processing flow of related technology one, for the input data to be processed, the attention mask matrix corresponding to the entire attention matrix is ​​generated outside the large operator according to the sequence length of the input data. Subsequently, when batch processing is performed according to the basic blocks in the attention matrix, the attention mask matrix corresponding to the basic block is obtained one by one according to the actual position of the basic block in the attention matrix. The basic block is masked according to the obtained attention mask matrix corresponding to the basic block, and the output of the neural network is obtained based on the mask calculation result. Figure 12 (2) Corresponding to the processing flow of related technology two, the attention mask matrix corresponding to the basic block is generated online inside the attention operator, and the basic block is masked to obtain the output of the neural network based on the mask calculation result. Figure 12 In related technologies one and two, for each basic block included in the attention matrix, the attention mask matrix corresponding to the basic block is obtained by default. (See also...) Figure 11 This application selectively obtains the attention mask matrix corresponding to the basic block, which is more flexible and effectively reduces the computational cost of obtaining the attention mask matrix.

[0115] The data processing method of the present application embodiments has been described above. Corresponding to the above method, the present application embodiments also provide a data processing device. Figure 13 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Based on Figure 13 The following modules are shown. Figure 13 The data processing device shown is capable of performing the above. Figure 5 All or part of the operations shown. It should be understood that the device may include more additional modules than those shown, or may omit some of the modules shown; this application embodiment does not impose limitations in this regard. For example... Figure 13 As shown, the device includes:

[0116] The acquisition module 1301 is used to acquire the auxiliary mask matrix of the neural network. The auxiliary mask matrix is ​​used for auxiliary mask calculation.

[0117] The acquisition module 1301 is also used to acquire the attention matrix of the input data, the attention matrix including multiple data blocks;

[0118] The processing module 1302 is used to, for any data block, if it is determined from the auxiliary mask matrix that any data block has a mask calculation requirement, obtain the attention mask matrix corresponding to any data block according to the auxiliary mask matrix, and obtain the result of processing the input data through the neural network based on the attention mask matrix corresponding to any data block.

[0119] In one possible implementation, the acquisition module 1301 is used to acquire the size of the basic computation block, which is used to divide the data blocks included in the attention matrix; determine the matrix shape according to the size of the basic computation block, the matrix shape indicating the number of rows and columns of the auxiliary mask matrix to be generated; and generate the auxiliary mask matrix according to the matrix shape.

[0120] In one possible implementation, any data block includes multiple correlation degrees, and any data block has a corresponding submatrix in the auxiliary mask matrix. The submatrix includes multiple elements, and the value of the element indicates whether to pay attention to the correlation degree corresponding to the element. The processing module 1302 is further used to determine that any data block does not have a mask calculation requirement when the values ​​of the multiple elements included in the submatrix are the same; or, to determine that any data block has a mask calculation requirement when the values ​​of the multiple elements included in the submatrix are different.

[0121] In one possible implementation, having multiple elements with the same value includes having multiple elements with the first value or having multiple elements with the second value. The first element is set to the first value to indicate the degree of relevance for which the first element is not being followed, and the second element is set to the second value to indicate the degree of relevance for which the second element is being followed. The first element and the second element can be any one of the multiple elements.

[0122] In one possible implementation, the processing module 1302 is further configured to determine the position of any data block in the attention matrix; map any data block to an auxiliary mask matrix according to the position of any data block in the attention matrix; and determine a submatrix based on the mapping result.

[0123] In one possible implementation, the processing module 1302 is further configured to stop performing processing operations after mask calculation on any data block when it is determined from the auxiliary mask matrix that no mask calculation is required for any data block and the multiple correlations included in any data block are not of concern; or, when it is determined from the auxiliary mask matrix that no mask calculation is required for any data block and the multiple correlations included in any data block are of concern, continue performing processing operations on any data block.

[0124] In one possible implementation, the processing module 1302 is used to process the sub-matrix corresponding to any data block in the auxiliary mask matrix according to the application scenario of the input data, and use the processed sub-matrix as the attention mask matrix corresponding to any data block; or, the sub-matrix corresponding to any data block is determined as the attention mask matrix corresponding to any data block.

[0125] In one possible implementation, the neural network is applied to at least one of AR, VR, AI, or application software.

[0126] The aforementioned device first determines whether a data block participates in subsequent mask calculations based on an auxiliary mask matrix. Only if the data block participates in mask calculations will the attention mask matrix used during the calculation process be obtained, offering high flexibility. This effectively avoids obtaining unnecessary attention mask matrices when the data block does not require mask calculations, reducing unnecessary operations, lowering computational load, and improving input data processing efficiency. Obtaining the attention mask matrix based on the auxiliary mask matrix is ​​a simple process with low computational cost.

[0127] It should be understood that the above Figure 13 The provided device, in implementing its functions, is only illustrated by the division of the above-described functional modules. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0128] See Figure 14 , Figure 14 A schematic diagram of the structure of a network device 1400 provided in an exemplary embodiment of this application is shown. Figure 14 The network device 1400 shown is used to perform the above. Figure 5 The data processing method shown involves the operations described. The network device 1400 is, for example, a switch, a router, etc., and can be implemented using a general bus architecture.

[0129] like Figure 14 As shown, the network device 1400 includes at least one processor 1401, a memory 1403, and at least one communication interface 1404.

[0130] Processor 1401 may be, for example, a general-purpose central processing unit (CPU), a digital signal processor (DSP), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the embodiments of this application. For example, processor 1401 includes an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A PLD may be, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof. It can implement or execute the various logic blocks, modules, and circuits described in connection with the embodiments of this application. A processor may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0131] Optionally, network device 1400 also includes a bus. The bus is used to transfer information between the components of network device 1400. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 14 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0132] Memory 1403 may be, for example, read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; electrically erasable programmable read-only memory (EEPROM); compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.); magnetic disk storage media or other magnetic storage devices; or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 1403 may exist independently and be connected to processor 1401 via a bus. Memory 1403 may also be integrated with processor 1401.

[0133] Communication interface 1404 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), or Wireless Local Area Network (WLAN). Communication interface 1404 may include wired and wireless communication interfaces. Specifically, communication interface 1404 may be an Ethernet interface, a Fast Ethernet (FE) interface, a Gigabit Ethernet (GE) interface, an Asynchronous Transfer Mode (ATM) interface, a WLAN interface, a cellular network communication interface, or a combination thereof. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. In this embodiment, communication interface 1404 can be used by network device 1400 to communicate with other devices.

[0134] In a specific implementation, as one example, the processor 1401 may include one or more CPUs, such as Figure 14 The CPU0 and CPU1 shown are examples of processors. Each of these processors can be a single-core processor or a multi-core processor. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0135] In a specific implementation, as one example, network device 1400 may include multiple processors, such as... Figure 14 The processors 1401 and 1405 shown are illustrated. Each of these processors can be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0136] In a specific implementation, as one embodiment, network device 1400 may further include output devices and input devices. The output device communicates with processor 1401 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with processor 1401 and can receive user input in various ways. For example, the input device may be a mouse, keyboard, touchscreen device, or sensor device, etc.

[0137] In some embodiments, memory 1403 is used to store program code 1410 for executing the solution of this application, and processor 1401 can execute the program code 1410 stored in memory 1403. That is, network device 1400 can implement the data processing method provided in the method embodiment through processor 1401 and program code 1410 in memory 1403. Program code 1410 may include one or more software modules. Optionally, processor 1401 itself may also store program code or instructions for executing the solution of this application.

[0138] In a specific embodiment, the network device 1400 of this application embodiment may correspond to the computing device in the above-described method embodiments.

[0139] in, Figure 5 Each step of the data processing method shown is completed through integrated logic circuits in the hardware or software instructions in the processor of the network device 1400. The steps of the method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. Since this storage medium is located in memory, the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method; to avoid repetition, these will not be described in detail here.

[0140] See Figure 15 , Figure 15 This illustration shows a schematic diagram of the structure of a network device 1500 provided in another exemplary embodiment of this application. Figure 15 The network device 1500 shown is used to perform the above. Figure 5 The data processing method shown involves all or part of the operations. The network device 1500 is, for example, a switch, a router, etc., and can be implemented using a general bus architecture.

[0141] like Figure 15 As shown, network device 1500 includes: main control board 1510 and interface board 1530.

[0142] The main control board, also known as the main processing unit (MPU) or route processor card, is used to control and manage the various components in the network device 1500, including routing calculation, device management, device maintenance, and protocol processing functions. The main control board 1510 includes a central processing unit 1511 and a memory 1512.

[0143] Interface board 1530 is also known as a line processing unit (LPU), linecard, or service board. Interface board 1530 provides various service interfaces and implements packet forwarding. Service interfaces include, but are not limited to, Ethernet interfaces, POS (Packet over SONET / SDH) interfaces, etc., with Ethernet interfaces including, for example, Flexible Ethernet Clients (FlexE Clients). Interface board 1530 includes: a central processing unit 1531, a network processor 1532, a forwarding table entry memory 1534, and a physical interface card (PIC) 1533.

[0144] The central processing unit 1531 on the interface board 1530 is used to control and manage the interface board 1530 and communicate with the central processing unit 1511 on the main control board 1510.

[0145] Network processor 1532 is used to implement packet forwarding processing. Network processor 1532 can be in the form of a forwarding chip. The forwarding chip can be a network processor (NP). In some embodiments, the forwarding chip can be implemented using an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). Specifically, network processor 1532 forwards received packets based on the forwarding table stored in forwarding table entry memory 1534. If the destination address of the packet is the address of network device 1500, the packet is sent to the CPU (such as central processing unit 1531) for processing; if the destination address of the packet is not the address of network device 1500, the next hop and outgoing interface corresponding to the destination address are looked up in the forwarding table according to the destination address, and the packet is forwarded to the outgoing interface corresponding to the destination address. Uplink packet processing may include: packet ingress interface processing, forwarding table lookup; downlink packet processing may include: forwarding table lookup, etc. In some embodiments, the central processing unit can also perform the functions of the forwarding chip, such as implementing software forwarding based on a general-purpose CPU, thus eliminating the need for a forwarding chip on the interface board.

[0146] The physical interface card 1533 is used to implement physical layer interfacing functions. Raw traffic enters the interface board 1530 through this card, and processed packets are sent out from the physical interface card 1533. The physical interface card 1533, also known as a daughter card, can be installed on the interface board 1530. It is responsible for converting photoelectric signals into packets, performing validity checks on the packets, and forwarding them to the network processor 1532 for processing. In some embodiments, the central processing unit 1531 can also perform the functions of the network processor 1532, such as implementing software forwarding based on a general-purpose CPU, thus eliminating the need for the network processor 1532 in the physical interface card 1533.

[0147] Optionally, network device 1500 includes multiple interface boards. For example, network device 1500 also includes interface board 1540, which includes: a central processing unit 1541, a network processor 1542, a forwarding table entry memory 1544, and a physical interface card 1543. The functions and implementation methods of each component in interface board 1540 are the same as or similar to those in interface board 1530, and will not be described again here.

[0148] Optionally, network device 1500 also includes a switching fabric board 1520. The switching fabric board 1520 can also be referred to as a switch fabric unit (SFU). When network device 1500 has multiple interface boards, the switching fabric board 1520 is used to complete data exchange between the interface boards. For example, interface boards 1530 and 1540 can communicate through the switching fabric board 1520.

[0149] The main control board 1510 and the interface boards are coupled. For example, the main control board 1510, interface boards 1530 and 1540, and the switching network board 1520 communicate with each other via a system bus connected to the system backplane. In one possible implementation, an inter-process communication (IPC) channel is established between the main control board 1510 and interface boards 1530 and 1540, and communication between the main control board 1510 and interface boards 1530 and 1540 is achieved through the IPC channel.

[0150] Logically, network device 1500 includes a control plane and a forwarding plane. The control plane includes a main control board 1510 and a central processing unit 1511, while the forwarding plane includes various components that perform forwarding, such as a forwarding table entry memory 1534, a physical interface card 1533, and a network processor 1532. The control plane performs functions such as router operation, generating forwarding tables, processing signaling and protocol messages, and configuring and maintaining the status of network devices. The control plane distributes the generated forwarding tables to the forwarding plane. In the forwarding plane, the network processor 1532 looks up and forwards messages received by the physical interface card 1533 based on the forwarding tables distributed by the control plane. The forwarding tables distributed by the control plane can be stored in the forwarding table entry memory 1534. In some embodiments, the control plane and the forwarding plane can be completely separated and not on the same network device.

[0151] It's worth noting that a network device may have one or more main control boards, including a primary and a backup main control board. It may also have one or more interface boards; the more powerful the network device's data processing capabilities, the more interface boards it provides. Each interface board may also have one or more physical interface cards. A switching board may or may not exist; multiple boards can share the load and provide redundancy. In a centralized forwarding architecture, the network device may not need a switching board, as the interface boards handle the entire system's business data processing. In a distributed forwarding architecture, the network device can have at least one switching board, which enables data exchange between multiple interface boards, providing high-capacity data exchange and processing capabilities. Therefore, the data access and processing capabilities of a distributed architecture network device are greater than those of a centralized architecture network device. Alternatively, the network device can also be a single board, without a switching board. The functions of the interface board and the main control board are integrated on this one board. In this case, the central processing unit (CPU) on the interface board and the CPU on the main control board can be combined into a single CPU to perform the combined functions. This type of network device has lower data exchange and processing capabilities (e.g., low-end switches or routers). The specific architecture adopted depends on the specific network deployment scenario, and no restrictions are imposed here.

[0152] In a specific embodiment, network device 1500 corresponds to the above. Figure 13 The data processing apparatus shown. In some embodiments, Figure 13 The processing module 1302 in the data processing device shown is equivalent to the central processing unit 1511 or network processor 1532 in the network device 1500.

[0153] This application embodiment also provides a communication device, which includes a transceiver, a memory, and a processor. The transceiver, the memory, and the processor communicate with each other via an internal connection path. The memory stores instructions, and the processor executes the instructions stored in the memory to control the transceiver to receive and transmit signals. When the processor executes the instructions stored in the memory, it causes the processor to perform... Figure 5 The data processing method shown.

[0154] It should be understood that the aforementioned processor can be a CPU, or other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0155] Furthermore, in an alternative embodiment, the memory described above may include read-only memory and random access memory, and provide instructions and data to the processor. The memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0156] The memory can be volatile or non-volatile, or may include both. Non-volatile memory can be ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), EEPROM, or flash memory. Volatile memory can be RAM, used as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0157] This application embodiment also provides a data processing device, the device including a processor, the processor being configured to load and execute at least one instruction to enable the data processing device to perform, as described above. Figure 5 The data processing method is illustrated. Optionally, the device further includes a memory coupled to the processor, the memory being used to store at least one instruction.

[0158] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to enable the computer to perform the following: Figure 5 The data processing method shown.

[0159] This application also provides a computer program (product) that, when executed by a computer, causes the processor or computer to perform the corresponding steps and / or processes in the above method embodiments.

[0160] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device equipped with the chip to perform actions such as... Figure 5 The data processing method shown.

[0161] This application embodiment also provides another chip, including: an input interface, an output interface, a processor, and a memory. The input interface, the output interface, the processor, and the memory are connected via internal interconnection paths. The processor is used to execute code in the memory. When the code is executed, the processor is used to perform actions such as... Figure 5 The data processing method shown.

[0162] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive).

[0163] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the attention matrix involved in this application was obtained with full authorization.

[0164] Those skilled in the art will recognize that the method steps and modules described in conjunction with the embodiments disclosed herein can be implemented in software, hardware, firmware, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0166] When implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer program instructions. As an example, the methods of this application embodiment can be described in the context of machine-executable instructions, such as program modules that execute on a device on a real or virtual processor of the target. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data structures. In various embodiments, the functionality of program modules can be combined or divided among the described program modules. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside on both local and remote storage media.

[0167] Computer program code used to implement the methods of the embodiments of this application may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the computer or other programmable data processing apparatus, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0168] In the context of the embodiments of this application, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0169] Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0170] A machine-readable medium can be any tangible medium that contains or stores programs for or relating to an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More detailed examples of machine-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0171] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0172] In the embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or modules, or they may be electrical, mechanical, or other forms of connection.

[0173] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0174] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0175] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the various examples described, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. Both the first image and the second image can be images, and in some cases, they can be separate and distinct images.

[0177] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0178] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple second messages refer to two or more second messages. The terms "system" and "network" are often used interchangeably in this document.

[0179] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0180] It should also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship.

[0181] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0182] It should also be understood that the terms “if” and “if” can be interpreted as meaning “when” or “upon”, or “in response to determination” or “in response to detection”. Similarly, depending on the context, the phrases “if determination…” or “if detection [the stated condition or event]” can be interpreted as meaning “when determination…”, or “in response to determination…”, or “when detection [the stated condition or event]” or “in response to detection [the stated condition or event]”.

[0183] It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0184] It should also be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the auxiliary mask matrix of the neural network, which is used to assist in mask calculation; Obtain the attention matrix of the input data, the attention matrix comprising multiple data blocks; For any data block, if it is determined that there is a mask calculation requirement for any data block based on the auxiliary mask matrix, the attention mask matrix corresponding to the data block is obtained based on the auxiliary mask matrix, and the result of processing the input data by the neural network is obtained based on the attention mask matrix corresponding to the data block.

2. The method according to claim 1, characterized in that, The process of obtaining the auxiliary mask matrix of the neural network includes: Obtain the size of the basic computation block, which is used to divide the data blocks included in the attention matrix; The matrix shape is determined based on the size of the basic computation block, and the matrix shape indicates the number of rows and columns of the auxiliary mask matrix to be generated; The auxiliary mask matrix is ​​generated according to the shape of the matrix.

3. The method according to claim 1 or 2, characterized in that, Each data block includes multiple correlation degrees, and each data block has a corresponding submatrix in the auxiliary mask matrix. The submatrix includes multiple elements, and the value of each element indicates whether to pay attention to the correlation degree corresponding to the element. After obtaining the attention matrix of the input data, the method further includes: If multiple elements in the submatrix have the same value, it is determined that no mask calculation is required for any of the data blocks. Alternatively, if the values ​​of the multiple elements included in the submatrix are different, it can be determined that any data block has a mask calculation requirement.

4. The method according to claim 3, characterized in that, The fact that the multiple elements have the same value includes either all of the multiple elements having a first value or all of the multiple elements having a second value. The first element having the first value is used to indicate that the correlation corresponding to the first element is not being paid attention to, and the second element having the second value is used to indicate that the correlation corresponding to the second element is being paid attention to. The first element and the second element are any one of the multiple elements.

5. The method according to claim 3 or 4, characterized in that, The method further includes: Determine the position of any data block in the attention matrix; Map any data block to the auxiliary mask matrix based on its position in the attention matrix, and determine the submatrix based on the mapping result.

6. The method according to any one of claims 1-5, characterized in that, After obtaining the attention matrix of the input data, the method further includes: If, based on the auxiliary mask matrix, it is determined that no mask calculation is required for any data block, and the multiple correlations included in any data block are not of concern, then the processing operations following the mask calculation for any data block shall be stopped. Alternatively, if it is determined from the auxiliary mask matrix that no mask calculation is required for any data block, and the multiple correlations included in any data block are of interest, the processing operation may continue to be performed on any data block.

7. The method according to any one of claims 1-6, characterized in that, The step of obtaining the attention mask matrix corresponding to any data block based on the auxiliary mask matrix includes: Based on the application scenario of the input data, process the sub-matrix corresponding to any data block in the auxiliary mask matrix, and use the processed sub-matrix as the attention mask matrix corresponding to any data block. Alternatively, the submatrix corresponding to any data block can be determined as the attention mask matrix corresponding to any data block.

8. The method according to any one of claims 1-7, characterized in that, The neural network is applied to at least one of augmented reality (AR), virtual reality (VR), artificial intelligence (AI), or application software.

9. A data processing apparatus, characterized in that, The device includes: An acquisition module is used to acquire an auxiliary mask matrix of the neural network, wherein the auxiliary mask matrix is ​​used to assist in mask calculation; The acquisition module is further configured to acquire an attention matrix of the input data, the attention matrix comprising multiple data blocks; The processing module is configured to, for any data block, if it is determined from the auxiliary mask matrix that any data block has a mask calculation requirement, obtain the attention mask matrix corresponding to the data block according to the auxiliary mask matrix, and obtain the result of processing the input data by the neural network based on the attention mask matrix corresponding to the data block.

10. The apparatus according to claim 9, characterized in that, The acquisition module is used to acquire the size of the basic computation block, which is used to divide the data blocks included in the attention matrix; and to determine the matrix shape based on the size of the basic computation block, wherein the matrix shape indicates the number of rows and columns of the auxiliary mask matrix to be generated. The auxiliary mask matrix is ​​generated according to the shape of the matrix.

11. The apparatus according to claim 9 or 10, characterized in that, Each data block includes multiple correlation degrees, and each data block has a corresponding submatrix in the auxiliary mask matrix. The submatrix includes multiple elements, and the value of each element indicates whether to pay attention to the correlation degree corresponding to the element. The processing module is further configured to determine that the data block does not have a mask calculation requirement when the values ​​of the multiple elements included in the submatrix are the same; or, determine that the data block has a mask calculation requirement when the values ​​of the multiple elements included in the submatrix are different.

12. The apparatus according to claim 11, characterized in that, The fact that the multiple elements have the same value includes either all of the multiple elements having a first value or all of the multiple elements having a second value. The first element having the first value is used to indicate that the correlation corresponding to the first element is not being paid attention to, and the second element having the second value is used to indicate that the correlation corresponding to the second element is being paid attention to. The first element and the second element are any one of the multiple elements.

13. The apparatus according to claim 11 or 12, characterized in that, The processing module is further configured to determine the position of any data block in the attention matrix; map any data block to the auxiliary mask matrix according to its position in the attention matrix; and determine the sub-matrix based on the mapping result.

14. The apparatus according to any one of claims 9-13, characterized in that, The processing module is further configured to: stop performing subsequent processing operations on any data block after the mask calculation if, based on the auxiliary mask matrix, it is determined that no mask calculation is required for any data block and the multiple correlations included in any data block are not of concern; or continue performing the processing operations on any data block if, based on the auxiliary mask matrix, it is determined that no mask calculation is required for any data block and the multiple correlations included in any data block are of concern.

15. The apparatus according to any one of claims 9-14, characterized in that, The processing module is used to process the sub-matrix corresponding to any data block in the auxiliary mask matrix according to the application scenario of the input data, and use the processed sub-matrix as the attention mask matrix corresponding to any data block. Alternatively, the submatrix corresponding to any data block can be determined as the attention mask matrix corresponding to any data block.

16. The apparatus according to any one of claims 9-15, characterized in that, The neural network is applied to at least one of augmented reality (AR), virtual reality (VR), artificial intelligence (AI), or application software.

17. A data processing device, characterized in that, The device includes a processor for loading and executing at least one instruction to enable the data processing device to implement the data processing method as described in any one of claims 1-8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the data processing method as described in any one of claims 1-8.

19. A chip, characterized in that, The chip includes a processor for running program instructions or code that cause a device containing the chip to perform the data processing method as described in any one of claims 1-8.

20. A computer program product, characterized in that, The computer program product includes a computer program / instruction that is executed by a processor to cause a computer to perform the data processing method as described in any one of claims 1-8.

Images