Data processing apparatus and method, storage medium, and chip

Abstract

The invention provides a data processing apparatus and method, a storage medium, and a chip. The data processing device comprises a multiplication accumulator. The multiplication accumulator is configured to obtain an input tensor and a sparse weight tensor, wherein the sparse weight tensor is obtained by performing sparse processing on an original weight tensor in at least one of a first dimension and a second dimension; and carrying out multiplication and addition operation on the sparse weight tensor and the input tensor. The computing power of the data processing device can be improved under the condition that the number of the multiplication accumulators is not increased.

Description

Technical Field

[0001] The present disclosure belongs to the field of data processing technology, and in particular, relates to a data processing device and method, a storage medium and a chip. Background Art

[0002] In some scientific computing and engineering applications, multiplication and addition is a basic operation. The multiplication and addition operation combines the result of the multiplication operation with another operand and adds them together to obtain the final result. In the prior art, the multiplication and addition operation is mainly implemented by the multiplication accumulator (MAC) in the data processing device. Convolution operation is a multiplication and addition operation that is currently widely used. Traditional convolution operations include input features, weights, biases, and output features. The input features and a set of weights are multiplied and added, and then added to the bias to obtain the output features. Summary of the invention

[0003] The present disclosure provides a data processing device and method, a storage medium and a chip, which are used to improve the ability of a data processing device to process multiplication and addition operations.

[0004] In a first aspect, an embodiment of the present disclosure provides a data processing device. The data processing device includes: a multiplication accumulator, the multiplication accumulator being configured to: obtain an input tensor and a sparse weight tensor, the sparse weight tensor being obtained by sparsely processing the original weight tensor in at least one of a first dimension and a second dimension; and perform multiplication and addition operations on the sparse weight tensor and the input tensor.

[0005] In an implementation manner of the first aspect, the input tensor includes data in the original input tensor corresponding to the reserved position of the sparse processing.

[0006] In an implementation of the first aspect, the data processing device also includes: an input circuit, which is configured to: obtain a position index, wherein the position index contains the reserved position of the sparse processing; and read the data corresponding to the reserved position in the original input tensor as the input tensor according to the position index.

[0007] In an implementation of the first aspect, the data processing device also includes: an input circuit, which is configured to read the original input tensor as the input tensor; wherein the multiplication accumulator is configured to: obtain a position index, which contains the reserved position of the sparse processing; and perform multiplication and addition operations on the data corresponding to the reserved position in the input tensor and the sparse weight tensor according to the position index.

[0008] In an implementation of the first aspect, the data processing device further includes: a controller configured to perform sparse processing on the original weight tensor to obtain the sparse weight tensor.

[0009] In an implementation of the first aspect, the input circuit is configured to read the sparse weight tensor, wherein the sparse weight tensor is obtained by sparse processing of the original weight tensor.

[0010] In an implementation of the first aspect, the original weight tensor is subjected to a first sparse processing in the first dimension to obtain the sparse weight tensor, and the first sparse processing adopts a first sparse granularity.

[0011] In an implementation of the first aspect, the first sparse granularity is m:n, and every n data of the original weight tensor in the first dimension are sparsely spaced into m data, where n and m are both positive integers and n>m.

[0012] In an implementation of the first aspect, the first sparse processing uses a position index of m×ceil(log2n) bits to represent the sparse position, and each sparse data uses a position index of ceil(log2n) bits to represent its position, where ceil is a round-up function.

[0013] In an implementation of the first aspect, the first sparse processing adopts To express A sparse method, and from the Select one of the sparse modes to execute, and ceil is the upward rounding function.

[0014] In an implementation of the first aspect, the first sparse processing uses ceil(log2N) bits to represent N sparse modes, and selects one of the N sparse modes for execution, where N is less than integer, ceil is a round-up function, and the N sparse methods are It is selected from a sparse method.

[0015] In an implementation of the first aspect, the original weight tensor is subjected to a second sparse processing in the second dimension to obtain the sparse weight tensor, and the second sparse processing adopts a second sparse granularity.

[0016] In an implementation of the first aspect, the second sparsity granularity is s:r, and every r data of the original weight tensor in the second dimension are sparsely spaced into s data, where r and s are both positive integers and r>s.

[0017] In an implementation of the first aspect, the original weight tensor is subjected to a first sparse processing and a second sparse processing in one of the first dimension and the second dimension to obtain an intermediate tensor, and the intermediate tensor is subjected to the other of the first sparse processing and the second sparse processing in another dimension of the first dimension and the second dimension to obtain the sparse weight tensor, the first sparse processing adopts a first sparse granularity, and the second sparse processing adopts a second sparse granularity.

[0018] In an implementation of the first aspect, the first dimension is the channel direction of the original weight tensor, and the second dimension is the kernel direction of the original weight tensor.

[0019] In an implementation of the first aspect, the original weight tensor is further subjected to a third sparse processing in a third dimension to obtain the sparse weight tensor, and the third sparse processing adopts a third sparse granularity.

[0020] In a second aspect, an embodiment of the present disclosure provides a data processing method. The data processing method comprises: obtaining an input tensor and a sparse weight tensor, wherein the sparse weight tensor is obtained by sparse processing the original weight tensor in at least one of the first dimension and the second dimension; and performing multiplication and addition operations on the sparse weight tensor and the input tensor.

[0021] In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. The computer program is executed to implement the data processing method according to the second aspect of the present disclosure.

[0022] In a fourth aspect, an embodiment of the present disclosure provides a chip, wherein the chip includes the data processing device described in any one of the first aspects of the present disclosure.

[0023] In the data processing device provided by the embodiment of the present disclosure, the multiplication accumulator uses the sparse weight tensor and the input tensor to perform multiplication and addition operations to obtain the multiplication and addition results of the original input tensor and the original weight tensor. Since the sparse weight tensor is obtained by sparse processing of the original weight tensor, the data in the sparse weight tensor is less than the original weight tensor. Therefore, compared with the solution of directly using the original input tensor and the original weight tensor for multiplication and addition operations, the embodiment of the present disclosure has less calculation amount when performing multiplication and addition operations, and can improve the computing power of the data processing device without increasing the multiplication accumulator.

[0024] In addition, compared with the prior art, the data processing device provided by the embodiment of the present disclosure has a smaller design area and power consumption while maintaining consistent computing power.

[0025] Furthermore, in the data processing device provided by the embodiment of the present disclosure, the multiplier accumulator only needs to read the sparse weight tensor and index, thereby reducing the weight reading bandwidth.

[0026] Furthermore, the data processing device provided by the embodiment of the present disclosure can flexibly configure different sparse parameters according to application scenarios and can be suitable for different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Shown is a structural schematic diagram of a data processing device in an embodiment of the present disclosure.

[0028] Figure 2 Shown is a flow chart of a data processing method in an embodiment of the present disclosure.

[0029] Figure 3A Shown is a flowchart for obtaining input tensors in an embodiment of the present disclosure.

[0030] Figure 3B Shown is an example diagram of the original weight tensor in an embodiment of the present disclosure.

[0031] Figure 3C Shown is an example diagram of a position index matrix in an embodiment of the present disclosure.

[0032] Figure 3D Shown is an example diagram of the original weight tensor in an embodiment of the present disclosure.

[0033] Figure 4 Shown is a flowchart of performing multiplication and addition operations in an embodiment of the present disclosure.

[0034] Figure 5 Shown is a schematic diagram of sparse processing in an embodiment of the present disclosure.

[0035] Figure 6 Shown is a schematic diagram of sparse processing in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] The following is an explanation of the embodiments of the present disclosure by specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0037] It should be noted that the illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present disclosure. Therefore, the drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0038] In practical applications, different data of a tensor are multiplied by different weights and then added to obtain the operation result, which is a widely used multiplication and addition operation. Such multiplication and addition operations include but are not limited to matrix multiplication, convolution operation, deconvolution operation, etc. Taking convolution operation as an example, the traditional convolution operation includes input features, weights, biases and output features. The input features and a set of weights are multiplied and added, and then the biases are added to obtain the output features. The specific formula is shown in Formula 1 below.

[0039]

[0040] in, Represents the complete output feature map of the convolution, N i represents a batch of feature maps, represents the output feature map channel, express The corresponding bias, C in Represents the input feature map channel, input(N i ,k) represents the complete input feature map.

[0041] In the field of artificial intelligence, convolution operations are usually implemented using multiplier-accumulators in neural-network processing units (NPUs). However, with the gradual popularization of artificial intelligence technology and the continuous updating and iteration of algorithms, the demand for NPU computing power is increasing, and the area and power consumption required by NPU are also increasing.

[0042] At least for the above problems, the embodiments of the present disclosure provide a data processing scheme for NPU. In the data processing device, the multiplication and accumulator uses the sparse weight tensor and the input tensor to perform multiplication and addition operations to obtain the multiplication and addition results of the original input tensor and the original weight tensor. The input tensor can be various types of data, including but not limited to image tensors, video tensors, voice signal tensors and / or text tensors in natural language processing. The output tensor is a feature map or feature map obtained after the input tensor is processed. Since the sparse weight tensor is obtained by sparse processing of the original weight tensor, the data in the sparse weight tensor is less than the original weight tensor. Therefore, compared with the scheme of directly using the original input tensor and the original weight tensor for multiplication and addition operations, the embodiment of the present disclosure requires less calculations during multiplication and addition operations, and can improve the computing power of the data processing device without increasing the multiplication and accumulator.

[0043] Figure 1 1 is a schematic diagram showing the structure of a data processing device 1 provided according to an embodiment of the present disclosure. The data processing device 1 may be any processing device capable of implementing multiplication and addition operations. For example, in some embodiments, the data processing device 1 may be an NPU, but the present disclosure is not limited thereto. Figure 1 As shown, the data processing device 1 provided in the embodiment of the present disclosure includes a multiplication accumulator 11. In some embodiments, the computing power of the data processing device is mainly determined by the specifications of the multiplication accumulator. For example, if the number of multiplication accumulators in the data processing device is 1024, each multiplication accumulator operates twice in one clock cycle, and its operating frequency is 1GHz, then the computing power of the data processing device is TOPs=1024×2×1GHz=2TOPs, indicating that the data processing device can perform 2T operations per second.

[0044] In the embodiment of the present disclosure, in order to expand the computing power of the multiplication accumulator 11, the multiplication accumulator 11 is configured to perform Figure 2 The data processing method shown in Figure 2 As shown, the data processing method includes the following steps S21 and S22.

[0045] In step S21, an input tensor and a sparse weight tensor are obtained. The sparse weight tensor is obtained by sparsely processing the original weight tensor in at least one of the first dimension and the second dimension, so that the sparse weight tensor contains less data than the original weight tensor. The first dimension and the second dimension are different.

[0046] In step S22, a multiplication-addition operation is performed on the sparse weight tensor and the input tensor to obtain a multiplication-addition operation result of the original input tensor and the original weight tensor.

[0047] In some possible implementations, the input tensor includes data corresponding to the reserved positions of the sparse processing in the original input tensor. Specifically, the data corresponding to the reserved positions in the original input tensor are retained, and data other than these data are deleted to obtain the input tensor.

[0048] In some possible implementations, the data processing device 1 may further include an output circuit 14 and a cache 15. The cache 15 is configured to store input tensors, sparse weight tensors, and results of multiplication and addition operations. The multiplication accumulator 11 is configured to read the input tensor and the sparse weight tensor from the cache 15 for multiplication and addition operations, and store the results of the multiplication and addition operations in the cache 15. The output circuit 14 is configured to output the results of the multiplication and addition operations in the cache 15.

[0049] According to an embodiment of the present disclosure, the data processing device 1 may further include an input circuit 12. The input circuit 12 may, for example, use a direct memory read access (RDMA) technology to read data, but the present disclosure is not limited thereto. In the embodiment of the present disclosure, the input circuit 12 is configured to execute Figure 3A Steps S31 and S32 are shown.

[0050] In step S31, a position index (index) is obtained, wherein the position index includes the reserved position for sparse processing. In some possible implementations, the position index can be represented by a position index matrix. In the position index matrix, 1 can be used to represent the reserved position, and 0 can be used to represent the sparse position.

[0051] In step S32, data corresponding to the reserved position in the original input tensor is read as the input tensor according to the position index.

[0052] Next, the above process will be introduced in detail through a specific example. Figure 3B is an example diagram showing the original input tensor according to this specific example, Figure 3B As shown, the size of the original input tensor in this example is 1×8×3×1 (based on NCHW arrangement). Figure 3C is an example diagram showing a position index matrix according to this specific example, which is used to represent the reserved positions and sparse positions in the channel (C) direction. In the position index matrix, the first row of data corresponds to the data 1_1_1, 1_2_1 and 1_3_1 in the original input tensor, the second row of data corresponds to the data 1_1_2, 1_2_2 and 1_3_2 in the original input tensor, and so on. Based on Figure 3B The original input tensor shown and Figure 3C According to the position index matrix shown, the input circuit 12 can read the corresponding data from the original input tensor as the input tensor according to the position of element 1 in the position index matrix. For example, in one reading process, the input circuit 12 can read the data 1_1_1 and 1_3_1 corresponding to element 1 in the first row of data in the position index matrix and skip the data 1_2_1 corresponding to element 0. In another reading process, the input circuit 12 can read the data 1_2_2 corresponding to element 1 in the second row of data in the position index matrix and skip the data 1_1_2 and 1_3_2 corresponding to element 0. In this way, the input circuit 12 can read 12 data in the original input tensor as the input tensor. The multiplication accumulator 11 can use the 12 data in the input tensor to perform multiplication and addition operations with the sparse weight tensor to obtain the multiplication and addition operation results of the original input tensor and the original weight matrix.

[0053] In some possible implementations, the data processing device 1 may be used to implement a convolution operation. Figure 3D is an example diagram showing the original input tensor in a convolution operation. Figure 3D As shown, the original input tensor is 1×8×5×5 (based on NCHW arrangement), and the pad of the convolution operation is 0 and the stride is 1. In these implementations, the input circuit 12 is based on Figure 3C The position index matrix shown reads 12 data in the first 1×8×3×1 matrix as one input tensor, reads 12 data in the second 1×8×3×1 matrix as another input tensor, reads 12 data in the third 1×8×3×1 matrix as another input tensor, and so on. The multiplication accumulator 11 uses the sparse weight tensor to perform multiplication and addition operations with each input tensor read by the input circuit and obtains the result of the convolution operation.

[0054] According to the above description, the data of the input tensor in the embodiment of the present disclosure is less than the data in the original input tensor, which is beneficial to reducing the amount of data stored in the cache 15, improving the utilization rate of the cache 15, and reducing the reading bandwidth of the input tensor.

[0055] According to an embodiment of the present disclosure, the input circuit 12 is configured to read the original input tensor as the input tensor. The multiplication accumulator 11 is configured to perform Figure 4 Steps S41 and S42 are shown.

[0056] In step S41, a position index is obtained, the position index including the reserved position for sparse processing.

[0057] In step S42, the data corresponding to the reserved position in the input tensor and the sparse weight tensor are multiplied and added according to the position index. Specifically, the multiplication accumulator 11 can obtain the sparse position according to the position index. Based on this, in step S42, the data corresponding to the reserved position can be read from the input tensor according to the position index and the data corresponding to the sparse position can be skipped, and the read data can be multiplied and added with the sparse weight tensor. In this way, the multiplication accumulator 11 can be equivalent to the amount of data calculation before sparseness, thereby realizing the improvement of the computing power of the multiplication accumulator 11.

[0058] According to one embodiment of the present disclosure, the input circuit 12 is further configured to read the sparse weight tensor and write the sparse weight tensor into the cache 15. The sparse weight tensor is obtained by sparse processing the original weight tensor. For example, in some possible implementations, the sparse weight tensor can be obtained by offline sparse processing of the original weight tensor. Specifically, when the original input tensor and the original weight tensor are used for multiplication and addition, redundant data exists in the original weight tensor. In the embodiment of the present disclosure, offline tools can be used and sparse algorithms can be used to perform redundancy removal operations on the original weight tensor in a pre-training manner to obtain a sparse weight tensor, and ensure that the results of the multiplication and addition operations using the sparse weight tensor and the input tensor are basically the same as the results of the operations of the original weight tensor and the original input tensor. In addition, offline tools can also be used to obtain position indexes in the embodiment of the present disclosure.

[0059] In some possible implementations, a sparse algorithm can be used through an offline tool to determine the sparse granularity, and the weights that need to be sparse and the weights that need to be retained in the original weight tensor can be determined based on the sparse granularity, and the original weight tensor can be sparsely processed to obtain a sparse weight tensor.

[0060] It should be noted that using an offline tool to obtain a sparse weight tensor is only one feasible method of the embodiment of the present disclosure, but the present disclosure is not limited thereto. In some other embodiments, other methods may also be used to obtain a sparse weight tensor.

[0061] According to an embodiment of the present disclosure, the data processing device 1 further includes a controller 13. The controller 13 is configured to perform sparse processing on the original weight tensor to obtain a sparse weight tensor.

[0062] According to one embodiment of the present disclosure, the original weight tensor is subjected to a first sparse processing in a first dimension to obtain a sparse weight tensor, wherein the first sparse processing adopts a first sparse granularity. In the convolution operation, the first dimension may be, for example, the number of output kernels, input channels, kernel height or kernel width, etc. For example, for the original weight tensor whose key features are evenly distributed in the input channel, the first sparse processing may be performed in the input channel dimension, and based on the result of the first sparse processing, the sparse granularity may be determined according to the channel size and its information.

[0063] In some possible implementations, the first sparse granularity is m:n. Every n data of the original weight tensor in the first dimension is sparsely spaced into m data, where n and m are both positive integers and n>m.

[0064] Next, the first sparse processing mentioned above will be introduced through a specific example. In this example, the first dimension is the channel dimension, and the first sparse granularity is configured as 2:4 (ie, n=4, m=2). Figure 5is an example diagram showing how to obtain a sparse weight tensor by sparsely processing the original weight tensor in this example. Figure 5 As shown, in this example, a sparse weight tensor of 1×4×1×3 is obtained by sparsely processing the original weight tensor of 1×8×1×3, where the sparse weight refers to the weight that is sparsely distributed, and the weight can be reconfigured to 0 or a fixed bias. Figure 6 is an example diagram showing that the sparse processing of the 8 weights of the channel dimension in the original weight tensor is performed with a sparse granularity of 2:4 in this example. Figure 6 As shown, among the 8 weights, every 4 weights are thinned into 2 weights, that is, weights 6_1 to 6_4 are thinned into 2 weights, and weights 6_5 to 6_8 are thinned into 2 weights, where the positions of the sparse weights and the retained weights can be determined by offline tools or by the controller 13.

[0065] In some possible implementations, the first sparse processing adopts The position index is used to represent the sparse position. Each data after sparseness is represented by The position index is used to indicate its position, and ceil is a function that rounds up to an integer. For example, if n = 4, then If n = 5, then Specifically, in order to restore the sparse positions, these implementations store the position of each retained weight. Each retained weight position needs to be stored using a ceil(log2 n) bit position index, and the first sparse processing process needs to store a total of m×ceil(log2n) position indexes. Figure 6 Taking the sparse processing process shown as an example, the retained weights 6_3 and 6_4 both require 2 bits to represent their positions in the four weights 6_1 to 6_4, so a total of 4 bits of position index need to be stored.

[0066] In some other possible implementations, the first sparse processing adopts To express A sparse method, and from this Select one of the sparse methods to execute, ceil is the upward rounding function. Specifically, n data are sparsely divided into m data, with a total of There are three ways, each of which corresponds to a combination of reserved positions and sparse positions. Figure 6 As an example of the sparse processing process shown in the figure, in one sparse mode, the reserved positions may be 6_3 and 6_4, and the sparse positions may be 6_1 and 6_2. In another sparse mode, the reserved positions may be 6_1 and 6_3, and the sparse positions may be 6_2 and 6_4. And so on. There are a total of A sparse method that requires storage The data represents the six sparse methods. Therefore, these implementation methods can reduce the demand for storage space and save hardware resources. This method can be applied to the scenario of sparse processing in one or a few dimensions, for example.

[0067] In some other possible implementations, the first sparse processing uses ceil(log2N) bits to represent N sparse modes, and selects one of the N sparse modes for execution. integer, ceil is the upward rounding function, the N sparse methods are Specifically, n data are sparsely divided into m data, with a total of There are three ways, each of which corresponds to a combination of reserved locations and sparse locations. A sparse method is used for further screening to remove the ones with a low proportion The above N sparse methods can be obtained by using the above N methods. The screening of sparse methods can be performed by offline analysis, but the present disclosure is not limited to this. The value of N can be, for example, an integer power of 2. Figure 6 As an example, the sparse processing process shown in the figure has a total of sparse methods. By screening by offline analysis and other methods, two methods with low proportions can be removed to obtain N = 4 sparse methods. At this time, only ceil (log24) = 2 bits of data need to be stored to represent the four sparse methods. Therefore, these implementation methods can further reduce the demand for storage space and save hardware resources. This method can be applied to scenarios where sparse processing is performed in multiple dimensions.

[0068] According to the above description, in the embodiment of the present disclosure, a sparse weight tensor is obtained by sparsely processing the original weight tensor, and the size of the sparse weight tensor is smaller than the original weight tensor. For example, in a convolution operation, for a weight of 8×64×3×3 (each dimension is the number of output kernels, input channels, kernel height, and kernel width), if the input channel dimension is sparsely processed with a sparse granularity of 2:4 (i.e., n=4, m=2), the sparse weight tensor input to the multiplication accumulator 11 is 8×32×3×3, and its size becomes 1 / 2 of the original weight tensor. For another example, for a weight of 8×64×3×3, if the output kernel number dimension is sparsely processed with a sparse granularity of 2:4, the sparse weight tensor input to the multiplication accumulator 11 is 4×64×3×3, and its size becomes 1 / 2 of the original weight tensor. In this way, the amount of calculation performed by the multiplication accumulator 11 can be reduced, the utilization rate of the multiplication accumulator 11 can be improved, and the reading bandwidth of some weights can be reduced.

[0069] According to one embodiment of the present disclosure, the original weight tensor is subjected to a second sparse processing on a second dimension to obtain a sparse weight tensor, and the second sparse processing adopts a second sparse granularity. In the convolution operation, the second dimension can be, for example, the number of output kernels, input channels, kernel height or kernel width, etc.

[0070] In some possible implementations, the second sparse granularity is s:r. Every r data of the original weight tensor in the second dimension is sparsely spaced into s data, where r and s are both positive integers and r>s.

[0071] It should be noted that the second sparse processing is similar to the first sparse processing, and will not be described in detail here.

[0072] According to one embodiment of the present disclosure, the original weight tensor is subjected to a first sparse processing and a second sparse processing in one dimension of the first dimension and the second dimension to obtain an intermediate tensor. The intermediate tensor is subjected to a first sparse processing and a second sparse processing in another dimension of the first dimension and the second dimension to obtain a sparse weight tensor. The first sparse processing adopts a first sparse granularity, and the second sparse processing adopts a second sparse granularity. The first sparse granularity and the second sparse granularity may be the same or different.

[0073] Next, the above process will be described in detail through a specific example. This specific example takes the convolution operation as an example, and the first dimension in this specific example is the channel dimension, and the second dimension is the kernel number dimension. For the original weight tensor of 8×64×3×3, the process of obtaining the sparse weight tensor includes: performing sparse processing (that is, the first sparse processing) on ​​the original weight tensor with a sparse granularity of 2:4 in the input channel dimension to obtain an intermediate tensor, and performing sparse processing (that is, the second sparse processing) on ​​the intermediate tensor with a sparse granularity of 2:4 in the output kernel number dimension to obtain the final sparse weight tensor, whose size is 4×32×3×3, which is 1 / 4 of the size of the original weight tensor. In this way, the amount of calculation performed by the multiplication accumulator 11 can be further reduced. In addition, in order to restore the sparse position, it is necessary to store a log2n+log2n=4bit position index for each retained weight in this example.

[0074] In some possible implementations, the second sparse processing may use a position index of s×ceil(log2r) bits to represent the sparse position, and each datum after sparse processing may use a position index of ceil(log2r) bits to represent its position.

[0075] In some other possible implementations, the second sparse processing may be performed by To express A sparse method, and from this Select one of the sparse modes to execute.

[0076] In some other possible implementations, the second sparse processing may use ceil(log2M) bits to represent M sparse modes, and select one of the M sparse modes for execution, where M is less than integer, the M sparse methods are According to an embodiment of the present disclosure, the first dimension may be the channel direction of the original weight tensor, and the second dimension may be the kernel direction of the original weight tensor, but the present disclosure is not limited thereto.

[0077] According to one embodiment of the present disclosure, the original weight tensor is further subjected to a third sparse processing in a third dimension to obtain the sparse weight tensor, and the third sparse processing adopts a third sparse granularity. The third dimension is a dimension other than the first dimension and the second dimension. For example, in a convolution operation, the third dimension can be the number of output kernels, input channels, kernel height or kernel width, etc.

[0078] An embodiment of the present disclosure further provides a computer-readable storage medium on which a computer program is stored. The computer program is executed to implement the data processing method provided by any embodiment of the present disclosure.

[0079] In some possible implementations, any combination of one or more storage media may be used. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0080] The embodiment of the present disclosure further provides a chip, which includes the data processing device provided by any embodiment of the present disclosure. In some embodiments, the chip is a neural network processor chip, but the present disclosure is not limited to this.

[0081] In summary, in the data processing device provided by the embodiment of the present disclosure, the multiplication accumulator uses the sparse weight tensor and the input tensor to perform multiplication and addition operations to obtain the multiplication and addition results of the original input tensor and the original weight tensor. Since the sparse weight tensor is obtained by sparse processing of the original weight tensor, the data in the sparse weight tensor is less than the original weight tensor. Therefore, compared with the solution of directly using the original input tensor and the original weight tensor for multiplication and addition operations, the embodiment of the present disclosure has less calculation amount when performing multiplication and addition operations, and can improve the computing power of the data processing device without increasing the multiplication accumulator. In addition, compared with the prior art, the data processing device provided by the embodiment of the present disclosure has a smaller design area and power consumption under the premise of maintaining consistent computing power. Furthermore, in the data processing device provided by the embodiment of the present disclosure, the multiplication accumulator only needs to read the sparse weight tensor and index, thereby reducing the weight reading bandwidth. Further, the data processing device provided by the embodiment of the present disclosure can flexibly configure different sparse parameters according to the application scenario, and can be applied to different application scenarios. Therefore, the present disclosure overcomes the various defects of the prior art and has a high application prospect.

[0082] The above embodiments are merely illustrative of the principles and effects of the present disclosure, and are not intended to limit the present disclosure. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present disclosure. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present disclosure shall still be covered by the claims of the present disclosure.

Claims

1. A data processing device, characterized in that: include: A multiplier accumulator, the multiplier accumulator being configured as: Obtain an input tensor and a sparse weight tensor, where the sparse weight tensor is obtained by sparsely processing the original weight tensor in at least one of the first dimension and the second dimension; as well as A multiplication-addition operation is performed on the sparse weight tensor and the input tensor.

2. The data processing device according to claim 1, characterized in that: The input tensor includes data in the original input tensor corresponding to the reserved locations for the sparse processing.

3. The data processing device according to claim 1, characterized in that: Also includes: An input circuit, the input circuit being configured as: Obtaining a position index, wherein the position index includes the reserved position for the sparse processing; as well as The data corresponding to the reserved position in the original input tensor is read according to the position index as the input tensor.

4. The data processing device according to claim 1, characterized in that: Also includes: An input circuit configured to read an original input tensor as the input tensor; The multiplier accumulator is configured as: Obtaining a position index, wherein the position index includes the reserved position for the sparse processing; and A multiplication and addition operation is performed on the data corresponding to the reserved position in the input tensor and the sparse weight tensor according to the position index.

5. The data processing device according to claim 3 or 4, characterized in that: Also includes: A controller is configured to perform sparse processing on the original weight tensor to obtain the sparse weight tensor.

6. The data processing device according to claim 3 or 4, characterized in that: The input circuit is configured to read the sparse weight tensor, wherein the sparse weight tensor is obtained by sparsely processing the original weight tensor.

7. The data processing device according to claim 1, characterized in that: The original weight tensor is subjected to a first sparse processing in the first dimension to obtain the sparse weight tensor, wherein the first sparse processing adopts a first sparse granularity.

8. The data processing device according to claim 7, characterized in that: The first sparse granularity is m:n, and every n data of the original weight tensor in the first dimension are sparsely spaced into m data, where m and n are both positive integers and n>m.

9. The data processing device according to claim 8, characterized in that: The first sparse processing uses a position index of m×ceil(log2n) bits to represent the sparse position, and each datum after sparse processing uses a position index of ceil(log2n) bits to represent its position, where ceil is a round-up function.

10. The data processing device according to claim 8, characterized in that: The first sparse processing adopts bit to indicate A sparse method, and from the Select one of the sparse modes to execute, and ceil is the upward rounding function.

11. The data processing device according to claim 8, characterized in that: The first sparse processing uses ceil(log2N) bits to represent N sparse modes, and selects one of the N sparse modes to execute, where N is less than integer, ceil is a round-up function, and the N sparse methods are It is selected from a sparse method.

12. The data processing device according to claim 1, characterized in that: The original weight tensor is subjected to a second sparse processing in the second dimension to obtain the sparse weight tensor, and the second sparse processing adopts a second sparse granularity.

13. The data processing device according to claim 12, characterized in that: The second sparse granularity is s:r, and every r data of the original weight tensor in the second dimension are sparsely spaced into s data, where r and s are both positive integers and r>s.

14. The data processing device according to claim 1, characterized in that: The original weight tensor is subjected to a first sparse processing or one of a second sparse processing in one of the first dimension and the second dimension to obtain an intermediate tensor, and the intermediate tensor is subjected to the other of the first sparse processing or the second sparse processing in another of the first dimension and the second dimension to obtain the sparse weight tensor, the first sparse processing adopts a first sparse granularity, and the second sparse processing adopts a second sparse granularity.

15. The data processing device according to any one of claims 1 to 14, characterized in that: The first dimension is the channel direction of the original weight tensor, and the second dimension is the kernel direction of the original weight tensor.

16. The data processing device according to claim 1, characterized in that: The original weight tensor is further subjected to a third sparse processing in a third dimension to obtain the sparse weight tensor, wherein the third sparse processing adopts a third sparse granularity.

17. A data processing method, characterized in that: include: Obtain an input tensor and a sparse weight tensor, where the sparse weight tensor is obtained by sparsely processing the original weight tensor in at least one of the first dimension and the second dimension; as well as A multiplication-addition operation is performed on the sparse weight tensor and the input tensor.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that: The computer program is executed to implement the data processing method according to any one of claims 16.

19. A chip, characterized in that: The chip includes the data processing device according to any one of claims 1 to 16.

Images