Data processing device, data processing method and related products

Abstract

The present disclosure discloses a data processing device, a data processing method, and related products. The data processing device can be implemented as a computing device included in a combined processing device, and the combined processing device can also include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete the computing operations specified by the user. The combined processing device can also include a storage device, which is connected to the computing device and the other processing devices respectively and is used to store data of the computing device and the other processing devices. The solution disclosed in the present disclosure provides dedicated instructions for operations related to structured sparse tensor data, which can simplify processing and improve the processing efficiency of the machine.

Description

Technical Field

[0001] The present disclosure generally relates to the field of processors and, more particularly, to data processing devices, data processing methods, chips, and boards. Background Art

[0002] In recent years, the rapid development of deep learning has led to significant advances in algorithm performance across a range of fields, including computer vision and natural language processing. However, deep learning algorithms are computationally and storage-intensive. As information processing tasks become increasingly complex and the demands for real-time and accurate algorithms continue to rise, neural networks are often designed to be deeper, increasing the computational load and storage requirements. This makes existing deep learning-based artificial intelligence technologies difficult to directly apply to hardware-constrained mobile phones, satellites, or embedded devices.

[0003] Therefore, the compression, acceleration, and optimization of deep neural network models have become extremely important. Numerous studies have attempted to reduce the computational and storage requirements of neural networks without compromising model accuracy. This is of crucial importance for the engineering application of deep learning technology on embedded and mobile devices. Sparsification is one such method for achieving model lightweighting.

[0004] Network parameter sparsification is to reduce the redundant components in a larger network by appropriate methods to reduce the network's computational load and storage space requirements. Existing hardware and / or instruction sets cannot effectively support sparsification processing. Summary of the Invention

[0005] In order to at least partially solve one or more technical problems mentioned in the background technology, the solution disclosed herein provides a data processing device, a data processing method, a chip and a board.

[0006] In a first aspect, the present disclosure discloses a data processing device, comprising: a control circuit configured to parse a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparsity, and at least one operand of the sparse instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of tensor data; a tensor interface circuit configured to parse the descriptor; a storage circuit configured to store information before and / or after sparsification; and an operation circuit configured to perform corresponding operations according to the sparse instruction based on the parsed descriptor.

[0007] In a second aspect, the present disclosure provides a chip comprising the data processing device of any embodiment of the aforementioned first aspect.

[0008] In a third aspect, the present disclosure provides a board comprising the chip of any one of the embodiments of the second aspect.

[0009] In a fourth aspect, the present disclosure provides a data processing method, which includes: parsing a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparsity, and at least one operand of the sparse instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of tensor data; parsing the descriptor; reading the corresponding operand based at least in part on the parsed descriptor; performing the operation related to structured sparsity on the operand; and outputting the operation result.

[0010] Through the data processing device, data processing method, integrated circuit chip and board provided above, the disclosed embodiment provides a sparse instruction for performing operations related to structured sparsity of tensor data, wherein the tensor data is described by a descriptor. In some embodiments, the sparse instruction may include an operation mode bit to indicate different operation modes of the sparse instruction, thereby performing different operations. In other embodiments, a plurality of sparse instructions may be included, each instruction corresponding to one or more different operation modes, thereby performing various operations related to structured sparsity. By providing specialized sparse instructions to perform operations related to structured sparsity of tensor data, processing can be simplified, thereby improving the processing efficiency of the machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0012] Figure 1 is a structural diagram showing a board according to an embodiment of the present disclosure;

[0013] Figure 2 is a structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram showing the internal structure of a multi-core computing device according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram showing the internal structure of a processor core according to an embodiment of the present disclosure;

[0017] Figure 6 A schematic diagram showing a data storage space according to an embodiment of the present disclosure;

[0018] Figure 7 A schematic diagram showing data blocks in a data storage space according to an embodiment of the present disclosure;

[0019] Figure 8 is a schematic diagram showing the structure of a data processing device according to an embodiment of the present disclosure;

[0020] Figure 9A is an exemplary pipeline operation circuit illustrating structured sparse processing according to an embodiment of the present disclosure;

[0021] Figure 9B is an exemplary pipeline operation circuit illustrating structured sparse processing according to another embodiment of the present disclosure; and

[0022] Figure 10 is an exemplary flow chart illustrating a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of this disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this disclosure, not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this disclosure.

[0024] It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0025] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0027] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0028] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC), or system-on-chip, integrated with one or more combined processing devices. The combined processing device is an artificial intelligence computing unit that supports various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud-based intelligent applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. Board 10 of this embodiment is suitable for cloud-based intelligent applications and has extensive off-chip storage, on-chip storage, and powerful computing capabilities.

[0029] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. Calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface formats, such as a PCIe interface.

[0030] Board 10 also includes a memory device 104 for storing data, which includes one or more storage units 105. Memory device 104 is connected to a control device 106 and chip 101 via a bus for data transmission. Control device 106 in board 10 is configured to control the state of chip 101. To this end, in one application scenario, control device 106 may include a microcontroller (MCU).

[0031] Figure 2 FIG. 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0032] The computing device 201 is configured to perform user-specified operations and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0033] Interface device 202 is used to transmit data and control instructions between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to a storage device on-chip of computing device 201. Furthermore, computing device 201 can obtain control instructions from processing device 203 via interface device 202 and write them to a control cache on-chip of computing device 201. Alternatively or optionally, interface device 202 can also read data from the storage device of computing device 201 and transmit it to processing device 203.

[0034] The processing device 203 is a general processing device that performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, which include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, with respect to the computing device 201 disclosed herein, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0035] The storage device 204 is used to store data to be processed, which may be DRAM or DDR memory, and is typically 16G or larger in size, for storing data of the computing device 201 and / or the processing device 203 .

[0036] Figure 3 The single-core computing device 301 is used to process input data for computer vision, speech, natural language, data mining, etc. The single-core computing device 301 includes three modules: a control module 31, a computing module 32, and a storage module 33.

[0037] The control module 31 coordinates and controls the operations of the computing module 32 and the storage module 33 to complete deep learning tasks. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 retrieves instructions from the processing device 203, while the instruction decode unit 312 decodes the retrieved instructions and sends the decoded results as control information to the computing module 32 and the storage module 33.

[0038] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations, supporting complex operations such as vector multiplication, addition, and nonlinear transformations. The matrix operation unit 322 is responsible for the core calculations of the deep learning algorithm, namely matrix multiplication and convolution.

[0039] The storage module 33 is used to store or transfer relevant data and includes a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access (DMA) module 333. NRAM 331 stores input neurons, output neurons, and intermediate computational results; WRAM 332 stores the convolution kernels (i.e., weights) of the deep learning network; and DMA 333, which connects to DRAM 204 via bus 34, transfers data between the single-core computing device 301 and DRAM 204.

[0040] Figure 4 The figure shows the internal structure of a multi-core computing device 201. The multi-core computing device 41 employs a layered design. As a system-on-chip (SoC), the multi-core computing device 41 includes at least one cluster, each of which includes multiple processor cores. In other words, the multi-core computing device 41 is structured in a hierarchy of SoC-cluster-processor cores.

[0041] At the system-on-chip level, Figure 4 As shown, the multi-core computing device 41 includes an external storage controller 401 , a peripheral communication module 402 , an on-chip interconnect module 403 , a synchronization module 404 and multiple clusters 405 .

[0042] There can be multiple external storage controllers 401, and two are shown in the figure as an example. They are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 2DRAM204 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 402 is used to receive control signals from the processing device 203 through the interface device 202, and start the computing device 201 to perform tasks. The on-chip interconnect module 403 connects the external storage controller 401, the peripheral communication module 402 and multiple clusters 405 to transmit data and control signals between each module. The synchronization module 404 is a global synchronization barrier controller (GBC) used to coordinate the work progress of each cluster and ensure information synchronization. Multiple clusters 405 are the computing cores of the multi-core computing device 41. Four are shown as an example in the figure. With the development of hardware, the multi-core computing device 41 disclosed in this disclosure can also include 8, 16, 64, or even more clusters 405. Clusters 405 are used to efficiently execute deep learning algorithms.

[0043] At the cluster level, Figure 4 As shown, each cluster 405 includes multiple processor cores (IPU cores) 406 and a memory core (MEM core) 407 .

[0044] The figure shows four processor cores 406 as an example, but the present disclosure does not limit the number of processor cores 406. Figure 5 Each processor core 406 is similar to Figure 3 The single-core computing device 301 also includes three major modules: a control module 51, a computing module 52, and a storage module 53. The functions and structures of the control module 51, computing module 52, and storage module 53 are roughly the same as those of the control module 31, computing module 32, and storage module 33, and will not be repeated here. It should be noted that the storage module 53 includes an input / output direct memory access module (IODMA) 533 and a move direct memory access module (MVDMA) 534. The IODMA 533 controls memory access between the NRAM 531 / WRAM 532 and the DRAM 204 via the broadcast bus 409; the MVDMA 534 is used to control memory access between the NRAM 531 / WRAM 532 and the storage unit (SRAM) 408.

[0045] Back to Figure 4The storage core 407 is primarily used for storage and communication, namely, storing shared data or intermediate results between the processor cores 406, and performing communication between the cluster 405 and the DRAM 204, between the clusters 405, and between the processor cores 406. In other embodiments, the storage core 407 has scalar operation capabilities and is used to perform scalar operations.

[0046] The storage core 407 includes SRAM 408, a broadcast bus 409, a cluster direct memory access module (CDMA) 410, and a global direct memory access module (GDMA) 411. SRAM 408 acts as a high-performance data transfer station. Data reused between different processor cores 406 within the same cluster 405 does not need to be obtained from DRAM 204 by each processor core 406. Instead, it is transferred between the processor cores 406 via SRAM 408. The storage core 407 only needs to quickly distribute the reused data from SRAM 408 to multiple processor cores 406, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip input / output access.

[0047] The broadcast bus 409, CDMA 410, and GDMA 411 are used for communication between the processor cores 406, communication between the clusters 405, and data transmission between the clusters 405 and the DRAM 204, respectively. Each of these will be described below.

[0048] Broadcast bus 409 facilitates high-speed communication between processor cores 406 within cluster 405. In this embodiment, broadcast bus 409 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point data transmission (e.g., from one processor core to another), multicast transfers a copy of data from SRAM 408 to a specific number of processor cores 406, and broadcast transfers a copy of data from SRAM 408 to all processor cores 406, a special case of multicast.

[0049] The CDMA 410 is used to control memory access to the SRAM 408 between different clusters 405 within the same computing device 201 .

[0050] GDMA 411 works in conjunction with external memory controller 401 to control memory access from cluster 405's SRAM 408 to DRAM 204, or to read data from DRAM 204 to SRAM 408. As previously mentioned, communication between DRAM 204 and NRAM 431 or WRAM 432 can be achieved via two channels. The first channel directly connects DRAM 204 and NRAM 431 or WRAM 432 via IODAM 433. The second channel first transfers data between DRAM 204 and SRAM 408 via GDMA 411, and then transfers data between SRAM 408 and NRAM 431 or WRAM 432 via MVDMA 534. While the second channel may appear to require more components and a longer data flow, in some embodiments, the bandwidth of the second channel is significantly greater than the first channel. Therefore, communication between DRAM 204 and NRAM 431 or WRAM 432 through the second channel may be more efficient. The embodiments of the present disclosure can select a data transmission channel according to the hardware conditions.

[0051] In other embodiments, the functions of GDMA 411 and IODMA 533 can be integrated into the same component. For ease of description, this disclosure treats GDMA 411 and IODMA 533 as separate components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those disclosed herein, they fall within the scope of protection of this disclosure. Furthermore, the functions of GDMA 411, IODMA 533, CDMA 410, and MVDMA 534 can also be implemented by the same component.

[0052] Traditional processor instructions are designed to perform basic single-data scalar operations. Here, single-data scalar operations refer to each operand of the instruction being a scalar data. However, with the development of artificial intelligence technology, in tasks such as image processing and pattern recognition, the operands are often multi-dimensional vectors (i.e., tensor data). Using only scalar operations cannot enable the hardware to efficiently complete the computing tasks. Therefore, how to efficiently perform multi-dimensional tensor data processing is also a problem that needs to be solved urgently in the current computing field.

[0053] In an embodiment of the present disclosure, a structured sparse instruction is provided for performing operations related to structured sparsity of tensor data. At least one operand of the structured sparse instruction includes at least one descriptor, through which information related to the tensor data can be obtained. Specifically, the descriptor can indicate at least one of the following information: shape information of the tensor data, spatial information of the tensor data. The shape information of the tensor data can be used to determine the data address of the tensor data corresponding to the operand in the data storage space. The spatial information of the tensor data can be used to determine the dependency between instructions, and thus determine, for example, the execution order of the instructions.

[0054] In one possible implementation, the spatial information of tensor data can be indicated by a spatial identifier (ID). The spatial ID can also be called a spatial alias, which refers to a spatial region used to store the corresponding tensor data. The spatial region can be a continuous space or multiple spaces. This disclosure does not limit the specific composition of the spatial region. Different spatial IDs indicate that there is no dependency between the spatial regions pointed to.

[0055] Various possible implementations of the shape information of tensor data will be described in detail below with reference to the accompanying drawings.

[0056] Tensors can contain a variety of data structures. Tensors can be of different dimensions. For example, a scalar can be considered a 0-dimensional tensor, a vector can be considered a 1-dimensional tensor, and a matrix can be a 2-dimensional or higher tensor. The shape of a tensor includes information such as the dimensions of the tensor and the size of each dimension of the tensor. For example, for a 3D tensor:

[0057] x3=[[[1,2,3],[4,5,6]];[[7,8,9],[10,11,12]]]

[0058] The shape or dimensions of this tensor can be expressed as X3 = (2, 2, 3), which means that the three parameters indicate that the tensor is a three-dimensional tensor with a first dimension of 2, a second dimension of 2, and a third dimension of 3. When storing tensor data in memory, the shape of the tensor data cannot be determined based on its data address (or storage area), and thus, related information such as the relationship between multiple tensor data cannot be determined, resulting in low processor access efficiency for tensor data.

[0059] In one possible implementation, a descriptor can be used to indicate the shape of N-dimensional tensor data, where N is a positive integer, such as N=1, 2, or 3, or zero. The three-dimensional tensor in the above example can be represented by a descriptor as (2, 2, 3). It should be noted that this disclosure does not limit the manner in which a descriptor indicates the shape of a tensor.

[0060] In one possible implementation, the value of N can be determined according to the dimension (also called order) of the tensor data, or it can be set according to the usage requirements of the tensor data. For example, when the value of N is 3, the tensor data is three-dimensional tensor data, and the descriptor can be used to indicate the shape of the three-dimensional tensor data in three dimensions (such as offset, size, etc.). It should be understood that those skilled in the art can set the value of N according to actual needs, and this disclosure does not limit this.

[0061] Although tensor data can be multi-dimensional, because the memory layout is always one-dimensional, there is a correspondence between tensors and storage on the memory. Tensor data is usually allocated in a continuous memory space, that is, tensor data can be expanded one-dimensionally (for example, row-major) and stored in the memory.

[0062] This relationship between a tensor and its underlying storage can be expressed through dimensions such as offset, size, and stride. A dimension's offset refers to the offset relative to a reference position in that dimension. A dimension's size refers to the size of that dimension, or the number of elements in that dimension. A dimension's stride refers to the spacing between adjacent elements in that dimension. For example, the stride of the three-dimensional tensor above is (6, 3, 1), meaning the stride of the first dimension is 6, the stride of the second dimension is 3, and the stride of the third dimension is 1.

[0063] Figure 6 Schematic diagram showing the data storage space according to the embodiment of the present disclosure. Figure 6 As shown, data storage space 61 stores two-dimensional data in a row-major manner, which can be represented by (x, y) (where the X axis is horizontally to the right and the Y axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure). The starting address PA_start (base address) of data storage space 61 is the physical address of the first data block 62. Data block 63 is a portion of the data in data storage space 61. Its offset 65 in the X-axis direction is represented by offset_x, its offset 64 in the Y-axis direction is represented by offset_y, its size in the X-axis direction is represented by size_x, and its size in the Y-axis direction is represented by size_y.

[0064] In one possible implementation, when a descriptor is used to define data block 63, the data reference point of the descriptor can be the first data block of data storage space 61, and the reference address of the descriptor can be agreed to be the starting address PA_start of data storage space 61. The content of the descriptor of data block 63 can then be determined by combining the size ori_x on the X axis and the size ori_y on the Y axis of data storage space 61, as well as the offset offset_y in the Y axis direction, the offset offset_x in the X axis direction, the size size_x in the X axis direction, and the size size_y in the Y axis direction of data block 63.

[0065] In a possible implementation, the following formula (1) can be used to express the content of the descriptor:

[0066]

[0067] It should be understood that although in the above examples, the content of the descriptor represents a two-dimensional space, those skilled in the art can set the specific dimension represented by the content of the descriptor according to actual conditions, and this disclosure does not limit this.

[0068] In one possible implementation, the base address of the data reference point of the descriptor in the data storage space can be agreed upon. Based on the base address, the content of the descriptor of the tensor data is determined according to the positions of at least two vertices at diagonal positions in N dimensional directions relative to the data reference point.

[0069] For example, the data reference point of the descriptor can be agreed to be the reference address PA_base in the data storage space. For example, a data (e.g., data at position (2, 2)) can be selected in the data storage space 61 as the data reference point, and the physical address of the data in the data storage space can be used as the reference address PA_base. The position of the two vertices at the diagonal position relative to the data reference point can be used to determine the reference address PA_base. Figure 6 The content of the descriptor of data block 63 in the data block 63 is determined. First, the positions of at least two vertices at the diagonal positions of the data block 63 relative to the data reference point are determined. For example, the positions of the diagonal vertices from the upper left to the lower right relative to the data reference point are used, where the relative position of the upper left vertex is (x_min, y_min) and the relative position of the lower right vertex is (x_max, y_max). Then, the content of the descriptor of data block 63 can be determined based on the reference address PA_base, the relative position of the upper left vertex (x_min, y_min), and the relative position of the lower right vertex (x_max, y_max).

[0070] In a possible implementation, the following formula (2) can be used to express the content of the descriptor (the base address is PA_base):

[0071]

[0072] It should be understood that although the vertices at the upper left corner and the lower right corner are used in the above example to determine the content of the descriptor, those skilled in the art can set the specific vertices of at least two diagonal positions according to actual needs, and this disclosure does not limit this.

[0073] In one possible implementation, the content of the tensor data descriptor can be determined based on the reference address of the descriptor's data reference point in the data storage space and the mapping relationship between the data description position and the data address of the tensor data indicated by the descriptor. The mapping relationship between the data description position and the data address can be set according to actual needs. For example, when the tensor data indicated by the descriptor is three-dimensional spatial data, the function f(x, y, z) can be used to define the mapping relationship between the data description position and the data address.

[0074] In a possible implementation, the following formula (3) can be used to express the content of the descriptor:

[0075]

[0076] In one possible implementation, the descriptor is further used to indicate the address of N-dimensional tensor data, wherein the content of the descriptor further includes at least one address parameter representing the address of the tensor data. For example, the content of the descriptor may be the following formula (4):

[0077]

[0078] Where PA is the address parameter. The address parameter can be a logical address or a physical address. When parsing the descriptor, PA can be used as any vertex, midpoint, or preset point of the vector shape, combined with the shape parameters in the X and Y directions to obtain the corresponding data address.

[0079] In a possible implementation, the address parameter of the tensor data includes a reference address of a data reference point of the descriptor in the data storage space of the tensor data, and the reference address includes a starting address of the data storage space.

[0080] In a possible implementation, the descriptor may further include at least one address parameter representing the address of the tensor data. For example, the content of the descriptor may be the following formula (5):

[0081]

[0082] PA_start is the base address parameter and will not be described in detail.

[0083] It should be understood that those skilled in the art can set the mapping relationship between the data description location and the data address according to actual conditions, and this disclosure does not limit this.

[0084] In one possible implementation, a predetermined reference address can be set within a task. All descriptors in instructions within this task use this reference address, and the descriptor content can include shape parameters based on this reference address. This reference address can be determined by setting the environment parameters for this task. A description of the reference address and its use can be found in the above embodiments. In this implementation, the descriptor content can be mapped to data addresses more quickly.

[0085] In one possible implementation, the base address can be included in the content of each descriptor, so that the base address of each descriptor can be different. Compared with the method of using environmental parameters to set a common base address, each descriptor in this method can describe data more flexibly and use a larger data address space.

[0086] In one possible implementation, the data address of the data corresponding to the operand of the processing instruction in the data storage space can be determined based on the content of the descriptor. The data address is calculated automatically by hardware, and the calculation method of the data address varies depending on the representation of the descriptor content. This disclosure does not limit the specific method for calculating the data address.

[0087] For example, the content of the descriptor in the operand is expressed using formula (1). The offsets of the tensor data indicated by the descriptor in the data storage space are offset_x and offset_y, and the size is size_x*size_y. Then, the starting data address PA1 of the tensor data indicated by the descriptor in the data storage space is (x,y) It can be determined using the following formula (6):

[0088] PA1 (x,y) =PA_start+(offset_y-1)*ori_x+offset_x (6)

[0089] The data starting address PA1 is determined according to the above formula (6) (x,y) , combined with the offsets offset_x and offset_y, and the sizes size_x and size_y of the storage area, the storage area of ​​the tensor data indicated by the descriptor in the data storage space can be determined.

[0090] In one possible implementation, when the operand also includes a data description location for a descriptor, the data address of the data corresponding to the operand in the data storage space can be determined based on the content of the descriptor and the data description location. In this way, partial data (e.g., one or more data) in the tensor data indicated by the descriptor can be processed.

[0091] For example, the content of the descriptor in the operand is expressed using formula (2). The offsets of the tensor data indicated by the descriptor in the data storage space are offset_x and offset_y respectively, and the size is size_x*size_y. The data description position for the descriptor included in the operand is (x q ,y q ), then the data address PA2 of the tensor data indicated by the descriptor in the data storage space (x,y) It can be determined using the following formula (7):

[0092] PA2 (x,y) =PA_start+(offset_y+y q -1)*ori_x+(offset_x+x q ) (7)

[0093] In one possible implementation, the descriptor can indicate data blocks. Data blocks can effectively speed up operations and improve processing efficiency in many applications. For example, in graphics processing, convolution operations often use data blocks for fast processing.

[0094] Figure 7 Schematic diagram showing data blocks in data storage space according to an embodiment of the present disclosure. Figure 7 As shown, the data storage space 700 also uses a row-first approach to store two-dimensional data, which can be represented by (x, y) (where the X axis is horizontal to the right and the Y axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure). Figure 6 Tensor data, Figure 7 The tensor data stored in consists of multiple data blocks.

[0095] In this case, the descriptor requires more parameters to represent these data blocks. Taking the X-axis (X dimension) as an example, the following parameters may be involved: ori_x, x.tile.size (the size 702 of the block), x.tile.stride (the stride 704 of the block, i.e., the distance between the first point of the first tile and the first point of the second tile), x.tile.num (the number of tiles, shown as 3 in the figure), x.stride (the overall stride, i.e., the distance between the first point of the first row and the first point of the second row), etc. Other dimensions can similarly include corresponding parameters.

[0096] In one possible implementation, a descriptor may include a descriptor identifier and / or descriptor content. The descriptor identifier is used to distinguish the descriptor, for example, the descriptor identifier may be a number; the descriptor content may include at least one shape parameter representing the shape of the tensor data. For example, if the tensor data is three-dimensional data, and the shape parameters of two of the three dimensions of the tensor data are fixed, the descriptor content may include the shape parameter representing the other dimension of the tensor data.

[0097] In one possible implementation, the identifier and / or content of the descriptor may be stored in a descriptor storage space (internal memory), such as a register, on-chip SRAM, or other media cache. The tensor data indicated by the descriptor may be stored in a data storage space (internal memory or external memory), such as an on-chip cache or off-chip memory. This disclosure does not limit the specific locations of the descriptor storage space and the data storage space.

[0098] In one possible implementation, the identifier, content of the descriptor, and the tensor data indicated by the descriptor can be stored in the same area of ​​the internal memory. For example, a continuous area of ​​the on-chip cache can be used to store the relevant content of the descriptor, and its address is ADDR0-ADDR1023. Among them, the address ADDR0-ADDR63 can be used as a descriptor storage space to store the identifier and content of the descriptor, and the address ADDR64-ADDR1023 can be used as a data storage space to store the tensor data indicated by the descriptor. In the descriptor storage space, the address ADDR0-ADDR31 can be used to store the identifier of the descriptor, and the address ADDR32-ADDR63 can be used to store the content of the descriptor. It should be understood that the address ADDR is not limited to 1 bit or 1 byte. It is used here to represent an address and is an address unit. Those skilled in the art can determine the descriptor storage space, data storage space and their specific addresses according to actual conditions, and this disclosure is not limited to this.

[0099] In one possible implementation, the descriptor identifier, content, and tensor data indicated by the descriptor can be stored in different areas of the internal memory. For example, registers can be used as descriptor storage space to store the descriptor identifier and content, and on-chip cache can be used as data storage space to store the tensor data indicated by the descriptor.

[0100] In one possible implementation, when registers are used to store the identifier and content of descriptors, the register number can be used to represent the identifier of the descriptor. For example, when the register number is 0, the identifier of the descriptor stored in it is set to 0. When the descriptor in the register is valid, an area in the cache space can be allocated to store the tensor data based on the size of the tensor data indicated by the descriptor.

[0101] In one possible implementation, the identifier and content of the descriptor may be stored in internal memory, and the tensor data indicated by the descriptor may be stored in external memory. For example, the identifier and content of the descriptor may be stored on-chip, while the tensor data indicated by the descriptor may be stored off-chip.

[0102] In one possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be divided for tensor data, and the starting address of each tensor data in the data storage space corresponds one-to-one to the descriptor. In this case, the circuit or module responsible for parsing the computing instruction (such as an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space based on the descriptor.

[0103] In one possible implementation, when the data address of the data storage space corresponding to the descriptor is a variable address, the descriptor can also be used to indicate the address of N-dimensional tensor data, wherein the content of the descriptor can also include at least one address parameter representing the address of the tensor data. For example, the tensor data is 3-dimensional data. When the descriptor points to the address of the tensor data, the content of the descriptor may include an address parameter representing the address of the tensor data, such as the starting physical address of the tensor data, or may include multiple address parameters of the address of the tensor data, such as the starting address + address offset of the tensor data, or the address parameters of the tensor data based on each dimension. Those skilled in the art can set the address parameters according to actual needs, and this disclosure does not limit this.

[0104] In one possible implementation, the address parameter of the tensor data may include the reference address of the descriptor's data reference point in the data storage space of the tensor data. The reference address may vary depending on the data reference point. This disclosure does not limit the selection of the data reference point.

[0105] In one possible implementation, the reference address may include the starting address of the data storage space. When the data reference point of the descriptor is the first data block in the data storage space, the reference address of the descriptor is the starting address of the data storage space. When the data reference point of the descriptor is data other than the first data block in the data storage space, the reference address of the descriptor is the address of the data block in the data storage space.

[0106] In one possible implementation, the shape parameters of the tensor data include at least one of the following: the size of the data storage space in at least one direction of the N-dimensional directions, the size of the storage area in at least one direction of the N-dimensional directions, the offset of the storage area in at least one direction of the N-dimensional directions, the positions of at least two vertices at diagonal positions in the N-dimensional directions relative to the data reference point, and the mapping relationship between the data description position of the tensor data indicated by the descriptor and the data address. The data description position is the mapping position of the point or area in the tensor data indicated by the descriptor. For example, when the tensor data is 3D data, the descriptor can use three-dimensional space coordinates (x, y, z) to represent the shape of the tensor data, and the data description position of the tensor data can be the position of the point or area mapped in the three-dimensional space represented by the three-dimensional space coordinates (x, y, z).

[0107] It should be understood that those skilled in the art can select shape parameters representing tensor data according to actual circumstances, and this disclosure does not limit this. By using descriptors in the data access process, associations between data can be established, thereby reducing the complexity of data access and improving instruction processing efficiency.

[0108] An embodiment of the present disclosure provides a data processing solution based on the aforementioned hardware environment, which performs operations related to structured sparsity of tensor data according to specialized sparse instructions.

[0109] Figure 8 FIG. 8 is a block diagram showing a data processing device 800 according to an embodiment of the present disclosure. The data processing device 800 may be implemented, for example, in Figure 2 As shown in the figure, the data processing device 800 may include a control circuit 810, a tensor interface circuit 812, a storage circuit 820 and an operation circuit 830.

[0110] The function of the control circuit 810 may be similar to Figure 3 The control module 31 or Figure 5 The control module 51 may include, for example, an instruction fetch unit for obtaining an instruction from, for example, Figure 2The processing device 203 of the embodiment of the present invention has an instruction, and an instruction decoding unit is used to decode the acquired instruction and send the decoding result as control information to the operation circuit 830 and the storage circuit 820.

[0111] In one embodiment, the control circuit 810 can be configured to parse a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparsity, and at least one operand of the sparse instruction includes at least one descriptor, the descriptor indicating at least one of the following information: shape information of tensor data and spatial information of the tensor data.

[0112] The tensor interface unit (TIU) 812 can be configured to implement operations associated with descriptors under the control of the control circuit 810. These operations may include, but are not limited to, registering, modifying, deregistering, and parsing descriptors; reading and writing descriptor contents, etc. This disclosure does not limit the specific hardware type of the tensor interface circuit. In this way, operations associated with descriptors can be implemented using dedicated hardware, further improving the access efficiency of tensor data.

[0113] In some embodiments, the tensor interface circuit 812 may be configured to parse shape information of tensor data included in an operand of an instruction to determine a data address of data corresponding to the operand in the data storage space.

[0114] Optionally or additionally, in some further embodiments, the tensor interface circuit 812 can be configured to compare the spatial information (e.g., spatial ID) of the tensor data included in the operands of two instructions to determine the dependency relationship between the two instructions, and further determine the out-of-order execution, synchronization, and other operations of the instructions.

[0115] Despite Figure 8 The control circuit 810 and the tensor interface circuit 812 are shown as two separate modules, but those skilled in the art will appreciate that these two units may also be implemented as one module or more modules, and the present disclosure is not limited in this regard.

[0116] The storage circuit 820 may be configured to store information before and / or after sparsification. In one embodiment, the operands of the sparse instruction are weights of the neural network. In this embodiment, the storage circuit may be, for example, Figure 3 WRAM332 or Figure 5 WRAM 532.

[0117] The operation circuit 830 may be configured to perform corresponding operations according to the sparse instruction based on the parsed descriptor.

[0118] In some embodiments, the operation circuit 830 may include one or more groups of pipeline operation circuits 831, wherein each group of pipeline operation circuits 831 may include one or more operators. When each group of pipeline operation circuits includes multiple operators, these multiple operators may be configured to perform multi-stage pipeline operations, that is, to form a multi-stage operation pipeline.

[0119] In some application scenarios, the pipelined operation circuit disclosed herein can support operations related to structured sparsity. For example, when performing structured sparsity processing, a multi-stage pipelined operation circuit composed of comparators and other circuits can be used to extract n data elements as valid data elements from every m data elements, where m>n. In one implementation, m=4 and n=2. In other implementations, n can also take other values, such as 1 or 3.

[0120] In one embodiment, the operation circuit 830 may further include an operation processing circuit 832, which may be configured to pre-process the data before the pipeline operation circuit 831 performs the operation or post-process the data after the operation according to the operation instruction. In some application scenarios, the aforementioned pre-processing and post-processing may, for example, include data splitting and / or data splicing operations. In structured sparse processing, the operation processing circuit may segment the data to be sparse according to every m data elements and then send it to the pipeline operation circuit 831 for processing.

[0121] Figure 9A FIG. 4 shows an exemplary operation pipeline of structured sparse processing according to an embodiment of the present disclosure. Figure 9A In the embodiment of FIG. 1 , when m=4 and n=2, a structured sparse processing is shown in which two data elements with larger absolute values ​​are selected from four data elements A, B, C, and D.

[0122] like Figure 9A As shown, the structured sparse processing can be performed using a four-stage pipeline operation circuit consisting of an absolute value operator and a comparator.

[0123] The first-stage pipeline operation circuit may include four absolute value operators 910 for synchronously performing absolute value operations on four input data elements A, B, C and D respectively.

[0124] The second-stage pipeline circuit may include two comparators for grouping and comparing the four absolute values ​​output by the previous stage. For example, the first comparator 921 may compare the absolute values ​​of data elements A and B and output the larger value, Max00, while the second comparator 922 may compare the absolute values ​​of data elements C and D and output the larger value, Max10.

[0125] The third stage pipeline operation circuit may include a third comparator 930 that compares the two larger values ​​Max00 and Max10 outputted from the previous stage and outputs the larger value Max0. This larger value Max0 is the value with the largest absolute value among the four data elements.

[0126] The fourth stage pipeline circuit may include a fourth comparator 940 that compares the smaller value Min0 in the previous stage with another value in the group containing the maximum value Max0 and outputs a larger value Max1. This larger value Max1 is the value with the second largest absolute value among the four data elements.

[0127] Therefore, structured sparse processing of four-choose-two can be achieved through a four-stage pipeline operation circuit.

[0128] Figure 9B FIG. 4 shows an exemplary operation pipeline of structured sparse processing according to another embodiment of the present disclosure. Figure 9B In the embodiment of FIG. 1 , when m=4 and n=2, a structured sparse processing is shown in which two data elements with larger absolute values ​​are selected from four data elements A, B, C, and D.

[0129] like Figure 9B As shown, the above-mentioned structured sparse processing can be performed using a multi-stage pipeline operation circuit composed of an absolute value operator, a comparator, etc.

[0130] The first pipeline stage may include m(4) absolute value operators 950 for synchronously performing absolute value operations on the four input data elements A, B, C, and D. To facilitate the final output of valid data elements, in some embodiments, the first pipeline stage simultaneously outputs the original data elements (i.e., A, B, C, and D) and the data after the absolute value operations (i.e., |A|, |B|, |C|, and |D|).

[0131] The second pipeline stage may include a permutation and combination circuit 960 for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values, and the positions of the m absolute values ​​in each group of data are different.

[0132] In some embodiments, the permutation and combination circuit can be a circular shifter that performs m-1 circular shifts on the permutation of m absolute values ​​(e.g., |A|, |B|, |C|, and |D|), thereby generating m groups of data. For example, in the example shown in the figure, four groups of data are generated, namely: {|A|, |B|, |C|, |D|}, {|B|, |C|, |D|, |A|}, {|C|, |D|, |A|, |B|}, and {|D|, |A|, |B|, |C|}. Similarly, when each group of data is output, the corresponding original data element is also output, and each group of data corresponds to one original data element.

[0133] The third pipeline stage includes a comparison circuit 970 for comparing the absolute values ​​of the m groups of data and generating a comparison result.

[0134] In some embodiments, the third pipeline stage may include m comparison circuits, each comparison circuit includes m-1 comparators (771, 772, 773), and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, where 1≤i≤m.

[0135] As can be seen from the figure, the third pipeline stage can also be considered as m-1 (3) sub-pipeline stages. Each sub-pipeline stage includes m comparators, which are used to compare its corresponding absolute value with other absolute values. The m-1 sub-pipeline stages thus sequentially compare their corresponding absolute value with the other m-1 absolute values.

[0136] For example, in the example shown in the figure, the four comparators 971 in the first sub-pipeline stage are used to compare the first absolute value with the second absolute value of the four sets of data, and output comparison results w0, x0, y0, and z0, respectively. The four comparators 972 in the second sub-pipeline stage are used to compare the first absolute value with the third absolute value of the four sets of data, and output comparison results w1, x1, y1, and z1, respectively. The four comparators 973 in the third sub-pipeline stage are used to compare the first absolute value with the fourth absolute value of the four sets of data, and output comparison results w2, x2, y2, and z2, respectively.

[0137] Thus, the comparison result of each absolute value with the other m-1 absolute values ​​can be obtained.

[0138] In some embodiments, the comparison result can be represented using a bitmap. For example, at the first comparator of the first comparison circuit, when |A| ≥ |B|, w0 = 1; at the second comparator of the first comparison circuit, when |A| < |C|, w1 = 0; at the third comparator of the first comparison circuit, when |A| ≥ |D|, w2 = 1. Therefore, the output result of the first comparison circuit is {A, w0, w1, w2}, which is {A, 1, 0, 1} at this time. Similarly, the output result of the second comparison circuit is {B, x0, x1, x2}, the output result of the third comparison circuit is {C, y0, y1, y2}, and the output result of the fourth comparison circuit is {D, z0, z1, z2}.

[0139] The fourth pipeline stage includes a filter circuit 980, which selects n data elements with larger absolute values ​​from the m data elements as valid data elements based on the comparison results of the third stage, and outputs these valid data elements and their corresponding indexes. The index indicates the position of these valid data elements within the m input data elements. For example, when A and C are selected from the four data elements A, B, C, and D, their corresponding indexes may be 0 and 2.

[0140] Based on the comparison results, appropriate logic can be designed to select n data elements with larger absolute values. Considering the possibility of multiple data elements with the same absolute value, in a further embodiment, when there are data elements with the same absolute value, they are selected according to a specified priority order. For example, the priority can be fixed from low to high in the index order, with A having the highest priority and D having the lowest priority. In one example, when the absolute values ​​of A, C, and D are the same and greater than the absolute value of B, the data selected are A and C.

[0141] From the previous comparison results, we can see that based on w0, w1, and w2, |A| is greater than {|B|, |C|, |D|}. If w0, w1, and w2 are all 1, |A| is greater than |B|, |C|, and |D|, and is the maximum of the four numbers, so A is selected. If two of w0, w1, and w2 are 1, |A| is the second largest of the four absolute values, so A is also selected. Otherwise, A is not selected. Therefore, in some embodiments, analysis and judgment can be made based on the number of occurrences of these values.

[0142] In one implementation, valid data elements can be selected based on the following logic. First, the number of times each data element is greater than other data elements can be counted. For example, define N A =sum_w=w0+w1+w2, N B =sum_x=x0+x1+x2, N C =sum_y=y0+y1+y2, N D=sum_z=z0+z1+z2. Next, make a judgment and selection according to the following conditions.

[0143] The condition for selecting A is: N A =3, or N A =2 and N B / N C / N D There is only one 3 in it;

[0144] The condition for selecting B is: N B =3, or N B =2 and N A / N C / N D There is only one 3 in N A ≠2;

[0145] The conditions for selecting C are: N C =3, and N A / N B There is at most one 3, or N C =2 and N A / N B / N D There is only one 3 in N A / N B There is no 2 in

[0146] The conditions for selecting D are: N D =3, and N A / N B / N C There is at most one 3, or N D =2 and N A / N B / N C There is only one 3 in N A / N B / N C There is no 2 in it.

[0147] Those skilled in the art will appreciate that in order to ensure selection according to a predetermined priority, there is a certain degree of redundancy in the above logic. Based on the size and order information provided by the comparison results, those skilled in the art can design other logic to implement the screening of valid data elements, and this disclosure is not limited in this respect. Figure 9B The multi-stage pipeline operation circuit can also realize four-choose-two structured sparse processing.

[0148] Those skilled in the art will appreciate that other forms of pipeline operation circuits may be designed to implement structured sparse processing, and the present disclosure is not limited in this regard.

[0149] As mentioned above, the operands of the sparse instruction can be data in the neural network, such as weights, neurons, etc. The data in the neural network usually contains multiple dimensions. For example, in a convolutional neural network, the data may have four dimensions: input channel, output channel, length and width. In some embodiments, the above-mentioned sparse instruction can be used for structured sparse processing of at least one dimension of multidimensional data in the neural network. Specifically, in one implementation, the sparse instruction can be used for structured sparse processing of the input channel dimension of multidimensional data in the neural network, such as in the inference process or forward training process of the neural network. In another implementation, the sparse instruction can be used to perform structured sparse processing on the input channel dimension and the output channel dimension of the multidimensional data in the neural network at the same time, such as in the reverse training process of the neural network.

[0150] In one embodiment, in response to receiving multiple sparse instructions, one or more multi-stage pipeline arithmetic circuits of the present disclosure may be configured to perform multiple data operations, such as single instruction multiple data ("SIMD") instructions. In another embodiment, the multiple operations performed by each stage of the arithmetic circuits are predetermined based on the functions supported by the multiple arithmetic circuits arranged stage by stage in the multi-stage arithmetic pipeline.

[0151] In the context of the present disclosure, the aforementioned multiple sparse instructions may be microinstructions or control signals running within one or more multi-stage operation pipelines, which may include (or indicate) one or more operations to be performed by the multi-stage operation pipeline. Depending on different operation scenarios, the operation may include but is not limited to arithmetic operations such as convolution operations and matrix multiplication operations, logical operations such as AND operations, XOR operations, and OR operations, shift operations, or any combination of the aforementioned types of operations.

[0152] Figure 10 FIG. 1 shows an exemplary flow chart of a data processing method 1000 according to an embodiment of the present disclosure.

[0153] like Figure 10 As shown, in step 1010, a sparse instruction is parsed, the sparse instruction indicating an operation related to structured sparseness, and at least one operand of the sparse instruction includes at least one descriptor, the descriptor indicating at least one of the following information: shape information of tensor data and spatial information of tensor data. This step can be performed, for example, by Figure 8 The control circuit 810 is used to execute.

[0154] Next, in step 1020, the descriptor is parsed. This step can be performed by, for example Figure 8Specifically, the data address of the tensor data corresponding to the operand in the data storage space can be determined based on the shape information of the tensor data; and / or the dependency relationship between instructions can be determined based on the space information of the tensor data.

[0155] Next, in step 1030, the corresponding operand is read based at least in part on the parsed descriptor. When the operand is tensor data, the data address can be obtained according to the parsed descriptor to read the corresponding data. Sparse instructions can indicate different operation modes, and the corresponding operands are also different, which will be described in detail later. This step can be, for example, Figure 8 The control circuit 810 is executed for the storage circuit 820.

[0156] Next, in step 1030, operations related to structured sparsity are performed on the read operands. This step can be performed by, for example Figure 8 The operation circuit 830 is used to execute.

[0157] Finally, in step 1040 , the operation result is output. For example, the operation result can be output by the operation circuit 830 to the storage circuit 820 for subsequent use.

[0158] There may be various operations related to structured sparsity, such as structured sparsity processing, anti-sparseness processing, etc. Various instruction schemes may be designed to implement operations related to structured sparsity.

[0159] In one solution, a sparse instruction may be designed, and the instruction may include an operation mode bit to indicate different operation modes of the sparse instruction, thereby performing different operations.

[0160] In another solution, multiple sparse instructions can be designed, each corresponding to one or more different operating modes, thereby performing different operations. In one implementation, a corresponding sparse instruction can be designed for each operating mode. In another implementation, the operating modes can be classified according to their characteristics, and a sparse instruction can be designed for each type of operating mode. Furthermore, when a certain type of operating mode includes multiple operating modes, an operating mode bit can be included in the sparse instruction to indicate the corresponding operating mode.

[0161] Regardless of which solution is adopted, the sparse instruction may indicate its corresponding operation mode through an operation mode bit and / or the instruction itself.

[0162] In one embodiment, the sparse instruction may indicate a first operating mode. In the first operating mode, the operand of the sparse instruction includes data to be sparsed. In this case, the operation circuit 830 may be configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and output the structured data after the sparse processing to the storage circuit 820.

[0163] The structured sparse processing in the first operating mode can be structured sparse processing based on a predetermined filtering rule, such as filtering out n data elements with larger absolute values ​​from every m data elements as valid data elements. The arithmetic circuit 830 can be configured, for example, as the pipeline arithmetic circuit described with reference to FIG. 9 to perform this structured sparse processing.

[0164] The result of the sparsification process consists of two parts: a data part and an index part. The data part includes the data after the sparsification process, that is, the valid data elements extracted according to the screening rules of the structured sparsification process. The index part is used to indicate the data after sparsification, that is, the location of the valid data elements in the data before sparsification (that is, the data to be sparsified).

[0165] The structure in the disclosed embodiments includes a data portion and an index portion that are bound to each other. In some embodiments, each bit in the index portion can correspond to a data element. For example, when the data type is fix8, a data element is 8 bits, and each bit in the index portion can correspond to 8 bits of data. In other embodiments, considering the hardware implementation when the structure is subsequently used, each bit in the index portion of the structure can be set to correspond to a position of N bits of data, where N is determined at least in part based on the hardware configuration. For example, each bit in the index portion of the structure can be set to correspond to a position of 4 bits of data. For example, when the data type is fix8, every 2 bits in the index portion correspond to a data element of the fix8 type. In some embodiments, the data portion of the structure can be aligned according to a first alignment requirement, and the index portion of the structure can be aligned according to a second alignment requirement, so that the entire structure also meets the alignment requirement. For example, the data portion can be aligned according to 64B, the index portion can be aligned according to 32B, and the entire structure can be aligned according to 96B (64B + 32B). This alignment requirement can reduce the number of memory accesses during subsequent use and improve processing efficiency.

[0166] By using this structure, the data and index components can be used together. Since the ratio of valid data elements to all data elements in structured sparse processing is fixed, for example, n / m, the size of the data after sparse processing is also fixed or predictable. As a result, the structure can be stored densely in the storage circuit without performance loss.

[0167] In another embodiment, the sparse instruction may indicate a second operation mode. The second operation mode differs from the first operation mode in that the second operation mode outputs only the data portion after structured sparse processing, but does not output the index portion.

[0168] Similarly, in the second operating mode, the operand of the sparse instruction includes the data to be sparsed. In this case, the operation circuit 830 can be configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output the data portion after the sparse processing to the storage circuit 820. The data portion includes the data after the sparse processing of the data to be sparsed. The data portion is densely stored in the storage circuit. The output data portion is aligned according to n elements. For example, in the example of m=4 and n=2, the input data to be sparsed is aligned according to 4 elements, while the output data portion is aligned according to 2 elements.

[0169] In another embodiment, the sparse instruction may indicate a third operation mode. The third operation mode differs from the first operation mode in that the output content is different, and the third operation mode only outputs the index part after structured sparse processing, but does not output the data part.

[0170] Similarly, in the third operating mode, the operand of the sparse instruction includes the data to be sparsified. At this time, the operation circuit 830 can be configured to perform structured sparse processing on the data to be sparsified according to the sparse instruction, and output the index part after the sparsification processing to the storage circuit 820. The index part indicates the original position of the sparse data in the data to be sparsified. The index part is densely stored in the storage circuit. Each bit in the output index part corresponds to the position of a data element. Since the index part can be used alone, for example, for structured sparsification of neurons in subsequent convolution processing, and the data type of the neurons may be uncertain, by corresponding each bit in the index part to the position of a data element, the independently stored index part can be applicable to various data types.

[0171] In another embodiment, the sparse instruction can indicate a fourth operating mode. The difference between the fourth operating mode and the first operating mode is that the fourth operating mode specifies the screening rules of the structured sparse processing, rather than performing the structured sparse processing according to a predetermined screening rule (for example, the previous absolute value larger rule). At this time, the sparse instruction has two operands: the data to be sparsed and the sparse index. The added sparse index operand is used to indicate the position of the valid data elements in the structured sparse processing to be executed, that is, to specify the screening rules of the structured sparse processing. Each bit in the sparse index corresponds to the position of a data element, and thus can be applied to data to be sparsed of various data types.

[0172] In the fourth operating mode, the operation circuit 830 can be configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and the location indicated by the sparse index, and output the result of the sparse processing to the storage circuit. In one implementation, the output result can be a structure after the sparse processing. In another implementation, the output result can be a portion of the data after the sparse processing.

[0173] The structure has the same meaning as in the first operating mode. It consists of a bound data portion and an index portion. The data portion contains the data after the sparsification process, and the index portion indicates the original position of the sparsified data within the data to be sparsified. The alignment requirements and corresponding relationships between the data and index portions in the structure are the same as in the first operating mode and are not repeated here.

[0174] The above four operation modes provide structured sparse processing of data, such as processing according to predetermined filtering rules or filtering rules specified by the instruction operands, and provide different output contents, such as outputting a structure, outputting only the data portion, outputting only the index portion, etc. The above instruction design can well support structured sparse processing and provides multiple output options to suit different scenario requirements. For example, when data and indexes need to be bound together, you can choose to output a structure, while when the index portion or data portion needs to be used separately, you can choose to output only the index portion or the data portion.

[0175] In another embodiment, the sparse instruction may indicate a fifth operation mode. The fifth operation mode does not require structured sparse processing, but only requires binding separate or independent data parts and index parts into a structure.

[0176] In the fifth operating mode, the operand of the sparse instruction includes the data portion after the sparsification process and the corresponding index portion. The data portion and the index portion are respectively in dense storage format, but are not bound. The input data portion is aligned according to n elements. For example, in the example of m=4, n=2, the input data portion is aligned according to 2 elements. The index portion indicates the original position of the data portion in the data before the sparsification process, where each bit of the index portion corresponds to a data element.

[0177] At this time, the operation circuit 830 can be configured to bind the data part and the index part into a structure according to the sparse instruction, and output the structure to the storage circuit. The meaning of the structure, the alignment requirements for the data part and the index part, the corresponding relationship, etc. are the same as those in the first operation mode and will not be repeated here. Depending on the data type of the data element, it is necessary to generate the index part in the structure accordingly based on the data type and the bit correspondence of the index part in the structure. For example, when the input index part is 0011, where each bit corresponds to one data element, if the data type is fix8, that is, each data element has 8 bits, then according to the corresponding relationship between each 1 bit of the index part in the structure and 4 bits of data, the index part in the structure should be: 00001111, that is, 2 bits correspond to one data element.

[0178] In yet another embodiment, the sparse instruction may indicate a sixth operation mode for performing a desparse process, that is, restoring the sparse data to the format or size of the data before the sparse process.

[0179] In the sixth operating mode, the operands of the sparse instruction include the data portion after the sparsification process and the corresponding index portion. The data portion and the index portion are respectively in dense storage format, but are not bound. The input data portion is aligned according to n elements. For example, in the example of m=4, n=2, the input data portion is aligned according to 2 elements, and the output data is aligned according to 4 elements. The index portion indicates the original position of the data portion in the data before the sparsification process, where each bit of the index portion corresponds to a data element.

[0180] At this time, the operation circuit 830 can be configured to perform desparse processing on the input data part according to the sparse instruction and the position indicated by the input index part to generate recovery data having the data format before the sparse processing, and output the recovery data to the storage circuit.

[0181] In one implementation, the desparseening process may include: placing each data element in the data part at a corresponding position in the data format before the sparseening process according to the position indicated by the index part and in accordance with the data format before the sparseening process, and filling predetermined information (for example, filling 0) at the remaining positions of the data format to generate recovered data.

[0182] As can be seen from the above description, the present disclosure provides a sparse instruction for performing operations related to structured sparsity. These operations may include forward structured sparse operations, may also include desparse operations, and may also include some related format conversion operations. In some embodiments, the sparse instruction may include an operation mode bit to indicate different operation modes of the sparse instruction, thereby performing different operations. In other embodiments, a plurality of sparse instructions may be directly provided, each instruction corresponding to one or more different operation modes, thereby performing various operations related to structured sparsity. By providing special sparse instructions to perform operations related to structured sparsity, processing can be simplified, thereby improving the processing efficiency of the machine.

[0183] According to different application scenarios, the electronic devices or devices disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, Internet of Things terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, vehicles, household appliances, and / or medical equipment. The vehicles include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs. The electronic devices or devices disclosed herein may also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical care and other fields. Furthermore, the electronic devices or devices disclosed herein may also be used in cloud, edge, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, electronic devices or apparatuses with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or apparatuses with low power consumption can be applied to terminal devices and / or edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0184] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teachings of the present disclosure, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present disclosure can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present disclosure. In addition, depending on the different schemes, the description of some embodiments of the present disclosure also has different emphases. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0185] In terms of specific implementation, based on the disclosure and teachings of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented in other ways not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them based on the consideration of logical functions, and there may be other ways of splitting them in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0186] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solution described in the embodiments of this disclosure. In addition, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically separately.

[0187] In other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, that is, as specific hardware circuits, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage circuits or storage devices may be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which may be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0188] The foregoing content can be better understood in accordance with the following terms:

[0189] Clause 1. A data processing apparatus comprising:

[0190] a control circuit configured to parse a sparse instruction, the sparse instruction indicating an operation related to structured sparsity, and at least one operand of the sparse instruction including at least one descriptor indicating at least one of the following information: shape information of tensor data and spatial information of the tensor data;

[0191] a tensor interface circuit configured to parse the descriptor;

[0192] a storage circuit configured to store information before and / or after sparsification; and

[0193] The operation circuit is configured to perform corresponding operations according to the sparse instruction based on the parsed descriptor.

[0194] Clause 2. The data processing apparatus according to clause 1, wherein:

[0195] The tensor interface circuit is configured to determine a data address of tensor data corresponding to the operand in a data storage space according to the shape information; and / or

[0196] The tensor interface circuit is configured to determine dependencies between instructions based on the spatial information.

[0197] Clause 3. The data processing apparatus according to any one of clauses 1-2, wherein the shape information of the tensor data includes at least one shape parameter representing the shape of N-dimensional tensor data, where N is a positive integer, and the shape parameter of the tensor data includes at least one of the following:

[0198] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

[0199] Clause 4. The data processing apparatus according to any one of clauses 1-2, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, where N is a positive integer, and the shape parameter includes at least one of the following:

[0200] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

[0201] Clause 5. A data processing apparatus according to any one of clauses 1 to 4, wherein the sparse instruction indicates a first operating mode, and an operand of the sparse instruction includes data to be sparsified,

[0202] The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output a structure after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0203] Clause 6. A data processing apparatus according to any one of clauses 1 to 4, wherein the sparse instruction indicates a second operating mode, and an operand of the sparse instruction includes data to be sparsified,

[0204] The operation circuit is configured to perform structured sparse processing on the data to be thinned out according to the sparse instruction, and output the data portion after the sparse processing to the storage circuit, where the data portion includes the data after the sparse processing of the data to be thinned out.

[0205] Clause 7. A data processing apparatus according to any one of clauses 1 to 4, wherein the sparse instruction indicates a third operating mode, and an operand of the sparse instruction includes data to be sparsified,

[0206] The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output an index part after the sparse processing to the storage circuit, where the index part indicates the position of the sparse data in the data to be sparse.

[0207] Clause 8. A data processing apparatus according to any one of clauses 1 to 4, wherein the sparse instruction indicates a fourth operating mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in the structured sparsification to be performed,

[0208] The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and the position indicated by the sparse index, and output the structure after sparse processing or the data part after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0209] Clause 9. A data processing apparatus according to any one of clauses 1 to 4, wherein the sparse instruction indicates a fifth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification processing,

[0210] The operation circuit is configured to bind the data part and the index part into a structure according to the sparse instruction, and output the structure to the storage circuit.

[0211] Clause 10. A data processing apparatus according to any one of clauses 1 to 4, wherein the sparse instruction indicates a sixth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification processing,

[0212] The operation circuit is configured to perform inverse sparsification processing on the data portion according to the sparse instruction and the position indicated by the index portion to generate restored data having the data format before the sparsification processing, and output the restored data to the storage circuit.

[0213] Clause 11. A data processing apparatus according to any one of clauses 5 to 8, wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

[0214] Item 12. A data processing device according to Item 11, wherein the operation circuit further includes: at least one multi-stage pipeline operation circuit, which includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

[0215] Clause 13. The data processing apparatus according to clause 12, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein:

[0216] The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values;

[0217] The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different;

[0218] The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; and

[0219] The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

[0220] Item 14. A data processing device according to Item 13, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

[0221] Clause 15. The data processing apparatus according to any one of clauses 13-14, wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them in a specified priority order.

[0222] Clause 16. The data processing apparatus according to clause 10, wherein the desparseness processing comprises:

[0223] According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

[0224] Clause 17. A data processing apparatus according to clause 5, 8 or 9, wherein

[0225] Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or

[0226] The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

[0227] Clause 18. A data processing apparatus according to any one of clauses 1 to 17, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

[0228] Clause 19. The data processing apparatus of clause 18, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

[0229] Clause 20. A data processing apparatus according to any one of clauses 1 to 19, wherein

[0230] The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or

[0231] The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

[0232] Clause 21. A chip comprising the data processing device according to any one of clauses 1-20.

[0233] Clause 22. A board comprising the chip according to clause 21.

[0234] Article 23. A data processing method comprising:

[0235] Parsing a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparsity, and at least one operand of the sparse instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of the tensor data;

[0236] parsing the descriptor;

[0237] Based at least in part on the parsed descriptor, reading a corresponding operand;

[0238] performing the structured sparsity-related operation on the operand; and

[0239] Output the operation results.

[0240] Clause 24. The data processing method according to Clause 23, wherein parsing the descriptor comprises:

[0241] Determining, based on the shape information, a data address of the tensor data corresponding to the operand in the data storage space; and / or

[0242] Dependencies between instructions are determined based on the spatial information.

[0243] Clause 25. The data processing method according to any one of Clauses 23-24, wherein the shape information of the tensor data includes at least one shape parameter representing the shape of N-dimensional tensor data, N is a positive integer, and the shape parameter of the tensor data includes at least one of the following:

[0244] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

[0245] Clause 26. The data processing method according to any one of clauses 23-24, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, N being a positive integer, and the shape parameter comprises at least one of the following:

[0246] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

[0247] Clause 27. A data processing method according to any one of clauses 23 to 26, wherein the sparse instruction indicates a first operating mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising:

[0248] According to the sparse instruction, performing structured sparse processing on the data to be sparsed; and

[0249] Output the structure after the sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after the sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0250] Clause 28. A data processing method according to any one of clauses 23 to 26, wherein the sparse instruction indicates a second operating mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising:

[0251] According to the sparse instruction, performing structured sparse processing on the data to be sparsed; and

[0252] Output the data portion after the thinning process, where the data portion includes the data after the thinning process of the data to be thinned.

[0253] Clause 29. A data processing method according to any one of clauses 23 to 26, wherein the sparse instruction indicates a third operating mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising:

[0254] According to the sparse instruction, performing structured sparse processing on the data to be sparsed; and

[0255] The index portion after the thinning process is output, where the index portion indicates the position of the thinned data in the data to be thinned.

[0256] Clause 30. A data processing method according to any one of clauses 23 to 26, wherein the sparse instruction indicates a fourth operating mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in the structured sparsification to be performed, and the method further comprises:

[0257] According to the sparse instruction, performing structured sparse processing on the data to be sparsed according to the position indicated by the sparse index; and

[0258] Output the structure after sparse processing or the data part after sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0259] Clause 31. A data processing method according to clauses 23-26, wherein the sparse instruction indicates a fifth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification process and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification process, and the method further comprises:

[0260] Binding the data portion and the index portion into a structure according to the sparse instruction; and

[0261] Output the structure.

[0262] Clause 32. A data processing method according to clauses 23-26, wherein the sparse instruction indicates a sixth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification process and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification process, and the method further comprises:

[0263] According to the sparse instruction, performing an inverse sparse process on the data portion according to the position indicated by the index portion to generate restored data having a data format before the sparse process; and

[0264] The restored data is output.

[0265] Clause 33. A data processing method according to any one of clauses 27-30, wherein the structured sparse processing includes selecting n data elements as valid data elements from every m data elements, where m>n.

[0266] Item 34. A data processing method according to Item 33, wherein the structured sparse processing is implemented using an operation circuit, the operation circuit comprising: at least one multi-stage pipeline operation circuit, which includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

[0267] Clause 35. The data processing method according to clause 34, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein:

[0268] The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values;

[0269] The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different;

[0270] The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; and

[0271] The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

[0272] Clause 36. A data processing method according to Clause 35, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

[0273] Clause 37. A data processing method according to any one of clauses 35-36, wherein the screening circuit is further configured to select data elements according to a specified priority order when there are data elements with the same absolute value.

[0274] Clause 38. The data processing method according to clause 32, wherein the desparseness processing comprises:

[0275] According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

[0276] Clause 39. A data processing method according to clause 27, 30 or 31, wherein:

[0277] Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or

[0278] The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

[0279] Clause 40. A data processing method according to any one of clauses 23-39, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

[0280] Clause 41. A data processing method according to clause 41, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

[0281] Clause 42. A data processing method according to any one of Clauses 23 to 41, wherein

[0282] The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or the sparse instruction includes multiple instructions, each instruction corresponding to one or more different operation modes.

[0283] The above is a detailed introduction to the embodiments of the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, based on the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A data processing device, comprising: a control circuit configured to parse a sparse instruction, the sparse instruction indicating an operation related to structured sparsity, and at least one operand of the sparse instruction including at least one descriptor indicating at least one of the following information: shape information of tensor data and spatial information of the tensor data; a tensor interface circuit configured to parse the descriptor; a storage circuit configured to store information before and / or after sparsification; as well as an operation circuit configured to perform a corresponding operation according to the sparse instruction based on the parsed descriptor; The structured sparse processing includes selecting n data elements from every m data elements as valid data elements, where m>n.

2. The data processing apparatus according to claim 1, wherein: The tensor interface circuit is configured to determine a data address of tensor data corresponding to the operand in a data storage space according to the shape information; and / or The tensor interface circuit is configured to determine dependencies between instructions based on the spatial information.

3. The data processing apparatus according to claim 1 , wherein the shape information of the tensor data comprises at least one shape parameter representing the shape of N-dimensional tensor data, where N is a positive integer, and the shape parameter of the tensor data comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

4. The data processing apparatus according to any one of claims 1 to 2, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, where N is a positive integer, and the shape parameter comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

5. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a first operation mode, and an operand of the sparse instruction includes data to be sparsed. The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output a structure after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

6. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a second operation mode, and an operand of the sparse instruction includes data to be sparsed. The operation circuit is configured to perform structured sparse processing on the data to be thinned out according to the sparse instruction, and output the data portion after the sparse processing to the storage circuit, where the data portion includes the data after the sparse processing of the data to be thinned out.

7. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a third operation mode, and an operand of the sparse instruction includes data to be sparsed. The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output an index part after the sparse processing to the storage circuit, where the index part indicates the position of the sparse data in the data to be sparse.

8. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a fourth operation mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in the structured sparse operation to be performed. The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and the position indicated by the sparse index, and output the structure after sparse processing or the data part after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

9. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a fifth operation mode, and an operand of the sparse instruction includes a data portion after the sparse processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparse processing. The operation circuit is configured to bind the data part and the index part into a structure according to the sparse instruction, and output the structure to the storage circuit.

10. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a sixth operation mode, and an operand of the sparse instruction includes a data portion after the sparse processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparse processing. The operation circuit is configured to perform inverse sparsification processing on the data portion according to the sparse instruction and the position indicated by the index portion to generate restored data having the data format before the sparsification processing, and output the restored data to the storage circuit.

11. The data processing device according to claim 5, wherein the operation circuit further comprises: At least one multi-stage pipeline operation circuit includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

12. The data processing apparatus according to claim 11, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein: The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values; The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different; The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; as well as The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

13. The data processing device according to claim 12, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m. 14 . The data processing apparatus according to claim 12 , wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them according to a specified priority order.

15. The data processing apparatus according to claim 10, wherein the desparseness processing comprises: According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

16. The data processing apparatus according to claim 5, wherein Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

17. The data processing apparatus according to any one of claims 1-2, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

18. The data processing apparatus according to claim 17, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

19. The data processing device according to any one of claims 1 to 2, wherein The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

20. A chip comprising the data processing device according to any one of claims 1 to 19.

21. A board comprising the chip according to claim 20.

22. A data processing method, comprising: Parsing a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparsity, and at least one operand of the sparse instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of the tensor data; parsing the descriptor; Based at least in part on the parsed descriptor, reading a corresponding operand; performing the structured sparsity-related operation on the operand; as well as Output the operation results; Structured sparse processing involves selecting n data elements from every m data elements as valid data elements, where m>n; The data processing method is executed by the data processing device according to any one of claims 1-19.

23. The data processing method according to claim 22, wherein: Parsing the descriptor includes: Determining, based on the shape information, a data address of the tensor data corresponding to the operand in the data storage space; and / or Dependencies between instructions are determined based on the spatial information.

24. The data processing method according to any one of claims 22-23, wherein the shape information of the tensor data comprises at least one shape parameter representing the shape of N-dimensional tensor data, N being a positive integer, and the shape parameter of the tensor data comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

25. The data processing method according to any one of claims 22 to 23, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, N is a positive integer, and the shape parameter comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

26. The data processing method according to any one of claims 22 to 23, wherein the sparse instruction indicates a first operation mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising: According to the sparse instruction, performing structured sparse processing on the data to be sparse; as well as Output the structure after the sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after the sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

27. The data processing method according to any one of claims 22 to 23, wherein the sparse instruction indicates a second operation mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising: According to the sparse instruction, performing structured sparse processing on the data to be sparse; as well as Output the data portion after the thinning process, where the data portion includes the data after the thinning process of the data to be thinned.

28. The data processing method according to any one of claims 22 to 23, wherein the sparse instruction indicates a third operation mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising: According to the sparse instruction, performing structured sparse processing on the data to be sparse; as well as The index portion after the thinning process is output, where the index portion indicates the position of the thinned data in the data to be thinned.

29. The data processing method according to any one of claims 22-23, wherein the sparse instruction indicates a fourth operation mode, and operands of the sparse instruction include data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in the structured sparse operation to be performed, and the method further comprises: According to the sparse instruction, and according to the position indicated by the sparse index, performing structured sparse processing on the data to be sparsed; as well as Output the structure after sparse processing or the data part after sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

30. The data processing method according to any one of claims 22-23, wherein the sparse instruction indicates a fifth operation mode, and an operand of the sparse instruction includes a data portion after the sparsification processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification processing, and the method further comprises: Binding the data portion and the index portion into a structure according to the sparse instruction; as well as Output the structure.

31. The data processing method according to any one of claims 22-23, wherein the sparse instruction indicates a sixth operation mode, and an operand of the sparse instruction includes a data portion after the sparsification processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification processing, and the method further comprises: According to the sparse instruction, performing an inverse sparsification process on the data portion according to the position indicated by the index portion to generate restored data having a data format before the sparsification process; as well as The restored data is output.

32. The data processing method according to claim 26, wherein the structured sparse processing is implemented using an operation circuit, the operation circuit comprising: At least one multi-stage pipeline operation circuit includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

33. The data processing method according to claim 32, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein: The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values; The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different; The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; as well as The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

34. The data processing method according to claim 33, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

35. The data processing method according to claim 33, wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them according to a specified priority order.

36. The data processing method according to claim 31, wherein the desparseness processing comprises: According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

37. The data processing method according to claim 26, wherein: Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

38. The data processing method according to any one of claims 22-23, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

39. The data processing method according to claim 38, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

40. The data processing method according to any one of claims 22-23, wherein The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

Images