Data processing device, data processing method and related products

Abstract

The present disclosure discloses a data processing device, a data processing method, and related products. The data processing device can be implemented as a computing device included in a combined processing device, and the combined processing device can also include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete the computing operations specified by the user. The combined processing device can also include a storage device, which is connected to the computing device and the other processing devices respectively and is used to store data from the computing device and the other processing devices. The solution disclosed in the present disclosure provides instructions for structured sparse convolution operations on tensor data, which can simplify processing and improve the processing efficiency of the machine.

Description

Technical Field

[0001] The present disclosure generally relates to the field of processors and, more particularly, to data processing devices, data processing methods, chips, and boards. Background Art

[0002] In recent years, the rapid development of deep learning has led to significant advances in algorithm performance across a range of fields, including computer vision and natural language processing. However, deep learning algorithms are computationally and storage-intensive. As information processing tasks become increasingly complex and the demands for real-time and accurate algorithms continue to rise, neural networks are often designed to be deeper, increasing the computational load and storage requirements. This makes existing deep learning-based artificial intelligence technologies difficult to directly apply to hardware-constrained mobile phones, satellites, or embedded devices.

[0003] Therefore, the compression, acceleration, and optimization of deep neural network models have become extremely important. Numerous studies have attempted to reduce the computational and storage requirements of neural networks without compromising model accuracy. This is of crucial importance for the engineering application of deep learning technology on embedded and mobile devices. Sparsification is one such method for achieving model lightweighting.

[0004] Network parameter sparsification is the process of reducing redundant components in a larger network through appropriate methods to reduce the network's computational load and storage requirements. Existing hardware and / or instruction sets cannot effectively support sparsification processing and operations related to sparsification. Summary of the Invention

[0005] In order to at least partially solve one or more technical problems mentioned in the background technology, the solution disclosed herein provides a data processing device, a data processing method, a chip and a board.

[0006] In a first aspect, the present disclosure discloses a data processing device, comprising: a control circuit configured to parse a convolution instruction, the convolution instruction including a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction including at least one descriptor, the descriptor indicating at least one of the following information: shape information of tensor data and spatial information of tensor data; a tensor interface circuit configured to parse the descriptor; a storage circuit configured to store information before and / or after convolution; and an operation circuit configured to perform a corresponding convolution operation according to the convolution instruction based on the parsed descriptor.

[0007] In a second aspect, the present disclosure provides a chip comprising the data processing device of any embodiment of the aforementioned first aspect.

[0008] In a third aspect, the present disclosure provides a board comprising the chip of any one of the embodiments of the second aspect.

[0009] In a fourth aspect, the present disclosure provides a data processing method, which includes: parsing a convolution instruction, the convolution instruction including a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction includes at least one descriptor, the descriptor indicating at least one of the following information: shape information of tensor data and spatial information of tensor data; reading a corresponding operand according to the convolution instruction; and performing a corresponding convolution operation on the operand according to the convolution instruction.

[0010] Through the data processing device, data processing method, integrated circuit chip and board provided above, the disclosed embodiment provides a convolution instruction for performing a structured sparse convolution operation of tensor data, wherein the tensor data is described by a descriptor. In some embodiments, the convolution instruction includes a sparse flag to indicate whether a structured sparse convolution operation is performed. By setting the sparse flag, the corresponding arithmetic circuit can be configured according to the value of the flag to perform the corresponding convolution operation. In some embodiments, when the sparse flag indicates the execution of a structured sparse convolution operation, the arithmetic circuit can be configured to perform structured sparse processing, and then convolution is performed on the sparsified data. By reusing the instruction field of the convolution instruction and adding a structured sparse enable flag, the processing can be simplified, thereby improving the processing efficiency of the machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an illustrative and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0012] Figure 1 is a structural diagram showing a board according to an embodiment of the present disclosure;

[0013] Figure 2 is a structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram showing the internal structure of a multi-core computing device according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram showing the internal structure of a processor core according to an embodiment of the present disclosure;

[0017] Figure 6 A schematic diagram showing a data storage space according to an embodiment of the present disclosure;

[0018] Figure 7 A schematic diagram showing data blocks in a data storage space according to an embodiment of the present disclosure;

[0019] Figure 8 is a schematic diagram showing the structure of a data processing device according to an embodiment of the present disclosure;

[0020] Figures 9A-9C A schematic diagram showing a portion of the structure of the arithmetic circuit according to an embodiment of the present disclosure;

[0021] Figure 10 An exemplary pipeline circuit diagram illustrating structured sparse processing according to an embodiment of the present disclosure; and

[0022] Figure 11 is an exemplary flow chart illustrating a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following will clearly and completely describe the technical solutions in the embodiments of this disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this disclosure, not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this disclosure.

[0024] It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0025] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0027] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0028] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1 As shown, board 10 includes chip 101, which is a system-on-chip (SoC), or system-on-chip, integrated with one or more combined processing devices. The combined processing device is an artificial intelligence computing unit that supports various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud-based intelligent applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. Board 10 of this embodiment is suitable for cloud-based intelligent applications and has extensive off-chip storage, on-chip storage, and powerful computing capabilities.

[0029] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. Calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface formats, such as a PCIe interface.

[0030] Board 10 also includes a memory device 104 for storing data, which includes one or more storage units 105. Memory device 104 is connected to a control device 106 and chip 101 via a bus for data transmission. Control device 106 in board 10 is configured to control the state of chip 101. To this end, in one application scenario, control device 106 may include a microcontroller (MCU).

[0031] Figure 2 FIG. 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0032] The computing device 201 is configured to perform user-specified operations and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0033] Interface device 202 is used to transmit data and control instructions between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to a storage device on-chip of computing device 201. Furthermore, computing device 201 can obtain control instructions from processing device 203 via interface device 202 and write them to a control cache on-chip of computing device 201. Alternatively or optionally, interface device 202 can also read data from the storage device of computing device 201 and transmit it to processing device 203.

[0034] The processing device 203 is a general processing device that performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, which include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, with respect to the computing device 201 disclosed herein, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0035] The storage device 204 is used to store data to be processed, which may be DRAM or DDR memory, and is typically 16G or larger in size, for storing data of the computing device 201 and / or the processing device 203 .

[0036] Figure 3The single-core computing device 301 is used to process input data for computer vision, speech, natural language, data mining, etc. The single-core computing device 301 includes three modules: a control module 31, a computing module 32, and a storage module 33.

[0037] The control module 31 coordinates and controls the operations of the computing module 32 and the storage module 33 to complete deep learning tasks. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 retrieves instructions from the processing device 203, while the instruction decode unit 312 decodes the retrieved instructions and sends the decoded results as control information to the computing module 32 and the storage module 33.

[0038] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations, supporting complex operations such as vector multiplication, addition, and nonlinear transformations. The matrix operation unit 322 is responsible for the core calculations of the deep learning algorithm, namely matrix multiplication and convolution.

[0039] The storage module 33 is used to store or transfer relevant data and includes a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access (DMA) module 333. NRAM 331 stores input neurons, output neurons, and intermediate computational results; WRAM 332 stores the convolution kernels (i.e., weights) of the deep learning network; and DMA 333, which connects to DRAM 204 via bus 34, transfers data between the single-core computing device 301 and DRAM 204.

[0040] Figure 4 The figure shows the internal structure of a multi-core computing device 201. The multi-core computing device 41 employs a layered design. As a system-on-chip (SoC), the multi-core computing device 41 includes at least one cluster, each of which includes multiple processor cores. In other words, the multi-core computing device 41 is structured in a hierarchy of SoC-cluster-processor cores.

[0041] At the system-on-chip level, Figure 4 As shown, the multi-core computing device 41 includes an external storage controller 401 , a peripheral communication module 402 , an on-chip interconnect module 403 , a synchronization module 404 and multiple clusters 405 .

[0042] There can be multiple external storage controllers 401, and two are shown in the figure as an example. They are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 2 DRAM204 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 402 is used to receive control signals from the processing device 203 through the interface device 202, and start the computing device 201 to perform tasks. The on-chip interconnect module 403 connects the external storage controller 401, the peripheral communication module 402 and multiple clusters 405 to transmit data and control signals between each module. The synchronization module 404 is a global synchronization barrier controller (GBC) used to coordinate the work progress of each cluster and ensure information synchronization. Multiple clusters 405 are the computing cores of the multi-core computing device 41. Four are shown as an example in the figure. With the development of hardware, the multi-core computing device 41 disclosed in this disclosure can also include 8, 16, 64, or even more clusters 405. Clusters 405 are used to efficiently execute deep learning algorithms.

[0043] At the cluster level, Figure 4 As shown, each cluster 405 includes multiple processor cores (IPU cores) 406 and a memory core (MEM core) 407 .

[0044] The figure shows four processor cores 406 as an example, but the present disclosure does not limit the number of processor cores 406. Figure 5 Each processor core 406 is similar to Figure 3 The single-core computing device 301 also includes three major modules: a control module 51, a computing module 52, and a storage module 53. The functions and structures of the control module 51, computing module 52, and storage module 53 are roughly the same as those of the control module 31, computing module 32, and storage module 33, and will not be repeated here. It should be noted that the storage module 53 includes an input / output direct memory access module (IODMA) 533 and a move direct memory access module (MVDMA) 534. The IODMA 533 controls memory access between the NRAM 531 / WRAM 532 and the DRAM 204 via the broadcast bus 409; the MVDMA 534 is used to control memory access between the NRAM 531 / WRAM 532 and the storage unit (SRAM) 408.

[0045] Back to Figure 4The storage core 407 is primarily used for storage and communication, namely, storing shared data or intermediate results between the processor cores 406, and performing communication between the cluster 405 and the DRAM 204, between the clusters 405, and between the processor cores 406. In other embodiments, the storage core 407 has scalar operation capabilities and is used to perform scalar operations.

[0046] The storage core 407 includes SRAM 408, a broadcast bus 409, a cluster direct memory access module (CDMA) 410, and a global direct memory access module (GDMA) 411. SRAM 408 acts as a high-performance data transfer station. Data reused between different processor cores 406 within the same cluster 405 does not need to be obtained from DRAM 204 by each processor core 406. Instead, it is transferred between the processor cores 406 via SRAM 408. The storage core 407 only needs to quickly distribute the reused data from SRAM 408 to multiple processor cores 406, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip input / output access.

[0047] The broadcast bus 409, CDMA 410, and GDMA 411 are used for communication between the processor cores 406, communication between the clusters 405, and data transmission between the clusters 405 and the DRAM 204, respectively. Each of these will be described below.

[0048] Broadcast bus 409 facilitates high-speed communication between processor cores 406 within cluster 405. In this embodiment, broadcast bus 409 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point data transmission (e.g., from one processor core to another), multicast transfers a copy of data from SRAM 408 to a specific number of processor cores 406, and broadcast transfers a copy of data from SRAM 408 to all processor cores 406, a special case of multicast.

[0049] The CDMA 410 is used to control memory access to the SRAM 408 between different clusters 405 within the same computing device 201 .

[0050] GDMA 411 works in conjunction with external memory controller 401 to control memory access from cluster 405's SRAM 408 to DRAM 204, or to read data from DRAM 204 to SRAM 408. As previously mentioned, communication between DRAM 204 and NRAM 431 or WRAM 432 can be achieved via two channels. The first channel directly connects DRAM 204 and NRAM 431 or WRAM 432 via IODAM 433. The second channel first transfers data between DRAM 204 and SRAM 408 via GDMA 411, and then transfers data between SRAM 408 and NRAM 431 or WRAM 432 via MVDMA 534. While the second channel may appear to require more components and a longer data flow, in some embodiments, the bandwidth of the second channel is significantly greater than the first channel. Therefore, communication between DRAM 204 and NRAM 431 or WRAM 432 through the second channel may be more efficient. The embodiments of the present disclosure can select a data transmission channel according to the hardware conditions.

[0051] In other embodiments, the functions of GDMA 411 and IODMA 533 can be integrated into the same component. For ease of description, this disclosure treats GDMA 411 and IODMA 533 as separate components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those disclosed herein, they fall within the scope of protection of this disclosure. Furthermore, the functions of GDMA 411, IODMA 533, CDMA 410, and MVDMA 534 can also be implemented by the same component.

[0052] Traditional processor instructions are designed to perform basic single-data scalar operations. Here, single-data scalar operations refer to each operand of the instruction being a scalar data. However, with the development of artificial intelligence technology, in tasks such as image processing and pattern recognition, the operands are often multi-dimensional vectors (i.e., tensor data). Using only scalar operations cannot enable the hardware to efficiently complete the computing tasks. Therefore, how to efficiently perform multi-dimensional tensor data processing is also a problem that needs to be solved urgently in the current computing field.

[0053] In an embodiment of the present disclosure, a convolution instruction is provided for performing convolution operations related to tensor data, especially structured sparse convolution operations on tensor data. At least one operand of the convolution instruction includes at least one descriptor, through which information related to the tensor data can be obtained. Specifically, the descriptor can indicate at least one of the following information: shape information of the tensor data, spatial information of the tensor data. The shape information of the tensor data can be used to determine the data address of the tensor data corresponding to the operand in the data storage space. The spatial information of the tensor data can be used to determine the dependency between instructions, and thus determine, for example, the execution order of the instructions.

[0054] In one possible implementation, the spatial information of tensor data can be indicated by a spatial identifier (ID). The spatial ID can also be called a spatial alias, which refers to a spatial region used to store the corresponding tensor data. The spatial region can be a continuous space or multiple spaces. This disclosure does not limit the specific composition of the spatial region. Different spatial IDs indicate that there is no dependency between the spatial regions pointed to.

[0055] Various possible implementations of the shape information of tensor data will be described in detail below with reference to the accompanying drawings.

[0056] Tensors can contain a variety of data structures. Tensors can be of different dimensions. For example, a scalar can be considered a 0-dimensional tensor, a vector can be considered a 1-dimensional tensor, and a matrix can be a 2-dimensional or higher tensor. The shape of a tensor includes information such as the dimensions of the tensor and the size of each dimension of the tensor. For example, for a 3D tensor:

[0057] x3=[[[1,2,3],[4,5,6]];[[7,8,9],[10,11,12]]]

[0058] The shape or dimensions of the tensor can be expressed as X3=(2, 2, 3), which means that the three parameters indicate that the tensor is a three-dimensional tensor, with the size of the first dimension being 2, the size of the second dimension being 2, and the size of the third dimension being 3. When storing tensor data in memory, the shape of the tensor data cannot be determined based on its data address (or storage area), and further, related information such as the relationship between multiple tensor data cannot be determined, resulting in low processor access efficiency to the tensor data.

[0059] In one possible implementation, a descriptor can be used to indicate the shape of N-dimensional tensor data, where N is a positive integer, such as N=1, 2, or 3, or zero. The three-dimensional tensor in the above example can be represented by a descriptor as (2, 2, 3). It should be noted that this disclosure does not limit the manner in which a descriptor indicates the shape of a tensor.

[0060] In one possible implementation, the value of N can be determined according to the dimension (also called order) of the tensor data, or it can be set according to the usage requirements of the tensor data. For example, when the value of N is 3, the tensor data is three-dimensional tensor data, and the descriptor can be used to indicate the shape of the three-dimensional tensor data in three dimensions (such as offset, size, etc.). It should be understood that those skilled in the art can set the value of N according to actual needs, and this disclosure does not limit this.

[0061] Although tensor data can be multi-dimensional, because the memory layout is always one-dimensional, there is a correspondence between tensors and storage on the memory. Tensor data is usually allocated in a continuous memory space, that is, tensor data can be expanded one-dimensionally (for example, row-major) and stored in the memory.

[0062] This relationship between a tensor and its underlying storage can be expressed through dimensions such as offset, size, and stride. A dimension's offset refers to the offset relative to a reference position in that dimension. A dimension's size refers to the size of that dimension, or the number of elements in that dimension. A dimension's stride refers to the spacing between adjacent elements in that dimension. For example, the stride of the three-dimensional tensor above is (6, 3, 1), meaning the stride of the first dimension is 6, the stride of the second dimension is 3, and the stride of the third dimension is 1.

[0063] Figure 6 Schematic diagram showing the data storage space according to the embodiment of the present disclosure. Figure 6 As shown, data storage space 61 stores two-dimensional data in a row-major manner, which can be represented by (x, y) (where the X axis is horizontally to the right and the Y axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure). The starting address PA_start (base address) of data storage space 61 is the physical address of the first data block 62. Data block 63 is a portion of the data in data storage space 61. Its offset 65 in the X-axis direction is represented by offset_x, its offset 64 in the Y-axis direction is represented by offset_y, its size in the X-axis direction is represented by size_x, and its size in the Y-axis direction is represented by size_y.

[0064] In one possible implementation, when a descriptor is used to define data block 63, the data reference point of the descriptor can be the first data block of data storage space 61, and the reference address of the descriptor can be agreed to be the starting address PA_start of data storage space 61. The content of the descriptor of data block 63 can then be determined by combining the size ori_x on the X axis and the size ori_y on the Y axis of data storage space 61, as well as the offset offset_y in the Y axis direction, the offset offset_x in the X axis direction, the size size_x in the X axis direction, and the size size_y in the Y axis direction of data block 63.

[0065] In a possible implementation, the following formula (1) can be used to express the content of the descriptor:

[0066]

[0067] It should be understood that although in the above examples, the content of the descriptor represents a two-dimensional space, those skilled in the art can set the specific dimension represented by the content of the descriptor according to actual conditions, and this disclosure does not limit this.

[0068] In one possible implementation, the base address of the data reference point of the descriptor in the data storage space can be agreed upon. Based on the base address, the content of the descriptor of the tensor data is determined according to the positions of at least two vertices at diagonal positions in N dimensional directions relative to the data reference point.

[0069] For example, the data reference point of the descriptor can be agreed to be the reference address PA_base in the data storage space. For example, a data (e.g., data at position (2, 2)) can be selected in the data storage space 61 as the data reference point, and the physical address of the data in the data storage space can be used as the reference address PA_base. The position of the two vertices at the diagonal position relative to the data reference point can be used to determine the reference address PA_base. Figure 6 The content of the descriptor of data block 63 in the data block 63 is determined. First, the positions of at least two vertices at the diagonal positions of the data block 63 relative to the data reference point are determined. For example, the positions of the diagonal vertices from the upper left to the lower right relative to the data reference point are used, where the relative position of the upper left vertex is (x_min, y_min) and the relative position of the lower right vertex is (x_max, y_max). Then, the content of the descriptor of data block 63 can be determined based on the reference address PA_base, the relative position of the upper left vertex (x_min, y_min), and the relative position of the lower right vertex (x_max, y_max).

[0070] In a possible implementation, the following formula (2) can be used to express the content of the descriptor (the base address is PA_base):

[0071]

[0072] It should be understood that although the vertices at the upper left corner and the lower right corner are used in the above example to determine the content of the descriptor, those skilled in the art can set the specific vertices of at least two diagonal positions according to actual needs, and this disclosure does not limit this.

[0073] In one possible implementation, the content of the tensor data descriptor can be determined based on the reference address of the descriptor's data reference point in the data storage space and the mapping relationship between the data description position and the data address of the tensor data indicated by the descriptor. The mapping relationship between the data description position and the data address can be set according to actual needs. For example, when the tensor data indicated by the descriptor is three-dimensional spatial data, the function f(x, y, z) can be used to define the mapping relationship between the data description position and the data address.

[0074] In a possible implementation, the following formula (3) can be used to express the content of the descriptor:

[0075]

[0076] In one possible implementation, the descriptor is further used to indicate the address of N-dimensional tensor data, wherein the content of the descriptor further includes at least one address parameter representing the address of the tensor data. For example, the content of the descriptor may be the following formula (4):

[0077]

[0078] Where PA is the address parameter. The address parameter can be a logical address or a physical address. When parsing the descriptor, PA can be used as any vertex, midpoint, or preset point of the vector shape, combined with the shape parameters in the X and Y directions to obtain the corresponding data address.

[0079] In a possible implementation, the address parameter of the tensor data includes a reference address of a data reference point of the descriptor in the data storage space of the tensor data, and the reference address includes a starting address of the data storage space.

[0080] In a possible implementation, the descriptor may further include at least one address parameter representing the address of the tensor data. For example, the content of the descriptor may be the following formula (5):

[0081]

[0082] PA_start is the base address parameter and will not be described in detail.

[0083] It should be understood that those skilled in the art can set the mapping relationship between the data description location and the data address according to actual conditions, and this disclosure does not limit this.

[0084] In one possible implementation, a predetermined reference address can be set within a task. All descriptors in instructions within this task use this reference address, and the descriptor content can include shape parameters based on this reference address. This reference address can be determined by setting the environment parameters for this task. A description of the reference address and its use can be found in the above embodiments. In this implementation, the descriptor content can be mapped to data addresses more quickly.

[0085] In one possible implementation, the base address can be included in the content of each descriptor, so that the base address of each descriptor can be different. Compared with the method of using environmental parameters to set a common base address, each descriptor in this method can describe data more flexibly and use a larger data address space.

[0086] In one possible implementation, the data address of the data corresponding to the operand of the processing instruction in the data storage space can be determined based on the content of the descriptor. The data address is calculated automatically by hardware, and the calculation method of the data address varies depending on the representation of the descriptor content. This disclosure does not limit the specific method for calculating the data address.

[0087] For example, the content of the descriptor in the operand is expressed using formula (1). The offsets of the tensor data indicated by the descriptor in the data storage space are offset_x and offset_y, and the size is size_x*size_y. Then, the starting data address PA1 of the tensor data indicated by the descriptor in the data storage space is (x，y) It can be determined using the following formula (6):

[0088] PA1 (x，y) =PA_start+(offset_y-1)*ori_x+offset_x (6)

[0089] The data starting address PA1 is determined according to the above formula (6) (x，y) , combined with the offsets offset_x and offset_y, and the sizes size_x and size_y of the storage area, the storage area of ​​the tensor data indicated by the descriptor in the data storage space can be determined.

[0090] In one possible implementation, when the operand also includes a data description location for a descriptor, the data address of the data corresponding to the operand in the data storage space can be determined based on the content of the descriptor and the data description location. In this way, partial data (e.g., one or more data) in the tensor data indicated by the descriptor can be processed.

[0091] For example, the content of the descriptor in the operand is expressed using formula (2). The offsets of the tensor data indicated by the descriptor in the data storage space are offset_x and offset_y respectively, and the size is size_x*size_y. The data description position for the descriptor included in the operand is (x q ,y q ), then the data address PA2 of the tensor data indicated by the descriptor in the data storage space (x，y) It can be determined using the following formula (7):

[0092] PA2 (x，y) =PA_start+(offset_y+y q -1)*ori_x+(offset_x+x q ) (7)

[0093] In one possible implementation, the descriptor can indicate data blocks. Data blocks can effectively speed up operations and improve processing efficiency in many applications. For example, in graphics processing, convolution operations often use data blocks for fast processing.

[0094] Figure 7 Schematic diagram showing data blocks in data storage space according to an embodiment of the present disclosure. Figure 7 As shown, the data storage space 700 also uses a row-first approach to store two-dimensional data, which can be represented by (x, y) (where the X axis is horizontal to the right and the Y axis is vertically downward). The size in the X-axis direction (the size of each row, or the total number of columns) is ori_x (not shown in the figure), and the size in the Y-axis direction (the total number of rows) is ori_y (not shown in the figure). Figure 6 Tensor data, Figure 7 The tensor data stored in consists of multiple data blocks.

[0095] In this case, the descriptor requires more parameters to represent these data blocks. Taking the X-axis (X dimension) as an example, the following parameters may be involved: ori_x, x.tile.size (the size 702 of the block), x.tile.stride (the stride 704 of the block, i.e., the distance between the first point of the first tile and the first point of the second tile), x.tile.num (the number of tiles, shown as 3 in the figure), x.stride (the overall stride, i.e., the distance between the first point of the first row and the first point of the second row), etc. Other dimensions can similarly include corresponding parameters.

[0096] In one possible implementation, a descriptor may include a descriptor identifier and / or descriptor content. The descriptor identifier is used to distinguish the descriptor, for example, the descriptor identifier may be a number; the descriptor content may include at least one shape parameter representing the shape of the tensor data. For example, if the tensor data is three-dimensional data, and the shape parameters of two of the three dimensions of the tensor data are fixed, the descriptor content may include the shape parameter representing the other dimension of the tensor data.

[0097] In one possible implementation, the identifier and / or content of the descriptor may be stored in a descriptor storage space (internal memory), such as a register, on-chip SRAM, or other media cache. The tensor data indicated by the descriptor may be stored in a data storage space (internal memory or external memory), such as an on-chip cache or off-chip memory. This disclosure does not limit the specific locations of the descriptor storage space and the data storage space.

[0098] In one possible implementation, the identifier, content of the descriptor, and the tensor data indicated by the descriptor can be stored in the same area of ​​the internal memory. For example, a continuous area of ​​the on-chip cache can be used to store the relevant content of the descriptor, and its address is ADDR0-ADDR1023. Among them, the address ADDR0-ADDR63 can be used as a descriptor storage space to store the identifier and content of the descriptor, and the address ADDR64-ADDR1023 can be used as a data storage space to store the tensor data indicated by the descriptor. In the descriptor storage space, the address ADDR0-ADDR31 can be used to store the identifier of the descriptor, and the address ADDR32-ADDR63 can be used to store the content of the descriptor. It should be understood that the address ADDR is not limited to 1 bit or 1 byte. It is used here to represent an address and is an address unit. Those skilled in the art can determine the descriptor storage space, data storage space and their specific addresses according to actual conditions, and this disclosure is not limited to this.

[0099] In one possible implementation, the descriptor identifier, content, and tensor data indicated by the descriptor can be stored in different areas of the internal memory. For example, registers can be used as descriptor storage space to store the descriptor identifier and content, and on-chip cache can be used as data storage space to store the tensor data indicated by the descriptor.

[0100] In one possible implementation, when registers are used to store the identifier and content of descriptors, the register number can be used to represent the identifier of the descriptor. For example, when the register number is 0, the identifier of the descriptor stored in it is set to 0. When the descriptor in the register is valid, an area in the cache space can be allocated to store the tensor data based on the size of the tensor data indicated by the descriptor.

[0101] In one possible implementation, the identifier and content of the descriptor may be stored in internal memory, and the tensor data indicated by the descriptor may be stored in external memory. For example, the identifier and content of the descriptor may be stored on-chip, while the tensor data indicated by the descriptor may be stored off-chip.

[0102] In one possible implementation, the data address of the data storage space corresponding to each descriptor can be a fixed address. For example, a separate data storage space can be divided for tensor data, and the starting address of each tensor data in the data storage space corresponds one-to-one to the descriptor. In this case, the circuit or module responsible for parsing the computing instruction (such as an entity outside the computing device of the present disclosure) can determine the data address of the data corresponding to the operand in the data storage space based on the descriptor.

[0103] In one possible implementation, when the data address of the data storage space corresponding to the descriptor is a variable address, the descriptor can also be used to indicate the address of N-dimensional tensor data, wherein the content of the descriptor can also include at least one address parameter representing the address of the tensor data. For example, the tensor data is 3-dimensional data. When the descriptor points to the address of the tensor data, the content of the descriptor may include an address parameter representing the address of the tensor data, such as the starting physical address of the tensor data, or may include multiple address parameters of the address of the tensor data, such as the starting address + address offset of the tensor data, or the address parameters of the tensor data based on each dimension. Those skilled in the art can set the address parameters according to actual needs, and this disclosure does not limit this.

[0104] In one possible implementation, the address parameter of the tensor data may include the reference address of the descriptor's data reference point in the data storage space of the tensor data. The reference address may vary depending on the data reference point. This disclosure does not limit the selection of the data reference point.

[0105] In one possible implementation, the reference address may include the starting address of the data storage space. When the data reference point of the descriptor is the first data block in the data storage space, the reference address of the descriptor is the starting address of the data storage space. When the data reference point of the descriptor is data other than the first data block in the data storage space, the reference address of the descriptor is the address of the data block in the data storage space.

[0106] In one possible implementation, the shape parameters of the tensor data include at least one of the following: the size of the data storage space in at least one direction of the N-dimensional directions, the size of the storage area in at least one direction of the N-dimensional directions, the offset of the storage area in at least one direction of the N-dimensional directions, the positions of at least two vertices at diagonal positions in the N-dimensional directions relative to the data reference point, and the mapping relationship between the data description position of the tensor data indicated by the descriptor and the data address. The data description position is the mapping position of the point or area in the tensor data indicated by the descriptor. For example, when the tensor data is 3D data, the descriptor can use three-dimensional space coordinates (x, y, z) to represent the shape of the tensor data, and the data description position of the tensor data can be the position of the point or area mapped in the three-dimensional space represented by the three-dimensional space coordinates (x, y, z).

[0107] It should be understood that those skilled in the art can select shape parameters representing tensor data according to actual circumstances, and this disclosure does not limit this. By using descriptors in the data access process, associations between data can be established, thereby reducing the complexity of data access and improving instruction processing efficiency.

[0108] An embodiment of the present disclosure provides a data processing solution based on the aforementioned hardware environment, which performs structured sparse convolution operations on tensor data according to the sparse flag included in the convolution instruction.

[0109] Figure 8 FIG. 8 is a block diagram showing a data processing device 800 according to an embodiment of the present disclosure. The data processing device 800 may be implemented, for example, in Figure 2 As shown in the figure, the data processing device 800 may include a control circuit 810, a tensor interface circuit 812, a storage circuit 820 and an operation circuit 830.

[0110] The function of the control circuit 810 may be similar to Figure 3 The control module 31 or Figure 5 The control module 51 may include, for example, an instruction fetch unit for obtaining an instruction from, for example, Figure 2The processing device 203 of the embodiment of the present invention has an instruction, and an instruction decoding unit is used to decode the acquired instruction and send the decoding result as control information to the operation circuit 830 and the storage circuit 820.

[0111] In one embodiment, the control circuit 810 may be configured to parse a convolution instruction, wherein the convolution instruction includes a sparse flag indicating whether a structured sparse convolution operation is to be performed, and at least one operand of the convolution instruction includes at least one descriptor indicating at least one of the following: shape information of tensor data and spatial information of the tensor data. In one implementation, the sparse flag may take a value of "1," indicating that the current convolution instruction performs a structured sparse convolution operation; correspondingly, the sparse flag may take a value of "0," indicating that the current convolution instruction performs a conventional convolution operation, and vice versa.

[0112] The tensor interface unit (TIU) 812 can be configured to implement operations associated with descriptors under the control of the control circuit 810. These operations may include, but are not limited to, registering, modifying, deregistering, and parsing descriptors; reading and writing descriptor contents, etc. This disclosure does not limit the specific hardware type of the tensor interface circuit. In this way, operations associated with descriptors can be implemented using dedicated hardware, further improving the access efficiency of tensor data.

[0113] In some embodiments, the tensor interface circuit 812 may be configured to parse shape information of tensor data included in an operand of an instruction to determine a data address of data corresponding to the operand in the data storage space.

[0114] Optionally or additionally, in some further embodiments, the tensor interface circuit 812 can be configured to compare the spatial information (e.g., spatial ID) of the tensor data included in the operands of two instructions to determine the dependency relationship between the two instructions, and further determine the out-of-order execution, synchronization, and other operations of the instructions.

[0115] Despite Figure 8 The control circuit 810 and the tensor interface circuit 812 are shown as two separate modules, but those skilled in the art will appreciate that these two circuits may also be implemented as one module or more modules, and the present disclosure is not limited in this respect.

[0116] The storage circuit 820 may be configured to store information before and / or after convolution. In one embodiment, the operands of the convolution instruction include weights of the convolution layer in the neural network and neuron data of the neural network. In this embodiment, the storage circuit may include, for example, Figure 3 WRAM 332 or Figure 5WRAM 532 for storing weights; and Figure 3 NRAM 331 or Figure 5 NRAM 531 for storing neuron data.

[0117] The operation circuit 830 may be configured to perform a corresponding convolution operation according to the convolution instruction.

[0118] In some embodiments, the operation circuit 830 may include a structured sparsification circuit 832 and a convolution circuit 833 .

[0119] When the sparse flag in the convolution instruction indicates that the current convolution instruction needs to perform a structured sparse convolution operation, the operation circuit 830 can be configured accordingly. For example, the structured sparse circuit 832 in the operation circuit 830 can be configured to perform structured sparse processing on at least one input data and output the sparse input data to the convolution circuit 833. The convolution circuit 833 can be configured to receive the data to be convolved and perform the convolution operation thereon. The data to be convolved includes at least the sparse input data received from the structured sparse circuit 832. Thus, when the sparse flag is set, structured sparse convolution processing can be implemented through the structured sparse circuit 832 and the convolution circuit 833. In some implementations, the input data may include neuron data and weights of a neural network.

[0120] Structured sparsification circuit 832 is configured to perform structured sparsification processing, which includes selecting n data elements from every m data elements as valid data elements, where m>n. In one implementation, m=4 and n=2. In other implementations, when m=4, n may also take other values, such as 1 or 3.

[0121] Convolution circuit 833 is used to perform a convolution operation on input data. When the sparsity flag is set to "1," i.e., a structured sparse convolution operation is being performed, the data received by convolution circuit 833 includes at least the sparsified input data from structured sparse circuit 832. When the sparsity flag is set to "0," i.e., a conventional convolution operation is being performed, the data received by convolution circuit 833 is unsparsed data.

[0122] Depending on different application scenarios, the input data to be convolved may exist in various forms, so the structured sparse circuit and the convolution circuit may need to perform structured sparse convolution processing according to different requirements.

[0123] Figures 9A-9CA partial structural diagram of an arithmetic circuit according to an embodiment of the present disclosure is shown. As shown in the figure, the structured sparse circuit 910 may include a first structured sparse subcircuit 912 and / or a second structured sparse subcircuit 914. The first structured sparse subcircuit 912 may be configured to perform structured sparse processing on input data according to a specified sparse mask. The second structured sparse subcircuit 914 may be configured to perform structured sparse processing on input data according to a predetermined sparse rule.

[0124] In the first scenario, one of the data to be convolved (assuming it is the first data) may have already been structured sparsified, while the other data to be convolved (assuming it is the second data) needs to be structured sparsified in the same manner as the first data. In this case, the first structured sparsification subcircuit can be used to perform sparsification on the second data.

[0125] Figure 9A A partial structural diagram of the operation circuit in the first scenario is shown. As shown in the figure, in this first scenario, the structured sparse circuit 910 includes a first structured sparse subcircuit 912, which receives the input second data and the index part of the first data that has been structured and sparse in advance. The first structured sparse subcircuit 912 uses the index part of the first data as a sparse mask to perform structured sparse processing on the second data. Specifically, the first structured sparse subcircuit 912 extracts the data at the corresponding position from the first data as valid data based on the valid data position indicated by the index part of the first data. In these embodiments, the first structured sparse subcircuit 912 can be implemented, for example, by circuits such as vector multiplication or matrix multiplication. The convolution circuit 920 receives the structured sparse first data and the sparsely processed second data output from the first structured sparse subcircuit 912, and performs convolution on the two.

[0126] In the second scenario, both data to be convolved have not been subjected to structured sparse processing, and structured sparse processing needs to be performed on both data before convolution. In this case, the second structured sparse subcircuit can be used to perform sparse processing on the first data and the second data.

[0127] Figure 9BA partial structural diagram of the operation circuit in the second scenario is shown. As shown in the figure, in this second scenario, the structured sparse circuit 910 may include two second structured sparse sub-circuits 914, which respectively receive the first data and the second data to be convolved, so as to simultaneously and independently perform structured sparse processing on the first data and the second data, and output the sparse data to the convolution circuit 920. The second structured sparse sub-circuit 914 can be configured to perform structured sparse processing according to a predetermined screening rule, for example, according to the rule of screening the largest absolute value, from every m data elements, n data elements with larger absolute values ​​are screened out as valid data elements. In these embodiments, the second structured sparse sub-circuit 914 can, for example, implement the above-mentioned processing by configuring a multi-stage operation pipeline composed of circuits such as comparators. It will be understood by those skilled in the art that the structured sparse circuit 910 may also include only one second structured sparse sub-circuit 914, which performs structured sparse processing on the first data and the second data in sequence.

[0128] In the third scenario, both data to be convolved have not undergone structured sparse processing, and structured sparse processing needs to be performed on both data separately before convolution, and one of the data (for example, the first data) needs to use the index part of the other data (for example, the second data) after sparse processing as a sparse mask.

[0129] Figure 9C A partial structural schematic diagram of the operation circuit in the third scenario is shown. As shown in the figure, in this third scenario, the structured sparse circuit 910 may include a first structured sparse subcircuit 912 and a second structured sparse subcircuit 914. The second structured sparse subcircuit 914 can be configured to perform structured sparse processing on, for example, the second data according to a predetermined screening rule, and provide the index portion of the sparsified second data to the first structured sparse subcircuit 912. The first structured sparse subcircuit 912 uses the index portion of the second data as a sparse mask to perform structured sparse processing on the first data. The convolution circuit 920 receives the structured sparse first data and the second processing from the first structured sparse subcircuit 912 and the second structured sparse subcircuit 914, respectively, and performs convolution on the two.

[0130] Those skilled in the art may also consider other application scenarios and design structured sparse circuits accordingly. For example, it may be necessary to apply the same sparse mask to two data to be convolved. In this case, the structured sparse circuit may include two first structured sparse sub-circuits to process the mask.

[0131] Figure 10 FIG. 1 shows an exemplary operation pipeline of structured sparse processing according to an embodiment of the present disclosure. The pipeline can be used to implement the aforementioned second structured sparse subcircuit, for example. Figure 10In the embodiment shown in FIG. , when m = 4 and n = 2, structured sparse processing is performed to select two data elements with larger absolute values ​​from four data elements A, B, C, and D. As shown in the figure, the structured sparse processing can be performed using a multi-stage pipeline circuit composed of absolute value operators, comparators, etc.

[0132] The first pipeline stage may include m(4) absolute value operators 1010 for synchronously performing absolute value operations on the four input data elements A, B, C, and D. To facilitate the final output of valid data elements, in some embodiments, the first pipeline stage simultaneously outputs the original data elements (i.e., A, B, C, and D) and the data after the absolute value operation (i.e., |A|, |B|, |C|, and |D|).

[0133] The second pipeline stage may include a permutation and combination circuit 1020 for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values, and the positions of the m absolute values ​​in each group of data are different.

[0134] In some embodiments, the permutation and combination circuit can be a circular shifter that performs m-1 circular shifts on the permutation of m absolute values ​​(e.g., |A|, |B|, |C|, and |D|), thereby generating m groups of data. For example, in the example shown in the figure, four groups of data are generated, namely: {|A|, |B|, |C|, |D|}, {|B|, |C|, |D|, |A|}, {|C|, |D|, |A|, |B|}, and {|D|, |A|, |B|, |C|}. Similarly, when each group of data is output, the corresponding original data element is also output, and each group of data corresponds to one original data element.

[0135] The third pipeline stage includes a comparison circuit 1030 for comparing the absolute values ​​of the m groups of data and generating a comparison result.

[0136] In some embodiments, the third pipeline stage may include m comparison circuits, each comparison circuit includes m-1 comparators (1031, 1032, 1033), and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, where 1≤i≤m.

[0137] As can be seen from the figure, the third pipeline stage can also be considered as m-1 (3) sub-pipeline stages. Each sub-pipeline stage includes m comparators, which are used to compare its corresponding absolute value with other absolute values. The m-1 sub-pipeline stages thus sequentially compare their corresponding absolute value with the other m-1 absolute values.

[0138] For example, in the example shown in the figure, the four comparators 1031 in the first sub-pipeline stage are used to compare the first absolute value of the four sets of data with the second absolute value, and output comparison results w0, x0, y0, and z0, respectively. The four comparators 1032 in the second sub-pipeline stage are used to compare the first absolute value of the four sets of data with the third absolute value, and output comparison results w1, x1, y1, and z1, respectively. The four comparators 1033 in the third sub-pipeline stage are used to compare the first absolute value of the four sets of data with the fourth absolute value, and output comparison results w2, x2, y2, and z2, respectively. In this way, the comparison results of each absolute value with the other m-1 absolute values ​​can be obtained.

[0139] In some embodiments, the comparison result can be represented using a bitmap. For example, at the first comparator of the first comparison circuit, when |A| ≥ |B|, w0 = 1; at the second comparator of the first comparison circuit, when |A| < |C|, w1 = 0; at the third comparator of the first comparison circuit, when |A| ≥ |D|, w2 = 1. Therefore, the output result of the first comparison circuit is {A, w0, w1, w2}, which is {A, 1, 0, 1} at this time. Similarly, the output result of the second comparison circuit is {B, x0, x1, x2}, the output result of the third comparison circuit is {C, y0, y1, y2}, and the output result of the fourth comparison circuit is {D, z0, z1, z2}.

[0140] The fourth pipeline stage includes a filter circuit 1040, which is used to select n data elements with larger absolute values ​​from the m data elements as valid data elements based on the comparison results of the third stage, and output these valid data elements and their corresponding indexes. The index indicates the position of these valid data elements in the m input data elements. For example, when A and C are selected from the four data elements A, B, C, and D, their corresponding indexes may be 0 and 2.

[0141] Based on the comparison results, appropriate logic can be designed to select n data elements with larger absolute values. Considering the possibility of multiple data elements with the same absolute value, in a further embodiment, when there are data elements with the same absolute value, they are selected according to a specified priority order. For example, the priority can be fixed from low to high in the index order, with A having the highest priority and D having the lowest priority. In one example, when the absolute values ​​of A, C, and D are the same and greater than the absolute value of B, the data selected are A and C.

[0142] From the previous comparison results, we can see that based on w0, w1, and w2, |A| is greater than {|B|, |C|, |D|}. If w0, w1, and w2 are all 1, |A| is greater than |B|, |C|, and |D|, and is the maximum of the four numbers, so A is selected. If two of w0, w1, and w2 are 1, |A| is the second largest of the four absolute values, so A is also selected. Otherwise, A is not selected. Therefore, in some embodiments, analysis and judgment can be made based on the number of occurrences of these values.

[0143] In one implementation, valid data elements can be selected based on the following logic. First, the number of times each data element is greater than other data elements can be counted. For example, define N A =sum_w=w0+w1+w2, N B =sum_x=x0+x1+x2, N C =sum_y=y0+y1+y2, N D =sum_z=z0+z1+z2. Next, make a judgment and selection according to the following conditions.

[0144] The condition for selecting A is: N A =3, or N A =2 and N B / N C / N D There is only one 3 in it;

[0145] The condition for selecting B is: N B =3, or N B =2 and N A / N C / N D There is only one 3 in N A ≠2;

[0146] The conditions for selecting C are: N C =3, and N A / N B There is at most one 3, or N C =2 and N A / N B / N D There is only one 3 in N A / N B There is no 2 in

[0147] The conditions for selecting D are: N D =3, and N A / N B / N C There is at most one 3, or N D =2 and N A / N B / N C There is only one 3 in N A / N B / N C There is no 2 in it.

[0148] Those skilled in the art will appreciate that in order to ensure selection according to a predetermined priority, there is a certain degree of redundancy in the above logic. Based on the size and order information provided by the comparison results, those skilled in the art can design other logic to implement the screening of valid data elements, and this disclosure is not limited in this respect. Figure 10 The multi-stage pipeline operation circuit can realize four-choose-two structured sparse processing.

[0149] Those skilled in the art will appreciate that other forms of pipeline operation circuits may be designed to implement structured sparse processing, and the present disclosure is not limited in this regard.

[0150] The result of the sparsification process consists of two parts: a data portion and an index portion. The data portion includes the data after sparsification, that is, the valid data elements extracted according to the screening rules of structured sparsification. The index portion is used to indicate the sparsified data, that is, the original position of the valid data elements in the original data before sparsification (that is, the data to be sparsified).

[0151] Various forms can be used to represent and / or store structured sparsely processed data. In one implementation, the structured sparsely processed data can be in the form of a structure. In this structure, the data portion and the index portion are bound to each other. In some embodiments, each bit in the index portion can correspond to a data element. For example, when the data type is fix8, a data element is 8 bits, and each bit in the index portion can correspond to 8 bits of data. In other embodiments, considering the hardware-level implementation when the structure is subsequently used, each bit in the index portion in the structure can be set to a position corresponding to N bits of data, where N is determined at least in part based on the hardware configuration. For example, each bit in the index portion in the structure can be set to a position corresponding to 4 bits of data. For example, when the data type is fix8, every 2 bits in the index portion correspond to a data element of the fix8 type. In some embodiments, the data portion in the structure can be aligned according to the first alignment requirement, and the index portion in the structure can be aligned according to the second alignment requirement, so that the entire structure also meets the alignment requirement. For example, the data portion can be aligned to 64B, the index portion can be aligned to 32B, and the entire structure can be aligned to 96B (64B + 32B). This alignment requirement can reduce the number of memory accesses in subsequent use and improve processing efficiency.

[0152] By using this structure, the data and index components can be used together. Since the ratio of valid data elements to original data elements in structured sparse processing is fixed, for example, n / m, the size of the data after sparse processing is also fixed or predictable. As a result, the structure can be stored densely in the storage circuit without performance loss.

[0153] In other implementations, the data portion and index portion obtained after the sparsification process can also be represented and / or stored separately for independent use. For example, the index portion of the second input data that has undergone structured sparsification can be provided to the first structured sparsification circuit 912 to be used as a mask to perform structured sparsification on the first input data. In this case, to accommodate different data types, each bit in the separately provided index portion can indicate whether a data element is valid.

[0154] The convolution circuit can use a variety of circuit configurations to implement the convolution operation. For example, the convolution circuit can share the same processing circuit for conventional convolution and structured sparse convolution, or the convolution circuit can allocate separate processing circuits for conventional convolution and structured sparse convolution operations. The disclosed embodiments are not limited in this respect.

[0155] Back to Figure 8 In some embodiments, the operation circuit 830 may further include a pre-processing circuit 831 and a post-processing circuit 834. The pre-processing circuit 831 may be configured to pre-process data before the structured sparsity circuit 832 and / or the convolution circuit 833 perform operations according to instructions; and the post-processing circuit 834 may be configured to post-process data after the convolution circuit 833 performs operations.

[0156] In some implementations, when the sparsity flag in the convolution instruction indicates that a structured sparse convolution operation is to be performed, the pre-processing circuit 831 may read the input data from the storage circuit 820 and output the input data to the structured sparse circuit 832 at a first rate. When the sparsity flag indicates a conventional convolution, the pre-processing circuit 831 may read the input data from the storage circuit 820 and output the input data to the convolution circuit 833 at a second rate. The first rate is greater than the second rate, and the ratio thereof is, for example, equal to the sparsity ratio in the structured sparse processing, such as m / n. For example, in a 2-out-of-4 structured sparse processing, the first rate is twice the second rate. Thus, the first rate is determined at least in part based on the processing capability of the convolution circuit 833 and the sparsity ratio of the structured sparse processing.

[0157] In some application scenarios, the aforementioned pre-processing and post-processing may also include, for example, data splitting and / or data concatenation operations. For example, the post-processing circuit 834 may perform fusion processing on the output results of the convolution circuit, such as addition, subtraction, multiplication, and the like.

[0158] As mentioned above, the operands of the convolution instruction can be data in the neural network, such as weights, neurons, etc., that is, the convolution instruction is used for structured sparse convolution operations in the neural network. The data in the neural network usually contains multiple dimensions. For example, in a convolutional neural network, the data may have four dimensions: input channels, output channels, length, and width. In some embodiments, the structured sparsity in the above convolution instruction can be performed on at least one dimension of the multidimensional data in the neural network. Specifically, in one implementation, the convolution instruction can be used for structured sparse convolution operations in the forward process of the neural network (for example, inference, or forward training), and the structured sparse processing is performed on the input channel dimension of the multidimensional data in the neural network. In another implementation, the convolution instruction can be used for structured sparse convolution operations in the reverse process of the neural network (for example, reverse training), and the structured sparse processing is performed simultaneously on the input channel dimension and the output channel dimension of the multidimensional data in the neural network.

[0159] In the context of the present disclosure, the aforementioned convolution instruction may be a microinstruction or control signal running inside one or more multi-stage operation pipelines, which may include (or indicate) one or more operation operations that need to be performed by the multi-stage operation pipelines.

[0160] Figure 11 FIG. 11 shows an exemplary flow chart of a data processing method 1100 according to an embodiment of the present disclosure.

[0161] like Figure 11 As shown, in step 1110, a convolution instruction is parsed, the convolution instruction includes a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction includes at least one descriptor, the descriptor indicating at least one of the following information: shape information of tensor data and spatial information of tensor data. This step can be performed, for example, by Figure 8 The control circuit 810 is used to execute.

[0162] Next, in step 1120, the descriptor is parsed. This step can be performed by, for example Figure 8 Specifically, the data address of the tensor data corresponding to the operand in the data storage space can be determined based on the shape information of the tensor data; and / or the dependency relationship between instructions can be determined based on the space information of the tensor data.

[0163] Next, in step 1130, the corresponding operand is read based at least in part on the parsed descriptor. When the operand is tensor data, the data address can be obtained according to the parsed descriptor to read the corresponding data. This step can be performed, for example, by Figure 8The control circuit 810 controls the storage circuit 820 to execute.

[0164] Finally, in step 1140, the corresponding convolution operation is performed on the read operand according to the convolution instruction. This step can be performed by Figure 8 The operation circuit 830 is used to execute.

[0165] Depending on the value of the sparse flag in the convolution instruction, different convolution operations can be performed. For example, when the sparse flag is set to "0", a regular convolution operation can be performed. When the sparse flag is set to "1", a structured sparse convolution operation is performed.

[0166] At this time, the structured sparse convolution operation may include: performing structured sparse processing on at least one input data; and performing a convolution operation on the sparse input data.

[0167] Specifically, in some implementations, performing structured sparse processing on at least one input data includes either: performing structured sparse processing on first input data and second input data to be convolved, respectively, and outputting the sparsely processed first input data and second input data to a convolution circuit to perform a convolution operation; or performing structured sparse processing on the second or first input data using an index portion corresponding to the structured sparse processed first or second input data as a sparse mask, and outputting the sparsely processed first input data to a convolution circuit to perform a convolution operation with the structured sparse processed second input data, wherein the index portion indicates the position of valid data elements in the structured sparse processing to be performed. Accordingly, the structured sparse processing includes selecting n data elements as valid data elements from every m data elements according to the indication of the index portion, where m>n.

[0168] The first or second input data that has been structured sparsely processed may be structured sparsely processed in advance and stored in a storage circuit, or the first or second input data that has been structured sparsely processed may be directly provided to the convolution operation after being structured sparsely processed online.

[0169] The structured sparse data can be provided in various forms. In one implementation, the structured sparse data is in the form of a structure, which includes a data portion and an index portion that are bound together. The data portion includes valid data elements after the structured sparse processing, and the index portion indicates the original position of the sparse data in the pre-sparse data.

[0170] Furthermore, before performing the structured sparse processing, the method may also include: transmitting input data at a first rate to perform the structured sparse processing, wherein the first rate is determined at least in part based on the processing capability of the hardware performing the convolution operation and the sparsity ratio of the structured sparse processing.

[0171] In some embodiments, the first input data may be neuron data of a neural network, and the second input data may be weights of a convolutional layer in the neural network; or vice versa.

[0172] It will be understood by those skilled in the art that the steps described in the method flow chart are combined with the above Figure 8 The various circuits of the described data processing device correspond to each other, so the features described above are also applicable to the method steps and will not be repeated here.

[0173] As can be seen from the above description, the disclosed embodiment provides a convolution instruction for performing a structured sparse convolution operation on tensor data, wherein the tensor data is described by a descriptor. In some embodiments, the convolution instruction includes a sparse flag to indicate whether a structured sparse convolution operation is performed. By setting the sparse flag, the corresponding arithmetic circuit can be configured according to the value of the flag to perform the corresponding convolution operation. In some embodiments, when the sparse flag indicates that a structured sparse convolution operation is to be performed, the arithmetic circuit can be configured to perform structured sparse processing, and then convolution is performed on the sparsified data. By reusing the instruction field of the convolution instruction and adding a structured sparse enable flag, the processing can be simplified, thereby improving the processing efficiency of the machine.

[0174] According to different application scenarios, the electronic devices or devices disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, Internet of Things terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, vehicles, household appliances, and / or medical equipment. The vehicles include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs. The electronic devices or devices disclosed herein may also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical care and other fields. Furthermore, the electronic devices or devices disclosed herein may also be used in cloud, edge, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, electronic devices or apparatuses with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or apparatuses with low power consumption can be applied to terminal devices and / or edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0175] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teachings of the present disclosure, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present disclosure can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present disclosure. In addition, depending on the different schemes, the description of some embodiments of the present disclosure also has different emphases. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0176] In terms of specific implementation, based on the disclosure and teachings of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented in other ways not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them based on the consideration of logical functions, and there may be other ways of splitting them in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0177] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solution described in the embodiments of this disclosure. In addition, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically separately.

[0178] In some other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, that is, as specific hardware circuits, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0179] The foregoing content can be better understood in accordance with the following terms:

[0180] Clause 1. A data processing apparatus comprising:

[0181] A control circuit configured to parse a convolution instruction, wherein the convolution instruction includes a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of the tensor data;

[0182] a tensor interface circuit configured to parse the descriptor;

[0183] a storage circuit configured to store pre-convolution and / or post-convolution information; and

[0184] The operation circuit is configured to perform a corresponding convolution operation according to the convolution instruction based on the parsed descriptor.

[0185] Clause 2. The data processing apparatus according to clause 1, wherein:

[0186] The tensor interface circuit is configured to determine a data address of tensor data corresponding to the operand in a data storage space according to the shape information; and / or

[0187] The tensor interface circuit is configured to determine dependencies between instructions based on the spatial information.

[0188] Clause 3. The data processing apparatus according to any one of Clauses 1-2, wherein the shape information of the tensor data includes at least one shape parameter representing the shape of N-dimensional tensor data, where N is a positive integer, and the shape parameter of the tensor data includes at least one of the following:

[0189] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

[0190] Clause 4. The data processing apparatus according to any one of clauses 1-2, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, where N is a positive integer, and the shape parameter comprises at least one of the following:

[0191] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

[0192] Clause 5. The data processing apparatus according to any one of clauses 1 to 4, wherein the operation circuit comprises a structured sparse circuit and a convolution circuit, and when the sparse flag indicates that a structured sparse convolution operation is to be performed,

[0193] The structured sparse circuit is configured to perform structured sparse processing on at least one input data and output the sparse input data to the convolution circuit; and

[0194] The convolution circuit is configured to receive data to be convolved and perform a convolution operation on the data, wherein the data to be convolved includes at least the sparsified input data.

[0195] Clause 6. The data processing apparatus of clause 5, wherein the structured sparse circuit comprises:

[0196] a first structured sparse subcircuit configured to perform structured sparse processing on input data according to a specified sparse mask; and / or

[0197] The second structured sparse subcircuit is configured to perform structured sparse processing on the input data according to a predetermined sparse rule.

[0198] Clause 7. The data processing apparatus according to clause 6, wherein the structured sparse circuit is further configured to perform any of the following:

[0199] performing structured sparse processing on the first input data and the second input data to be convolved respectively using a second structured sparse subcircuit, and outputting the sparsified first input data and the second input data to the convolution circuit to perform a convolution operation; or

[0200] Using the first structured sparse subcircuit, the index part corresponding to the first or second input data that has been structured sparsely processed is used as a sparse mask to perform structured sparse processing on the second or first input data, and the sparsified first input data is output to the convolution circuit to perform a convolution operation with the second input data that has been structured sparsely processed, wherein the index part indicates the position of the valid data element in the structured sparse processing to be performed.

[0201] Clause 8. The data processing apparatus according to clause 7, wherein:

[0202] The first or second input data that has undergone structured sparse processing is pre-structured sparse processing and stored in the storage circuit, or

[0203] The first or second input data that has undergone structured sparse processing is generated by performing structured sparse processing online using the second structured sparse subcircuit.

[0204] Clause 9. A data processing device according to any one of clauses 7-8, wherein the first or second input data that has been structured sparsely processed is in the form of a structure, the structure includes a data part and an index part that are bound to each other, the data part includes valid data elements that have been structured sparsely processed, and the index part is used to indicate the position of the data after sparse processing in the data before sparse processing.

[0205] Clause 10. A data processing apparatus according to any one of clauses 6 to 9, wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

[0206] Clause 11. A data processing device according to any one of clauses 6 to 10, wherein the second structured sparse subcircuit further comprises: at least one multi-stage pipeline operation circuit, which includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements.

[0207] Clause 12. The data processing apparatus according to clause 11, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein:

[0208] The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values;

[0209] The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different;

[0210] The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; and

[0211] The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

[0212] Item 13. A data processing device according to Item 12, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

[0213] Clause 14. The data processing apparatus according to any one of clauses 12-13, wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them in accordance with a specified priority order.

[0214] Clause 15. The data processing device according to any one of clauses 5 to 14, wherein the operation circuit further comprises a pre-processing circuit, and when the sparse flag indicates to perform a structured sparse convolution operation,

[0215] The pre-processing circuit reads input data from the storage circuit and outputs the input data to the structured sparse circuit at a first rate, wherein the first rate is at least partially based on the processing capability of the convolution circuit and the sparsity ratio of the structured sparse processing.

[0216] Clause 16. A data processing device according to any one of clauses 5 to 15, wherein the input data includes neuron data and weights of a neural network.

[0217] Clause 17. A data processing device according to any one of clauses 1 to 16, wherein the convolution instruction is used for a structured sparsity convolution operation in a neural network, and the structured sparsity is performed on at least one dimension of multidimensional data in the neural network.

[0218] Clause 18. The data processing apparatus according to clause 17, wherein:

[0219] The at least one dimension is selected from an input channel dimension and an output channel dimension.

[0220] Clause 19. A chip comprising the data processing device according to any one of clauses 1 to 18.

[0221] Clause 20. A board comprising the chip according to clause 19.

[0222] Article 21. A data processing method comprising:

[0223] Parsing a convolution instruction, wherein the convolution instruction includes a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of the tensor data;

[0224] parsing the descriptor;

[0225] Based at least in part on the parsed descriptor, reading a corresponding operand; and

[0226] A corresponding convolution operation is performed on the operand according to the convolution instruction.

[0227] Clause 22. The data processing method according to clause 21, wherein parsing the descriptor comprises:

[0228] Determining, based on the shape information, a data address of the tensor data corresponding to the operand in the data storage space; and / or

[0229] Dependencies between instructions are determined based on the spatial information.

[0230] Clause 23. The data processing method according to any one of Clauses 21-22, wherein the shape information of the tensor data includes at least one shape parameter representing the shape of N-dimensional tensor data, where N is a positive integer, and the shape parameter of the tensor data includes at least one of the following:

[0231] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

[0232] Clause 24. The data processing method according to any one of clauses 21-22, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, N being a positive integer, and the shape parameter comprising at least one of the following:

[0233] The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

[0234] Clause 25. The data processing method according to any one of Clauses 21 to 24, wherein when the sparsity flag indicates that a structured sparse convolution operation is to be performed, the method further comprises:

[0235] Performing structured sparse processing on at least one input data using a structured sparse circuit to obtain sparse input data; and

[0236] A convolution operation is performed on the data to be convolved using a convolution single path, where the data to be convolved includes at least the sparsified input data.

[0237] Clause 26. The data processing method according to Clause 25, wherein the structured sparse processing comprises:

[0238] Using a first structured sparse subcircuit, perform structured sparse processing on the input data according to a specified sparse mask; and / or

[0239] The second structured sparse subcircuit is used to perform structured sparse processing on the input data according to a predetermined sparse rule.

[0240] Clause 27. The data processing method according to Clause 26, wherein the structured sparse processing further comprises any of the following:

[0241] performing structured sparse processing on the first input data and the second input data to be convolved respectively using a second structured sparse subcircuit, and outputting the sparsified first input data and the second input data to the convolution circuit to perform a convolution operation; or

[0242] Using the first structured sparse subcircuit, the index part corresponding to the first or second input data that has been structured sparsely processed is used as a sparse mask to perform structured sparse processing on the second or first input data, and the sparsified first input data is output to the convolution circuit to perform a convolution operation with the second input data that has been structured sparsely processed, wherein the index part indicates the position of the valid data element in the structured sparse processing to be performed.

[0243] Clause 28. A data processing method according to Clause 27, wherein:

[0244] The first or second input data that has undergone structured sparse processing is pre-structured sparse processing and stored in a storage circuit, or

[0245] The first or second input data that has undergone structured sparse processing is generated by performing structured sparse processing online using the second structured sparse subcircuit.

[0246] Clause 29. A data processing method according to any one of Clauses 27-28, wherein the first or second input data that has been structured sparsely processed is in the form of a structure, the structure includes a data part and an index part that are bound to each other, the data part includes valid data elements that have been structured sparsely processed, and the index part is used to indicate the position of the data after sparse processing in the data before sparse processing.

[0247] Clause 30. A data processing method according to any one of clauses 26 to 29, wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

[0248] Clause 31. A data processing method according to any one of clauses 26-30, wherein the second structured sparse subcircuit further comprises: at least one multi-stage pipeline operation circuit, which includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements.

[0249] Clause 32. The data processing method according to clause 31, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein:

[0250] The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values;

[0251] The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different;

[0252] The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; and

[0253] The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

[0254] Clause 33. A data processing method according to Clause 32, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

[0255] Clause 34. A data processing method according to any one of clauses 32-33, wherein the screening circuit is further configured to select data elements according to a specified priority order when there are data elements with the same absolute value.

[0256] Clause 35. A data processing method according to any one of Clauses 25 to 34, further comprising:

[0257] The input data is fed at a first rate to perform the structured sparse processing, wherein the first rate is based at least in part on a processing capability of hardware performing a convolution operation and a sparsity ratio of the structured sparse processing.

[0258] Clause 36. A data processing method according to any one of clauses 25-25, wherein the input data includes neuron data and weights of a neural network.

[0259] Clause 37. A data processing method according to any one of clauses 21-36, wherein the convolution instruction is used for a structured sparsity convolution operation in a neural network, and the structured sparsity is performed on at least one dimension of multidimensional data in the neural network.

[0260] Clause 38. A data processing method according to Clause 37, wherein:

[0261] The at least one dimension is selected from an input channel dimension and an output channel dimension.

[0262] The above is a detailed introduction to the embodiments of the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, based on the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A data processing device, comprising: A control circuit configured to parse a convolution instruction, wherein the convolution instruction includes a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of the tensor data; a tensor interface circuit configured to parse the descriptor; a storage circuit configured to store pre-convolution and / or post-convolution information; as well as The operation circuit is configured to perform a corresponding convolution operation according to the convolution instruction based on the parsed descriptor.

2. The data processing apparatus according to claim 1, wherein: The tensor interface circuit is configured to determine a data address of tensor data corresponding to the operand in a data storage space according to the shape information; and / or The tensor interface circuit is configured to determine dependencies between instructions based on the spatial information.

3. The data processing apparatus according to claim 1 , wherein the shape information of the tensor data comprises at least one shape parameter representing the shape of N-dimensional tensor data, where N is a positive integer, and the shape parameter of the tensor data comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

4. The data processing apparatus according to claim 1 , wherein the shape information of the tensor data indicates at least one shape parameter of a shape of N-dimensional tensor data including a plurality of data blocks, N being a positive integer, and the shape parameter comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

5. The data processing device according to claim 1 , wherein the operation circuit comprises a structured sparse circuit and a convolution circuit, and when the sparse flag indicates that a structured sparse convolution operation is to be performed, The structured sparse circuit is configured to perform structured sparse processing on at least one input data and output the sparse input data to the convolution circuit; and The convolution circuit is configured to receive data to be convolved and perform a convolution operation on the data, wherein the data to be convolved includes at least the sparsified input data.

6. The data processing apparatus according to claim 5, wherein the structured sparse circuit comprises: a first structured sparse subcircuit configured to perform structured sparse processing on input data according to a specified sparse mask; and / or The second structured sparse subcircuit is configured to perform structured sparse processing on the input data according to a predetermined sparse rule.

7. The data processing apparatus according to claim 6, wherein the structured sparse circuit is further configured to perform any one of the following: performing structured sparse processing on the first input data and the second input data to be convolved respectively using a second structured sparse subcircuit, and outputting the sparsified first input data and the second input data to the convolution circuit to perform a convolution operation; or Using the first structured sparse subcircuit, the index part corresponding to the first or second input data that has been structured sparsely processed is used as a sparse mask to perform structured sparse processing on the second or first input data, and the sparsified first input data is output to the convolution circuit to perform a convolution operation with the second input data that has been structured sparsely processed, wherein the index part indicates the position of the valid data element in the structured sparse processing to be performed.

8. The data processing apparatus according to claim 7, wherein: The first or second input data that has undergone structured sparse processing is pre-structured sparse processing and stored in the storage circuit, or The first or second input data that has undergone structured sparse processing is generated by performing structured sparse processing online using the second structured sparse subcircuit.

9. A data processing device according to claim 7, wherein the first or second input data that has been structured sparsely processed is in the form of a structure, the structure includes a data part and an index part that are bound to each other, the data part includes valid data elements after structured sparse processing, and the index part is used to indicate the position of the data after sparse processing in the data before sparse processing. 10 . The data processing apparatus according to claim 6 , wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

11. The data processing apparatus according to claim 6, wherein the second structured sparse subcircuit further comprises: At least one multi-stage pipeline operation circuit includes a plurality of operators arranged in stages and is configured to perform a structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements.

12. The data processing apparatus according to claim 11, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein: The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values; The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different; The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; as well as The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

13. The data processing device according to claim 12, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m. 14 . The data processing apparatus according to claim 12 , wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them according to a specified priority order.

15. The data processing device according to claim 5, wherein the operation circuit further comprises a pre-processing circuit, and when the sparse flag indicates to perform a structured sparse convolution operation, The pre-processing circuit reads input data from the storage circuit and outputs the input data to the structured sparse circuit at a first rate, wherein the first rate is at least partially based on the processing capability of the convolution circuit and the sparsity ratio of the structured sparse processing.

16. The data processing apparatus according to claim 5, wherein the input data comprises neuron data and weights of a neural network.

17. The data processing apparatus according to claim 1, wherein the convolution instruction is used for a convolution operation of structured sparsity in a neural network, and the structured sparsity is performed on at least one dimension of multidimensional data in the neural network.

18. The data processing apparatus according to claim 17, wherein: The at least one dimension is selected from an input channel dimension and an output channel dimension.

19. A chip comprising the data processing device according to any one of claims 1 to 18.

20. A board comprising the chip according to claim 19.

21. A data processing method, executed by the data processing device according to any one of claims 1 to 18, comprising: Parsing a convolution instruction, wherein the convolution instruction includes a sparse flag for indicating whether to perform a structured sparse convolution operation, and at least one operand of the convolution instruction includes at least one descriptor, wherein the descriptor indicates at least one of the following information: shape information of tensor data and spatial information of the tensor data; parsing the descriptor; Based at least in part on the parsed descriptor, reading a corresponding operand; as well as A corresponding convolution operation is performed on the operand according to the convolution instruction.

22. The data processing method according to claim 21, wherein: Parsing the descriptor includes: Determining, based on the shape information, a data address of the tensor data corresponding to the operand in the data storage space; and / or Dependencies between instructions are determined based on the spatial information.

23. The data processing method according to claim 21, wherein the shape information of the tensor data comprises at least one shape parameter representing the shape of N-dimensional tensor data, N is a positive integer, and the shape parameter of the tensor data comprises at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​the tensor data in at least one direction of the N dimensions, the offset of the storage area in at least one direction of the N dimensions, the positions of at least two vertices at diagonal positions in the N dimensions relative to the data reference point, and the mapping relationship between the data description position of the tensor data and the data address.

24. The data processing method according to claim 21, wherein the shape information of the tensor data indicates at least one shape parameter of the shape of N-dimensional tensor data including a plurality of data blocks, N is a positive integer, and the shape parameter includes at least one of the following: The size of the data storage space where the tensor data is located in at least one direction of the N dimensions, the size of the storage area of ​​a single data block in at least one direction of the N dimensions, the block step of the data block in at least one direction of the N dimensions, the number of data blocks in at least one direction of the N dimensions, and the overall step of the data block in at least one direction of the N dimensions.

25. The data processing method according to claim 21, when the sparse flag indicates to perform a structured sparse convolution operation, the method further comprises: Performing structured sparse processing on at least one input data using a structured sparse circuit to obtain sparse input data; as well as A convolution operation is performed on the data to be convolved using a convolution single path, where the data to be convolved includes at least the sparsified input data.

26. The data processing method according to claim 25, wherein the structured sparse processing comprises: Using the first structured sparse subcircuit, perform structured sparse processing on the input data according to the specified sparse mask; and / or The second structured sparse subcircuit is used to perform structured sparse processing on the input data according to a predetermined sparse rule.

27. The data processing method according to claim 26, wherein the structured sparse processing further comprises any one of the following: performing structured sparse processing on the first input data and the second input data to be convolved respectively using a second structured sparse subcircuit, and outputting the sparsified first input data and the second input data to the convolution circuit to perform a convolution operation; or Using the first structured sparse subcircuit, the index part corresponding to the first or second input data that has been structured sparsely processed is used as a sparse mask to perform structured sparse processing on the second or first input data, and the sparsified first input data is output to the convolution circuit to perform a convolution operation with the second input data that has been structured sparsely processed, wherein the index part indicates the position of the valid data element in the structured sparse processing to be performed.

28. The data processing method according to claim 27, wherein: The first or second input data that has undergone structured sparse processing is pre-structured sparse processing and stored in a storage circuit, or The first or second input data that has undergone structured sparse processing is generated by performing structured sparse processing online using the second structured sparse subcircuit.

29. The data processing method according to claim 27, wherein the first or second input data that has been structured sparsely processed is in the form of a structure, the structure includes a data part and an index part that are bound to each other, the data part includes valid data elements that have been structured sparsely processed, and the index part is used to indicate the position of the data after sparse processing in the data before sparse processing. 30 . The data processing method according to claim 26 , wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, wherein m>n.

31. The data processing method according to claim 26, wherein the second structured sparse subcircuit further comprises: At least one multi-stage pipeline operation circuit includes a plurality of operators arranged in stages and is configured to perform a structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements.

32. The data processing method according to claim 31, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein: The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values; The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different; The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; as well as The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

33. The data processing method according to claim 32, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

34. The data processing method according to claim 32, wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them according to a specified priority order.

35. The data processing method according to claim 25, further comprising: The input data is fed at a first rate to perform the structured sparse processing, wherein the first rate is based at least in part on a processing capability of hardware performing a convolution operation and a sparsity ratio of the structured sparse processing.

36. The data processing method according to claim 25, wherein the input data includes neuron data and weights of a neural network.

37. The data processing method according to claim 21, wherein the convolution instruction is used for a structured sparsity convolution operation in a neural network, and the structured sparsity is performed on at least one dimension of multidimensional data in the neural network.

38. The data processing method according to claim 37, wherein: The at least one dimension is selected from an input channel dimension and an output channel dimension.

Images