Data processing device, data processing method and related products

Abstract

This disclosure discloses a data processing device, a data processing method, and related products. The data processing device can be implemented as a computing device included in a combined processing device, which can also include an interface device and other processing devices. The computing device interacts with the other processing devices to jointly complete user-specified computing operations. The combined processing device can also include a storage device, which is connected to the computing device and the other processing devices, respectively, for storing data from the computing device and the other processing devices. The disclosed solution provides dedicated instructions for operations related to structured sparsity, which can simplify processing and improve machine processing efficiency.

Description

Technical Field

[0001] The present disclosure generally relates to the field of processors and, more particularly, to data processing devices, data processing methods, chips, and boards. Background Art

[0002] In recent years, the rapid development of deep learning has led to significant advances in algorithm performance across a range of fields, including computer vision and natural language processing. However, deep learning algorithms are computationally and storage-intensive. As information processing tasks become increasingly complex and the demands for real-time and accurate algorithms continue to rise, neural networks are often designed to be deeper, increasing the computational load and storage requirements. This makes existing deep learning-based artificial intelligence technologies difficult to directly apply to hardware-constrained mobile phones, satellites, or embedded devices.

[0003] Therefore, the compression, acceleration, and optimization of deep neural network models have become extremely important. Numerous studies have attempted to reduce the computational and storage requirements of neural networks without compromising model accuracy. This is of crucial importance for the engineering application of deep learning technology on embedded and mobile devices. Sparsification is one such method for achieving model lightweighting.

[0004] Network parameter sparsification is to reduce the redundant components in a larger network by appropriate methods to reduce the network's computational load and storage space requirements. Existing hardware and / or instruction sets cannot effectively support sparsification processing. Summary of the Invention

[0005] In order to at least partially solve one or more technical problems mentioned in the background technology, the solution disclosed herein provides a data processing device, a data processing method, a chip and a board.

[0006] In a first aspect, the present disclosure discloses a data processing device, comprising: a control circuit configured to parse sparse instructions, wherein the sparse instructions indicate operations related to structured sparsity; a storage circuit configured to store information before and / or after sparsification; and an operation circuit configured to perform corresponding operations according to the sparse instructions.

[0007] In a second aspect, the present disclosure provides a chip comprising the data processing device of any embodiment of the aforementioned first aspect.

[0008] In a third aspect, the present disclosure provides a board comprising the chip of any one of the embodiments of the second aspect.

[0009] In a fourth aspect, the present disclosure provides a data processing method, which includes: parsing a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparsity; reading a corresponding operand according to the sparse instruction; performing the operation related to structured sparsity on the operand; and outputting the operation result.

[0010] Through the data processing device, data processing method, integrated circuit chip and board provided above, the embodiment of the present disclosure provides a sparse instruction for performing operations related to structured sparsity. In some embodiments, the sparse instruction may include an operation mode bit to indicate different operation modes of the sparse instruction, thereby performing different operations. In other embodiments, a plurality of sparse instructions may be included, each instruction corresponding to one or more different operation modes, thereby performing various operations related to structured sparsity. By providing a special sparse instruction to perform operations related to structured sparsity, processing can be simplified, thereby improving the processing efficiency of the machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an illustrative and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0012] Figure 1 is a structural diagram showing a board according to an embodiment of the present disclosure;

[0013] Figure 2 is a structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0014] Figure 3 is a schematic diagram showing the internal structure of a single-core computing device according to an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram showing the internal structure of a multi-core computing device according to an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram showing the internal structure of a processor core according to an embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram showing the structure of a data processing device according to an embodiment of the present disclosure;

[0018] Figure 7A is an exemplary pipeline operation circuit illustrating structured sparse processing according to an embodiment of the present disclosure;

[0019] Figure 7B is an exemplary pipeline operation circuit illustrating structured sparse processing according to another embodiment of the present disclosure; and

[0020] Figure 8 is an exemplary flow chart illustrating a data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of this disclosure in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this disclosure, not all of them. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this disclosure.

[0022] It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, specification, and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0023] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0024] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context.

[0025] The specific embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1As shown, board 10 includes chip 101, which is a system-on-chip (SoC), or system-on-chip, integrated with one or more combined processing devices. The combined processing device is an artificial intelligence computing unit that supports various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud-based intelligent applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. Board 10 of this embodiment is suitable for cloud-based intelligent applications and has extensive off-chip storage, on-chip storage, and powerful computing capabilities.

[0027] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. Calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface formats, such as a PCIe interface.

[0028] Board 10 also includes a memory device 104 for storing data, which includes one or more storage units 105. Memory device 104 is connected to a control device 106 and chip 101 via a bus for data transmission. Control device 106 in board 10 is configured to control the state of chip 101. To this end, in one application scenario, control device 106 may include a microcontroller (MCU).

[0029] Figure 2 FIG. 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0030] The computing device 201 is configured to perform user-specified operations and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0031] Interface device 202 is used to transmit data and control instructions between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to a storage device on-chip of computing device 201. Furthermore, computing device 201 can obtain control instructions from processing device 203 via interface device 202 and write them to a control cache on-chip of computing device 201. Alternatively or optionally, interface device 202 can also read data from the storage device of computing device 201 and transmit it to processing device 203.

[0032] The processing device 203 is a general processing device that performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, which include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, with respect to the computing device 201 disclosed herein, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0033] The storage device 204 is used to store data to be processed, which may be DRAM or DDR memory, and is typically 16G or larger in size, for storing data of the computing device 201 and / or the processing device 203 .

[0034] Figure 3 The single-core computing device 301 is used to process input data for computer vision, speech, natural language, data mining, etc. The single-core computing device 301 includes three modules: a control module 31, a computing module 32, and a storage module 33.

[0035] The control module 31 coordinates and controls the operations of the computing module 32 and the storage module 33 to complete deep learning tasks. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 retrieves instructions from the processing device 203, while the instruction decode unit 312 decodes the retrieved instructions and sends the decoded results as control information to the computing module 32 and the storage module 33.

[0036] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations, supporting complex operations such as vector multiplication, addition, and nonlinear transformations. The matrix operation unit 322 is responsible for the core calculations of the deep learning algorithm, namely matrix multiplication and convolution.

[0037] The storage module 33 is used to store or transfer relevant data and includes a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access (DMA) module 333. NRAM 331 stores input neurons, output neurons, and intermediate computational results; WRAM 332 stores the convolution kernels (i.e., weights) of the deep learning network; and DMA 333, which connects to DRAM 204 via bus 34, transfers data between the single-core computing device 301 and DRAM 204.

[0038] Figure 4 The figure shows the internal structure of a multi-core computing device 201. The multi-core computing device 41 employs a layered design. As a system-on-chip (SoC), the multi-core computing device 41 includes at least one cluster, each of which includes multiple processor cores. In other words, the multi-core computing device 41 is structured in a hierarchy of SoC-cluster-processor cores.

[0039] At the system-on-chip level, Figure 4 As shown, the multi-core computing device 41 includes an external storage controller 401 , a peripheral communication module 402 , an on-chip interconnect module 403 , a synchronization module 404 and multiple clusters 405 .

[0040] There can be multiple external storage controllers 401, and two are shown in the figure as an example. They are used to respond to access requests issued by the processor core and access external storage devices, such as Figure 2DRAM204 in the chip, thereby reading data from outside the chip or writing data. The peripheral communication module 402 is used to receive control signals from the processing device 203 through the interface device 202, and start the computing device 201 to perform tasks. The on-chip interconnect module 403 connects the external storage controller 401, the peripheral communication module 402 and multiple clusters 405 to transmit data and control signals between each module. The synchronization module 404 is a global synchronization barrier controller (GBC) used to coordinate the work progress of each cluster and ensure information synchronization. Multiple clusters 405 are the computing cores of the multi-core computing device 41. Four are shown as an example in the figure. With the development of hardware, the multi-core computing device 41 disclosed in this disclosure can also include 8, 16, 64, or even more clusters 405. Clusters 405 are used to efficiently execute deep learning algorithms.

[0041] At the cluster level, Figure 4 As shown, each cluster 405 includes multiple processor cores (IPU cores) 406 and a memory core (MEM core) 407 .

[0042] The figure shows four processor cores 406 as an example, but the present disclosure does not limit the number of processor cores 406. Figure 5 Each processor core 406 is similar to Figure 3 The single-core computing device 301 also includes three major modules: a control module 51, a computing module 52, and a storage module 53. The functions and structures of the control module 51, computing module 52, and storage module 53 are roughly the same as those of the control module 31, computing module 32, and storage module 33, and will not be repeated here. It should be noted that the storage module 53 includes an input / output direct memory access module (IODMA) 533 and a move direct memory access module (MVDMA) 534. The IODMA 533 controls memory access between the NRAM 531 / WRAM 532 and the DRAM 204 via the broadcast bus 409; the MVDMA 534 is used to control memory access between the NRAM 531 / WRAM 532 and the storage unit (SRAM) 408.

[0043] Back to Figure 4The storage core 407 is primarily used for storage and communication, namely, storing shared data or intermediate results between the processor cores 406, and performing communication between the cluster 405 and the DRAM 204, between the clusters 405, and between the processor cores 406. In other embodiments, the storage core 407 has scalar operation capabilities and is used to perform scalar operations.

[0044] The storage core 407 includes SRAM 408, a broadcast bus 409, a cluster direct memory access module (CDMA) 410, and a global direct memory access module (GDMA) 411. SRAM 408 acts as a high-performance data transfer station. Data reused between different processor cores 406 within the same cluster 405 does not need to be obtained from DRAM 204 by each processor core 406. Instead, it is transferred between the processor cores 406 via SRAM 408. The storage core 407 only needs to quickly distribute the reused data from SRAM 408 to multiple processor cores 406, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip input / output access.

[0045] The broadcast bus 409, CDMA 410, and GDMA 411 are used for communication between the processor cores 406, communication between the clusters 405, and data transmission between the clusters 405 and the DRAM 204, respectively. Each of these will be described below.

[0046] Broadcast bus 409 facilitates high-speed communication between processor cores 406 within cluster 405. In this embodiment, broadcast bus 409 supports inter-core communication methods including unicast, multicast, and broadcast. Unicast refers to point-to-point data transmission (e.g., from one processor core to another), multicast transfers a copy of data from SRAM 408 to a specific number of processor cores 406, and broadcast transfers a copy of data from SRAM 408 to all processor cores 406, a special case of multicast.

[0047] The CDMA 410 is used to control memory access to the SRAM 408 between different clusters 405 within the same computing device 201 .

[0048] GDMA 411 works in conjunction with external memory controller 401 to control memory access from cluster 405's SRAM 408 to DRAM 204, or to read data from DRAM 204 to SRAM 408. As previously mentioned, communication between DRAM 204 and NRAM 431 or WRAM 432 can be achieved via two channels. The first channel directly connects DRAM 204 and NRAM 431 or WRAM 432 via IODAM 433. The second channel first transfers data between DRAM 204 and SRAM 408 via GDMA 411, and then transfers data between SRAM 408 and NRAM 431 or WRAM 432 via MVDMA 534. While the second channel may appear to require more components and a longer data flow, in some embodiments, the bandwidth of the second channel is significantly greater than the first channel. Therefore, communication between DRAM 204 and NRAM 431 or WRAM 432 through the second channel may be more efficient. The embodiments of the present disclosure can select a data transmission channel according to the hardware conditions.

[0049] In other embodiments, the functions of GDMA 411 and IODMA 533 can be integrated into the same component. For ease of description, this disclosure treats GDMA 411 and IODMA 533 as separate components. For those skilled in the art, as long as the functions implemented and the technical effects achieved are similar to those disclosed herein, they fall within the scope of protection of this disclosure. Furthermore, the functions of GDMA 411, IODMA 533, CDMA 410, and MVDMA 534 can also be implemented by the same component.

[0050] An embodiment of the present disclosure provides a data processing solution based on the aforementioned hardware environment, which performs operations related to structured sparsity according to specialized sparse instructions.

[0051] Figure 6 FIG. 6 is a block diagram showing a data processing device 600 according to an embodiment of the present disclosure. The data processing device 600 may be implemented, for example, in Figure 2 As shown in the figure, the data processing device 600 may include a control circuit 610, a storage circuit 620 and an operation circuit 630.

[0052] The function of the control circuit 610 may be similar to Figure 3 The control module 31 or Figure 5 The control module 51 may include, for example, an instruction fetch unit for obtaining an instruction from, for example, Figure 2The processing device 203 receives the instruction, and the instruction decoding unit is used to decode the acquired instruction and send the decoding result as control information to the operation circuit 630 and the storage circuit 620.

[0053] In one embodiment, the control circuitry 610 may be configured to parse sparse instructions, wherein the sparse instructions indicate operations related to structured sparsity.

[0054] The storage circuit 620 may be configured to store information before and / or after sparsification. In one embodiment, the operands of the sparse instruction are data in a neural network, such as weights, neurons, etc. In this embodiment, the storage circuit may be, for example, Figure 3 WRAM 332, NRAM 331 or Figure 5 WRAM 532, NRAM 531.

[0055] The operation circuit 630 may be configured to perform corresponding operations according to the sparse instruction.

[0056] In some embodiments, the operation circuit 630 may include one or more groups of pipeline operation circuits 631, wherein each group of pipeline operation circuits 631 may include one or more operators. When each group of pipeline operation circuits includes multiple operators, these multiple operators may be configured to perform multi-stage pipeline operations, that is, to form a multi-stage operation pipeline.

[0057] In some application scenarios, the pipelined operation circuit disclosed herein can support operations related to structured sparsity. For example, when performing structured sparsity processing, a multi-stage pipelined operation circuit composed of comparators and other circuits can be used to extract n data elements as valid data elements from every m data elements, where m>n. In one implementation, m=4 and n=2. In other implementations, n can also take other values, such as 1 or 3.

[0058] In one embodiment, the operation circuit 630 may further include an operation processing circuit 632, which may be configured to pre-process the data before the pipeline operation circuit 631 performs the operation or post-process the data after the operation according to the operation instruction. In some application scenarios, the aforementioned pre-processing and post-processing may, for example, include data splitting and / or data splicing operations. In structured sparse processing, the operation processing circuit may segment the data to be sparsed according to every m data elements and then send it to the pipeline operation circuit 631 for processing.

[0059] Figure 7A FIG. 4 shows an exemplary operation pipeline of structured sparse processing according to an embodiment of the present disclosure. Figure 7AIn the embodiment of FIG. 1 , when m=4 and n=2, a structured sparse processing is shown in which two data elements with larger absolute values ​​are selected from four data elements A, B, C, and D.

[0060] like Figure 7A As shown, the structured sparse processing can be performed using a four-stage pipeline operation circuit consisting of an absolute value operator and a comparator.

[0061] The first-stage pipeline operation circuit may include four absolute value operators 710 for synchronously performing absolute value operations on four input data elements A, B, C and D respectively.

[0062] The second-stage pipeline circuit may include two comparators for grouping and comparing the four absolute values ​​output by the previous stage. For example, the first comparator 721 may compare the absolute values ​​of data elements A and B and output the larger value, Max00, while the second comparator 722 may compare the absolute values ​​of data elements C and D and output the larger value, Max10.

[0063] The third stage pipeline operation circuit may include a third comparator 730 that compares the two larger values ​​Max00 and Max10 outputted from the previous stage and outputs the larger value Max0. This larger value Max0 is the value with the largest absolute value among the four data elements.

[0064] The fourth stage pipeline circuit may include a fourth comparator 740 that compares the smaller value Min0 in the previous stage with another value in the group containing the maximum value Max0 and outputs a larger value Max1. This larger value Max1 is the value with the second largest absolute value among the four data elements.

[0065] Therefore, structured sparse processing of four-choose-two can be achieved through a four-stage pipeline operation circuit.

[0066] Figure 7B FIG. 4 shows an exemplary operation pipeline of structured sparse processing according to another embodiment of the present disclosure. Figure 7B In the embodiment of FIG. 1 , when m=4 and n=2, a structured sparse processing is shown in which two data elements with larger absolute values ​​are selected from four data elements A, B, C, and D.

[0067] like Figure 7B As shown, the above-mentioned structured sparse processing can be performed using a multi-stage pipeline operation circuit composed of an absolute value operator, a comparator, etc.

[0068] The first pipeline stage may include m(4) absolute value operators 750 for synchronously performing absolute value operations on the four input data elements A, B, C, and D. To facilitate the final output of valid data elements, in some embodiments, the first pipeline stage simultaneously outputs the original data elements (i.e., A, B, C, and D) and the data after the absolute value operations (i.e., |A|, |B|, |C|, and |D|).

[0069] The second pipeline stage may include a permutation and combination circuit 760 for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values, and the positions of the m absolute values ​​in each group of data are different.

[0070] In some embodiments, the permutation and combination circuit can be a circular shifter that performs m-1 circular shifts on the permutation of m absolute values ​​(e.g., |A|, |B|, |C|, and |D|), thereby generating m groups of data. For example, in the example shown in the figure, four groups of data are generated, namely: {|A|, |B|, |C|, |D|}, {|B|, |C|, |D|, |A|}, {|C|, |D|, |A|, |B|}, and {|D|, |A|, |B|, |C|}. Similarly, when each group of data is output, the corresponding original data element is also output, and each group of data corresponds to one original data element.

[0071] The third pipeline stage includes a comparison circuit 770 for comparing the absolute values ​​of the m groups of data and generating a comparison result.

[0072] In some embodiments, the third pipeline stage may include m comparison circuits, each comparison circuit includes m-1 comparators (771, 772, 773), and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, where 1≤i≤m.

[0073] As can be seen from the figure, the third pipeline stage can also be considered as m-1 (3) sub-pipeline stages. Each sub-pipeline stage includes m comparators, which are used to compare its corresponding absolute value with other absolute values. The m-1 sub-pipeline stages thus sequentially compare their corresponding absolute value with the other m-1 absolute values.

[0074] For example, in the example shown in the figure, the four comparators 771 in the first sub-pipeline stage are used to compare the first absolute value with the second absolute value of the four sets of data, and output comparison results w0, x0, y0, and z0, respectively. The four comparators 772 in the second sub-pipeline stage are used to compare the first absolute value with the third absolute value of the four sets of data, and output comparison results w1, x1, y1, and z1, respectively. The four comparators 773 in the third sub-pipeline stage are used to compare the first absolute value with the fourth absolute value of the four sets of data, and output comparison results w2, x2, y2, and z2, respectively.

[0075] Thus, the comparison result of each absolute value with the other m-1 absolute values ​​can be obtained.

[0076] In some embodiments, the comparison result can be represented using a bitmap. For example, at the first comparator of the first comparison circuit, when |A| ≥ |B|, w0 = 1; at the second comparator of the first comparison circuit, when |A| < |C|, w1 = 0; at the third comparator of the first comparison circuit, when |A| ≥ |D|, w2 = 1. Therefore, the output result of the first comparison circuit is {A, w0, w1, w2}, which is {A, 1, 0, 1} at this time. Similarly, the output result of the second comparison circuit is {B, x0, x1, x2}, the output result of the third comparison circuit is {C, y0, y1, y2}, and the output result of the fourth comparison circuit is {D, z0, z1, z2}.

[0077] The fourth pipeline stage includes a filter circuit 780, which selects n data elements with larger absolute values ​​from the m data elements as valid data elements based on the comparison results of the third stage, and outputs these valid data elements and their corresponding indexes. The index indicates the position of these valid data elements within the m input data elements. For example, when A and C are selected from the four data elements A, B, C, and D, their corresponding indexes may be 0 and 2.

[0078] Based on the comparison results, appropriate logic can be designed to select n data elements with larger absolute values. Considering the possibility of multiple data elements with the same absolute value, in a further embodiment, when there are data elements with the same absolute value, they are selected according to a specified priority order. For example, the priority can be fixed from low to high in the index order, with A having the highest priority and D having the lowest priority. In one example, when the absolute values ​​of A, C, and D are the same and greater than the absolute value of B, the data selected are A and C.

[0079] From the previous comparison results, we can see that based on w0, w1, and w2, |A| is greater than {|B|, |C|, |D|}. If w0, w1, and w2 are all 1, |A| is greater than |B|, |C|, and |D|, and is the maximum of the four numbers, so A is selected. If two of w0, w1, and w2 are 1, |A| is the second largest of the four absolute values, so A is also selected. Otherwise, A is not selected. Therefore, in some embodiments, analysis and judgment can be made based on the number of occurrences of these values.

[0080] In one implementation, valid data elements can be selected based on the following logic. First, the number of times each data element is greater than other data elements can be counted. For example, define N A =sum_w=w0+w1+w2, N B =sum_x=x0+x1+x2, N C =sum_y=y0+y1+y2, N D =sum_z=z0+z1+z2. Next, make a judgment and selection according to the following conditions.

[0081] The condition for selecting A is: N A =3, or N A =2 and N B / N C / N D There is only one 3 in it;

[0082] The condition for selecting B is: N B =3, or N B =2 and N A / N C / N D There is only one 3 in N A ≠2;

[0083] The conditions for selecting C are: N C =3, and N A / N B There is at most one 3, or N C =2 and N A / N B / N D There is only one 3 in N A / N B There is no 2 in

[0084] The conditions for selecting D are: N D =3, and N A / N B / N C There is at most one 3, or N D =2 and N A / N B / N C There is only one 3 in N A / N B / N C There is no 2 in it.

[0085] Those skilled in the art will appreciate that in order to ensure selection according to a predetermined priority, there is a certain degree of redundancy in the above logic. Based on the size and order information provided by the comparison results, those skilled in the art can design other logic to implement the screening of valid data elements, and this disclosure is not limited in this respect. Figure 7B The multi-stage pipeline operation circuit can also realize four-choose-two structured sparse processing.

[0086] Those skilled in the art will appreciate that other forms of pipeline operation circuits may be designed to implement structured sparse processing, and the present disclosure is not limited in this regard.

[0087] As mentioned above, the operands of the sparse instruction can be data in the neural network, such as weights, neurons, etc. The data in the neural network usually contains multiple dimensions. For example, in a convolutional neural network, the data may have four dimensions: input channel, output channel, length and width. In some embodiments, the above-mentioned sparse instruction can be used for structured sparse processing of at least one dimension of multidimensional data in the neural network. Specifically, in one implementation, the sparse instruction can be used for structured sparse processing of the input channel dimension of multidimensional data in the neural network, such as in the inference process or forward training process of the neural network. In another implementation, the sparse instruction can be used to perform structured sparse processing on the input channel dimension and the output channel dimension of the multidimensional data in the neural network at the same time, such as in the reverse training process of the neural network.

[0088] In one embodiment, in response to receiving multiple sparse instructions, one or more multi-stage pipeline arithmetic circuits of the present disclosure may be configured to perform multiple data operations, such as single instruction multiple data ("SIMD") instructions. In another embodiment, the multiple operations performed by each stage of the arithmetic circuits are predetermined based on the functions supported by the multiple arithmetic circuits arranged stage by stage in the multi-stage arithmetic pipeline.

[0089] In the context of the present disclosure, the aforementioned multiple sparse instructions may be microinstructions or control signals running within one or more multi-stage operation pipelines, which may include (or indicate) one or more operations to be performed by the multi-stage operation pipeline. Depending on different operation scenarios, the operation may include but is not limited to arithmetic operations such as convolution operations and matrix multiplication operations, logical operations such as AND operations, XOR operations, and OR operations, shift operations, or any combination of the aforementioned types of operations.

[0090] Figure 8FIG. 8 is a flow chart illustrating an exemplary data processing method 800 according to an embodiment of the present disclosure.

[0091] like Figure 8 As shown, in step 810, parse the sparse instruction, which indicates an operation related to structured sparse. This step can be performed by Figure 6 The control circuit 610 is used to execute.

[0092] Next, in step 820, the corresponding operands are read according to the sparse instruction. The sparse instruction can indicate different operation modes, and the corresponding operands are also different, which will be described in detail later. This step can be performed by, for example Figure 6 The control circuit 610 is executed for the storage circuit 620.

[0093] Next, in step 830, operations related to structured sparsity are performed on the read operands. This step can be performed by, for example Figure 6 The operation circuit 630 is used to execute.

[0094] Finally, in step 840 , the operation result is output. For example, the operation result can be output by the operation circuit 630 to the storage circuit 620 for subsequent use.

[0095] There may be various operations related to structured sparsity, such as structured sparsity processing, anti-sparseness processing, etc. Various instruction schemes may be designed to implement operations related to structured sparsity.

[0096] In one solution, a sparse instruction may be designed, and the instruction may include an operation mode bit to indicate different operation modes of the sparse instruction, thereby performing different operations.

[0097] In another solution, multiple sparse instructions can be designed, each corresponding to one or more different operating modes, thereby performing different operations. In one implementation, a corresponding sparse instruction can be designed for each operating mode. In another implementation, the operating modes can be classified according to their characteristics, and a sparse instruction can be designed for each type of operating mode. Furthermore, when a certain type of operating mode includes multiple operating modes, an operating mode bit can be included in the sparse instruction to indicate the corresponding operating mode.

[0098] Regardless of which solution is adopted, the sparse instruction may indicate its corresponding operation mode through an operation mode bit and / or the instruction itself.

[0099] In one embodiment, the sparse instruction may indicate a first operating mode. In the first operating mode, the operand of the sparse instruction includes data to be sparsed. In this case, the operation circuit 630 may be configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and output the structured data after the sparse processing to the storage circuit 620.

[0100] The structured sparse processing in the first operation mode can be a structured sparse processing according to a predetermined screening rule, for example, according to the rule of screening the largest absolute value, from every m data elements, n data elements with larger absolute values ​​are screened as valid data elements. The operation circuit 630 can be configured to refer to Figure 7A and Figure 7B The pipeline circuit described is used to perform this structured sparse processing.

[0101] The result of the sparsification process consists of two parts: a data part and an index part. The data part includes the data after the sparsification process, that is, the valid data elements extracted according to the screening rules of the structured sparsification process. The index part is used to indicate the data after sparsification, that is, the location of the valid data elements in the data before sparsification (that is, the data to be sparsified).

[0102] The structure in the disclosed embodiments includes a data portion and an index portion that are bound to each other. In some embodiments, each bit in the index portion can correspond to a data element. For example, when the data type is fix8, a data element is 8 bits, and each bit in the index portion can correspond to 8 bits of data. In other embodiments, considering the hardware implementation when the structure is subsequently used, each bit in the index portion of the structure can be set to correspond to a position of N bits of data, where N is determined at least in part based on the hardware configuration. For example, each bit in the index portion of the structure can be set to correspond to a position of 4 bits of data. For example, when the data type is fix8, every 2 bits in the index portion correspond to a data element of the fix8 type. In some embodiments, the data portion of the structure can be aligned according to a first alignment requirement, and the index portion of the structure can be aligned according to a second alignment requirement, so that the entire structure also meets the alignment requirement. For example, the data portion can be aligned according to 64B, the index portion can be aligned according to 32B, and the entire structure can be aligned according to 96B (64B + 32B). This alignment requirement can reduce the number of memory accesses during subsequent use and improve processing efficiency.

[0103] By using this structure, the data and index components can be used together. Since the ratio of valid data elements to all data elements in structured sparse processing is fixed, for example, n / m, the size of the data after sparse processing is also fixed or predictable. As a result, the structure can be stored densely in the storage circuit without performance loss.

[0104] In another embodiment, the sparse instruction may indicate a second operation mode. The second operation mode differs from the first operation mode in that the second operation mode outputs only the data portion after structured sparse processing, but does not output the index portion.

[0105] Similarly, in the second operating mode, the operand of the sparse instruction includes the data to be sparsed. In this case, the operation circuit 630 can be configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output the data portion after the sparse processing to the storage circuit 620. The data portion includes the data after the sparse processing of the data to be sparsed. The data portion is densely stored in the storage circuit. The output data portion is aligned according to n elements. For example, in the example of m=4 and n=2, the input data to be sparsed is aligned according to 4 elements, while the output data portion is aligned according to 2 elements.

[0106] In another embodiment, the sparse instruction may indicate a third operation mode. The third operation mode differs from the first operation mode in that the output content is different, and the third operation mode only outputs the index part after structured sparse processing, but does not output the data part.

[0107] Similarly, in the third operating mode, the operand of the sparse instruction includes the data to be sparsified. At this time, the operation circuit 630 can be configured to perform structured sparse processing on the data to be sparsified according to the sparse instruction, and output the index part after the sparse processing to the storage circuit 620. The index part indicates the position of the sparse data in the data to be sparsified. The index part is densely stored in the storage circuit. Each bit in the output index part corresponds to the position of a data element. Since the index part can be used alone, for example, for structured sparsification of neurons in subsequent convolution processing, and the data type of the neurons may be uncertain, by corresponding each bit in the index part to the position of a data element, the independently stored index part can be applicable to various data types.

[0108] In another embodiment, the sparse instruction can indicate a fourth operating mode. The difference between the fourth operating mode and the first operating mode is that the fourth operating mode specifies the screening rules of the structured sparse processing, rather than performing the structured sparse processing according to a predetermined screening rule (for example, the previous absolute value larger rule). At this time, the sparse instruction has two operands: the data to be sparsed and the sparse index. The added sparse index operand is used to indicate the position of the valid data elements in the structured sparse processing to be executed, that is, to specify the screening rules of the structured sparse processing. Each bit in the sparse index corresponds to the position of a data element, and thus can be applied to data to be sparsed of various data types.

[0109] In the fourth operating mode, the operation circuit 630 can be configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and the location indicated by the sparse index, and output the result of the sparse processing to the storage circuit. In one implementation, the output result can be a structure after the sparse processing. In another implementation, the output result can be a portion of the data after the sparse processing.

[0110] The structure has the same meaning as in the first operating mode. It includes a bound data portion and an index portion. The data portion contains the data after the sparsification process, and the index portion indicates the location of the sparsified data within the data to be sparsified. The alignment requirements and corresponding relationships between the data and index portions in the structure are the same as in the first operating mode and are not repeated here.

[0111] The above four operation modes provide structured sparse processing of data, such as processing according to predetermined filtering rules or filtering rules specified by the instruction operands, and provide different output contents, such as outputting a structure, outputting only the data portion, outputting only the index portion, etc. The above instruction design can well support structured sparse processing and provides multiple output options to suit different scenario requirements. For example, when data and indexes need to be bound together, you can choose to output a structure, while when the index portion or data portion needs to be used separately, you can choose to output only the index portion or the data portion.

[0112] In another embodiment, the sparse instruction may indicate a fifth operation mode. The fifth operation mode does not require structured sparse processing, but only requires binding separate or independent data parts and index parts into a structure.

[0113] In the fifth operating mode, the operand of the sparse instruction includes the data portion after the sparsification process and the corresponding index portion. The data portion and the index portion are respectively in dense storage format, but are not bound. The input data portion is aligned according to n elements. For example, in the example of m=4, n=2, the input data portion is aligned according to 2 elements. The index portion indicates the position of the data portion in the data before the sparsification process, where each bit of the index portion corresponds to a data element.

[0114] At this time, the operation circuit 630 can be configured to bind the data part and the index part into a structure according to the sparse instruction, and output the structure to the storage circuit. The meaning of the structure, the alignment requirements for the data part and the index part, the corresponding relationship, etc. are the same as those in the first operation mode and will not be repeated here. Depending on the data type of the data element, it is necessary to generate the index part in the structure accordingly based on the data type and the bit correspondence of the index part in the structure. For example, when the input index part is 0011, where each bit corresponds to one data element, if the data type is fix8, that is, each data element has 8 bits, then according to the corresponding relationship between each 1 bit of the index part in the structure and 4 bits of data, the index part in the structure should be: 00001111, that is, 2 bits correspond to one data element.

[0115] In yet another embodiment, the sparse instruction may indicate a sixth operation mode for performing a desparse process, that is, restoring the sparse data to the format or size of the data before the sparse process.

[0116] In the sixth operating mode, the operands of the sparse instruction include the data portion after the sparsification process and the corresponding index portion. The data portion and the index portion are respectively in dense storage format, but are not bound. The input data portion is aligned according to n elements. For example, in the example of m=4, n=2, the input data portion is aligned according to 2 elements, and the output data is aligned according to 4 elements. The index portion indicates the position of the data portion in the data before the sparsification process, where each bit of the index portion corresponds to a data element.

[0117] At this time, the operation circuit 630 can be configured to perform desparse processing on the input data part according to the sparse instruction and the position indicated by the input index part to generate recovery data having the data format before the sparse processing, and output the recovery data to the storage circuit.

[0118] In one implementation, the desparseening process may include: placing each data element in the data part at a corresponding position in the data format before the sparseening process according to the position indicated by the index part and in accordance with the data format before the sparseening process, and filling predetermined information (for example, filling 0) at the remaining positions of the data format to generate recovered data.

[0119] As can be seen from the above description, the present disclosure provides a sparse instruction for performing operations related to structured sparsity. These operations may include forward structured sparse operations, may also include desparse operations, and may also include some related format conversion operations. In some embodiments, the sparse instruction may include an operation mode bit to indicate different operation modes of the sparse instruction, thereby performing different operations. In other embodiments, a plurality of sparse instructions may be directly provided, each instruction corresponding to one or more different operation modes, thereby performing various operations related to structured sparsity. By providing special sparse instructions to perform operations related to structured sparsity, processing can be simplified, thereby improving the processing efficiency of the machine.

[0120] According to different application scenarios, the electronic devices or devices disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, Internet of Things terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, vehicles, household appliances, and / or medical equipment. The vehicles include airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging (MRI), ultrasound machines and / or electrocardiographs. The electronic devices or devices disclosed herein may also be applied to the Internet, Internet of Things, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, medical care and other fields. Furthermore, the electronic devices or devices disclosed herein may also be used in cloud, edge, terminal and other application scenarios related to artificial intelligence, big data and / or cloud computing. In one or more embodiments, electronic devices or apparatuses with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or apparatuses with low power consumption can be applied to terminal devices and / or edge devices (such as smartphones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0121] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will understand that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teachings of the present disclosure, those skilled in the art will understand that some of the steps therein can be performed in other orders or simultaneously. Further, those skilled in the art will understand that the embodiments described in the present disclosure can be regarded as optional embodiments, that is, the actions or modules involved therein are not necessarily necessary for the implementation of one or more schemes of the present disclosure. In addition, depending on the different schemes, the description of some embodiments of the present disclosure also has different emphases. In view of this, those skilled in the art will understand that the parts that are not described in detail in a certain embodiment of the present disclosure may also refer to the relevant descriptions of other embodiments.

[0122] In terms of specific implementation, based on the disclosure and teachings of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented in other ways not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them based on the consideration of logical functions, and there may be other ways of splitting them in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. With respect to the connection relationship between different units or components, the connection discussed above in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0123] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network elements. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the solution described in the embodiments of this disclosure. In addition, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically separately.

[0124] In some other implementation scenarios, the above-mentioned integrated units can also be implemented in the form of hardware, that is, as specific hardware circuits, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which can be, for example, resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), ROM and RAM, etc.

[0125] The foregoing content can be better understood in accordance with the following terms:

[0126] Clause 1. A data processing apparatus comprising:

[0127] control circuitry configured to parse sparse instructions indicating structured sparsity-related operations;

[0128] a storage circuit configured to store information before and / or after sparsification; and

[0129] The operation circuit is configured to perform corresponding operations according to the sparse instruction.

[0130] Clause 2. The data processing apparatus of clause 1, wherein the sparse instruction indicates a first operating mode, and an operand of the sparse instruction includes data to be sparsified,

[0131] The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output a structure after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0132] Clause 3. The data processing apparatus of clause 1, wherein the sparse instruction indicates a second operating mode, and an operand of the sparse instruction includes data to be sparsified,

[0133] The operation circuit is configured to perform structured sparse processing on the data to be thinned out according to the sparse instruction, and output the data portion after the sparse processing to the storage circuit, where the data portion includes the data after the sparse processing of the data to be thinned out.

[0134] Clause 4. The data processing apparatus of clause 1, wherein the sparse instruction indicates a third operating mode, and an operand of the sparse instruction includes data to be sparsified,

[0135] The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output an index part after the sparse processing to the storage circuit, where the index part indicates the position of the sparse data in the data to be sparse.

[0136] Clause 5. The data processing apparatus of clause 1, wherein the sparse instruction indicates a fourth operating mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, the sparse index indicating a position of a valid data element in the structured sparsification to be performed,

[0137] The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and the position indicated by the sparse index, and output the structure after sparse processing or the data part after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0138] Clause 6. The data processing apparatus according to clause 1, wherein the sparse instruction indicates a fifth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification process and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification process,

[0139] The operation circuit is configured to bind the data part and the index part into a structure according to the sparse instruction, and output the structure to the storage circuit.

[0140] Clause 7. The data processing apparatus according to clause 1, wherein the sparse instruction indicates a sixth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification process and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification process,

[0141] The operation circuit is configured to perform inverse sparsification processing on the data portion according to the sparse instruction and the position indicated by the index portion to generate restored data having the data format before the sparsification processing, and output the restored data to the storage circuit.

[0142] Clause 8. A data processing apparatus according to any one of clauses 2 to 5, wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

[0143] Item 9. A data processing device according to Item 8, wherein the operation circuit further includes: at least one multi-stage pipeline operation circuit, which includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

[0144] Clause 10. The data processing apparatus according to clause 9, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein:

[0145] The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values;

[0146] The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different;

[0147] The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; and

[0148] The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

[0149] Item 11. A data processing device according to Item 10, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

[0150] Clause 12. The data processing apparatus according to any one of clauses 10-11, wherein the screening circuit is further configured to select data elements according to a specified priority order when there are data elements with the same absolute value.

[0151] Clause 13. The data processing apparatus according to clause 7, wherein the desparseness processing comprises:

[0152] According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

[0153] Clause 14. A data processing apparatus according to clause 2, 5 or 6, wherein:

[0154] Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or

[0155] The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

[0156] Clause 15. A data processing apparatus according to any one of clauses 1 to 14, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

[0157] Clause 16. The data processing apparatus of clause 15, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

[0158] Clause 17. A data processing apparatus according to any one of clauses 1 to 16, wherein

[0159] The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or

[0160] The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

[0161] Clause 18. A chip comprising the data processing device according to any one of Clauses 1 to 17.

[0162] Clause 19. A board comprising the chip according to clause 18.

[0163] Article 20. A data processing method comprising:

[0164] Parsing a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparseness;

[0165] Reading corresponding operands according to the sparse instruction;

[0166] performing the structured sparsity-related operation on the operand; and

[0167] Output the operation results.

[0168] Clause 21. The data processing method according to clause 20, wherein the sparse instruction indicates a first operating mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising:

[0169] According to the sparse instruction, performing structured sparse processing on the data to be sparsed; and

[0170] Output the structure after the sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after the sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0171] Clause 22. The data processing method according to Clause 20, wherein the sparse instruction indicates a second operating mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising:

[0172] According to the sparse instruction, performing structured sparse processing on the data to be sparsed; and

[0173] Output the data portion after the thinning process, where the data portion includes the data after the thinning process of the data to be thinned.

[0174] Clause 23. The data processing method according to Clause 20, wherein the sparse instruction indicates a third operating mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising:

[0175] According to the sparse instruction, performing structured sparse processing on the data to be sparsed; and

[0176] The index portion after the thinning process is output, where the index portion indicates the position of the thinned data in the data to be thinned.

[0177] Clause 24. The data processing method of clause 20, wherein the sparse instruction indicates a fourth operating mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in the structured sparsification to be performed, the method further comprising:

[0178] According to the sparse instruction, performing structured sparse processing on the data to be sparsed according to the position indicated by the sparse index; and

[0179] Output the structure after sparse processing or the data part after sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

[0180] Clause 25. The data processing method according to Clause 20, wherein the sparse instruction indicates a fifth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification processing, and the method further comprises:

[0181] Binding the data portion and the index portion into a structure according to the sparse instruction; and

[0182] Output the structure.

[0183] Clause 26. The data processing method according to Clause 20, wherein the sparse instruction indicates a sixth operating mode, and an operand of the sparse instruction includes a data portion after the sparsification processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification processing, and the method further comprises:

[0184] According to the sparse instruction, performing an inverse sparse process on the data portion according to the position indicated by the index portion to generate restored data having a data format before the sparse process; and

[0185] The restored data is output.

[0186] Clause 27. A data processing method according to any one of clauses 21 to 24, wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

[0187] Clause 28. A data processing method according to Clause 27, wherein the structured sparse processing is implemented using an arithmetic circuit, the arithmetic circuit comprising: at least one multi-stage pipeline arithmetic circuit, which comprises a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

[0188] Clause 29. The data processing method according to clause 28, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein:

[0189] The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values;

[0190] The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different;

[0191] The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; and

[0192] The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

[0193] Clause 30. A data processing method according to Clause 29, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

[0194] Clause 31. A data processing method according to any one of clauses 29-30, wherein the screening circuit is further configured to select data elements according to a specified priority order when there are data elements with the same absolute value.

[0195] Clause 32. The data processing method according to Clause 26, wherein the desparseness processing comprises:

[0196] According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

[0197] Clause 33. A data processing method according to clause 21, 24 or 25, wherein:

[0198] Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or

[0199] The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

[0200] Clause 34. A data processing method according to any one of clauses 20-33, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

[0201] Clause 35. A data processing method according to clause 34, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

[0202] Clause 36. A data processing method according to any one of clauses 20 to 35, wherein

[0203] The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or

[0204] The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

[0205] The above is a detailed introduction to the embodiments of the present disclosure. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core ideas of the present disclosure. At the same time, for those skilled in the art, based on the ideas of the present disclosure, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A data processing device, comprising: control circuitry configured to parse sparse instructions indicating structured sparsity-related operations; a storage circuit configured to store information before and / or after sparsification; as well as an operation circuit configured to perform corresponding operations according to the sparse instruction; The sparse instruction indicates a first operation mode, and an operand of the sparse instruction includes data to be sparsed, The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output a structure after the sparse processing to the storage circuit, wherein the structure includes a data portion and an index portion bound to each other, the data portion includes the data after the sparse processing of the data to be sparsed, and the index portion is used to indicate the position of the sparse data in the data to be sparsed; Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

2. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates the second operation mode, and an operand of the sparse instruction includes data to be sparsed, The operation circuit is configured to perform structured sparse processing on the data to be thinned out according to the sparse instruction, and output the data portion after the sparse processing to the storage circuit, where the data portion includes the data after the sparse processing of the data to be thinned out.

3. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a third operation mode, and an operand of the sparse instruction includes data to be sparsed, The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction, and output an index part after the sparse processing to the storage circuit, where the index part indicates the position of the sparse data in the data to be sparse.

4. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a fourth operation mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in structured sparsification to be performed. The operation circuit is configured to perform structured sparse processing on the data to be sparsed according to the sparse instruction and the position indicated by the sparse index, and output the structure after sparse processing or the data part after sparse processing to the storage circuit, wherein the structure includes a data part and an index part bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

5. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a fifth operation mode, and an operand of the sparse instruction includes a data portion after the sparse processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparse processing. The operation circuit is configured to bind the data part and the index part into a structure according to the sparse instruction, and output the structure to the storage circuit.

6. The data processing apparatus according to claim 1 , wherein the sparse instruction indicates a sixth operation mode, and an operand of the sparse instruction includes a data portion after the sparse processing and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparse processing. The operation circuit is configured to perform inverse sparsification processing on the data portion according to the sparse instruction and the position indicated by the index portion to generate restored data having the data format before the sparsification processing, and output the restored data to the storage circuit. 7 . The data processing apparatus according to claim 1 , wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

8. The data processing device according to claim 7, wherein the operation circuit further comprises: At least one multi-stage pipeline operation circuit includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

9. The data processing apparatus according to claim 8, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein: The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values; The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different; The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; as well as The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

10. The data processing device according to claim 9, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

11. The data processing apparatus according to any one of claims 9 to 10, wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them according to a specified priority order.

12. The data processing apparatus according to claim 6, wherein the desparseness processing comprises: According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

13. The data processing device according to any one of claims 1 to 6, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network. The data processing apparatus according to claim 13 , wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

15. The data processing device according to any one of claims 1 to 6, wherein The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

16. A chip comprising the data processing device according to any one of claims 1 to 15.

17. A board comprising the chip according to claim 16.

18. A data processing method comprising: Parsing a sparse instruction, wherein the sparse instruction indicates an operation related to structured sparseness; Reading corresponding operands according to the sparse instruction; performing the structured sparsity-related operation on the operand; as well as Output the operation results; The sparse instruction indicates a first operation mode, and an operand of the sparse instruction includes data to be sparsed, and the method further includes: According to the sparse instruction, performing structured sparse processing on the data to be sparse; as well as Outputting a structure after the sparse processing, the structure comprising a data portion and an index portion bound to each other, the data portion comprising the data after the sparse processing of the data to be sparsed, and the index portion being used to indicate the position of the sparse data in the data to be sparsed; Each bit in the index portion of the structure corresponds to a position of N bits of data, where N is determined at least in part based on hardware configuration; and / or The data portion of the structure is aligned according to a first alignment requirement, and the index portion of the structure is aligned according to a second alignment requirement.

19. The data processing method according to claim 18, wherein the sparse instruction indicates the second operation mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising: According to the sparse instruction, performing structured sparse processing on the data to be sparse; as well as Output the data portion after the thinning process, where the data portion includes the data after the thinning process of the data to be thinned.

20. The data processing method according to claim 18, wherein the sparse instruction indicates a third operation mode, and an operand of the sparse instruction includes data to be sparsified, the method further comprising: According to the sparse instruction, performing structured sparse processing on the data to be sparse; as well as The index portion after the thinning process is output, where the index portion indicates the position of the thinned data in the data to be thinned.

21. The data processing method according to claim 18, wherein the sparse instruction indicates a fourth operation mode, and an operand of the sparse instruction includes data to be sparsified and a sparse index, wherein the sparse index indicates a position of a valid data element in structured sparsification to be performed, the method further comprising: According to the sparse instruction, and according to the position indicated by the sparse index, performing structured sparse processing on the data to be sparsed; as well as Output the structure after sparse processing or the data part after sparse processing, the structure includes a data part and an index part that are bound to each other, the data part includes the data after sparse processing of the data to be sparsed, and the index part is used to indicate the position of the sparse data in the data to be sparsed.

22. The data processing method according to claim 18, wherein the sparse instruction indicates a fifth operation mode, and an operand of the sparse instruction includes a data portion after the sparsification process and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification process, the method further comprising: Binding the data portion and the index portion into a structure according to the sparse instruction; as well as Output the structure.

23. The data processing method according to claim 18, wherein the sparse instruction indicates a sixth operation mode, and an operand of the sparse instruction includes a data portion after the sparsification process and a corresponding index portion, wherein the index portion indicates a position of the data portion in the data before the sparsification process, the method further comprising: According to the sparse instruction, performing an inverse sparsification process on the data portion according to the position indicated by the index portion to generate restored data having a data format before the sparsification process; as well as The restored data is output.

24. The data processing method according to any one of claims 18 to 21, wherein the structured sparse processing comprises selecting n data elements from every m data elements as valid data elements, where m>n.

25. The data processing method according to claim 24, wherein the structured sparse processing is implemented using an operation circuit, the operation circuit comprising: At least one multi-stage pipeline operation circuit includes a plurality of operators arranged in stages and is configured to perform structured sparse processing of selecting n data elements with larger absolute values ​​from m data elements as valid data elements according to the sparse instruction.

26. The data processing method according to claim 25, wherein the multi-stage pipeline operation circuit comprises four pipeline stages, wherein: The first pipeline stage includes m absolute value operators for respectively taking absolute values ​​of the m data elements to be sparsified to generate m absolute values; The second pipeline stage includes a permutation and combination circuit for permuting and combining the m absolute values ​​to generate m groups of data, wherein each group of data includes the m absolute values ​​and positions of the m absolute values ​​in each group of data are different; The third pipeline stage includes m comparison circuits for comparing the absolute values ​​of the m groups of data and generating a comparison result; as well as The fourth pipeline stage includes a screening circuit for selecting n data elements with larger absolute values ​​as valid data elements according to the comparison result, and outputting the valid data elements and corresponding indexes, where the indexes indicate positions of the valid data elements in the m data elements.

27. The data processing method according to claim 26, wherein each comparison circuit in the third pipeline stage includes m-1 comparators, and the m-1 comparators in the i-th comparison circuit are used to compare an absolute value in the i-th group of data with the other three absolute values ​​in sequence and generate a comparison result, 1≤i≤m.

28. The data processing method according to any one of claims 26-27, wherein the screening circuit is further configured to, when data elements with the same absolute value exist, select them according to a specified priority order.

29. The data processing method according to claim 23, wherein the desparseness processing comprises: According to the position indicated by the index part and in accordance with the data format before the sparse processing, each data element in the data part is placed at the corresponding position of the data format before the sparse processing, and predetermined information is filled in the remaining positions of the data format to generate the restored data.

30. The data processing method according to any one of claims 18 to 23, wherein the sparse instruction is used for structured sparse processing of at least one dimension of multidimensional data in a neural network.

31. The data processing method according to claim 30, wherein the at least one dimension is selected from an input channel dimension and an output channel dimension.

32. The data processing method according to any one of claims 18 to 23, wherein The sparse instruction includes an operation mode bit to indicate the operation mode of the sparse instruction, or The sparse instructions include multiple instructions, each instruction corresponding to one or more different operation modes.

Images