Data processing device, processor, chip, board card and data processing method

Abstract

The invention discloses a data transmission device, a processor, a chip, a board card and a data transmission method. The data transmission device may be included in a computing device, which may be included in a combined processing device, which may also include an interface device and other processing devices. The computing device interacts with other processing devices to jointly complete computing operation specified by a user. The combined processing device can further comprise a storage device, and the storage device is connected with the computing device and the other processing devices and used for storing data of the computing device and the other processing devices. According to the scheme disclosed by the invention, an optimization scheme of convolution operation is provided, the requirement on the pendulum number of software can be saved, and the high-speed computing power of a processor is fully utilized.

Description

Technical Field

[0001] The present disclosure generally relates to the field of computers. More specifically, the present disclosure relates to a data processing device, a processor, a chip, a board, and a data processing method. Background Art

[0002] As the computing power of chips increases, the bandwidth of the arithmetic unit and the number of arithmetic units are also increasing. When the hardware performs a certain mode of operation, in order to achieve higher utilization of the arithmetic unit, there are usually certain alignment requirements for the dimension size of the data. For example, it is required that the size of a certain dimension of the data be aligned to 64B, etc., so that the computing power of the arithmetic unit can be fully utilized and high utilization can be achieved.

[0003] In order to ensure programmers have more flexibility in use, although the hardware has alignment requirements, it generally supports scenarios where the input data is of any size. When the dimension size of the data does not meet the hardware alignment requirements, various measures can be taken to meet them, such as padding the data with zeros to achieve alignment.

[0004] In order to meet the hardware alignment requirement, additional invalid data will be generated in the actual data, including invalid data introduced by zero padding before the operation, and invalid operation results generated by performing operations on these invalid data during the operation. Therefore, a solution is urgently needed to process such invalid data to minimize the impact of invalid data in the entire operation process and reduce processing overhead. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present disclosure proposes a data processing solution and related products in multiple aspects.

[0006] In a first aspect, the present disclosure provides a data processing device, comprising a decoder, a write control circuit and a storage circuit, wherein: the decoder is configured to generate validity indication information for the operation output data of each write back cycle of executing the operation instruction in response to a non-aligned operation instruction, and send it to the write control circuit; and the write control circuit is configured to control the operation output data of the corresponding write back cycle to be written back to the storage circuit according to the validity indication information, so that only the operation output data indicated as valid by the validity indication information is written back to the storage circuit.

[0007] In a second aspect, the present disclosure provides a processor, comprising the data processing device of the first aspect.

[0008] In a third aspect, the present disclosure provides a chip comprising the processor of the aforementioned second aspect.

[0009] In a fourth aspect, the present disclosure provides a board comprising the chip of the third aspect.

[0010] In a fifth aspect, the present disclosure provides a data processing method, comprising: a decoder generates validity indication information for the operation output data of each write back cycle of executing the operation instruction in response to an unaligned operation instruction, and sends the information to a write control circuit; and the write control circuit controls the operation output data of the corresponding write back cycle to be written back to the storage circuit according to the validity indication information, so that only the operation output data indicated as valid by the validity indication information is written back to the storage circuit.

[0011] Through the data processing device, processor, chip, board and data processing method provided above, the disclosed embodiment provides a scheme for centrally processing invalid data after the operation result is output, so as to simplify the overall processing flow. Specifically, by sensing which of the operation output data of each write-back cycle is invalid through the decoder, and providing indication information to the data write-back path through the interface, invalid data can be avoided from being written to the storage circuit. The scheme of the disclosed embodiment can be applied to various operation instructions with alignment requirements, and is particularly suitable for small convolution operation mode and width folding (Wfold) operation mode in convolution operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0013] Figure 1 A structural diagram of a board according to an embodiment of the present disclosure is shown;

[0014] Figure 2 A structural diagram showing a combined processing device according to an embodiment of the present disclosure;

[0015] Figure 3 A schematic diagram showing the internal structure of a processing core when the computing device is a single-core or multi-core device;

[0016] Figure 4 A schematic block diagram of a computing device that can perform various operations is shown;

[0017] Figure 5 An exemplary structural block diagram of a data processing device for implementing the invalid data processing solution of the disclosed embodiment is shown;

[0018] Figure 6a shows the input neuron scale in an exemplary application scenario according to an embodiment of the present disclosure;

[0019] Figure 6b Shows Figure 6a The output neuron scale in the exemplary application scenario;

[0020] Figure 7a shows the input neuron scale in another exemplary application scenario according to an embodiment of the present disclosure;

[0021] Figure 7b Shows Figure 7a The output neuron scale in the exemplary application scenario;

[0022] Figure 8 An exemplary flow chart showing a data processing method of a data processing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0024] It should be understood that the terms "first", "second", "third" and "fourth" etc. that may appear in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0025] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0026] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0027] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings. Exemplary Hardware Environment

[0028] Figure 1 FIG. 1 is a schematic diagram showing the structure of a board 10 according to an embodiment of the present disclosure. Figure 1 As shown, the board 10 includes one or more chips 101, which is a system-on-chip, or system on chip (SoC), integrated with one or more combined processing devices. The combined processing device is an artificial intelligence computing unit that supports various deep learning and machine learning algorithms to meet the intelligent processing needs in complex scenarios in the fields of computer vision, speech, natural language processing, data mining, etc. In particular, deep learning technology is widely used in the field of cloud intelligence. A notable feature of cloud-based intelligent applications is the large amount of input data, which places high demands on the storage capacity and computing power of the platform. The board 10 of this embodiment is suitable for cloud-based intelligent applications, and has huge off-chip storage, on-chip storage and powerful computing power.

[0029] The chip 101 is connected to an external device 103 via an external interface device 102. The external device 103 is, for example, a server, a computer, a camera, a display, a mouse, a keyboard, a network card, or a wifi interface. The data to be processed can be transmitted from the external device 103 to the chip 101 via the external interface device 102. The calculation result of the chip 101 can be transmitted back to the external device 103 via the external interface device 102. According to different application scenarios, the external interface device 102 can have different interface forms, such as a PCIe (Peripheral Component Interconnect express) interface.

[0030] The board 10 also includes a storage device 104 for storing data, which includes one or more storage units 105. The storage device 104 is connected to the control device 106 and the chip 101 through a bus and transmits data. The control device 106 in the board 10 is configured to control the state of the chip 101. To this end, in an application scenario, the control device 106 may include a single chip microcomputer, also known as a micro controller unit (MCU).

[0031] Figure 2 1 is a block diagram showing the combined processing device in the chip 101 of this embodiment. Figure 2 As shown in , the combined processing device 20 includes a computing device 201 , an interface device 202 , a processing device 203 and a storage device 204 .

[0032] The computing device 201 is configured to execute user-specified operations, and is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0033] The interface device 202 is used to transmit data and control instructions between the computing device 201 and the processing device 203. For example, the computing device 201 can obtain input data from the processing device 203 via the interface device 202 and write it into a storage device on the computing device 201 chip. Further, the computing device 201 can obtain control instructions from the processing device 203 via the interface device 202 and write them into a control cache on the computing device 201 chip. Alternatively or optionally, the interface device 202 can also read data from the storage device on the computing device 201 chip and transmit it to the processing device 203.

[0034] The processing device 203, as a general processing device, performs basic controls including but not limited to data handling, starting and / or stopping the computing device 201. Depending on the implementation, the processing device 203 can be a central processing unit (CPU), a graphics processing unit (GPU) or one or more types of processors in other general and / or special processors, these processors include but are not limited to digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing device 201 disclosed in the present invention, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing device 201 and the processing device 203 are integrated and considered together, the two are regarded as forming a heterogeneous multi-core structure.

[0035] The storage device 204 is used to store data to be processed, which may be DRAM (Dynamic Random Access Memory), which is DDR (Double Data Rate) memory, and is usually 16G or larger in size, and is used to store data of the computing device 201 and / or the processing device 203 .

[0036] Figure 3 Shows Figure 2 The internal structure diagram of the processing core when the computing device 201 in FIG. 2 is a single-core or multi-core device. The computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 301 includes three modules: a control module 31 (also called a controller), an operation module 32 (also called an operator), and a storage module 33 (also called a memory).

[0037] The control module 31 is used to coordinate and control the operation of the operation module 32 and the storage module 33 to complete the deep learning task, and includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 is used to obtain instructions from the processing device 203, and the instruction decode unit 312 decodes the obtained instructions and sends the decoding results to the operation module 32 and the storage module 33 as control information.

[0038] The operation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformation; the matrix operation unit 322 is responsible for the core calculation of the deep learning algorithm, namely matrix multiplication and convolution.

[0039] The storage module 33 is used to store or transfer relevant data, including a neuron RAM (NRAM) 331, a weight RAM (WRAM) 332, and a direct memory access module (DMA) 333. NRAM 331 is used to store input neurons, output neurons, and intermediate results after calculation; WRAM 332 is used to store the convolution kernel of the deep learning network, that is, the weight; DMA 333 is connected to DRAM 204 via bus 34, and is responsible for data transfer between the computing device 301 and DRAM 204. It should be noted that the NRAM and WRAM here can be two storage areas formed by dividing the same memory in the logical storage space, or they can be two independent memories, which are not specifically limited here. Exemplary Computing Devices

[0040] Figure 4 FIG. 4 is a schematic block diagram of a computing device 400 that can perform various operations. It can be understood that the structure can be regarded as Figure 3 The computing device 400 can be configured to perform various operations, including but not limited to various convolution operation modes, such as the small convolution operation mode and the width folding operation mode described later.

[0041] exist Figure 4 In a computing device, the processing circuits are grouped and placed, thereby improving the area utilization of the device. Furthermore, based on this hardware layout, a data transmission mechanism is provided to improve the data transmission efficiency through multiple data transmission channels, thereby meeting the data needs of the processing circuits with increasing computing power.

[0042] As shown in the figure, the computing device 400 includes a storage circuit 410 for storing data, such as input neurons, weights, output neurons, etc. in the convolution operation. By storing all input and output data of the convolution operation on the same storage circuit 410, it is convenient to uniformly manage the storage space on the storage circuit. In this regard, the storage circuit 410 is equivalent to Figure 3 The merger of NRAM and WRAM in .

[0043] The computing device 400 also includes a plurality of parallel processing circuits (PP), which are mainly responsible for the core calculations of deep learning algorithms, such as matrix multiplication and convolution operations. These parallel processing circuits can be organized and divided into groups to form a plurality of parallel processing circuit groups (PPB). Different parallel processing circuit groups PPB are located separately from each other at different positions of the circuit board, so as to facilitate full filling of the space on the circuit board and improve area utilization. Four parallel processing circuit groups PPB0 to PPB3 420 are shown in the figure, and are schematically shown as being located around the storage circuit 410. Those skilled in the art will appreciate that the number of parallel processing circuit groups may be more or less, depending on the specific hardware configuration, and the disclosed embodiments are not limited in this respect.

[0044] The storage circuit 410 and the plurality of processing circuit groups PPB may support a variety of data transmission mechanisms to support different multiplexing methods of input neurons and weights, thereby reducing the amount of data access during operation and improving processing efficiency.

[0045] The computing device 400 may also include one or more pre-processing circuits and / or post-processing circuits (not shown in the figure). In some implementations, there may be a pre-processing circuit between the storage circuit 410 and the plurality of processing circuit groups PPB to convert data from the storage circuit 410 into the data type required by the processing circuit group PPB, such as converting float32 data into int16, etc. Optionally or additionally, in some implementations, a post-processing circuit may also be included to perform some vector operations to support complex operations such as vector multiplication, addition, and nonlinear transformation, such as performing operations such as addition and subtraction of bias, or activation function operations on the results output by the processing circuit group PPB, and then writing them back to the storage circuit 410.

[0046] Furthermore, each parallel processing circuit group includes multiple parallel processing circuits, and the figure shows that the parallel processing circuit group PPB0 420 includes four parallel processing circuits PP0-PP3 421. Those skilled in the art will appreciate that the number of parallel processing circuits in each parallel processing circuit group may be more or less, depending on the specific hardware configuration, and the disclosed embodiments are not limited in this respect.

[0047] In addition to the parallel processing circuits PP0-PP3 421 contained in the parallel processing circuit group PPB0, a shared cache 422 is included for caching data from the storage circuit 410 for use by the parallel processing circuits PP0-PP3 therein. When applied to convolution operations, the shared cache 422 can be used to store neuron data.

[0048] Furthermore, each parallel processing circuit PP includes a plurality of operation circuits CU, and the figure shows that the parallel processing circuit PP3 421 includes four operation circuits CU0-CU3 431. Those skilled in the art will appreciate that the number of operation circuits in each parallel processing circuit PP may be more or less, depending on the specific hardware configuration, and the disclosed embodiments are not limited in this respect.

[0049] In addition to the included operation circuits CU0-CU3 431, the parallel processing circuit PP3 also includes a local cache 432 for caching data from the storage circuit 410 for use by the operation circuits CU0-CU3 therein. When applied to convolution operations, the local cache 432 can be used to store weight data.

[0050] Figure 4 4 also shows different transmission routes of neuron data and weight data in some embodiments, where the thick lines are neuron data and the thin lines are weight data. As can be seen from the figure, the neuron data read out from the storage circuit 410 is cached in the shared cache 422 and then distributed to the operation circuit CU in each parallel processing circuit PP; while the weight data read out from the storage circuit 410 is directly cached in the local cache 432 in each parallel processing circuit PP and then distributed to each operation circuit CU therein. It can be understood that in these implementations, there is a hard-wired connection path between the local cache 432 and the storage circuit 410 to directly cache the data from the storage circuit 410. Example Hardware Alignment Requirements

[0051] In the disclosed embodiments, the dimensions of the multidimensional data involved are characterized as (N, H, W, C) or (Co, H, W, Ci), which represents the storage order of the data in the memory. It can be understood that although the multidimensional data has multiple dimensions, because the layout of the memory is always one-dimensional, there is a corresponding relationship between the multidimensional data and the storage order on the memory. Multidimensional data is usually allocated in a continuous storage space, that is, the multidimensional data can be expanded in one dimension and stored in sequence on the memory. Adjacent dimensions refer to dimensions that are adjacent to each other in the dimensional information representation of multidimensional data. For example, W and Ci are adjacent, and adjacent dimensions can also be called continuous dimensions.

[0052] Hardware stores data in rows (cache rows) and read, write and compute operations are most efficient when the entire row is aligned. Therefore, in order to fully utilize bandwidth and adapt to the memory access requirements of the operator array, it is usually necessary to vectorize and align the data. The design of artificial intelligence chips usually takes the input channel Ci dimension as the lowest dimension for multi-dimensional data, and arranges them in the order of N (batch) H (height) W (width) C (channel). The data on the Ci dimension is continuous. Therefore, vectorization alignment requires that the size of the Ci dimension be aligned to a specified value, such as the alignment value M, so that the data is accessed in units of the alignment value M. Based on different hardware designs, M can have different values, such as 64B, 128B, 256B, etc. When the Ci dimension of the input neuron is large, it is easier to meet the above alignment requirements. When the Ci dimension of the input neuron is small, for example, less than the size of a cache row, the Ci dimension needs to be padded to a row of data (for example, 64B), for example, by filling invalid data 0. This padding will cause a lot of redundant calculations, resulting in resource waste and reduced computing efficiency. The applicant has proposed a variety of optimization schemes for convolution operations with smaller Ci dimensions to reduce hardware alignment requirements and thus reduce redundant calculations. Several alignment optimization schemes are briefly introduced below.

[0053] Alignment optimization solution 1: small convolution operation mode

[0054] In the small convolution operation mode, the data in the W (width) and H (height) dimensions are folded into the C (channel) dimension, thereby fully utilizing the bandwidth of the operator. Specifically, the data to be operated will be divided into blocks according to the specified block format, and then provided to the operator for operation. The block format defines the size of the split data blocks, the dimensional storage order within the data blocks, and the dimensional storage order between data blocks. In some implementations, when the hardware-required alignment value M is 64B, the size of the split data blocks is, for example, 4×4×4B (HWC), the dimensional storage order within the data blocks is CHW, where W is the lowest dimension, and the dimensional storage order between data blocks is HWC, where C is the lowest dimension. The size of the split data blocks just meets the alignment requirements, so that when the operation is performed in units of the split data blocks, the computing power of the operator can be fully utilized.

[0055] By folding the data in the W and H dimensions into the C dimension, the alignment requirements for the C dimension can be greatly reduced, but there are still alignment constraints overall. In the latest known small convolution operation mode, the alignment requirements are unified for various convolution operation types. These convolution operation types include conventional 3D convolution operations with C-dimensional accumulation involved in the forward operation on the neural network (called Forward4), depth-wise convolution operations without C-dimensional accumulation (called Forward1), and reverse depth-wise convolution operations without C-dimensional accumulation involved in the reverse operation on the neural network (called Update1), and cross-product convolution operations equivalent to the reverse operation of conventional 3D convolutions (called Update4).

[0056] Table 1 below exemplifies the data storage format and alignment requirements under the Forward1 type. Table 1: Data storage format and alignment requirements for Forward1 type

[0057] In Table 1, stride indicates the stride of the dimension, which refers to the interval between adjacent elements in the current dimension in a higher dimension than the current dimension. For example, ci_stride indicates the stride of the input neuron in the ci dimension, that is, the interval between adjacent elements in the wi dimension in the ci dimension; wi_stride indicates the stride of the input neuron in the wi dimension, that is, the interval between adjacent elements in the hi dimension in the wi dimension; syn_ci_stride indicates the stride of the weight in the ci dimension, that is, the interval between adjacent elements in the kw dimension in the ci dimension; kw_stride indicates the stride of the weight in the kw dimension, that is, the interval between adjacent elements in the kh dimension in the kw dimension; co_stride indicates the stride of the output neuron in the co dimension, that is, the interval between adjacent elements in the wo dimension in the co dimension; wo_stride indicates the stride of the output neuron in the wo dimension, that is, the interval between adjacent elements in the ho dimension in the wo dimension.

[0058] Table 2 exemplifies the data storage format and alignment requirements under the Forward4 type. Table 2: Data storage format and alignment requirements for Forward4 type

[0059] The data storage format and alignment requirements for other convolution operation types are similar, but they vary slightly depending on the specific algorithm and are not elaborated here.

[0060] As can be seen from the table above, all data is stored in the original format on the on-chip storage circuit (e.g. Figure 4 The data is stored on the storage circuit 410 of the chip, and only the step size requirements are required in the C and W dimensions. When the C and / or W dimensions in the original format of the data do not meet the step size requirements, it can be required to be processed according to the above-mentioned dimensional step size. For example, for the part of the C dimension that is not aligned to the dimensional step size, the software can be required to empty this part of the storage space. In this way, when reading from the on-chip storage circuit, the read address can be conveniently controlled. Under the Forward4 type, since the C-dimensional data needs to be accumulated, the hardware will automatically flush the non-aligned part of the C dimension during the calculation process. Under the Forward1 type, since the C-dimensional data does not need to be accumulated, the hardware does not need to flush the non-aligned part of the C dimension. It can be noted that the hardware will not modify the space of the unaligned part of the C dimension of the input neuron data in the on-chip storage. Since the W dimension needs to participate in multiplication and accumulation in the calculation, for the part of the W dimension that is not aligned to the dimensional step size, the software can be required to empty this part of the storage space and the hardware will perform a flush operation.

[0061] In the above small convolution operation mode, in order to facilitate reading multiple discrete addresses at a time when reading from the on-chip storage circuit, there are certain constraints on the dimension stride of the C dimension and the W dimension. In general, it is required that ci_stride×wi_stride= 128B*N+64B, N is a natural number.

[0062] Table 3 shows the dimension step constraints of neurons and weight data under the Forward1 type. Table 3: Dimensional step constraints of neurons and weight data under Forward1 type

[0063] The output neuron data also meets the storage constraints similar to those of the input neuron data, so that it can be used as the input neuron data of the next convolutional layer. Considering that the data types of input and output on the storage circuit RAM may be different, the co_stride of the output neuron is determined by the ci_stride of the input neuron and the bit width ratio of the data type of the input and output neurons on the RAM. Table 4 shows the co_stride constraints of the output neuron under the Forward1 type. Table 4: co_stride constraints for output neurons under Forward1 type

[0064] The constraint of wo_stride of the output neuron is determined by co_stride. Table 5 shows the wo_stride constraint of the output neuron under the Forward1 type. co_stride size wo_stride size co_stride = 16B 8Q+4(Q≥0) co_stride = 32B 4Q+2(Q≥0) co_stride = 128B*M+64B(M≥0) 2Q+1(Q≥0) Table 5: wo_stride constraints for output neurons under Forward1 type

[0065] In the Forward4 type, since the convolution operation will be accumulated in the ci direction, in order to avoid the complexity of the circuit due to the interaction between the processing circuit groups, ci_stride can be limited to 16B, that is, only one case is processed. At this time, the constraints of the step size of each dimension are as follows: ci_stride=syn_ci_stride=16B, wi_stride=8N+4 (N≥0), kw_stride=8M+4 (M≥0).

[0066] There is no step size constraint on the kh and co dimensions of the weights, so they can be calculated by default as follows: syn_kh_stride=ram_input_synapse_data_width*syn_ci_stride*kw_stride syn_co_stride=ram_input_synapse_data_width*syn_ci_stride*kw_stride*kh

[0067] The co_stride and wo_stride constraints of the output neurons are the same as in Table 5.

[0068] It can be seen that the small convolution operation mode can reduce the alignment requirements for the C dimension by folding the H and W dimensions, but the input neuron data, weight data, and output neuron data still have alignment constraints in the C and W dimensions. When this alignment constraint is not met, invalid data will be introduced.

[0069] Alignment optimization solution 2: Width folding (Wfold) convolution operation mode

[0070] Different from the small convolution operation mode, in the Wfold mode, only the data on the W (width) dimension is folded into the C (channel) dimension to maximize the bandwidth of the operator. This is suitable for situations where the W dimension is large. For the specific implementation scheme of the Wfold mode, please refer to Chinese patent application No. 202111400054.3 submitted by the applicant. Only the parts related to the disclosed embodiment are briefly described here. The Wfold mode is mainly used to implement conventional 3D convolution operations.

[0071] In the Wfold mode, the input channel Ci dimension of the input neuron data is aligned to the input channel split granularity Pci, and the data of each split granularity is combined with Ws times the data of the W dimension to form an input neuron data row, which is transmitted to the operation circuit to perform a convolution operation, where Pci*Ws=M, M is the hardware alignment requirement. On the data reading path, alignment padding is performed on the part that is not aligned to Pci. The input channel Ci dimension of the weight data also needs to be aligned to Pci, so the corresponding alignment padding will also be performed on the data reading path. Pci can be, for example, 16B, 32B, 64B, etc., determined according to the hardware alignment requirements and the size of the input channel dimension Ci. Depending on the operation task scheduling of the processing circuit in the Wfold mode, there are also alignment requirements for the output channel Co dimension of the weight. The output channel alignment value Aco is based on, for example, the number of schedulable processing circuits, such as alignment to a multiple of the number of processing circuits. For example, in Figure 4 In the exemplary computing device of , the number of processing circuits is 16. Similarly, according to the scheduling of computing tasks in the Wfold mode, the output data after the operation also has alignment requirements accordingly. For example, the Wo dimension of the output data has alignment requirements. For example, according to the number of computing circuits in the hardware processing circuit, the Wo dimension is required to be aligned to Ncu, where Ncu is the number of computing circuits in a single processing circuit. Figure 4 In the exemplary computing device of , Ncu=4. Similar alignment requirements exist for the Co and Ho dimensions of the output data.

[0072] In summary, although the above alignment optimization scheme reduces the hardware alignment requirements, alignment requirements still exist, and invalid data will inevitably be introduced. Exemplary invalid data handling scheme

[0073] A known solution for dealing with invalid data introduced by alignment requirements is to use a mask to mark which data in the read data is invalid on the data read path that supplies the data to be operated to the operator, and provide the relevant information to the operator. The operator then does not perform operations on invalid data based on this information to save power consumption. The operator accordingly generates a mask and provides it to the subsequent circuit to inform which output data is invalid. However, this processing solution is very complicated to implement because it is necessary to provide corresponding masks at multiple nodes in the entire calculation process, the processing is very scattered, and the overhead is also large. When switching different operation modes, it is necessary to adjust the masks at multiple nodes.

[0074] In view of this, a processing scheme for invalid data is provided in the disclosed embodiment, which concentrates the processing of invalid data in one place, thereby simplifying the entire calculation process. More specifically, the data reading process and the operation process of the operator can be processed normally as valid data without sensing invalid data, and the decoder can sense which data is valid and which is invalid in each beat of the output data, and then provide this validity indication information to the output data write back path. On the output data write back path, write back control is performed according to the validity indication information provided by the decoder, and only valid data in the output data is written back to the storage circuit.

[0075] Figure 5 The exemplary structural block diagram of a data processing device for implementing the invalid data processing scheme of the disclosed embodiment is shown. As shown in the figure, the data processing device 500 includes a decoder 510, a write control circuit 520 and a storage circuit 530.

[0076] Decoder 510 may be, for example, Figure 3 The instruction decoding unit IDU 312 in the processor is used to decode the acquired instruction and send the decoding result as control information to related circuits, such as operation circuits, data read paths, data write paths, etc. In some embodiments, the decoder 510 can generate validity indication information for the operation output data of each write-back cycle of executing the operation instruction in response to the non-aligned operation instruction, and send it to the write control circuit 520.

[0077] Specifically, the decoder can identify operation instructions with alignment requirements, such as convolution operation instructions for executing the above small convolution operation mode or WFold operation mode. Furthermore, the decoder can determine whether the hardware alignment requirements of the operation are met based on the scale of the input data. If the scale of the input data does not meet the hardware alignment requirements, it means that invalid data will be generated in the operation due to the alignment requirements. At this time, based on the specific operation task scheduling of the operation instruction, that is, the output data distribution of each write-back cycle, validity indication information can be generated for the operation output data of each write-back cycle.

[0078] In some embodiments, the validity indication information may indicate the amount of valid data in one or more dimensions of the operation output data of the corresponding write-back cycle. From the description of the previous alignment optimization scheme, it can be seen that different alignment optimization schemes may have alignment requirements in different dimensions in terms of input data and output data, so the operation output data may have invalid data in one or more dimensions. For example, in the small convolution operation mode, the output neuron has invalid data in the output channel Co dimension due to alignment requirements; in the WFold operation mode, the output neuron has invalid data in the Co dimension and the Wo dimension.

[0079] In some embodiments, based on the alignment requirements and storage format requirements for the operation output data, the above-mentioned validity indication information may include one or more of the following fields: Co dimension segment size co_seg_size, the number of Co segments under each write address co_seg_num, the size of the first part of invalid data of each Co segment co_vacant_size, and the actual valid Co data real_co in each Co segment.

[0080] In the small convolution operation mode, for the output neuron data, it is required to write data to multiple (for example, 4) memory addresses at one time, so as to facilitate the use as the input neuron data of the next convolution layer and support the subsequent one-time reading of multiple discrete addresses. In this case, there is a first address alignment requirement for these multiple memory addresses. For example, when writing 4 addresses at the same time, each address is required to be aligned to 64B. This data writing requirement can be referred to as "four address aligned write".

[0081] In the WFold operation mode, for output neuron data, the actual valid data in the Co and Wo dimensions within each Co segment is required to be densely placed in the storage circuit. This data writing requirement can be referred to as "one address dense writing".

[0082] The Co dimension segment size is used to indicate the Co dimension segment mode. Based on different operation instructions or operation modes, the Co dimension segment size can have different meanings. For example, in the small convolution operation mode, the Co dimension segment size refers to the Co dimension step size when the output neuron is stored in the storage circuit, which can be determined according to the specific parameters involved in the operation instruction. For example, the content of the previous Table 4 can be referred to, and the co_stride of the output neuron is determined based on the ci_stride of the input neuron and the bit width ratio of the data type of the input and output neurons on the RAM, thereby determining the Co dimension segment size co_seg_size. As mentioned above, the possible values ​​of co_seg_size include 16B, 32B, 64B, and so on. For another example, in the WFold operation mode, the Co dimension segment size refers to the size of the space occupied when the Co dimension is aligned as required, for example, aligned to the output channel alignment value Aco. The possible values ​​of co_seg_size include 32B, 64B, 128B, 256B, and so on.

[0083] The number of Co segments co_seg_num at each write address refers to the actual size (wo_real) of the width Wo dimension. In the output neuron, the Co dimension and the Wo dimension are adjacent dimensions, so the number of Co segments is equivalent to the number of data in the Wo dimension. The possible values ​​of co_seg_num include natural numbers such as 0, 1, 2, 3, 4, etc. For example, in the WFold operation mode, depending on the split granularity Pci of the Ci dimension, the number of data in the calculated Wo dimension will be different. For example, when Pci = 16B, each operation circuit calculates 4 Wo, when Pci = 32B, each operation circuit calculates 2 Wo, and when Pci = 64B, each operation circuit calculates 1 Wo. Therefore, the number of Co segments co_seg_num can indicate the valid number of Wo data wo_real. Similarly, in the small convolution operation mode, depending on the size of the co_stride of the output neuron and whether it is valid, the actual number of Wo written at each write address will be different.

[0084] The invalid data size co_vacant_size of the first part of each Co segment refers to the size of the offset of the first part of the Co segment in order to ensure that the first address of the written data meets the alignment requirements. For example, in the small convolution operation mode, in order to support the simultaneous reading of multiple discrete addresses on the storage circuit at one time, the read first address is required to be aligned to a specified value, such as 64B. Therefore, when the output neuron is written back to the storage circuit, for example, as the input neuron of the next convolutional layer, it is also stored in accordance with the above-mentioned storage requirements of the input neuron to simplify the processing. In the WFold operation mode, there is no such discrete read first address alignment requirement, so there is no need to use the co_vacant_size field, that is, this field is invalid in the WFold operation mode.

[0085] The actual valid Co data real_co in each Co segment refers to the amount of real and valid Co data in each Co segment. This is because Co has alignment requirements, such as alignment to co_stride, and invalid data may exist in order to meet the alignment requirements. Therefore, real_co is used to indicate the amount of real and valid Co data.

[0086] According to the above field information, it is possible to determine which data in the operation output data is valid and which data is invalid. For example, in the small convolution operation mode, the tail invalid data can be determined according to co_seg_size-(co_vacant_size+real_co).

[0087] The field information provided above takes into account both alignment requirements and storage requirements, and can provide indication information for the effective storage of the operation output results in various operation instructions. According to the needs of the actual scenario, the above field information only needs about 10 to 20 bits to indicate the validity of the output data (e.g., 256B) of a write-back cycle, and the implementation efficiency is very high.

[0088] continue Figure 5 The write control circuit 520 can be configured to control the write back of the operation output data of the corresponding write back cycle to the storage circuit 530 according to the validity indication information provided by the decoder 510, so that only the operation output data indicated as valid by the validity indication information is written back to the storage circuit.

[0089] In some embodiments, the write control circuit 520 can generate a mask for the operation output data of the corresponding write-back cycle according to the validity indication information to indicate which of the operation output data of the current write-back cycle is to be written into the storage circuit. In some implementations, the mask can be a bitmap, in which each bit can indicate whether a unit of data in the operation output data is valid. For example, if the bit is "1", it indicates that the unit data is valid and needs to be written to the storage circuit. If the bit is "0", it indicates that the unit data is invalid and does not need to be written to the storage circuit. Vice versa. In some implementations, each bit in the mask can correspond to 1B of data in the operation output data, that is, the unit is 1B, so as to better adapt to various data types with different bit widths.

[0090] Therefore, the decoder senses the validity of the operation output data of each write-back cycle, and thereby instructs the data write-back path to control the writing of the output data back to the storage circuit, which can effectively solve the problem of invalid data in the operation output data and balance the overall processing performance.

[0091] Furthermore, in response to the non-aligned operation instruction, the decoder can schedule the data read path to read data according to the alignment requirement, and schedule the operator to perform operations according to the alignment requirement. In this way, the data reading process and the operation process of the operator do not need to perceive invalid data, and all data are treated as valid data and processed normally.

[0092] For different convolution types, depending on whether invalid data will pollute the operation results, the invalid data part can be set to zero / flush to zero by hardware during the data reading process. For example, in the small convolution operation mode, the Forward1 type is a depth-wise convolution, and the product result is not accumulated in the Ci dimension. Therefore, even if there is invalid data introduced in the Ci dimension due to alignment requirements, it will not pollute the final operation result. You only need to discard these invalid data when the output data is written back. The Forward4 type is a regular 3D convolution operation. The product result needs to be accumulated in the Ci dimension. Therefore, the invalid data in the Ci dimension will affect the final operation result and needs to be flushed to avoid polluting the operation result.

[0093] The following description is made through specific examples to better understand the technical solution of the disclosed embodiment.

[0094] Figure 6a The input neuron scale in an exemplary application scenario according to an embodiment of the present disclosure is shown. In this example, it is assumed that the operation involved in the convolution instruction is a small convolution operation mode, the Forward1 type, the data type of the input neuron and the weight is int8, the data type of the output neuron is fp16, the input channel Ci=13, the input neuron scale is Wi=Hi=18, the weight scale is kw=kh=4, and the convolution step is sw=sh=1. At this time, according to the requirements of the small convolution operation mode, ci_stride=16B, the input neuron Wi is aligned to 20, and Hi is aligned to 20. Therefore, when reading data, the data will be taken according to the alignment requirements, for example, 20×20×16B input neurons will be read in each operation round. Figure 610 shows the input neuron data read in each operation round, where the shaded part 611 represents the invalid data introduced due to the alignment requirements. It can be understood that there is also invalid data in the 16B on the Ci dimension, and the valid data is 13B. The figure also shows that 4 discrete addresses are read at a time through a read port, and 64B of data is read at each address.

[0095] Figure 6b Shows Figure 6a The output neuron scale in the exemplary application scenario of FIG. 620 shows the output neuron data generated in each operation round, where the shaded portion represents invalid data caused by invalid operations introduced due to alignment requirements. Referring to the previous Table 4, it can be determined that the Co dimension step size co_stride of the output neuron is 32B. The Wo and Ho dimensions of the output neuron are both 16, but only the first 15 are valid.

[0096] Based on the above information, the validity of the output data of each write-back cycle can be determined.

[0097] For example, for the data output by the write-back cycle indicated by 621, there is invalid data only in the Co dimension. Therefore, the corresponding validity indication information is as follows: co_seg_size = 32B, co_seg_num = 2 (wo_real), co_vacant_size = 0B (assuming the address is aligned), real_co = 26B. When writing to the storage circuit, it is written according to four addresses, and each address will affect 64B of storage space. The written data corresponding to different addresses are shown in different grayscales in the figure.

[0098] For another example, for the data outputted by the write-back cycle indicated by 622, in addition to invalid data in the Co dimension, there is also invalid data in the Wo dimension. Therefore, the corresponding validity indication information is as follows: co_seg_size=32B, co_seg_num=1(wo_real), co_vacant_size=0B (assuming the address is aligned), real_co=26B.

[0099] For another example, for the data output by the write-back cycle indicated by 623, in addition to the invalid data in the Co and Wo dimensions, there is also invalid data in the Ho dimension. Therefore, the corresponding validity indication information is as follows: co_seg_size = 32B, co_seg0_1_2_num = 1 (wo_real), co_seg3_num = 0 (wo_real), co_vacant_size = 0B (assuming the address is aligned), real_co = 26B. Note that information is provided here for the number of co segments of each write address, for example, the number of co segments of the first three write addresses is 1, that is, the number of valid Wo is 1, and the number of co segments of the last write address is 0, that is, there is no Wo valid data. In some optional implementations, additional information can also be used to indicate how many of the four write addresses are valid, thereby avoiding the need to set a co segment number field for each write address. For example, an additional 2 bits of information can be added to indicate that the number of valid addresses is 1, 2, 3, and 4, respectively.

[0100] Figure 7a The input neuron scale in another exemplary application scenario according to the embodiment of the present disclosure is shown. This application scenario adopts the WFold operation mode. In this example, it is assumed that the operation mode is standard convolution, that is, conventional 3D convolution operation, the data type of the input neuron and weight is int8, the data type of the output neuron is fp16, the input channel Ci=13, the output channel Co=10, the input neuron scale is Wi=17, Hi=1, the weight scale is kw=3, kh=1, and the convolution step size is sw=sh=1. At this time, the real and effective output neuron scale should be Wo=15, Ho=1, Co=10.

[0101] 710 in the figure shows the input neuron. In WFold mode, the input neurons are densely stored in the storage circuit, so each read port can read 8 numbers in the Wi dimension in each cycle, that is, read 13B*8 data stored continuously in the storage circuit. When two read ports are used to read simultaneously, all input neurons can be read through 2 cycles. 711 in the figure shows the data read by the read port r0 in the first beat, and 712 in the figure shows the data read by the read port r1 in the first beat.

[0102] In the current example, according to the requirements of the WFold operation mode, the Ci dimension needs to be aligned to Pci=16B, and the Co dimension needs to be aligned to 16. Therefore, after the data is read out, each 13B of the read Ci channel will be filled with zeros to expand to 16B, and then provided to the operator for operation. The operator will execute the data as valid data.

[0103] Figure 7b Shows Figure 7a The output neuron scale in the exemplary application scenario of FIG. 720 shows the output neuron data generated in each operation round, where the shaded part represents the invalid data caused by the invalid operation introduced due to the alignment requirement. Figure 4 In the hardware architecture, the computing tasks are distributed among the four PPBs. For example, PPB0 calculates the output data of Wo index 0 to 3 and Co index 0 to 15, PPB1 calculates the output data of Wo index 4 to 7 and Co index 0 to 15, PPB2 calculates the output data of Wo index 8 to 11 and Co index 0 to 15, and PPB3 calculates the output data of Wo index 12 to 15 and Co index 0 to 15. It can be seen that due to the alignment requirements for Wo and Co during the calculation, invalid data is introduced into the output result, and the output neuron scale containing invalid data is Wo = 16 (of which 15 are valid), Ho = 1, Co = 16 (of which 10 are valid). When writing data, the computing output data of the first write-back cycle is Wo index 0 to 7 and Co index 0 to 15, and the computing output data of the second write-back cycle is Wo index 8 to 15 and Co index 0 to 15.

[0104] For the first write-back cycle 721, the data in the Wo dimension of the calculation output data are all valid, and there are invalid data in the Co dimension. Therefore, the validity indication information provided by the decoder is as follows: co_seg_size = 32B, co_seg_num = 8 (wo_real), co_vacant_size is invalid, real_co = 10*2B = 20B. Dense writing is performed through one address, and the amount of written data is Wo*Co*data_width = 8*10*2B = 160B. Specifically, on the write data path, the data of real_co is taken at intervals of co_seg_size, and wo_real is taken for a total of times, and these data are densely placed on the storage circuit.

[0105] Similarly, for the second write-back cycle 722, only 7 data in the Wo dimension of the operation output data are valid, and there are also invalid data in the Co dimension. Therefore, the validity indication information provided by the decoder is as follows: co_seg_size = 32B, co_seg_num = 7 (wo_real), co_vacant_size is invalid, real_co = 10*2B = 20B. Dense writing is performed through one address, and the amount of written data is Wo*Co*data_width = 7*10*2B = 140B.

[0106] Thus, the invalid data processing scheme of the disclosed embodiment is described in detail above through two specific examples. It can be seen that this scheme can be applied to various computing scenarios in which invalid data exists in the output data, including but not limited to the above small convolution computing mode and WFold computing mode. Based on the specific computing instructions, the decoder can sense the invalid data that may exist in the output data of each write-back cycle and indicate it. The write control circuit only needs to control the write-back data according to the instructions of the decoder. This method concentrates the processing of invalid data on the output data write-back end, avoids the cumbersome operations and overhead caused by the decentralized processing in each link of the calculation process, and improves the overall computing performance. In addition, the indication information provided by the decoder can take into account various storage format requirements, including but not limited to multiple discrete address alignment writes, one address dense write, and so on. This indication information only requires 10 to 20 bits of lines, and the hardware overhead and area overhead are very small.

[0107] Figure 8 An exemplary flow chart of a data processing method of a data processing device according to an embodiment of the present disclosure is shown.

[0108] As shown in the figure, in step 810, the decoder generates validity indication information for the operation output data of each write-back cycle of executing the operation instruction in response to the non-aligned operation instruction, and sends it to the write control circuit.

[0109] In some embodiments, the validity indication information indicates the amount of valid data in one or more dimensions of the operation output data of the corresponding write-back cycle. Furthermore, the validity indication information may also indicate the storage format of the operation output data on the storage circuit. In some embodiments, the validity indication information may include one or more of the following fields: Co dimension segment size co_seg_size; Co segment number co_seg_num under each write address; used to indicate the size of the invalid data of the front part of each Co segment co_vacant_size; and the actual valid Co data real_co in each Co segment; wherein Co represents the output channel dimension. Under different operation instructions, the specific meaning of each of the above fields can refer to the previous description and will not be repeated here.

[0110] In step 820, the write control circuit controls the operation output data of the corresponding write-back cycle to be written back to the storage circuit according to the received validity indication information, so that only the operation output data indicated as valid by the validity indication information is written back to the storage circuit.

[0111] In some embodiments, the write control circuit may generate a mask for the operation output data of the corresponding write-back cycle based on the validity indication information to indicate the valid data in the operation output data of the write-back cycle that needs to be written into the storage circuit.

[0112] In some implementations, the mask may be a bitmap, where each bit indicates whether a unit of data in the operation output data is valid. In some implementations, the unit is 1B, where B represents a byte.

[0113] Furthermore, in addition to controlling the write-back path, the decoder can also, in response to the non-aligned operation instruction, schedule the data read path to perform data reading according to the alignment requirement, and schedule the operator to perform the operation according to the alignment requirement. In some embodiments, when the decoder detects an operation instruction whose non-aligned portion will pollute the operation output result, the data read path can be instructed to perform a zeroing operation on the non-aligned portion.

[0114] Those skilled in the art will appreciate that the various features of the computing device and the data transmission device described above in conjunction with the accompanying drawings can be similarly applied to Figure 8 The data processing method is described in detail in the data processing method, so it will not be repeated here.

[0115] Those skilled in the art can understand that although the above scheme is described by taking the small convolution operation mode and the WFold operation mode as examples, the above scheme is not limited thereto, but can be applied to any other scenario with similar invalid data discarding requirements. In addition, although the above scheme is a solution proposed for an artificial intelligence processor, it can also be applied to general or special processors such as CPUs and GPUs, and the disclosed embodiments are not limited in this respect.

[0116] The present disclosure also provides a processor, including the aforementioned data processing device. The present disclosure also provides a chip, which may include the processor of any embodiment described above in conjunction with the accompanying drawings. Further, the present disclosure also provides a board, which may include the aforementioned chip.

[0117] According to different application scenarios, the electronic equipment or device disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, transportation, household appliances, and / or medical equipment. The transportation includes airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs. The electronic equipment or device disclosed herein may also be applied to the Internet, IoT, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and medical fields. Further, the electronic equipment or device disclosed herein may also be used in cloud, edge, and terminal applications related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0118] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art may appreciate the parts that are not described in detail in a certain embodiment of the present disclosure, and may also refer to the relevant descriptions of other embodiments.

[0119] In terms of specific implementation, based on the disclosure and teaching of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article splits them on the basis of considering the logical functions, and there may be other splitting methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in the units or components can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0120] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed on multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0121] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs, etc. Further, the aforementioned storage unit or storage device may be any appropriate storage medium (including magnetic storage media or magneto-optical storage media, etc.), which may be, for example, a variable resistive memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

[0122] The embodiments of the present disclosure are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method of the present disclosure and its core idea. At the same time, for those skilled in the art, according to the ideas of the present disclosure, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

1. A data processing device, comprising a decoder, a write control circuit and a storage circuit, wherein: The decoder is configured to generate validity indication information for operation output data of each write-back cycle of executing the operation instruction in response to the non-aligned operation instruction, and send the validity indication information to the write control circuit; and The write control circuit is configured to control the operation output data of the corresponding write-back cycle to be written back to the storage circuit according to the validity indication information, so that only the operation output data indicated as valid by the validity indication information is written back to the storage circuit. 2 . The data processing apparatus according to claim 1 , wherein the validity indication information indicates the valid data amount of the operation output data of the corresponding write-back cycle in one or more dimensions.

3. The data processing device according to any one of claims 1-2, wherein the validity indication information also indicates a storage format of the operation output data in the storage circuit.

4. The data processing device according to any one of claims 1 to 3, wherein the validity indication information comprises one or more of the following fields: Co dimension segment size co_seg_size; Co segment number co_seg_num under each write address; co_vacant_size is used to indicate the size of the invalid data in the front part of each Co segment; and The actual valid Co data real_co in each Co segment; Where Co represents the output channel dimension.

5. A data processing device according to any one of claims 1-4, wherein the operation instruction is a small convolution operation instruction, and the validity indication information is indicated based on a storage format requirement that multiple addresses are written simultaneously and the first address meets the alignment requirement.

6. The data processing apparatus according to claim 5, wherein the validity indication information comprises: Co dimension segment size co_seg_size, used to indicate the Co dimension step size of the output channel when the operation output data is stored in the storage circuit; The number of Co segments at each write address, co_seg_num, is used to indicate the number of valid data in the width Wo dimension that needs to be written at each write address; The size of the invalid data in the front part of each Co segment, co_vacant_size, is used to indicate the size of the storage space that needs to be vacated to meet the alignment requirement of the first address; and The actual valid Co data real_co in each Co segment.

7. A data processing device according to any one of claims 1-4, wherein the operation instruction is a WFold operation instruction, and the validity indication information is indicated based on a storage format requirement of writing to a single address at a time and densely placing valid data.

8. The data processing apparatus according to claim 7, wherein the validity indication information comprises: Co dimension segment size co_seg_size is used to indicate the size of the space occupied when the output data Co dimension is aligned as required; The number of Co segments co_seg_num under each write address is used to indicate the number of valid data of the width Wo dimension that needs to be written under the write address in the current write-back cycle; and The actual valid Co data real_co in each Co segment.

9. The data processing device according to any one of claims 1 to 8, wherein: The write control circuit is further configured to generate a mask for the operation output data of the corresponding write-back cycle based on the validity indication information to indicate the valid data in the operation output data of the write-back cycle that needs to be written into the storage circuit.

10. The data processing apparatus according to claim 9, wherein the mask is a bitmap, wherein each bit indicates whether a unit of data in the operation output data is valid.

11. The data processing apparatus according to claim 10, wherein the unit is 1B, where B represents a byte.

12. The data processing device according to any one of claims 1 to 11, wherein: The decoder is further configured to, in response to the non-aligned operation instruction, schedule the data read path to perform data reading according to the alignment requirement, and schedule the operator to perform operation according to the alignment requirement.

13. The data processing apparatus according to claim 12, wherein: The decoder is further configured to, in response to an operation instruction in which the non-aligned portion may pollute the operation output result, instruct the data read path to perform a zeroing operation on the non-aligned portion.

14. A processor, comprising the data transmission device according to any one of claims 1-23.

15. A chip comprising the processor according to claim 24.

16. A board comprising the chip according to claim 25.

17. A data processing method, comprising: The decoder generates validity indication information for the operation output data of each write-back cycle of executing the operation instruction in response to the non-aligned operation instruction, and sends the validity indication information to the write control circuit; as well as The write control circuit controls the operation output data of the corresponding write-back cycle to be written back to the storage circuit according to the validity indication information, so that only the operation output data indicated as valid by the validity indication information is written back to the storage circuit. 18 . The data processing method according to claim 17 , wherein the validity indication information indicates the valid data amount of the operation output data corresponding to the write-back cycle in one or more dimensions.

19. The data processing method according to any one of claims 17-18, wherein the validity indication information further indicates a storage format of the operation output data in the storage circuit.

20. The data processing method according to any one of claims 17-18, wherein the validity indication information comprises one or more of the following fields: Co dimension segment size co_seg_size; Co segment number co_seg_num under each write address; co_vacant_size is used to indicate the size of the invalid data in the front part of each Co segment; and The actual valid Co data real_co in each Co segment; Where Co represents the output channel dimension.

21. The data processing method according to any one of claims 17 to 20, further comprising: The write control circuit generates a mask for the operation output data of the corresponding write-back cycle based on the validity indication information to indicate the valid data in the operation output data of the write-back cycle that needs to be written into the storage circuit.

22. The data processing method according to claim 21, wherein the mask is a bitmap, wherein each bit indicates whether a unit of data in the operation output data is valid.

23. The data processing method according to claim 22, wherein the unit is 1B, where B represents byte.

24. The data processing method according to any one of claims 17 to 23, further comprising: The decoder responds to the non-aligned operation instruction, schedules the data read path to perform data reading according to the alignment requirement, and schedules the operator to perform operation according to the alignment requirement.

25. The data processing method according to claim 24, further comprising: In response to an operation instruction in which a non-aligned portion may pollute an operation output result, the decoder instructs a data read path to perform a zeroing operation on the non-aligned portion.

Images