Data processing method, neural network accelerator and electronic equipment

Abstract

The invention discloses a data processing method, a neural network accelerator and electronic equipment.The method comprises the steps that input data of a neural network is cached to a storage module, the storage module comprises a plurality of storage queues organized in an array mode, and the input data comprises a plurality of data queues arranged in an array; the data queues with the consistent row attributes are correspondingly stored in the storage queues with the corresponding column attributes respectively; on the basis of the data reading period, controlling a reading enabling state of each storage queue in the to-be-read column according to a preset rule; and based on the data reading period, the to-be-read data in the storage queues in the enabled state is selected and output through a plurality of partially overlapped selection windows and is output to the processing module, the processing module is used for processing the input data according to a preset processing mode, and each selection window corresponds to a plurality of storage queues.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, a neural network accelerator and an electronic device. Background Art

[0002] In the calculation process of deep learning models (such as convolutional neural networks), in order to improve the calculation efficiency, a common method is to convert the operation of the deep learning model into a matrix multiplication form. This conversion requires flattening the input data into a matrix form. During the flattening process, some input data may be copied multiple times. This data processing not only increases the demand for storage space, but also increases the power consumption of data reading and transmission. Summary of the invention

[0003] The technical solutions provided by this application are as follows:

[0004] The first aspect of the present application provides a data processing method, comprising:

[0005] Cache the input data of the neural network into a storage module, wherein the storage module includes a plurality of storage queues organized in an array form, and the input data includes a plurality of data queues arranged in an array, and each data queue with the same row attribute is stored in each storage queue with the corresponding column attribute;

[0006] Based on the data reading cycle, the read enable state of each storage queue in the to-be-read column is controlled according to a preset rule; the preset rule matches the preset processing method of the input data;

[0007] Based on the data reading cycle, the data to be read in the storage queue in the enabled state is selected and output through a number of partially overlapping selection windows, and output to the processing module. The processing module is used to process the input data according to the preset processing method, and each of the selection windows corresponds to multiple storage queues.

[0008] Based on the data reading cycle, the read enable state of each storage queue in the to-be-read column is controlled according to preset rules, including:

[0009] Obtaining initial control data; the initial control data includes a plurality of control bits corresponding to the arrangement order of each storage queue in the corresponding to-be-read column, each control bit is used to control the read enable state of each storage queue in the corresponding to-be-read column, each control bit includes first data or second data, the first data corresponds to entering the enable state, and the second data corresponds to exiting the enable state;

[0010] Based on the initial control data, the first data or the second data of each control bit is adjusted according to a preset rule based on a data reading cycle to obtain target control data; the initial control data matches the preset processing method;

[0011] The read enable state of each storage queue in the to-be-read column is controlled based on the target control data.

[0012] The processing module includes: a computing array;

[0013] The adjusting the first data or the second data of each of the control bits according to a preset rule includes:

[0014] Based on the data arrangement mode of the input data corresponding to the computing array, the insertion period corresponding to each of the control bits is determined; the data arrangement mode corresponds to the preset processing mode; the data arrangement mode is characterized in that the end position of the data is delayed row by row as the number of rows increases, and the end positions of the data in different rows are kept aligned after reaching the set number of rows;

[0015] Based on the data reading cycle, controlling the first data in each of the control bits to move bit by bit in a direction from a start control bit to an end control bit;

[0016] When it is determined based on the insertion period that the data reading period reaches an insertion timing, the first data is inserted into a start control bit of each of the control bits.

[0017] The processing module includes: a computing array;

[0018] The adjusting the first data or the second data of each of the control bits according to a preset rule includes:

[0019] Based on the data arrangement mode of the input data corresponding to the computing array, when the adjustment timing is reached, each of the control bits is adjusted from the first data to the second data at the same time; the data arrangement mode is characterized in that the starting positions of data in different rows are always aligned and the end positions of the data are always aligned.

[0020] The method of selecting and outputting the to-be-read data in the storage queue in the enabled state through a plurality of partially overlapping selection windows based on the data reading cycle includes:

[0021] Based on the data reading cycle, each selection window in a plurality of selection windows corresponds to a plurality of continuous control bits in the target control data; and there is a partial overlap between the plurality of continuous control bits corresponding to every two adjacent selection windows;

[0022] Selecting a target control bit corresponding to a first first data from the plurality of consecutive control bits;

[0023] Output the data to be read in the storage queue corresponding to the target control bit.

[0024] The method of selecting and outputting the to-be-read data in the storage queue in the enabled state through a plurality of partially overlapping selection windows based on the data reading cycle includes:

[0025] Based on the data reading cycle, a plurality of receiving ends of each selection window in a plurality of selection windows respectively correspond to a plurality of continuous storage queues in the to-be-read column; and there is a partial overlap between the plurality of continuous storage queues corresponding to every two adjacent selection windows;

[0026] Selecting a first target storage queue in an enabled state from the plurality of consecutive storage queues;

[0027] Output the data to be read in the target storage queue.

[0028] The data queue includes: data in a plurality of depth directions;

[0029] The outputting the to-be-read data in the storage queue corresponding to the target control bit includes:

[0030] The data to be read is located in the storage queue corresponding to the target control bit by using the position index corresponding to the storage queue corresponding to the target control bit; the position index is used to point to the position of the next data to be read in the storage queue;

[0031] The data to be read is outputted, and the position index is incremented.

[0032] On the other hand, the present application provides a neural network accelerator, comprising:

[0033] A storage module, used for caching input data of the neural network; the storage module includes a plurality of storage queues organized in an array form, the input data includes a plurality of data queues arranged in an array, and each data queue with the same row attribute is stored in each storage queue with the corresponding column attribute;

[0034] A control module, used to control the read enable state of each storage queue in the to-be-read column according to a preset rule based on a data read cycle; the preset rule matches a preset processing method of the input data;

[0035] A reading module is used to select and output the to-be-read data in the storage queue in the enabled state through a number of partially overlapping selection windows based on the data reading cycle, and output it to the processing module. The processing module is used to process the input data according to the preset processing method, and each of the selection windows corresponds to multiple storage queues.

[0036] The control module comprises:

[0037] a shift register, used for storing initial control data, and controlling the bitwise shifting of first data in each control bit in the initial control data based on the data reading cycle; the initial control data comprises a plurality of control bits, each control bit is used to control the read enable state of each storage queue in the corresponding to-be-read column, each control bit comprises the first data or the second data, the first data corresponds to an entry enable state, and the second data corresponds to an exit enable state;

[0038] A cycle counter, used for generating a trigger signal according to the insertion cycle corresponding to each of the control bits, and sending the trigger signal to the shift register;

[0039] The shift register is further configured to insert the first data into a starting control bit of each of the control bits in response to the trigger signal.

[0040] A third aspect of the present application provides an electronic device, including: any one of the neural network accelerators described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0042] Figure 1 A schematic diagram of data tiling provided for this application;

[0043] Figure 2 A flowchart of a data processing method provided in Example 1 of the present application;

[0044] Figure 3 A schematic diagram of a cache of a storage module provided in this application;

[0045] Figure 4 A schematic diagram of a plurality of partially overlapping selection windows provided by the present application for selecting and outputting data to be read in a storage queue in an enabled state;

[0046] Figure 5A flowchart of a data processing method provided in Example 2 of the present application;

[0047] Figure 6 A schematic diagram of adjusting the first data or the second data of each control bit provided by the present application;

[0048] Figure 7 A schematic diagram of a data arrangement method corresponding to a Transformer provided in this application;

[0049] Figure 8 A schematic diagram of a data arrangement method corresponding to a convolutional neural network provided in this application;

[0050] Fig. 9 A schematic diagram of inserting a start control bit into first data corresponding to a data arrangement mode of a Transformer provided in the present application;

[0051] Fig.10 A schematic diagram of a data arrangement method corresponding to a sparse Transformer provided in this application;

[0052] Fig.11 A schematic diagram of adjusting the first data or the second data of each control bit corresponding to a sparse Transformer provided in the present application;

[0053] Fig.12 A schematic diagram of selected output data corresponding to a convolutional neural network provided in this application;

[0054] Fig.13 A schematic diagram of selecting output data corresponding to a Transformer provided in this application;

[0055] Fig.14 Another schematic diagram of selecting output data corresponding to a convolutional neural network provided in this application;

[0056] Fig.15 Another schematic diagram of selecting output data corresponding to a Transformer provided in this application;

[0057] Fig.16 A schematic diagram of data output including position index provided by this application;

[0058] Fig.17 A schematic diagram of the structure of a neural network accelerator provided in this application. DETAILED DESCRIPTION

[0059] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0060] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0061] The terms "first", "second" etc. in the specification of the application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable in appropriate circumstances, and this is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0062] In order to improve the computational efficiency of matrix calculations, especially at the hardware implementation level, the matrix systolic array (Processing Element, PE) technology has been widely used. This technology faces a key problem in practical applications: the input data of PE needs to be flattened. For example, for convolution calculations, flattening is to reorganize the input data according to the size of the convolution kernel, so that each column of data corresponds to a specific area in the input data and the data required for the convolution operation with the convolution kernel. However, this process will cause data expansion, and the specific impacts are as follows:

[0063] First, from the perspective of storage space, the tiling operation significantly increases the size of the matrix that needs to be stored. Taking a 3×3 convolution kernel as an example, when using a 32×32 matrix systolic array, it is tiled into a 34×9 data matrix, that is, a data matrix with 34 rows and 9 columns, where the rows contain the data that is repeatedly copied to form a complete convolution operation. Figure 1 C6, C7, and C8 in the red box appear three times in the tiled data, which means they are expanded three times. This expansion directly causes the corresponding storage space (such as SRAM) to expand three times. Secondly, from the perspective of data reading and transmission power consumption, as the amount of data increases, the power consumption of storing and reading this data will also increase accordingly. Especially in Figure 1In the example shown, if certain data (such as C6, C7, C8, etc.) are expanded 3 times during the tiling process (meaning they appear multiple times in the tiled matrix in some form), this means that more power consumption will be consumed in reading and writing these repeated data during data transmission.

[0064] Therefore, the present application provides a data processing method to achieve efficient data reuse and ensure that a piece of data is copied only once, thereby effectively reducing the burden of storage space and power consumption caused by data expansion.

[0065] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0066] Reference Figure 2 , is a flow chart of a data processing method provided in Example 1 of the present application, such as Figure 2 As shown, the method may include but is not limited to the following steps:

[0067] Step S101, cache the input data of the neural network into a storage module, the storage module includes a plurality of storage queues organized in an array form, the input data includes a plurality of data queues arranged in an array, and each data queue with consistent row attributes is respectively stored in each storage queue with corresponding column attributes.

[0068] The array form of several storage queues can be understood as the arrangement of logical addresses, not the direct array layout of physical entities. The arrangement of logical addresses can be reflected by the allocation of storage addresses. For example, storage queues in the same row share the same base address, which can represent the starting position of the storage queue in the memory. Although the storage queues in the row share this base address, each queue has a unique offset on the base address to distinguish different storage queues. Storage queues in different rows have different base addresses.

[0069] In this embodiment, the number of storage queues organized in array form can be set based on the scale of all input data of the neural network.

[0070] The number of storage queues organized in array form can also be configured according to the amount of input data actually involved in the convolution operation with the computing array as the execution carrier. For example, when the computing array is a matrix systolic array, the size of the matrix systolic array is 32×32, and the convolution kernel is 3×3, since the bit width of the matrix systolic array is 32, 32 rows of tiled data need to be input. Accordingly, the convolution kernel needs to slide at least 32 times in the input data. When the width of the convolution kernel is 3, the width of the input data needs to be 34 columns. This is because when the convolution kernel slides on the input data, it needs to cover at least 3 columns of input data each time (i.e., in the W direction). When the convolution kernel slides to the edge of the input data, in order to ensure that the convolution kernel can still cover 3 columns of data in the last slide, 1 column (a total of 2 columns) needs to be reserved on both sides of the input data as a boundary. Therefore, the total number of input data columns is 34 (i.e., wph+kw-1) columns. 34 columns of input data require 34 rows of storage queues, and a convolution operation requires 3 rows of input data, corresponding to 3 columns of storage queues. Therefore, the number of storage queues can be set to 3×34, where 3 represents the number of rows directly related to each convolution operation, and 34 can represent the number of data queues consistent with the row attributes of the matrix systolic array to adapt. That is, the number of storage queues for a certain column attribute is equal to wph+kw-1, wph represents the bit width of the matrix systolic array, and kw represents the width of the convolution kernel.

[0071] For example, Figure 3 As shown, the data queues with the row attribute H1 include C0, C3, C6, ..., C99, and C0, C3, C6, ..., C99 can be stored in each storage queue with the column attribute Buffer-w1; the data queues with the row attribute H2 include C1, C4, C7, ..., C100, and the data queues C1, C4, C7, ..., C100 can be stored in each storage queue with the column attribute Buffer-w2; the data queues with the row attribute H3 include C2, C5, C8, ..., C101, and the data queues C2, C5, C8, ..., C101 can be stored in each storage queue with the column attribute Buffer-w3.

[0072] If the bit width of the storage queue is 256 bits and the data bit width is 8 bits, the length of the data queue can be 32, that is, the data queue contains 32 data of the input data on the channel Chanel (C direction).

[0073] Of course, the length of the data queue can also be greater than 32 data bit widths. When it is greater than 32, 32 data can be read from the data queues C0, C3, C6, ..., C99 and stored in each storage queue with the column attribute Buffer-w1, and 32 data can be read from the data queues C1, C4, C7, ..., C100 and stored in each storage queue with the column attribute Buffer-w2, and 32 data can be read from the data queues C2, C5, C8, ..., C101 and stored in each storage queue with the column attribute Buffer-w3.

[0074] For example, data queue C0 includes data C0-0, C0-1, C0-2, ..., C0-63, data queue C3 includes data C3-0, C3-1, C3-2, ..., C3-63, data queue C6 includes data C6-0, C6-1, C6-2, ..., C6-64, ..., data queue C99 includes data C99-0, C99-1, C99-2, ..., C99-64, and data C0-0, C0-1, C0-2, ..., C0-31 can be read from data queue C0 to the column attribute Buffer -w1's Buffer0; read data C3-0, C3-1, C3-2, ..., C3-31 from data queue C3 into Buffer1 with column attribute Buffer-w1; read data C6-0, C6-1, C6-2, ..., C6-31 from data queue C6 into Buffer2 with column attribute Buffer-w1; ...; read data C99-0, C99-1, C99-2, ..., C99-31 from data queue C99 into Buffer33 with column attribute Buffer-w1. After the calculation of the first 32 data above is completed, read the remaining part and store it in the corresponding Buffer. At this time, the data of each data queue has been divided into two parts according to the channel, and the calculation results corresponding to the two divided parts can be spliced ​​in order.

[0075] Step S102 : Based on the data reading cycle, the read enable state of each storage queue in the to-be-read column is controlled according to a preset rule; the preset rule is the preset processing method of the input data.

[0076] The column to be read may be one or more columns of storage queues in a plurality of storage queues organized in an array form.

[0077] The read enable state can be used to indicate whether the data in the storage queue can be read. When the read enable state is enabled, it means that the data in the storage queue can be read during the data read cycle. When the read enable state is disabled, it means that the data in the storage queue will not be read during the data read cycle.

[0078] The read enable state of each storage queue in the to-be-read column is controlled according to a preset rule, in order to ensure that the reading of data matches the preset processing method of the input data.

[0079] Different types of neural networks have different preset processing methods for corresponding input data. For example, the neural network may include a convolutional neural network, and the corresponding preset processing method may include: performing a convolution operation based on a convolution kernel.

[0080] Alternatively, the neural network may include: Transformer, and the corresponding preset processing method may include: calculating the relationship between different elements in the input data based on the attention matrix.

[0081] Alternatively, the neural network may include: a sparse Transformer, the corresponding preset processing method differs from the preset processing method corresponding to the Transformer mainly in that the attention matrix of the sparse Transformer is designed to be sparse to reduce computation and memory usage.

[0082] Although different types of neural networks have different preset processing methods for input data, the data flow speed from the storage module to the processing module can be the same, so the data reading cycle can remain unchanged.

[0083] Step S103: Based on the data reading cycle, the data to be read in the storage queue in the enabled state is selected and output through a number of partially overlapping selection windows, and output to the processing module, the processing module is used to process the input data according to the preset processing method, and each of the selection windows corresponds to multiple storage queues.

[0084] Although each selection window corresponds to multiple storage queues, it does not mean that the data in all the corresponding storage queues will be read. Among the multiple storage queues corresponding to the selection window, only the data in the storage queues that are in the enabled state will be selected and output.

[0085] Since there is a partial overlap between the selection windows, some storage queues in an enabled state can be selected multiple times during a data reading cycle, thereby achieving data multiplexing.

[0086] For example, the preset processing method of the input data may include: performing a convolution operation based on a convolution kernel of size 3×3, corresponding to Figure 3 In the data queue with the row attribute H1, the columns to be read can be the storage queues with the column attribute Buffer-w1, such as Figure 4As shown, in the data reading cycle T0, Buffer0 with the column attribute Buffer-w1 is in the enabled state, and C0-0 is read from Buffer0 with the column attribute Buffer-w1; in the data reading cycle T1, Buffer1 with the column attribute Buffer-w1 is in the enabled state, and C3-0 is read from Buffer1 with the column attribute Buffer-w1; in the data reading cycle T2, Buffer2 with the column attribute Buffer-w1 is in the enabled state, and C6-0 is read from Buffer2 with the column attribute Buffer-w1; in the data reading cycle T3, Buffer0 and Buffer3 with the column attribute Buffer-w1 are in the enabled state, and C0-1 is read from Buffer0 with the column attribute Buffer-w1 and C9-0 is read from Buffer3 with the column attribute Buffer-w1;.....

[0087] Selecting window 1 can correspond to Buffer0, Buffer1 and Buffer2 whose column attribute is Buffer-w1; selecting window 2 can correspond to Buffer1, Buffer2 and Buffer3 whose column attribute is Buffer-w1; selecting window 3 can correspond to Buffer2, Buffer3 and Buffer4 whose column attribute is Buffer-w1; selecting window 4 can correspond to Buffer3, Buffer4 and Buffer5 whose column attribute is Buffer-w1; selecting window 5 can correspond to Buffer4, Buffer5 and Buffer6 whose column attribute is Buffer-w1…

[0088] Corresponding to the convolution kernel of size kw×kh, the length of each selection window = kw (that is, each selection window corresponds to kw consecutive buffers), and the latter selection window is offset by 1 buffer compared to the previous selection window.

[0089] In the data reading cycle T0, Buffer0 is in the enabled state, and Buffer0 has only selection window 1 corresponding to it. Therefore, based on selection window 1, C0-0 in Buffer0 can be selected for output, that is, C0-0 is not multiplexed.

[0090] In the data reading cycle T1, Buffer1 is in the enabled state, and Buffer1 has selection windows 1 and 2 corresponding to it. Therefore, based on selection windows 1 and 2, C3-0 in Buffer1 can be selected for output, that is, C3-0 is reused twice.

[0091] In the data reading cycle T2, Buffer2 is in the enabled state, and Buffer2 has selection windows 1, 2, and 3 corresponding to it. Therefore, based on selection windows 1, 2, and 3, C6-0 in Buffer3 can be selected for output, that is, C6-0 is reused 3 times.

[0092] In the data reading cycle T3, Buffer0 and Buffer3 are enabled. Buffer0 only has selection window 1 corresponding to it. Therefore, based on selection window 1, C0-1 in Buffer0 can be selected for output. Buffer3 has selection windows 2, 3, and 4 corresponding to it. Therefore, based on selection windows 2, 3, and 4, C9-0 in Buffer3 can be selected for output. That is, C9-0 is reused 3 times...

[0093] In the subsequent data reading cycle, corresponding data multiplexing is performed with reference to the above-mentioned method of selecting output, which will not be described in detail here.

[0094] Reference Figure 4 , it can be seen that the data reuse is related to the way the 3×3 convolution kernel slides in the input data during the convolution operation. Data queue C0 is located at the head of the input data of row attribute H1. The convolution kernel will traverse the data queue C0 when it slides for the first time. As the convolution kernel slides, the data queue C0 will not be traversed again. Therefore, the data in the data queue C0 does not need to be reused.

[0095] Data queue C3 is located at the second position of the input data of row attribute H1. As the convolution kernel slides, data queue C3 will be traversed twice for convolution operation. Therefore, the data in data queue C3 will be reused twice.

[0096] Data queue C6 is located at the third position of the input data of row attribute H1. As the convolution kernel slides, data queue C6 will be traversed three times for convolution operation. Therefore, the data in data queue C6 will be reused 3 times.

[0097] The situation is similar for the data in the subsequent positions of the data queue C9. As the convolution kernel slides, the subsequent data will be traversed three times, twice, or once, and accordingly, the data will be reused 3 times, 2 times, or no reuse is required.

[0098] For a convolution kernel of size kw×kh, each data queue can be traversed between 1 and kw times, and the specific number of traversals is determined by the convolution kernel width, input data width, and sliding step size. When the number of traversals is not less than 2, the number of times the data can be reused is between 2 and kw times; when the number of traversals is equal to 1, the data does not need to be reused.

[0099] In summary, in each data reading cycle, each selection window outputs one bit of data, forming a column of data to be processed that is input to the processing module. The number of selection windows can be related to the input of the subsequent processing module. For a systolic array of pw×ph, the number of selection windows n=pw; the number of buffers corresponding to each selection window = the width of the convolution kernel (kw), and the next selection window is offset by 1 buffer from the previous selection window, so that data corresponding to the number of traversals equal to 1 does not need to be reused, and data corresponding to the number of traversals not less than 2 can be reused between 2 and kw times.

[0100] It should be noted that for different calculation types, the number of data reuse times and the selection window configuration may be different and need to be adjusted according to the specific calculation mode.

[0101] In this embodiment, the input data of the neural network is cached to a storage module, the storage module includes a plurality of storage queues organized in an array form, the input data includes a plurality of data queues arranged in an array, and each data queue with the same row attribute is stored in each storage queue with the corresponding column attribute, respectively, so that the data is not repeated in each storage queue. This storage method can not only effectively utilize the storage space, but also reduce the amount of data read from the Cache (SRAM), thereby reducing the power consumption of transmitting the data to be processed from the Cache to the storage queue. On this basis, based on the data reading cycle of reading data from the storage queue, the read enable state of each storage queue in the column to be read is controlled according to a preset rule, and based on the data reading cycle, the data to be read in the storage queue in the enabled state is selected and output through a plurality of partially overlapping selection windows, so that data reuse can be realized, ensuring that the reading of data matches the preset processing method of the input data. Then, the multiplexed data can be output to the processing module, so that the processing module can process the input data according to the preset processing method.

[0102] As another optional embodiment of the present application, refer to Figure 5 , is a flow chart of a data processing method provided in Example 2 of the present application. This example is mainly an implementation of step S102 described in the above Example 1. Step S102 may include but is not limited to the following steps:

[0103] Step S1021, obtaining initial control data; the initial control data includes a plurality of control bits corresponding to the arrangement order of each storage queue in the corresponding to-be-read column, each control bit is used to control the read enable state of each storage queue in the corresponding to-be-read column.

[0104] In this embodiment, each control bit may include first data or second data, the first data may correspond to entering an enabled state, and the second data may correspond to exiting an enabled state.

[0105] The initial control data may match the preset processing method. The length of the initial control data, i.e., the number of control bits included, is consistent with the number of storage queues in the corresponding column to be read. For example, the neural network may include: a convolutional neural network, the preset processing method may include: performing a convolution operation based on a convolution kernel, and accordingly, the initial control data may include: a number of control bits, each of which may include second data. For example, the second data may be 0, the first data may be 1, and each control bit in the initial control data is 0.

[0106] Alternatively, the neural network may include: Transformer, and the corresponding preset processing method may include: calculating the relationship between different elements in the input data based on the attention matrix. Accordingly, the initial control data may include: a plurality of control bits, each of which may include the second data. For example, the second data may be 0, the first data may be 1, and each control bit in the initial control data is 0.

[0107] Alternatively, the neural network may include: a sparse Transformer, and the corresponding preset processing method may include: calculating the relationship between different elements in the input data based on a sparse attention matrix. Accordingly, the initial control data may include: a plurality of control bits, each of which may include the first data. For example, the second data may be 0, the first data may be 1, and each control bit in the initial control data is 1.

[0108] Step S1022: Based on the initial control data and based on a data reading cycle, the first data or the second data of each control bit is adjusted according to a preset rule to obtain target control data.

[0109] In the target control data, corresponding to the preset rule, compared with the data of each control bit in the previous data reading cycle, the data of some control bits may be changed or the data of all control bits may be changed.

[0110] For example, the preset processing method of the input data may include: performing a convolution operation based on a convolution kernel of size 3×3, corresponding to Figure 2 In the data queue with the row attribute H1, the columns to be read may be the storage queues with the column attribute Buffer-w1. The initial control data may include 34 control bits, all of which are 0. Figure 6As shown, in the data reading cycle T0, the starting control bit of the 34 control bits is adjusted from the second data to the first data according to a preset rule to obtain the target control data (ie, 10000000 . . . ).

[0111] In the data reading cycle T1, the starting control bit of the 34 control bits is adjusted from the first data to the second data according to a preset rule, and the second control bit is adjusted from the second data to the first data, to obtain the target control data (ie, 01000000...).

[0112] In the data reading cycle T2, the second control bit of the 34 control bits is adjusted from the first data to the second data according to the preset rule, and the third control bit is adjusted from the second data to the first data, to obtain the target control data (ie, 00100000...).

[0113] In the data reading cycle T3, the starting control bit of the 34 control bits is adjusted from the second data to the first data according to the preset rule, and the third control bit is adjusted from the first data to the second data, and the fourth control bit is adjusted from the second data to the first data, to obtain the target control data (ie, 10010000…).

[0114] In the data reading cycle T4, according to the preset rule, the starting control bit of the 34 control bits is adjusted from the first data to the second data, and the second control bit is adjusted from the second data to the first data, and the fourth control bit is adjusted from the first data to the second data, and the fifth control bit is adjusted from the second data to the first data, to obtain the target control data (i.e., 01001000…).

[0115] For each subsequent data reading cycle, the control bit will continue to be adjusted according to the preset rules, which will not be described in detail here.

[0116] Step S1023 : controlling the read enable state of each storage queue in the to-be-read column based on the target control data.

[0117] On the basis that several control bits correspond to the arrangement order of each storage queue in the corresponding column to be read, if the control bit corresponding to the storage queue is the first data, the storage queue is controlled to enter an enabled state; if the control bit corresponding to the storage queue is the second data, the storage queue is controlled to exit the enabled state or remain in a disabled state.

[0118] Continue to refer to Figure 6 For example, in the data reading cycle T0, the target control data is 10000000…, then the storage queue Buffer0 with the control column attribute Buffer-w1 enters the enabled state, and controls other storage queues to remain in the disabled state.

[0119] In the data reading cycle T1, the target control data is 01000000..., then the storage queue Buffer0 with the column attribute Buffer-w1 is controlled to exit the enabled state, and the storage queue Buffer1 with the column attribute Buffer-w1 enters the enabled state, and other storage queues remain in the disabled state.

[0120] In the data reading cycle T2, the target control data is 00100000..., then the storage queue Buffer1 with the column attribute Buffer-w1 is controlled to exit the enabled state, and the storage queue Buffer2 with the column attribute Buffer-w1 enters the enabled state, and other storage queues remain in the disabled state.

[0121] In the data reading cycle T3, the target control data is 10010000…, then the storage queue Buffer0 with the column attribute Buffer-w1 is controlled to enter the enabled state, and the storage queue Buffer2 with the column attribute Buffer-w1 exits the enabled state, and the storage queue Buffer3 with the column attribute Buffer-w1 enters the enabled state, and other storage queues remain in the disabled state.

[0122] In the data reading cycle T4, the target control data is 01001000…, then the storage queue Buffer0 with the column attribute Buffer-w1 is controlled to exit the enabled state, and the storage queue Buffer1 with the column attribute Buffer-w1 enters the enabled state, and the storage queue Buffer3 with the column attribute Buffer-w1 exits the enabled state, and the storage queue Buffer4 with the column attribute Buffer-w1 enters the enabled state, and other storage queues remain in the disabled state.

[0123] For each subsequent data reading cycle, the read enable state of each storage queue in the to-be-read column will continue to be controlled based on the corresponding target control data, which will not be described in detail here.

[0124] In this embodiment, the setting of the initial control data matches the preset processing mode, ensuring that the initial state of the data reading is consistent with the computing requirements of the neural network. By providing an independent control bit corresponding to each storage queue, the read enable state of each storage queue can be accurately controlled. In addition, by dynamically adjusting the control bit according to the data reading cycle and the preset rules, it can be ensured that the read enable state of the storage queue matches the computing requirements of the neural network, and then based on the data reading cycle, the data to be read in the storage queue in the enabled state is selected and output through a number of partially overlapping selection windows, ensuring that the data reuse matches the computing requirements of the neural network. Then, the multiplexed data can be output to the processing module, so that the processing module can process the input data according to the preset processing mode.

[0125] As another optional embodiment of the present application, a data processing method is provided for embodiment 3 of the present application. This embodiment is mainly an implementation method of the processing module described in the above embodiment 2 and adjusting the first data or the second data of each control bit according to preset rules. The processing module may include: a computing array.

[0126] Adjusting the first data or the second data of each of the control bits according to a preset rule may include but is not limited to the following steps:

[0127] Step S11: determining an insertion period corresponding to each of the control bits based on a data arrangement mode of the input data corresponding to the calculation array.

[0128] The data arrangement method corresponds to the preset processing method. The data arrangement method can be expressed as the end position of the data is delayed line by line as the number of lines increases, and the end positions of the data in different lines are kept aligned after reaching the set number of lines. Figure 7 The Transformer shown and Figure 8 The convolutional neural network with a convolution kernel of size 3×3 shown in the figure all conforms to this data arrangement. However, the difference between the Transformer and the convolutional neural network is that the data arrangement of the convolutional neural network is periodic.

[0129] Corresponding to Figure 7 According to the data arrangement shown, the insertion cycle corresponding to each control bit can be synchronized with the data reading cycle.

[0130] Corresponding to Figure 8 In the data arrangement shown, the insertion period corresponding to each control bit can be related to the width of the convolution kernel, that is, the insertion period = n × kw + 1 (that is, kw is the width of the convolution kernel, and n is an integer not less than 0).

[0131] Step S12: Based on the data reading cycle, control the first data in each of the control bits to move bit by bit in a direction from the start control bit to the end control bit.

[0132] In this embodiment, the start control bit corresponds to the storage queue that is first accessed in the column to be read (i.e., the start storage queue), and the end control bit corresponds to the storage queue that is last accessed in the column to be read (i.e., the end storage queue). Under such a setting, when the data read cycle is started, the first data in each of the control bits moves bit by bit in the direction from the start control bit to the end control bit, which can accurately reflect the reading order of the storage queues in the column to be read.

[0133] In this embodiment, a shift register may be provided, the bit width of the shift register is equal to the length of the initial control data, and the shift direction of the shift register matches the direction from the start control bit to the end control bit.

[0134] The initial control data may be stored in a shift register, and each bit in the shift register corresponds to the first data or the second data of each control bit in the initial control data. In each data reading cycle, the shift register may be shifted according to the shift direction and the step length (i.e., 1 bit) to achieve the first data in each of the control bits to be shifted bit by bit along the direction from the starting control bit to the end control bit. It is understandable that the shift operation in the shift register is performed on the data stored therein, rather than on the control bit itself.

[0135] Step S13: When it is determined based on the insertion cycle that the data reading cycle reaches an insertion timing, insert the first data into a starting control bit of each of the control bits.

[0136] Corresponding to the implementation of the shift register, when the data reading cycle reaches the insertion timing, the first data can be inserted into the bit corresponding to the start control bit in the shift register.

[0137] Similar to Figure 7 In the data arrangement shown, the insertion period corresponding to each control bit can be synchronized with the data reading period, which means that each time a new data reading period starts, it is considered to have arrived at the insertion timing.

[0138] For example, the initial control data of the Transformer may include 34 control bits, all of which are 0, such as Fig. 9 As shown, in the data reading cycle T0, 1 is inserted into the start control bit (ie, an implementation of the first data) to obtain the target control data (ie, 10000000 . . . ).

[0139] In the data reading cycle T1, the 1 of the start control bit is moved bit by bit in the direction from the start control bit to the end control bit, and 1 is inserted into the start control bit to obtain the target control data (ie, 11000000 ...).

[0140] In the data reading cycle T2, the 1 of the start control bit and the 1 of the second control bit are moved bit by bit in the direction from the start control bit to the end control bit, and 1 is inserted into the start control bit to obtain the target control data (ie, 11100000…).

[0141] In the data reading cycle T3, the 1 of the start control bit, the 1 of the second control bit, and the 1 of the third control bit are moved bit by bit in the direction from the start control bit to the end control bit, and 1 is inserted into the start control bit to obtain the target control data (ie, 11110000…).

[0142] In the data reading cycle T4, the 1 of the starting control bit, the 1 of the second control bit, the 1 of the third control bit, and the 1 of the fourth control bit are moved bit by bit in the direction from the starting control bit to the end control bit, and 1 is inserted into the starting control bit to obtain the target control data (i.e., 11111000...).

[0143] For each subsequent data reading cycle, the first data in each control bit will continue to be moved bit by bit in the direction from the start control bit to the end control bit, and 1 will be inserted into the start control bit, which will not be repeated here.

[0144] Corresponding to Figure 8 In the data arrangement shown, the insertion period corresponding to each control bit can be related to the width of the convolution kernel, that is, the insertion period = n×kw+1. When the new data reading period is equal to n×kw+1, it is considered that the insertion time has arrived. For example, the initial control data may include 34 control bits, all of which are 0. Figure 6 As shown, in the data reading cycle T0 (ie, the first data reading cycle), 1 is inserted into the start control bit, in the data reading cycle T3 (ie, the fourth data reading cycle), 1 is inserted into the start control bit, and so on.

[0145] In this embodiment, based on the data arrangement mode of the computing array corresponding to the input data, the insertion period corresponding to each of the control bits is determined, and based on the data reading period, the first data in each of the control bits is controlled to move bit by bit in the direction from the starting control bit to the ending control bit. When it is determined based on the insertion period that the data reading period reaches the insertion timing, the first data is inserted into the starting control bit of each of the control bits. This can adapt to various neural networks that require delayed output, such as convolutional neural networks and Transformers, and facilitate the control of data array output similar to a parallelogram shape.

[0146] As another optional embodiment of the present application, a data processing method is provided for embodiment 4 of the present application. This embodiment is mainly an implementation method of the processing module described in the above embodiment 2 and adjusting the first data or the second data of each control bit according to a preset rule. The processing module may include: a computing array. The computing array may include multiple processing units, and the multiple processing units may be arranged in a matrix form. Multiple processing units can perform computing tasks in parallel and can share and transmit data through internal connections.

[0147] Adjusting the first data or the second data of each of the control bits according to a preset rule may include but is not limited to the following steps:

[0148] Step S21: Based on the data arrangement of the input data corresponding to the computing array, when an adjustment timing is reached, simultaneously adjust each of the control bits from the first data to the second data.

[0149] The data arrangement method corresponds to the preset processing method. The data arrangement method can be expressed as the starting positions of data in different rows are always aligned and the end positions of data are always aligned. For example, the neural network can include a sparse Transformer, such as Fig.10 As shown, data in different rows are aligned both horizontally and vertically.

[0150] In this embodiment, the adjustment timing may be when all processing units in the computing array have completed their respective computing tasks.

[0151] For example, Fig.11 As shown, in each data reading cycle before the adjustment timing is reached, each control bit is always kept at 1 (ie, an implementation method of the first data); when the adjustment timing is reached, each control bit is adjusted from 1 to 0 at the same time (ie, an implementation method of the second data).

[0152] In this embodiment, based on the data arrangement of the input data corresponding to the computing array, when the adjustment timing is reached, each of the control bits is adjusted from the first data to the second data at the same time, which can adapt to neural networks such as sparse Transformer that require synchronous output, and facilitate the control of data array output similar to a rectangular shape.

[0153] As another optional embodiment of the present application, a data processing method is provided in Embodiment 5 of the present application. This embodiment is mainly an implementation of step S103 described in the above Embodiment 2. Step S103 may include but is not limited to:

[0154] Step S1031 : Based on the data reading cycle, each selection window in a plurality of selection windows corresponds to a plurality of continuous control bits in the target control data; and there is a partial overlap between the plurality of continuous control bits corresponding to every two adjacent selection windows.

[0155] In this embodiment, the number of the selection windows can match the size of the calculation array.

[0156] The length of each selection window can be kept consistent. For example, the neural network can include a convolutional neural network, and the length of the selection window can match the width of the convolution kernel. For example, the size of the convolution kernel can be 3×3, and accordingly, the length of the selection window can be 3; or, the size of the convolution kernel can be 7×7, and accordingly, the length of the selection window can be 7. Alternatively, the neural network can include: a Transformer or a sparse Transformer, and the length of the selection window can match the size of the attention window of the Transformer or the sparse Transformer.

[0157] The length of the selection window can determine the number of consecutive control bits it corresponds to. For example, the length of the selection window can be 3, then the selection window can correspond to 3 consecutive control bits in the target control data; or, the length of the selection window can be 7, then the selection window can correspond to 7 consecutive control bits in the target control data.

[0158] The number of overlapping control bits between the plurality of consecutive control bits corresponding to every two adjacent selection windows may be the length of the selection window minus one.

[0159] Step S1032: Select a target control bit corresponding to the first first data from the plurality of consecutive control bits.

[0160] The several consecutive control bits may include one first data or multiple first data or no first data. If the several consecutive control bits include one first data or multiple first data, a target control bit corresponding to the first first data can be selected from the several consecutive control bits.

[0161] Step S1033: output the data to be read in the storage queue corresponding to the target control bit.

[0162] Since each selection window corresponds to several consecutive control bits in the target control data, one or more target control bits may be selected based on multiple selection windows. For multiple target control bits, two output methods can be used: one is to read bit by bit, that is, output the data corresponding to each target control bit in turn; the other is to read at one time, that is, output all the data corresponding to all target control bits at one time. The specific output method can be flexibly selected according to the actual application scenario and requirements.

[0163] In this embodiment, the data reading cycle T6 of the convolutional neural network with a convolution kernel of size 3×3 is taken as an example to illustrate the process of selecting output data based on the selection window. Fig.12 As shown, the calculation array may include a matrix systolic array, the size of the matrix systolic array is 32×32, and accordingly, there are 32 selection windows, selection window 0 corresponds to control bit 0 (i.e., the starting control bit) to control bit 2, selection window 1 corresponds to control bit 1-control bit 3, selection window 2 corresponds to control bit 2 to control bit 4, selection window 3 corresponds to control bit 3 to control bit 5, selection window 4 corresponds to control bit 4 to control bit 6, selection window 5 corresponds to control bit 5 to control bit 7, selection window 6 corresponds to control bit 6 to control bit 8, and so on.

[0164] In the data reading cycle T6, 1 is inserted into control bit 0 (i.e., an implementation of the first data). After inserting 1, the target control bit corresponding to the first 1 is selected from control bits 0 to 2 corresponding to selection window 0, i.e., control bit 0, and the data C0-2 to be read in the storage queue Buffer0 (storage queue Buffer0 stores C0-0 to C0-n) corresponding to the output control bit 0 (i.e., an implementation of the target control bit) is output.

[0165] Select the target control bit corresponding to the first 1 from the control bits 1 to 3 corresponding to the selection window 1, that is, control bit 3, and output the to-be-read data C9-1 in the storage queue Buffer3 (storage queue Buffer3 stores C9-0 to C9-n) corresponding to the control bit 3 (that is, an implementation of the target control bit).

[0166] Select the first target control bit corresponding to 1 from control bits 2 to 4 corresponding to selection window 2, that is, control bit 3, and output the to-be-read data C9-1 in the storage queue Buffer3 (storage queue Buffer3 stores C9-0 to C9-n) corresponding to control bit 3 (that is, an implementation of the target control bit).

[0167] Select the first target control bit corresponding to 1 from control bits 3 to 5 corresponding to selection window 3, that is, control bit 3, and output the to-be-read data C9-1 in the storage queue Buffer3 (storage queue Buffer3 stores C9-0 to C9-n) corresponding to control bit 3 (that is, an implementation of the target control bit).

[0168] Select the first target control bit corresponding to 1 from control bits 4 to 6 corresponding to selection window 4, that is, control bit 6, and output the to-be-read data C18-0 in the storage queue Buffer6 (storage queue Buffer6 stores C18-0 to C18-n) corresponding to control bit 6 (that is, an implementation of the target control bit).

[0169] Select the first target control bit corresponding to 1, i.e., control bit 6, from control bits 5 to 7 corresponding to selection window 5, and output the to-be-read data C18-0 in storage queue Buffer6 (storage queue Buffer6 stores C18-0 to C18-n) corresponding to control bit 6 (i.e., an implementation of the target control bit).

[0170] Select the first target control bit corresponding to 1 from control bits 6 to 8 corresponding to selection window 6, that is, control bit 6, and output the to-be-read data C18-0 in the storage queue Buffer6 (storage queue Buffer6 stores C18-0 to C18-n) corresponding to control bit 6 (that is, an implementation of the target control bit).

[0171] In this embodiment, the data reading cycle T3 of the Transformer is taken as an example to illustrate the process of selecting output data based on the selection window. For example, the initial control data of the Transformer may include 34 control bits, and all 34 control bits are 0, such as Fig.13 As shown, the calculation array may include a matrix systolic array, the size of the matrix systolic array is 32×32, and accordingly, there are 32 selection windows, selection window 0 corresponds to control bit 0 (i.e., the starting control bit) to control bit 2, selection window 1 corresponds to control bit 1-control bit 3, selection window 2 corresponds to control bit 2 to control bit 4, selection window 3 corresponds to control bit 3 to control bit 5, selection window 4 corresponds to control bit 4 to control bit 6, selection window 5 corresponds to control bit 5 to control bit 7, selection window 6 corresponds to control bit 6 to control bit 8, and so on.

[0172] In the data reading cycle T3, 1 is inserted into control bit 0 (i.e., an implementation of the first data). After inserting 1, although control bits 0 to 2 corresponding to selection window 0 are all 1, the target control bit corresponding to the first 1 is selected from control bits 0 to 2 corresponding to selection window 0, i.e., control bit 0, and the to-be-read data M1 in the storage queue Buffer0 corresponding to the output control bit 0 (i.e., an implementation of the target control bit) is output.

[0173] Although the control bits 1 to 3 corresponding to the selection window 1 are all 1, the target control bit corresponding to the first 1 is selected from the control bits 1 to 3 corresponding to the selection window 1, that is, control bit 1, and the to-be-read data M2 in the storage queue Buffer1 corresponding to the output control bit 1 (that is, an implementation of the target control bit) is output.

[0174] Although control bits 2 and 3 in control bits 2 to 4 corresponding to selection window 2 are both 1, the first target control bit corresponding to 1 is selected from control bits 2 to 4 corresponding to selection window 2, i.e., control bit 2, and the to-be-read data M2 in the storage queue Buffer2 corresponding to the output control bit 2 (i.e., an implementation of the target control bit) is output.

[0175] The target control bit corresponding to the first 1, namely control bit 3, is selected from control bits 3 to 5 corresponding to selection window 3, and the to-be-read data M4 in storage queue Buffer3 corresponding to control bit 3 (ie, one implementation of the target control bit) is output.

[0176] In this embodiment, the setting of the initial control data matches the preset processing mode, ensuring that the initial state of the data reading is consistent with the computing requirements of the neural network. By corresponding an independent control bit to each storage queue, the read enable state of each storage queue can be accurately controlled. In addition, by dynamically adjusting the control bit according to the data reading cycle and the preset rules, it can be ensured that the read enable state of the storage queue matches the computing requirements of the neural network, and then based on the data reading cycle, each selection window in a plurality of selection windows corresponds to a plurality of continuous control bits in the target control data, and there is a partial overlap between the plurality of continuous control bits corresponding to each two adjacent selection windows. The target control bit corresponding to the first first data is selected from the plurality of continuous control bits, which can ensure that the data reuse matches the computing requirements of the neural network. Then, the data to be read in the storage queue corresponding to the target control bit is output, so that the computing array can process the input data according to the preset processing mode.

[0177] Furthermore, by combining the target control data and several selection windows, each selection window corresponds to several consecutive control bits in the target control data, and there is a partial overlap between the several consecutive control bits corresponding to every two adjacent selection windows, it is possible to adapt to various neural networks that require delayed output, such as convolutional neural networks and Transformers, and neural networks that require synchronous output, such as sparse Transformers, to facilitate the control of data array outputs of different shapes and reduce the complexity of circuit design.

[0178] Furthermore, using the target control data to control the read enable state of the storage queue and performing data selection output based on a number of selection windows can be processed in parallel. This parallel processing method can reduce waiting time and improve system performance.

[0179] As another optional embodiment of the present application, a data processing method is provided in Embodiment 6 of the present application. This embodiment is mainly an implementation of step S103 described in the above Embodiment 1. Step S103 may include but is not limited to:

[0180] Step S1034: Based on the data reading cycle, a plurality of receiving ends of each selection window in a plurality of selection windows respectively correspond to a plurality of continuous storage queues in the to-be-read column.

[0181] In this embodiment, the number of the selection windows can match the size of the calculation array.

[0182] The number of multiple receivers of each selection window can be kept consistent. For example, the neural network can include a convolutional neural network, and the number of multiple receivers of the selection window can match the width of the convolution kernel. For example, the size of the convolution kernel can be 3×3, and accordingly, the selection window can include 3 receivers; or, the size of the convolution kernel can be 7×7, and accordingly, the selection window can include 7 receivers. Alternatively, the neural network can include: a Transformer or a sparse Transformer, and the number of multiple receivers of the selection window can match the size of the attention window of the Transformer or the sparse Transformer.

[0183] The number of multiple receiving ends of the selection window can determine the number of the corresponding consecutive storage queues. For example, the selection window can include 3 receiving ends, and the selection window can correspond to 3 consecutive storage queues in the to-be-read column; or, the selection window can include 7 receiving ends, and the selection window can correspond to 7 consecutive storage queues in the to-be-read column.

[0184] There is a partial overlap between the plurality of continuous storage queues corresponding to every two adjacent selection windows.

[0185] The number of overlaps between the plurality of continuous storage queues corresponding to every two adjacent selection windows may be the length of the selection window minus one.

[0186] Step S1035: Select a first target storage queue in an enabled state from the plurality of continuous storage queues.

[0187] The plurality of consecutive storage queues may include one or more storage queues in an enabled state, or may not include a storage queue in an enabled state.

[0188] If the plurality of consecutive storage queues may include one or more storage queues in an enabled state, a first target storage queue in an enabled state may be selected from the plurality of consecutive storage queues.

[0189] Step S1036: output the data to be read in the target storage queue.

[0190] In this embodiment, the data reading cycle T6 of the convolutional neural network with a convolution kernel of size 3×3 is taken as an example to illustrate the process of selecting output data based on the selection window. Fig.14 As shown, the computing array may include a matrix systolic array, the size of the matrix systolic array is 32×32, and accordingly, there are 32 selection windows, the three receiving ends of selection window 0 correspond to Buffer0 to Buffer2, the three receiving ends of selection window 1 correspond to Buffer1-Buffer3, the three receiving ends of selection window 2 correspond to Buffer2 to Buffer4, the three receiving ends of selection window 3 correspond to Buffer3 to Buffer5, the three receiving ends of selection window 4 correspond to Buffer4 to Buffer6, the three receiving ends of selection window 5 correspond to Buffer5 to Buffer7, the three receiving ends of selection window 6 correspond to Buffer6 to Buffer8, and so on.

[0191] In the data reading cycle T6, Buffer0, Buffer3 and Buffer6 are all in the enabled state, and the other buffers are in the disabled state. In the data reading cycle T6, the first target buffer in the enabled state is selected from Buffer0 to 2 corresponding to the selection window 0, that is, Buffer0, and the data to be read C0-2 in the storage queue Buffer0 (the storage queue Buffer0 stores C0-0 to C0-n) is output.

[0192] The first target buffer in the enabled state, namely Buffer3, is selected from Buffer1 to Buffer3 corresponding to selection window 1, and the to-be-read data C9-1 in the storage queue Buffer3 (the storage queue Buffer3 stores C9-0 to C9-n) is output.

[0193] The first target buffer in the enabled state, namely Buffer3, is selected from Buffer2 to Buffer4 corresponding to selection window 2, and the to-be-read data C9-1 in the storage queue Buffer3 (the storage queue Buffer3 stores C9-0 to C9-n) is output.

[0194] The first target buffer in the enabled state, namely Buffer3, is selected from Buffer3 to 5 corresponding to selection window 3, and the to-be-read data C9-1 in the storage queue Buffer3 (the storage queue Buffer3 stores C9-0 to C9-n) is output.

[0195] The first target buffer in the enabled state, namely Buffer6, is selected from Buffer4 to Buffer6 corresponding to the selection window 4, and the to-be-read data C18-0 in the storage queue Buffer6 (the storage queue Buffer6 stores C18-0 to C18-n) is output.

[0196] The first target buffer in the enabled state, namely Buffer6, is selected from Buffer5 to Buffer7 corresponding to selection window 5, and the to-be-read data C18-0 in storage queue Buffer6 (storage queue Buffer6 stores C18-0 to C18-n) is output.

[0197] The first target buffer in the enabled state, namely Buffer6, is selected from Buffer6 to 8 corresponding to selection window 6, and the to-be-read data C18-0 in storage queue Buffer6 (storage queue Buffer6 stores C18-0 to C18-n) is output.

[0198] In this embodiment, the data reading cycle T3 of the Transformer is taken as an example to illustrate the process of selecting output data based on the selection window. For example, the initial control data of the Transformer may include 34 buffers, and all 34 buffers are 0, such as Fig.15 As shown, the computing array may include a matrix systolic array, the size of the matrix systolic array is 32×32, and accordingly, there are 32 selection windows, selection window 0 corresponds to Buffer0 (i.e., the starting buffer) to Buffer2, the three receiving ends of selection window 1 correspond to Buffer1-Buffer3 respectively, the three receiving ends of selection window 2 correspond to Buffer2 to Buffer4 respectively, the three receiving ends of selection window 3 correspond to Buffer3 to Buffer5 respectively, the three receiving ends of selection window 4 correspond to Buffer4 to Buffer6 respectively, the three receiving ends of selection window 5 correspond to Buffer5 to Buffer7 respectively, the three receiving ends of selection window 6 correspond to Buffer6 to Buffer8 respectively, and so on.

[0199] In the data reading cycle T3, Buffer0, Buffer1, Buffer2 and Buffer3 are all in the enabled state, and the other buffers are in the disabled state.

[0200] In the data reading cycle T3, although Buffer0 to 2 corresponding to the selection window 0 are all in the enabled state, the first target Buffer in the enabled state, namely Buffer0, is selected from Buffer0 to 2 corresponding to the selection window 0, and the data M1 to be read in Buffer0 (i.e., an implementation method of the target Buffer) is output.

[0201] In the data reading cycle T3, although Buffers 1 to 3 corresponding to selection window 1 are all in the enabled state, the first target Buffer 1 in the enabled state, i.e., Buffer 1, is selected from Buffers 1 to 3 corresponding to selection window 1, and the data M2 to be read in Buffer 1 (i.e., an implementation method of the target Buffer) is output.

[0202] In the data reading cycle T3, although Buffer2 and 3 in Buffer2 to 4 corresponding to selection window 2 are both in the enabled state, the first target Buffer in the enabled state, namely Buffer2, is selected from Buffer2 to 4 corresponding to selection window 2, and the data M2 to be read in Buffer2 (i.e., an implementation method of the target Buffer) is output.

[0203] In the data reading cycle T3, the first target buffer in the enabled state, ie, Buffer3, is selected from Buffer3 to 5 corresponding to the selection window 3, and the to-be-read data M4 in Buffer3 (ie, an implementation method of the target buffer) is output.

[0204] In this embodiment, based on the data reading cycle, multiple receiving ends of each selection window in a plurality of selection windows correspond to a plurality of continuous storage queues in the to-be-read column, and there is a partial overlap between the plurality of continuous storage queues corresponding to each two adjacent selection windows. The first target storage queue in an enabled state is selected from the plurality of continuous storage queues, so that data reuse can be matched with the computing requirements of the neural network. Then, the to-be-read data in the target storage queue is output, so that the computing array can process the input data according to the preset processing method.

[0205] In addition, by combining the use of multiple receiving ends of several selection windows, it is possible to adapt to various neural networks that require delayed output, such as convolutional neural networks and Transformers, and neural networks that require synchronous output, such as sparse Transformers, to facilitate the control of data array outputs of different shapes and reduce the complexity of circuit design.

[0206] As another optional embodiment of the present application, a data processing method is provided in Embodiment 7 of the present application. This embodiment is mainly an implementation of step S1033 described in the above Embodiment 1. In this embodiment, the data queue may include: data in a plurality of depth directions;

[0207] For example, Figure 3 As shown, data queues C0 and the like may respectively include data in several C directions (ie, one implementation of the depth direction).

[0208] Step S1033 may include but is not limited to:

[0209] Step S31 , locating the data to be read in the storage queue corresponding to the target control bit through the position index corresponding to the storage queue corresponding to the target control bit.

[0210] The position index may be used to point to the position of the next data to be read in the storage queue.

[0211] The initial position index corresponding to the storage queue corresponding to the target control bit may point to the first data in the storage queue corresponding to the target control bit. For example, the data stored in the storage queue Buffer0 may include: 32 data in the depth direction contained in the data queue C0, namely C0-0 to C0-31. The initial position index may point to C0-0, which is the next data to be read.

[0212] Step S32: output the data to be read, and increment the position index.

[0213] In this embodiment, the input data of the convolutional neural network with a convolution kernel of size 3×3 is taken as an example to illustrate step S31. For example, the data queue C0 contains 32 data in the depth direction, namely C0-0 to C0-31; the data queue C3 contains 32 data in the depth direction, namely C3-0 to C3-31; the data queue C6 contains 32 data in the depth direction, namely C6-0 to C6-31; the data queue C9 contains 32 data in the depth direction, namely C9-0 to C9-31…

[0214] In the data reading cycle T0, the control bit 0 corresponding to the selection window 0 is the target control bit, the position index corresponding to the storage queue Buffer0 corresponding to the control bit 0 points to C0-0, C0-0 is output at T0, and the position index corresponding to Buffer0 is incremented, and the new position index points to C0-1.

[0215] In the data reading cycle T1, the control bit 1 corresponding to the selection window 0 and the control bit 1 corresponding to the selection window 1 are the target control bits respectively. The position index corresponding to the storage queue Buffer1 corresponding to the control bit 1 points to C3-0. In T1, both selection windows 0 and 1 output C3-0, and the position index corresponding to Buffer1 is incremented. The new position index points to C3-1.

[0216] In the data reading cycle T2, the control bit 2 corresponding to selection window 0, the control bit 2 corresponding to selection window 1, and the control bit 2 corresponding to selection window 2 are the target control bits respectively. The position index corresponding to the storage queue Buffer2 corresponding to control bit 2 points to C6-0. At T2, selection windows 0 to 2 all output C6-0, and the position index corresponding to Buffer2 is incremented, and the new position index points to C6-1.

[0217] In the data reading cycle T3, the control bit 1 corresponding to the selection window 0, the control bit 3 corresponding to the selection window 1, the control bit 3 corresponding to the selection window 2, and the control bit 3 corresponding to the selection window 3 are respectively the target control bits, and the position index corresponding to the storage queue Buffer0 corresponding to the control bit 0 points to C0-1. In the data reading cycle T3, the selection window 0 outputs C0-1, and the position index corresponding to Buffer0 is incremented, and the new position index executes C0-2. The position index corresponding to the storage queue Buffer3 corresponding to the control bit 3 points to C9-0. In the data reading cycle T3, the selection windows 1 to 3 all output C9-0, and the position index corresponding to Buffer3 is incremented, and the new position index points to C9-1.

[0218] For each subsequent data reading cycle, the position index corresponding to the storage queue corresponding to the target control bit will continue to be used to locate the data to be read in the storage queue corresponding to the target control bit, output the data to be read, and increment the position index, which will not be repeated here.

[0219] It can be understood that for the position index corresponding to each storage queue, its update (i.e., increment processing) cycle and insertion cycle can match. To illustrate this more intuitively, a specific example can be referred to: for example, the neural network can include a convolutional neural network with a convolution kernel of size 7×7. In the data reading cycle T0, the target control data can be 10000000…, the control bit 0 corresponding to the selection window 0 (i.e., the selection window corresponding to buffer0-buffer6) is the target control bit, and the position index corresponding to the storage queue buffer0 corresponding to the control bit 0 points to buffer0-0, i.e. C0-0, and C0-0 is output at T0, and the position index corresponding to buffer0 is incremented, and the new position index points to buffer0-1, i.e. C0-1.

[0220] like Fig.16 As shown, as time goes by, when the data reading cycle T7 is reached, the control bit 0 corresponding to the selection window 0 is again the target control bit, and the position index corresponding to the storage queue buffer0 corresponding to the control bit 0 points to buffer0-1, that is, C0-1, and C0-1 is output at T7, and the position index corresponding to buffer0 is incremented, and the new position index points to buffer0-2, that is, C0-2.

[0221] However, it should be noted that in this example, the detailed process of selecting and outputting some data is omitted for the sake of brevity and clarity.

[0222] In this embodiment, the position index in the storage queue corresponding to the target control bit can be used to quickly and accurately locate the position of the next data to be read in the storage queue, avoiding the need to traverse the entire storage queue to find data, thereby significantly improving the efficiency of data reading.

[0223] Next, the neural network accelerator provided by the present application is introduced. The neural network accelerator introduced below and the data processing method introduced above can be referenced to each other.

[0224] Reference Fig.17 The neural network accelerator may include: a storage module 100, a control module 200 and a reading module 300.

[0225] The storage module 100 is used to cache the input data of the neural network; the storage module includes a plurality of storage queues organized in an array form, and the input data includes a plurality of data queues arranged in an array, and each data queue with the same row attribute is respectively stored in each storage queue with the corresponding column attribute.

[0226] The control module 200 is used to control the read enable state of each storage queue in the to-be-read column according to a preset rule based on a data read cycle; the preset rule matches a preset processing method of the input data.

[0227] The reading module 300 is used to select and output the to-be-read data in the storage queue in the enabled state through a number of partially overlapping selection windows based on the data reading cycle, and output it to the processing module. The processing module is used to process the input data according to the preset processing method, and each of the selection windows corresponds to multiple storage queues.

[0228] The control module 200 may specifically include:

[0229] a shift register, used for storing initial control data, and controlling the bitwise shifting of first data in each control bit in the initial control data based on the data reading cycle; the initial control data comprises a plurality of control bits, each control bit is used to control the read enable state of each storage queue in the corresponding to-be-read column, each control bit comprises the first data or the second data, the first data corresponds to an entry enable state, and the second data corresponds to an exit enable state;

[0230] A cycle counter, used for generating a trigger signal according to the insertion cycle corresponding to each of the control bits, and sending the trigger signal to the shift register;

[0231] The shift register is further configured to insert the first data into a starting control bit of each of the control bits in response to the trigger signal.

[0232] The reading module 300 may specifically include:

[0233] Multiple selection units and output units.

[0234] Each of the selection units is respectively used for:

[0235] Based on the data reading cycle, the plurality of continuous control bits in the target control data are respectively corresponding; there is a partial overlap between the plurality of continuous control bits corresponding to every two adjacent selection windows; and

[0236] A target control bit corresponding to a first first data is selected from the plurality of consecutive control bits.

[0237] The output unit is used to output the data to be read in the storage queue corresponding to the target control bit.

[0238] In this embodiment, the selection unit may include multiple receiving ends, and the multiple receiving ends of each selection unit correspond to a plurality of continuous storage queues in the to-be-read column; there is a partial overlap between the plurality of continuous storage queues corresponding to every two adjacent selection windows; and,

[0239] A first target storage queue in an enabled state is selected from the plurality of consecutive storage queues.

[0240] Neural network accelerators may also include:

[0241] The position register is used to store the position index corresponding to the storage queue corresponding to the target control bit; the position index is used to point to the position of the next data to be read in the storage queue.

[0242] The output unit is used to read the position index from the position register, locate the data to be read in the storage queue corresponding to the target control bit based on the position index, output the data to be read, and increment the position index.

[0243] In another embodiment of the present application, an electronic device is provided, which may include the above-mentioned neural network accelerator. The neural network accelerator may be used to improve the computing efficiency of the neural network.

[0244] In this embodiment, an NPU (neural processing unit) or other types of AI chips can be used as a neural network accelerator.

[0245] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0246] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0247] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0248] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A data processing method, comprising: Cache the input data of the neural network into a storage module, wherein the storage module includes a plurality of storage queues organized in an array form, and the input data includes a plurality of data queues arranged in an array, and each data queue with the same row attribute is stored in each storage queue with the corresponding column attribute; Based on the data reading cycle, the read enable state of each storage queue in the to-be-read column is controlled according to preset rules; The preset rule matches the preset processing method of the input data; Based on the data reading cycle, the data to be read in the storage queue in the enabled state is selected and output through a number of partially overlapping selection windows, and output to the processing module. The processing module is used to process the input data according to the preset processing method, and each of the selection windows corresponds to multiple storage queues.

2. The data processing method according to claim 1, based on the data reading cycle, controlling the read enable state of each storage queue in the to-be-read column according to a preset rule, comprising: Obtaining initial control data; The initial control data includes a plurality of control bits corresponding to the arrangement order of each storage queue in the corresponding to-be-read column, each control bit is used to control the read enable state of each storage queue in the corresponding to-be-read column, each control bit includes first data or second data, the first data corresponds to entering the enable state, and the second data corresponds to exiting the enable state; Based on the initial control data and based on a data reading cycle, adjusting the first data or the second data of each control bit according to a preset rule to obtain target control data; The initial control data matches the preset processing mode; The read enable state of each storage queue in the to-be-read column is controlled based on the target control data.

3. The data processing method according to claim 2, wherein the processing module comprises: Compute arrays; The adjusting the first data or the second data of each of the control bits according to a preset rule includes: Based on the data arrangement mode of the input data corresponding to the computing array, the insertion period corresponding to each of the control bits is determined; the data arrangement mode corresponds to the preset processing mode; the data arrangement mode is characterized in that the end position of the data is delayed row by row as the number of rows increases, and the end positions of the data in different rows are kept aligned after reaching the set number of rows; Based on the data reading cycle, controlling the first data in each of the control bits to move bit by bit in a direction from a start control bit to an end control bit; When it is determined based on the insertion period that the data reading period reaches an insertion timing, the first data is inserted into a start control bit of each of the control bits.

4. The data processing method according to claim 2, wherein the processing module comprises: Compute arrays; The adjusting the first data or the second data of each of the control bits according to a preset rule includes: Based on the data arrangement mode of the input data corresponding to the computing array, when the adjustment timing is reached, each of the control bits is adjusted from the first data to the second data at the same time; the data arrangement mode is characterized in that the starting positions of data in different rows are always aligned and the end positions of the data are always aligned.

5. The data processing method according to claim 2, wherein based on the data reading cycle, the method selects and outputs the to-be-read data in the storage queue in the enabled state through a plurality of partially overlapping selection windows, comprising: Based on the data reading cycle, each selection window in a plurality of selection windows corresponds to a plurality of continuous control bits in the target control data; There is a partial overlap between the plurality of continuous control bits corresponding to every two adjacent selection windows; Selecting a target control bit corresponding to a first first data from the plurality of consecutive control bits; Output the data to be read in the storage queue corresponding to the target control bit.

6. The data processing method according to claim 1, wherein based on the data reading cycle, the method selects and outputs the to-be-read data in the storage queue in the enabled state through a plurality of partially overlapping selection windows, comprising: Based on the data reading cycle, a plurality of receiving ends of each selection window in a plurality of selection windows respectively correspond to a plurality of continuous storage queues in the to-be-read column; and there is a partial overlap between the plurality of continuous storage queues corresponding to every two adjacent selection windows; Selecting a first target storage queue in an enabled state from the plurality of consecutive storage queues; Output the data to be read in the target storage queue.

7. The data processing method according to claim 5, wherein the data queue comprises: Data in several depth directions; The outputting the to-be-read data in the storage queue corresponding to the target control bit includes: The data to be read is located in the storage queue corresponding to the target control bit by using the position index corresponding to the storage queue corresponding to the target control bit; the position index is used to point to the position of the next data to be read in the storage queue; The data to be read is outputted, and the position index is incremented.

8. A neural network accelerator, comprising: A storage module, used for caching input data of the neural network; the storage module includes a plurality of storage queues organized in an array form, the input data includes a plurality of data queues arranged in an array, and each data queue with the same row attribute is stored in each storage queue with the corresponding column attribute; A control module, used to control the read enable state of each storage queue in the to-be-read column according to preset rules based on a data read cycle; The preset rule matches the preset processing method of the input data; A reading module is used to select and output the to-be-read data in the storage queue in the enabled state through a number of partially overlapping selection windows based on the data reading cycle, and output it to the processing module. The processing module is used to process the input data according to the preset processing method, and each of the selection windows corresponds to multiple storage queues.

9. The neural network accelerator according to claim 8, wherein the control module comprises: a shift register, used for storing initial control data, and controlling the bitwise shifting of first data in each control bit in the initial control data based on the data reading cycle; the initial control data includes a plurality of control bits, each control bit is used to control the read enable state of each storage queue in the corresponding to-be-read column, each control bit includes the first data or the second data, the first data corresponds to an entry enable state, and the second data corresponds to an exit enable state; A cycle counter, used for generating a trigger signal according to the insertion cycle corresponding to each of the control bits, and sending the trigger signal to the shift register; The shift register is further configured to insert the first data into a starting control bit of each of the control bits in response to the trigger signal.

10. An electronic device comprising: A neural network accelerator as described in any one of claims 8-9.

Images