A data processing method, a neural network accelerator and an electronic device

Abstract

The application discloses a data processing method, a neural network accelerator and an electronic device. The method comprises the following steps: buffering input data of a neural network to a storage module, the storage module comprises a plurality of storage queues in an array form, the input data comprises a plurality of data queues arranged in an array, and each data queue with consistent row attributes is stored in a corresponding storage queue with consistent column attributes; based on a data reading period, controlling the read enable state of each storage queue in a to-be-read column according to a preset rule; based on the data reading period, selecting to-be-read data in the storage queue in the enable state through a plurality of selection windows with partial overlap, and outputting the to-be-read data to a processing module, the processing module is used for processing the input data according to a preset processing mode, and each selection window corresponds to a plurality of storage queues.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a data processing method, a neural network accelerator and an electronic device. BACKGROUND

[0002] In the calculation process of a deep learning model (such as a convolutional neural network), in order to improve the calculation efficiency, a common method is to convert the operation of the deep learning model into a matrix multiplication form, which requires the input data to be tiled into a matrix form. In the tiling process, part of the input data may be copied multiple times. Such data processing not only increases the demand for storage space, but also increases the power consumption of data reading and transmission. SUMMARY

[0003] The technical scheme provided by the present application is as follows:

[0004] The first aspect of the present application provides a data processing method, comprising:

[0005] Caching input data of a neural network to a storage module, the storage module comprising a plurality of storage queues organized in an array form, the input data comprising a plurality of data queues arranged in an array, each data queue with consistent row attributes being stored in a corresponding storage queue with consistent column attributes;

[0006] Based on a data reading period, controlling the read enable state of each storage queue in the to-be-read column according to a preset rule; the preset rule matches a preset processing mode of the input data;

[0007] Based on the data reading period, selecting to-be-read data in the storage queue in the enabled state through a plurality of selection windows with partial overlap, and outputting to a processing module, the processing module being configured to process the input data according to the preset processing mode, each selection window corresponding to a plurality of storage queues.

[0008] Based on a data reading period, controlling the read enable state of each storage queue in the to-be-read column according to a preset rule, comprising:

[0009] Obtaining initial control data; the initial control data comprising a plurality of control bits corresponding to the arrangement order of each storage queue in the corresponding to-be-read column, each control bit being configured to control the read enable state of each storage queue in the corresponding to-be-read column, each control bit comprising a first data or a second data, the first data corresponding to entering the enabled state, and the second data corresponding to exiting the enabled state;

[0010] On the basis of the initial control data, the first data or the second data of each control bit is adjusted according to a preset rule based on a data reading period to obtain target control data; the initial control data matches the preset processing mode;

[0011] The target control data is used to control the read enable state of each storage queue in the column to be read.

[0012] The processing module comprises a calculation array.

[0013] The first data or the second data of each control bit is adjusted according to a preset rule, comprising:

[0014] Based on a data arrangement mode corresponding to the calculation array of the input data, a corresponding insertion period of each control bit is determined; the data arrangement mode corresponds to a preset processing mode; the data arrangement mode is that the end position of data is delayed row by row backward with the increase of row number, and the end positions of data of different rows are kept aligned after reaching a set row number;

[0015] Based on the data reading period, the first data in each control bit is moved bit by bit in the direction from the starting control bit to the end control bit;

[0016] When the data reading period reaches an insertion opportunity based on the insertion period, the first data is inserted into the starting control bit in each control bit.

[0017] The processing module comprises a calculation array.

[0018] The first data or the second data of each control bit is adjusted according to a preset rule, comprising:

[0019] Based on a data arrangement mode corresponding to the calculation array of the input data, each control bit is simultaneously adjusted from the first data to the second data when reaching an adjustment opportunity; the data arrangement mode is that the starting positions of data of different rows are always kept aligned and the end positions of data are always kept aligned.

[0020] The target control data is used to control the read enable state of each storage queue in the column to be read.

[0021] Based on the data reading period, each selection window of a plurality of selection windows respectively corresponds to a plurality of continuous control bits in the target control data; there is a partial overlap between the plurality of continuous control bits corresponding to each two adjacent selection windows;

[0022] selecting a target control bit corresponding to a first data from the plurality of continuous control bits;

[0023] outputting data to be read in a storage queue corresponding to the target control bit.

[0024] The data to be read in the storage queue in the enabled state is outputted through a plurality of selection windows with partial overlap based on the data read period, including:

[0025] Each selection window of the plurality of selection windows corresponds to a plurality of receiving ends, and the plurality of receiving ends correspond to a plurality of continuous storage queues in the column to be read based on the data read period; there is partial overlap between the plurality of continuous storage queues corresponding to each two adjacent selection windows.

[0026] selecting a target storage queue in the enabled state from the plurality of continuous storage queues;

[0027] outputting data to be read in the target storage queue.

[0028] The data queue includes data in a plurality of depth directions.

[0029] The data to be read in the storage queue corresponding to the target control bit is outputted, including:

[0030] locating the data to be read in the storage queue corresponding to the target control bit through a position index corresponding to the storage queue corresponding to the target control bit; the position index is used to point to the position of the next data to be read in the storage queue.

[0031] outputting the data to be read and incrementing the position index.

[0032] Another aspect of the present application provides a neural network accelerator, including:

[0033] a storage module configured to cache input data of a neural network; the storage module includes a plurality of storage queues in an array form; the input data includes a plurality of data queues arranged in an array, and each data queue with consistent row attributes is stored in a corresponding storage queue with consistent column attributes;

[0034] a control module configured to control a read enable state of each storage queue in a column to be read based on a data read period according to a preset rule; the preset rule matches a preset processing mode of the input data.

[0035] The reading module is configured to select, based on the data reading period, the to-be-read data in the storage queue in an enabled state through a plurality of selection windows with partial overlap, and output the to-be-read data to the processing module, wherein the processing module is configured to process the input data according to the preset processing mode, and each selection window corresponds to a plurality of storage queues.

[0036] The control module comprises:

[0037] The shift register is configured to store initial control data, and control a first data in each control bit in the initial control data to move based on the data reading period, wherein the initial control data comprises a plurality of control bits, each control bit is configured to control a read enable state of each storage queue in a corresponding to-be-read column, and each control bit comprises the first data or second data, the first data corresponds to an entering enabled state, and the second data corresponds to an exiting enabled state.

[0038] The period counter is configured to generate a trigger signal according to an insertion period corresponding to each control bit, and send the trigger signal to the shift register.

[0039] The shift register is further configured to insert the first data into a start control bit in each control bit in response to the trigger signal.

[0040] The third aspect of the present application provides an electronic device, comprising: any one of the neural network accelerators described above. BRIEF DESCRIPTION OF DRAWINGS

[0041] The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent by describing in detail the following specific embodiments thereof with reference to the attached drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0042] Figure 1 A schematic diagram of data tiling is provided for the present application;

[0043] Figure 2 A flowchart of a data processing method is provided for Embodiment 1 of the present application;

[0044] Figure 3 A cache diagram of a storage module is provided for the present application;

[0045] Figure 4 A schematic diagram of a plurality of selection windows with partial overlap selecting to-be-read data in a storage queue in an enabled state is provided for the present application;

[0046] Figure 5A flowchart of a data processing method provided for Embodiment 2 of the present application;

[0047] Figure 6 An adjustment diagram of first data or second data of each control bit provided for the present application;

[0048] Figure 7 A diagram of a data arrangement mode corresponding to a Transformer provided for the present application;

[0049] Figure 8 A diagram of a data arrangement mode corresponding to a convolutional neural network provided for the present application;

[0050] Figure 9 A diagram of insertion of first data into a start control bit corresponding to a data arrangement mode of a Transformer provided for the present application;

[0051] Figure 10 A diagram of a data arrangement mode corresponding to a sparse Transformer provided for the present application;

[0052] Figure 11 An adjustment diagram of first data or second data of each control bit corresponding to a sparse Transformer provided for the present application;

[0053] Figure 12 A diagram of selection of output data corresponding to a convolutional neural network provided for the present application;

[0054] Figure 13 A diagram of selection of output data corresponding to a Transformer provided for the present application;

[0055] Figure 14 Another diagram of selection of output data corresponding to a convolutional neural network provided for the present application;

[0056] Figure 15 Another diagram of selection of output data corresponding to a Transformer provided for the present application;

[0057] Figure 16 A diagram of data output containing a position index provided for the present application;

[0058] Figure 17 A structural diagram of a neural network accelerator provided for the present application. DETAILED DESCRIPTION

[0059] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0060] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0061] The terms "first," "second," etc., used in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0062] To improve the computational efficiency of matrix calculations, especially at the hardware implementation level, Processing Element (PE) technology has been widely used. This technology faces a key problem in practical applications: the input data of the PE needs to be tiled. For example, in convolution calculations, tiling reorganizes the input data according to the size of the convolution kernel, so that each column of data corresponds to the data needed for a specific region of the input data to be convolved with the kernel. However, this process leads to data bloat, with the following specific impacts:

[0063] First, from a storage perspective, tiling significantly increases the size of the matrix that needs to be stored. Taking a 3×3 convolution kernel as an example, when using a 32×32 matrix systolic array, tiling results in a 34×9 data matrix—that is, a 34-row, 9-column data matrix. These rows contain the data that is repeatedly copied to form the complete convolution operation. For example... Figure 1 The C6, C7, and C8 components (highlighted in red) appear three times in the tiled data, meaning they are expanded threefold. This expansion directly results in a threefold increase in the corresponding storage space (e.g., SRAM). Secondly, from the perspective of data read and transfer power consumption, the increased data volume leads to a corresponding increase in power consumption for storing and retrieving this data. Especially... Figure 1In the example shown, if some data (such as C6, C7, C8, etc.) are expanded by a factor of 3 during the tiling process (meaning that they appear multiple times in the tiling matrix in some form), this means that more power will be consumed in reading and writing these repeated data during data transmission.

[0064] Therefore, this application provides a data processing method to achieve efficient data reuse, ensuring that a piece of data is copied only once, thereby effectively reducing the burden of storage space and power consumption caused by data expansion.

[0065] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0066] Reference Figure 2 This is a flowchart illustrating a data processing method provided in Embodiment 1 of this application, as shown below. Figure 2 As shown, the method may include, but is not limited to, the following steps:

[0067] Step S101: Cache the input data of the neural network into the storage module. The storage module includes several storage queues organized in an array. The input data includes several data queues arranged in an array. Each data queue with the same row attribute is stored in a corresponding storage queue with the corresponding column attribute.

[0068] An array of storage queues can be understood as an arrangement of logical addresses, not a direct array layout of physical entities. The arrangement of logical addresses can be represented by the allocation of storage addresses. For example, storage queues in the same row share the same base address, which represents the starting position of that row of queues in memory. Although the queues in that row share this base address, each queue has a unique offset from the base address to distinguish between different queues. Storage queues in different rows, however, have different base addresses.

[0069] In this embodiment, the number of several storage queues organized in an array can be set based on the scale of all input data of the neural network.

[0070] The number of storage queues organized in an array can also be configured based on the actual amount of input data involved in the convolution operation performed on the computation array. Taking a 32×32 matrix systolic array and a 3×3 convolution kernel as an example, since the bit width of the matrix systolic array is 32, 32 rows of tiled data are required as input. Correspondingly, the convolution kernel needs to slide at least 32 times across the input data. When the width of the convolution kernel is 3, the width of the input data needs to be 34 columns. This is because, when the convolution kernel slides across the input data, it needs to cover at least 3 columns of input data each time (i.e., in the W direction). When the convolution kernel slides to the edge of the input data, to ensure that the kernel can still cover 3 columns of data on the last slide, one column (2 columns in total) needs to be reserved on each side of the input data as a boundary. Therefore, the total number of input data columns is 34 (i.e., wph + kw - 1). 34 columns of input data require 34 rows of storage queues, while a single convolution operation requires 3 rows of input data, requiring 3 columns of storage queues. Therefore, the number of storage queues can be set to 3×34, where 3 represents the number of rows directly related to each convolution operation, and 34 can represent the number of data queues that are consistent with the row attributes of the matrix systolic array. That is, the number of storage queues for a certain column attribute is equal to wph+kw-1, where wph represents the bit width of the matrix systolic array and kw represents the width of the convolution kernel.

[0071] For example, such as Figure 3 As shown, the data queue with row attribute H1 includes C0, C3, C6, ..., C99, and C0, C3, C6, ..., C99 can be stored in the various storage queues with column attribute Buffer-w1; the data queue with row attribute H2 includes C1, C4, C7, ..., C100, and data queues C1, C4, C7, ..., C100 can be stored in the various storage queues with column attribute Buffer-w2; the data queue with row attribute H3 includes C2, C5, C8, ..., C101, and data queues C2, C5, C8, ..., C101 can be stored in the various storage queues with column attribute Buffer-w3.

[0072] If the storage queue has a bit width of 256 bits and the data bit width is 8 bits, the length of the data queue can be 32, meaning the data queue contains 32 pieces of input data on the channel (C direction).

[0073] Of course, the length of the data queue can also be greater than 32 data bits. In the case of greater than 32, 32 data bits can be read from data queues C0, C3, C6, ..., C99 and stored into the respective storage queues with column attribute Buffer-w1, 32 data bits can be read from data queues C1, C4, C7, ..., C100 and stored into the respective storage queues with column attribute Buffer-w2, and 32 data bits can be read from data queues C2, C5, C8, ..., C101 and stored into the respective storage queues with column attribute Buffer-w3.

[0074] For example, data queue C0 includes data C0-0, C0-1, C0-2, ..., C0-63; data queue C3 includes data C3-0, C3-1, C3-2, ..., C3-63; data queue C6 includes data C6-0, C6-1, C6-2, ..., C6-64, ...; and data queue C99 includes data C99-0, C99-1, C99-2, ..., C99-64. Data C0-0, C0-1, C0-2, ..., C0-31 can be read from data queue C0 into a column with the attribute Buffer. The data is read from data queue C3 (C3-0, C3-1, C3-2, ..., C3-31) into Buffer1 (column attribute Buffer-w1); from data queue C6 (C6-0, C6-1, C6-2, ..., C6-31) into Buffer2 (column attribute Buffer-w1); ...; from data queue C99 (C99-0, C99-1, C99-2, ..., C99-31) into Buffer33 (column attribute Buffer-w1). After the first 32 data items are calculated, the remaining data is read and stored in the corresponding buffers. At this point, the data in each data queue has been divided into two parts by channel, and the calculation results corresponding to the two parts can be concatenated sequentially.

[0075] Step S102: Based on the data reading cycle, control the read enable state of each storage queue in the column to be read according to preset rules; the preset rules are the same as the preset processing method of the input data.

[0076] The column to be read can be one or more of several storage queues organized in an array.

[0077] The read enable state indicates whether data in a storage queue can be read. When the read enable state is enabled, it means that the data in the storage queue can be read during the data read cycle. When the read enable state is disabled, it means that the data in the storage queue will not be read during the data read cycle.

[0078] The system controls the read enable status of each storage queue in the column to be read according to preset rules, aiming to ensure that the reading of data matches the preset processing method of the input data.

[0079] Different types of neural networks have different preset processing methods for their input data. For example, neural networks can include convolutional neural networks, and the preset processing methods can include convolution operations based on convolution kernels.

[0080] Alternatively, a neural network may include a Transformer, and the corresponding preset processing method may include calculating the relationship between different elements in the input data based on the attention matrix.

[0081] Alternatively, neural networks can include sparse Transformers. The main difference between the default processing method for sparse Transformers and the default processing method for Transformers is that the attention matrix of a sparse Transformer is designed to be sparse in order to reduce computation and memory usage.

[0082] Although different types of neural networks have different preset processing methods for input data, the data flow speed from the storage module to the processing module can be the same, so the data reading cycle can remain unchanged.

[0083] Step S103: Based on the data reading cycle, select the data to be read in the storage queue that is in the enabled state through several partially overlapping selection windows, and output it to the processing module. The processing module is used to process the input data according to the preset processing method. Each selection window corresponds to multiple storage queues.

[0084] Although each selection window corresponds to multiple storage queues, this does not mean that data in all of its corresponding storage queues will be read. Among the multiple storage queues corresponding to a selection window, only the data in those storage queues that are in the enabled state will be selected and output.

[0085] Because there is some overlap between the selection windows, some enabled storage queues can be selected multiple times during the data reading cycle, thus achieving data reuse.

[0086] For example, the preset processing method for input data may include: performing a convolution operation based on a convolution kernel of size 3×3, corresponding to Figure 3 The data queue with row attribute H1 can have its columns to be read as various storage queues with column attribute Buffer-w1, such as... Figure 4As shown, in data read cycle T0, Buffer0 with column attribute Buffer-w1 is enabled, and C0-0 is read from Buffer0 with column attribute Buffer-w1; in data read cycle T1, Buffer1 with column attribute Buffer-w1 is enabled, and C3-0 is read from Buffer1 with column attribute Buffer-w1; in data read cycle T2, Buffer2 with column attribute Buffer-w1 is enabled, and C6-0 is read from Buffer2 with column attribute Buffer-w1; in data read cycle T3, Buffer0 and Buffer3 with column attribute Buffer-w1 are enabled, and C0-1 is read from Buffer0 with column attribute Buffer-w1 and C9-0 is read from Buffer3 with column attribute Buffer-w1;.....

[0087] Selection window 1 corresponds to Buffer0, Buffer1, and Buffer2 with column attribute Buffer-w1; selection window 2 corresponds to Buffer1, Buffer2, and Buffer3 with column attribute Buffer-w1; selection window 3 corresponds to Buffer2, Buffer3, and Buffer4 with column attribute Buffer-w1; selection window 4 corresponds to Buffer3, Buffer4, and Buffer5 with column attribute Buffer-w1; selection window 5 corresponds to Buffer4, Buffer5, and Buffer6 with column attribute Buffer-w1…

[0088] For a convolution kernel of size kw×kh, the length of each selection window is kw (that is, each selection window corresponds to kw consecutive buffers), and the subsequent selection window is offset by 1 buffer from the previous selection window.

[0089] During the data reading cycle T0, Buffer0 is enabled. Buffer0 has only selection window 1 corresponding to it. Therefore, based on selection window 1, C0-0 in output Buffer0 can be selected, that is, C0-0 is not reused.

[0090] During the data reading cycle T1, Buffer1 is enabled. Buffer1 has selection windows 1 and 2 corresponding to it. Therefore, based on selection windows 1 and 2, C3-0 in Buffer1 can be selected for output, that is, C3-0 is reused twice.

[0091] During the data reading cycle T2, Buffer2 is enabled. Buffer2 has selection windows 1, 2 and 3 corresponding to it. Therefore, based on selection windows 1, 2 and 3, C6-0 in output Buffer3 can be selected, that is, C6-0 is reused 3 times.

[0092] During data read cycle T3, Buffer0 and Buffer3 are enabled. Buffer0 only has selection window 1 corresponding to it, therefore, based on selection window 1, C0-1 in Buffer0 can be selected for output. Buffer3 has selection windows 2, 3, and 4 corresponding to it, therefore, based on selection windows 2, 3, and 4, C9-0 in Buffer3 can be selected for output, meaning C9-0 is reused 3 times…

[0093] During subsequent data reading cycles, data reuse will be performed in accordance with the above-mentioned output selection method, which will not be elaborated further here.

[0094] Reference Figure 4 It can be seen that the data reuse situation is related to the sliding method of the 3×3 convolution kernel in the input data during the convolution operation. The data queue C0 is located at the beginning of the input data of row attribute H1. The convolution kernel will traverse the data queue C0 during the first slide. As the convolution kernel slides, the data queue C0 will not be traversed again. Therefore, the data in the data queue C0 does not need to be reused.

[0095] Data queue C3 is located at the second position of the input data in row attribute H1. As the convolution kernel slides, data queue C3 will be traversed twice for convolution operations. Therefore, the data in data queue C3 will be reused twice.

[0096] Data queue C6 is located at the third position of the input data in row attribute H1. As the convolution kernel slides, data queue C6 will be traversed three times for convolution operations. Therefore, the data in data queue C6 will be reused three times.

[0097] The situation is similar for data in subsequent positions such as C9 in the data queue. As the convolution kernel slides, the subsequent data will be traversed three times, two times, or once. Accordingly, the data will be reused three times, two times, or not at all.

[0098] For a convolution kernel of size kw×kh, each data queue can be traversed between 1 and kw times. The specific number of traversals is determined by the kernel width, the input data width, and the stride. When the number of traversals is not less than 2, the data can be reused between 2 and kw times; when the number of traversals is equal to 1, the data does not need to be reused.

[0099] In summary, during each data reading cycle, each selection window outputs one bit of data, forming a sequence of data to be processed by the processing module. The number of selection windows can be related to the input of subsequent processing modules. For a pw×ph systolic array, the number of selection windows n=pw; the number of buffers corresponding to each selection window is equal to the width of the convolution kernel (kw). Each subsequent selection window is offset by one buffer from the previous one. This ensures that data with a traversal count of 1 does not need to be reused, while data with a traversal count of at least 2 can be reused between 2 and kw times.

[0100] It should be noted that the number of data reuses and the selection window configuration may differ for different calculation types and need to be adjusted according to the specific calculation mode.

[0101] In this embodiment, the input data of the neural network is cached in a storage module. The storage module includes several storage queues organized in an array. The input data includes several data queues arranged in an array. Data queues with consistent row attributes are stored in corresponding storage queues with specific column attributes, ensuring that data is not duplicated within each storage queue. This storage method not only effectively utilizes storage space but also reduces the amount of data read from the cache (SRAM), thereby reducing the power consumption of transferring data to be processed from the cache to the storage queues. Based on this, according to the data read cycle from the storage queues, the read enable state of each storage queue within the column to be read is controlled according to preset rules. Based on the data read cycle, several partially overlapping selection windows are used to select and output the data to be read from the enabled storage queues, achieving data reuse and ensuring that the data read matches the preset processing method of the input data. Then, the reused data can be output to the processing module, allowing the processing module to process the input data according to the preset processing method.

[0102] As another optional embodiment of this application, refer to Figure 5 This is a flowchart illustrating a data processing method provided in Embodiment 2 of this application. This embodiment is mainly an implementation of step S102 described in Embodiment 1 above. Step S102 may include, but is not limited to, the following steps:

[0103] Step S1021: Obtain initial control data; the initial control data includes several control bits corresponding to the arrangement order of each storage queue in the corresponding column to be read, and each control bit is used to control the read enable state of each storage queue in the corresponding column to be read.

[0104] In this embodiment, each control bit may include first data or second data, where the first data may correspond to entering the enable state and the second data may correspond to exiting the enable state.

[0105] The initial control data can be matched with the preset processing method. The length of the initial control data, i.e., the number of control bits it contains, is consistent with the number of storage queues in the corresponding column to be read. For example, the neural network may include a convolutional neural network, and the preset processing method may include convolution operations based on convolution kernels. Accordingly, the initial control data may include several control bits, and each control bit may include second data. For example, the second data can be 0, the first data can be 1, and all control bits in the initial control data are 0.

[0106] Alternatively, the neural network may include a Transformer, and the corresponding pre-defined processing method may include calculating the relationship between different elements in the input data based on an attention matrix. Accordingly, the initial control data may include several control bits, each of which may include second data. For example, the second data can be 0, the first data can be 1, and all control bits in the initial control data are 0.

[0107] Alternatively, the neural network may include a sparse Transformer, and the corresponding pre-defined processing method may include calculating the relationship between different elements in the input data based on a sparse attention matrix. Accordingly, the initial control data may include several control bits, each of which may contain first data. For example, the second data may be 0, the first data may be 1, and all control bits in the initial control data may be 1.

[0108] Step S1022: Based on the initial control data, and according to the data reading cycle, adjust the first data or second data of each control bit according to a preset rule to obtain the target control data.

[0109] In the target control data, corresponding to the preset rules, compared with the data of each control bit in the previous data reading cycle, the data of some control bits may change or the data of all control bits may change.

[0110] For example, the preset processing method for input data may include: performing a convolution operation based on a convolution kernel of size 3×3, corresponding to Figure 2 The data queue with row attribute H1 can be read from various storage queues with column attribute Buffer-w1. The initial control data can include 34 control bits, all of which are 0. Figure 6As shown, during the data reading cycle T0, the starting control bit among the 34 control bits is adjusted from the second data to the first data according to the preset rules to obtain the target control data (i.e., 10000000...).

[0111] During the data reading cycle T1, the starting control bit among the 34 control bits is adjusted from the first data to the second data according to the preset rules, and the second control bit is adjusted from the second data to the first data to obtain the target control data (i.e., 01000000...).

[0112] During the data reading cycle T2, the second control bit among the 34 control bits is adjusted from the first data to the second data according to the preset rules, and the third control bit is adjusted from the second data to the first data to obtain the target control data (i.e., 00100000...).

[0113] During the data reading cycle T3, the starting control bit among the 34 control bits is adjusted from the second data to the first data according to the preset rules, and the third control bit is adjusted from the first data to the second data, and the fourth control bit is adjusted from the second data to the first data, so as to obtain the target control data (i.e., 10010000...).

[0114] During the data reading cycle T4, the starting control bit among the 34 control bits is adjusted from the first data to the second data according to the preset rules, and the second control bit is adjusted from the second data to the first data, and the fourth control bit is adjusted from the first data to the second data, and the fifth control bit is adjusted from the second data to the first data, so as to obtain the target control data (i.e., 01001000...).

[0115] For each subsequent data reading cycle, the control bits will continue to be adjusted according to the preset rules, which will not be elaborated here.

[0116] Step S1023: Control the read enable state of each storage queue in the column to be read based on the target control data.

[0117] Based on the correspondence between several control bits and the arrangement order of each storage queue in the column to be read, if the control bit corresponding to the storage queue is the first data, the storage queue is controlled to enter the enabled state; if the control bit corresponding to the storage queue is the second data, the storage queue is controlled to exit the enabled state or remain in the disabled state.

[0118] Continue to refer to Figure 6 For example, in the data reading cycle T0, if the target control data is 10000000…, then the storage queue Buffer0 with the control column attribute Buffer-w1 will enter the enabled state, while other storage queues will remain in the disabled state.

[0119] During the data reading cycle T1, if the target control data is 01000000…, then the storage queue Buffer0 with the control column attribute Buffer-w1 exits the enabled state, the storage queue Buffer1 with the control column attribute Buffer-w1 enters the enabled state, and other storage queues remain in the disabled state.

[0120] During the data reading cycle T2, if the target control data is 00100000…, then the storage queue Buffer1 with the control column attribute Buffer-w1 will exit the enabled state, and the storage queue Buffer2 with the control column attribute Buffer-w1 will enter the enabled state, while other storage queues will remain in the disabled state.

[0121] During the data reading cycle T3, if the target control data is 10010000…, then the storage queue Buffer0 with the control column attribute Buffer-w1 will enter the enabled state, the storage queue Buffer2 with the control column attribute Buffer-w1 will exit the enabled state, the storage queue Buffer3 with the control column attribute Buffer-w1 will enter the enabled state, and the other storage queues will remain in the disabled state.

[0122] During data reading cycle T4, if the target control data is 01001000…, then the storage queue Buffer0 with column attribute Buffer-w1 will exit the enabled state, the storage queue Buffer1 with column attribute Buffer-w1 will enter the enabled state, the storage queue Buffer3 with column attribute Buffer-w1 will exit the enabled state, the storage queue Buffer4 with column attribute Buffer-w1 will enter the enabled state, and the other storage queues will remain in the disabled state.

[0123] For each subsequent data reading cycle, the read enable status of each storage queue in the column to be read will continue to be controlled based on the corresponding target control data, which will not be elaborated here.

[0124] In this embodiment, the initial control data settings are matched with the preset processing method, ensuring that the initial state of data reading is consistent with the computational requirements of the neural network. By assigning an independent control bit to each storage queue, the read enable state of each storage queue can be precisely controlled. Furthermore, by dynamically adjusting the control bit according to the data reading cycle and preset rules, the read enable state of the storage queue can be kept in line with the computational requirements of the neural network. Based on the data reading cycle, several partially overlapping selection windows are used to select and output the data to be read from the enabled storage queues, ensuring that data reuse matches the computational requirements of the neural network. Then, the reused data can be output to the processing module, allowing the processing module to process the input data according to the preset processing method.

[0125] As another optional embodiment of this application, a data processing method is provided in embodiment 3 of this application. This embodiment is mainly an implementation of the processing module described in embodiment 2 above and the first data or second data of each control bit adjusted according to preset rules. The processing module may include: a computing array.

[0126] Adjusting the first or second data of each control bit according to preset rules may include, but is not limited to, the following steps:

[0127] Step S11: Based on the data arrangement of the input data corresponding to the computing array, determine the insertion period corresponding to each of the control bits.

[0128] The data arrangement method corresponds to the preset processing method. The data arrangement method can be characterized by the data's end position being delayed row by row as the number of rows increases, and after reaching a set number of rows, the end positions of the data in different rows remain aligned. For example, as... Figure 7 The Transformer shown and such Figure 8 The convolutional neural networks shown all conform to this data arrangement method, with 3×3 kernels. However, the difference between Transformers and convolutional neural networks is that the data arrangement method in convolutional neural networks exhibits periodicity.

[0129] Corresponding to, for example Figure 7 The data arrangement shown allows the insertion cycle for each control bit to be synchronized with the data reading cycle.

[0130] Corresponding to, for example Figure 8 The data arrangement shown allows the insertion period for each control bit to be related to the width of the convolution kernel, i.e., insertion period = n × kw + 1 (where kw is the width of the convolution kernel and n is an integer not less than 0).

[0131] Step S12: Based on the data reading cycle, control the first data in each of the control bits to move bit by bit in the direction from the start control bit to the end control bit.

[0132] In this embodiment, the start control bit corresponds to the first accessed storage queue in the column to be read (i.e., the start storage queue), while the end control bit corresponds to the last accessed storage queue in the column to be read (i.e., the end storage queue). With this setting, when the data reading cycle starts, the first data in each of the control bits moves bit by bit in the direction from the start control bit to the end control bit, which can accurately reflect the reading order of the storage queues in the column to be read.

[0133] In this embodiment, a shift register can be set, the bit width of which is equal to the length of the initial control data, and the shift direction of the shift register matches the direction from the start control bit to the end control bit.

[0134] The initial control data can be stored in a shift register, where each bit corresponds to either the first or second data of each control bit in the initial control data. During each data read cycle, the shift register can shift according to the shift direction and step size (i.e., 1 bit) to move the first data of each control bit bit by bit along the direction from the start control bit to the end control bit. It is understood that the shift operation in the shift register is performed on the data stored therein, not on the control bits themselves.

[0135] Step S13: When the data reading cycle reaches the insertion timing based on the insertion cycle, insert the first data into the start control bit of each of the control bits.

[0136] In the implementation of a shift register, the first data can be inserted into the bit corresponding to the start control bit in the shift register when the data reading cycle reaches the insertion timing.

[0137] Similar to Figure 7 The data arrangement shown allows the insertion cycle corresponding to each control bit to be synchronized with the data reading cycle. This means that whenever a new data reading cycle begins, it is considered that the insertion opportunity has arrived.

[0138] For example, the initial control data for a Transformer can include 34 control bits, all of which are 0. Figure 9 As shown, during the data reading cycle T0, a 1 is inserted into the start control bit (i.e., one implementation of the first data) to obtain the target control data (i.e., 10000000...).

[0139] During the data reading cycle T1, the 1 in the start control bit is moved bit by bit along the direction from the start control bit to the end control bit, and a 1 is inserted into the start control bit to obtain the target control data (i.e., 11000000…).

[0140] During the data reading cycle T2, the 1 in the start control bit and the 1 in the second control bit are moved bit by bit along the direction from the start control bit to the end control bit, and a 1 is inserted into the start control bit to obtain the target control data (i.e., 11100000…).

[0141] During the data reading cycle T3, the 1 in the start control bit, the 1 in the second control bit, and the 1 in the third control bit are moved bit by bit along the direction from the start control bit to the end control bit, and a 1 is inserted into the start control bit to obtain the target control data (i.e., 11110000…).

[0142] During the data reading cycle T4, the 1 in the start control bit, the 1 in the second control bit, the 1 in the third control bit, and the 1 in the fourth control bit are moved bit by bit along the direction from the start control bit to the end control bit, and a 1 is inserted into the start control bit to obtain the target control data (i.e., 11111000...).

[0143] For each subsequent data reading cycle, the first data in each control bit will continue to be moved bit by bit in the direction from the start control bit to the end control bit, and a 1 will be inserted into the start control bit. This will not be elaborated further here.

[0144] Corresponding to, for example Figure 8 The data arrangement shown allows the insertion period for each control bit to be related to the width of the convolution kernel; that is, the insertion period = n × kw + 1. When the new data reading period equals n × kw + 1, it is considered that the insertion opportunity has arrived. For example, the initial control data may include 34 control bits, all of which are 0. Figure 6 As shown, a 1 is inserted into the start control bit during data read cycle T0 (i.e., the first data read cycle), and a 1 is inserted into the start control bit during data read cycle T3 (i.e., the fourth data read cycle), and so on.

[0145] In this embodiment, based on the data arrangement of the input data corresponding to the computing array, the insertion period corresponding to each of the control bits is determined. Based on the data reading period, the first data in each of the control bits is controlled to move bit by bit in the direction from the start control bit to the end control bit. When the data reading period is determined to reach the insertion timing based on the insertion period, the first data is inserted into the start control bit of each of the control bits. This can be adapted to various neural networks that require delayed output, such as convolutional neural networks and Transformers, and is convenient for controlling the output of data arrays with a shape similar to a parallelogram.

[0146] As another optional embodiment of this application, this embodiment provides a data processing method for embodiment 4. This embodiment is mainly an implementation of the processing module described in embodiment 2 above and the first or second data of each control bit adjusted according to preset rules. The processing module may include a computing array. The computing array may include multiple processing units, which may be arranged in a matrix. The multiple processing units may execute computing tasks in parallel and may share and transmit data through internal connections.

[0147] Adjusting the first or second data of each control bit according to preset rules may include, but is not limited to, the following steps:

[0148] Step S21: Based on the data arrangement of the input data corresponding to the computing array, when the adjustment time is reached, each of the control bits is simultaneously adjusted from the first data to the second data.

[0149] The data arrangement method corresponds to the preset processing method. The data arrangement method can be characterized by ensuring that the starting positions of data in different rows are always aligned and the ending positions of data are always aligned. For example, a neural network may include a sparse Transformer, such as... Figure 10 As shown, the data in different rows are aligned in both the horizontal and vertical directions.

[0150] In this embodiment, the adjustment timing can be when all processing units in the computing array have completed their respective computing tasks.

[0151] For example, such as Figure 11 As shown, during each data reading cycle before the adjustment timing is reached, each control bit remains at 1 (i.e., one implementation of the first data); when the adjustment timing is reached, each control bit is simultaneously adjusted from 1 to 0 (i.e., one implementation of the second data).

[0152] In this embodiment, based on the data arrangement of the input data corresponding to the computing array, when the adjustment time is reached, each of the control bits is simultaneously adjusted from the first data to the second data. This can adapt to neural networks that require synchronous output, such as sparse Transformers, and facilitate the control of data array outputs with a rectangular shape.

[0153] As another optional embodiment of this application, this embodiment provides a data processing method for embodiment 5 of this application. This embodiment is mainly an implementation of step S103 described in embodiment 2 above. Step S103 may include, but is not limited to:

[0154] Step S1031: Based on the data reading cycle, each of the several selection windows corresponds to several consecutive control bits in the target control data; there is partial overlap between the several consecutive control bits corresponding to every two adjacent selection windows.

[0155] In this embodiment, the number of selection windows can be matched with the size of the computing array.

[0156] The length of each selection window can remain consistent. For example, the neural network can include a convolutional neural network, where the length of the selection window can match the width of the convolutional kernel. For instance, the kernel size could be 3×3, and the corresponding selection window length could be 3; or the kernel size could be 7×7, and the corresponding selection window length could be 7. Alternatively, the neural network can include a Transformer or a sparse Transformer, where the length of the selection window can match the size of the attention window of the Transformer or sparse Transformer.

[0157] The length of the selection window determines the number of consecutive control bits it corresponds to. For example, if the length of the selection window is 3, then the selection window can correspond to 3 consecutive control bits in the target control data; or, if the length of the selection window is 7, then the selection window can correspond to 7 consecutive control bits in the target control data.

[0158] The number of overlapping control bits between any two adjacent selection windows corresponding to the plurality of consecutive control bits can be the length of the selection window minus one.

[0159] Step S1032: Select the target control bit corresponding to the first data from the plurality of consecutive control bits.

[0160] The plurality of consecutive control bits may contain one or more first data or no dialing first data. If the plurality of consecutive control bits contain one or more first data, the target control bit corresponding to the first first data can be selected from the plurality of consecutive control bits.

[0161] Step S1033: Output the data to be read in the storage queue corresponding to the target control bit.

[0162] Since each selection window corresponds to several consecutive control bits in the target control data, multiple selection windows may select one or more target control bits. For multiple target control bits, two output methods can be used: one is bit-by-bit reading, which outputs the data corresponding to each target control bit sequentially; the other is all-at-once reading, which outputs all the data corresponding to all target control bits at once. The specific output method can be flexibly selected according to the actual application scenario and requirements.

[0163] In this embodiment, taking a data reading cycle T6 of a convolutional neural network with a 3×3 kernel as an example, the process of selecting output data based on a selection window is explained. For example, as Figure 12 As shown, the computation array can include a matrix systolic array with a size of 32×32. Correspondingly, there are 32 selection windows. Selection window 0 corresponds to control bits 0 (i.e., the start control bit) to control bit 2, selection window 1 corresponds to control bits 1 to control bits 3, selection window 2 corresponds to control bits 2 to control bits 4, selection window 3 corresponds to control bits 3 to control bits 5, selection window 4 corresponds to control bits 4 to control bits 6, selection window 5 corresponds to control bits 5 to control bits 7, selection window 6 corresponds to control bits 6 to control bits 8, and so on.

[0164] During the data reading cycle T6, insert 1 into control bit 0 (i.e., one implementation of the first data). After inserting 1, select the target control bit corresponding to the first 1 from control bits 0 to 2 corresponding to selection window 0, i.e. control bit 0, and output the data to be read C0-2 in the storage queue Buffer0 (which stores C0-0 to C0-n) corresponding to control bit 0 (i.e., one implementation of the target control bit).

[0165] Select the first target control bit corresponding to the first 1 from the control bits 1 to 3 corresponding to the selection window 1, namely control bit 3, and output the data C9-1 to be read in the storage queue Buffer3 (which stores C9-0 to C9-n) corresponding to the output control bit 3 (i.e., one implementation of the target control bit).

[0166] Select the first 1 corresponding to the target control bit from the control bits 2 to 4 corresponding to the selection window 2, that is, control bit 3, and output the data C9-1 to be read in the storage queue Buffer3 (which stores C9-0 to C9-n) corresponding to the output control bit 3 (that is, one implementation of the target control bit).

[0167] Select the first 1 corresponding to the target control bit from the control bits 3 to 5 corresponding to the selection window 3, i.e., control bit 3, and output the data C9-1 to be read in the storage queue Buffer3 (which stores C9-0 to C9-n) corresponding to the output control bit 3 (i.e., one implementation of the target control bit).

[0168] Select the first 1 corresponding to the target control bit from the control bits 4 to 6 corresponding to the selection window 4, namely control bit 6, and output the data C18-0 to be read in the storage queue Buffer 6 (which stores C18-0 to C18-n) corresponding to the output control bit 6 (i.e., one implementation of the target control bit).

[0169] Select the first 1 corresponding to the target control bit from the control bits 5 to 7 corresponding to the selection window 5, namely control bit 6, and output the data C18-0 to be read in the storage queue Buffer 6 (which stores C18-0 to C18-n) corresponding to the output control bit 6 (i.e., one implementation of the target control bit).

[0170] Select the first 1 corresponding to the target control bit from the control bits 6 to 8 corresponding to the selection window 6, i.e., control bit 6, and output the data C18-0 to be read in the storage queue Buffer 6 (which stores C18-0 to C18-n) corresponding to the output control bit 6 (i.e., one implementation of the target control bit).

[0171] In this embodiment, the process of selecting output data based on the selection window is further illustrated using Transformer data read cycle T3 as an example. For instance, the initial control data of the Transformer may include 34 control bits, all of which are 0, such as... Figure 13 As shown, the computation array can include a matrix systolic array with a size of 32×32. Correspondingly, there are 32 selection windows. Selection window 0 corresponds to control bits 0 (i.e., the start control bit) to control bit 2, selection window 1 corresponds to control bits 1 to control bits 3, selection window 2 corresponds to control bits 2 to control bits 4, selection window 3 corresponds to control bits 3 to control bits 5, selection window 4 corresponds to control bits 4 to control bits 6, selection window 5 corresponds to control bits 5 to control bits 7, selection window 6 corresponds to control bits 6 to control bits 8, and so on.

[0172] During the data reading cycle T3, a 1 is inserted into control bit 0 (i.e., one implementation of the first data). After inserting the 1, although control bits 0 to 2 corresponding to selection window 0 are all 1, the target control bit corresponding to the first 1 is selected from control bits 0 to 2 corresponding to selection window 0, i.e., control bit 0, and the data M1 to be read in the storage queue Buffer0 corresponding to control bit 0 (i.e., one implementation of the target control bit) is output.

[0173] Although all three control bits 1 to 3 corresponding to selection window 1 are 1, the target control bit corresponding to the first 1 is selected from the three control bits 1 to 3 corresponding to selection window 1. That is, control bit 1, and the data M2 to be read in the storage queue Buffer1 corresponding to control bit 1 (that is, one implementation of the target control bit) is output.

[0174] Although control bits 2 and 3 in the control bits 2 to 4 corresponding to selection window 2 are both 1, the target control bit corresponding to the first 1 in the control bits 2 to 4 corresponding to selection window 2 is selected, namely control bit 2, and the data M2 to be read in the storage queue Buffer2 corresponding to the output control bit 2 (i.e., one implementation of the target control bit) is output.

[0175] Select the first 1 corresponding to the target control bit from the control bits 3 to 5 corresponding to the selection window 3, i.e., control bit 3, and output the data M4 to be read in the storage queue Buffer 3 corresponding to the control bit 3 (i.e., one implementation of the target control bit).

[0176] In this embodiment, the initial control data settings are matched with the preset processing method, ensuring that the initial state of data reading is consistent with the computational requirements of the neural network. By assigning an independent control bit to each storage queue, the read enable state of each storage queue can be precisely controlled. Furthermore, dynamically adjusting the control bit according to the data reading cycle and preset rules ensures that the read enable state of the storage queue matches the computational requirements of the neural network. Based on the data reading cycle, each of the several selection windows corresponds to several consecutive control bits in the target control data. There is partial overlap between the consecutive control bits corresponding to every two adjacent selection windows. Selecting the target control bit corresponding to the first piece of data from these consecutive control bits ensures that data reuse matches the computational requirements of the neural network. Then, the data to be read in the storage queue corresponding to the target control bit is output, allowing the computing array to process the input data according to the preset processing method.

[0177] Furthermore, by combining the use of target control data and several selection windows, each selection window corresponds to several consecutive control bits in the target control data, and there is partial overlap between the several consecutive control bits corresponding to every two adjacent selection windows, it can adapt to various neural networks that require delayed output, such as convolutional neural networks and Transformers, as well as neural networks that require synchronous output, such as sparse Transformers, which facilitates the control of data array output of different shapes and reduces circuit design complexity.

[0178] Furthermore, the read enable state of the storage queue controlled by the target control data and the data selection output based on several selection windows can be processed in parallel. This parallel processing method can reduce waiting time and improve system performance.

[0179] As another optional embodiment of this application, this embodiment provides a data processing method for embodiment 6 of this application. This embodiment is mainly an implementation of step S103 described in embodiment 1 above. Step S103 may include, but is not limited to:

[0180] Step S1034: Based on the data reading cycle, the multiple receiving ends of each selection window in the multiple selection windows correspond to several consecutive storage queues in the column to be read.

[0181] In this embodiment, the number of selection windows can be matched with the size of the computing array.

[0182] The number of receivers in each selection window can remain consistent. For example, the neural network can include a convolutional neural network, where the number of receivers in the selection window can match the width of the convolutional kernel. For instance, the kernel size can be 3×3, and correspondingly, the selection window can include 3 receivers; or, the kernel size can be 7×7, and correspondingly, the selection window can include 7 receivers. Alternatively, the neural network can include a Transformer or a sparse Transformer, where the number of receivers in the selection window can match the size of the attention window of the Transformer or sparse Transformer.

[0183] The number of receivers in a selection window determines the number of consecutive storage queues it corresponds to. For example, a selection window may include 3 receivers, in which case the selection window may correspond to 3 consecutive storage queues in the column to be read; or, a selection window may include 7 receivers, in which case the selection window may correspond to 7 consecutive storage queues in the column to be read.

[0184] There is partial overlap between the several consecutive storage queues corresponding to each pair of adjacent selection windows.

[0185] The number of overlaps between the several consecutive storage queues corresponding to any two adjacent selection windows can be the length of the selection window minus one.

[0186] Step S1035: Select the first target storage queue that is in an enabled state from the plurality of consecutive storage queues.

[0187] The plurality of consecutive storage queues may include one or more storage queues in an enabled state, or may not include any storage queues in an enabled state.

[0188] If the plurality of consecutive storage queues may contain one or more storage queues in an enabled state, the first target storage queue in an enabled state may be selected from the plurality of consecutive storage queues.

[0189] Step S1036: Output the data to be read in the target storage queue.

[0190] In this embodiment, taking a data reading cycle T6 of a convolutional neural network with a 3×3 kernel as an example, the process of selecting output data based on a selection window is explained. For example, as Figure 14 As shown, the computation array can include a matrix systolic array with a size of 32×32. Correspondingly, there are 32 selection windows. The three receivers of selection window 0 correspond to Buffer0 to Buffer2, the three receivers of selection window 1 correspond to Buffer1 to Buffer3, the three receivers of selection window 2 correspond to Buffer2 to Buffer4, the three receivers of selection window 3 correspond to Buffer3 to Buffer5, the three receivers of selection window 4 correspond to Buffer4 to Buffer6, the three receivers of selection window 5 correspond to Buffer5 to Buffer7, the three receivers of selection window 6 correspond to Buffer6 to Buffer8, and so on.

[0191] During data read cycle T6, Buffer0, Buffer3, and Buffer6 are all enabled, while the other buffers are disabled. During data read cycle T6, the first enabled target buffer, Buffer0, is selected from Buffer0 to Buffer2 corresponding to selection window 0, and the data to be read, C0-2, is output from storage queue Buffer0 (which stores C0-0 to C0-n).

[0192] Select the first enabled target Buffer, namely Buffer3, from Buffer1 to 3 corresponding to selection window 1, and output the data C9-1 to be read in the storage queue Buffer3 (the storage queue Buffer3 stores C9-0 to C9-n).

[0193] Select the first enabled target Buffer, namely Buffer3, from Buffer2 to 4 corresponding to selection window 2, and output the data C9-1 to be read in the storage queue Buffer3 (the storage queue Buffer3 stores C9-0 to C9-n).

[0194] Select the first enabled target Buffer, i.e., Buffer3, from Buffer3 to 5 corresponding to selection window 3, and output the data C9-1 to be read in the storage queue Buffer3 (the storage queue Buffer3 stores C9-0 to C9-n).

[0195] Select the first enabled target Buffer, namely Buffer6, from Buffer4 to 6 corresponding to selection window 4, and output the data C18-0 to be read in the storage queue Buffer6 (the storage queue Buffer6 stores C18-0 to C18-n).

[0196] Select the first enabled target Buffer, namely Buffer6, from Buffer5 to 7 corresponding to selection window 5, and output the data C18-0 to be read in the storage queue Buffer6 (the storage queue Buffer6 stores C18-0 to C18-n).

[0197] Select the first enabled target Buffer, i.e., Buffer6, from Buffer6 to 8 corresponding to selection window 6, and output the data C18-0 to be read in the storage queue Buffer6 (the storage queue Buffer6 stores C18-0 to C18-n).

[0198] In this embodiment, the process of selecting output data based on the selection window is further illustrated using Transformer data reading cycle T3 as an example. For instance, the initial control data of the Transformer may include 34 buffers, all of which are 0. Figure 15 As shown, the computation array can include a matrix systolic array with a size of 32×32. Correspondingly, there are 32 selection windows. Selection window 0 corresponds to Buffer0 (i.e., the starting Buffer) to Buffer2. The three receivers of selection window 1 correspond to Buffer1-Buffer3 respectively. The three receivers of selection window 2 correspond to Buffer2 to Buffer4 respectively. The three receivers of selection window 3 correspond to Buffer3 to Buffer5 respectively. The three receivers of selection window 4 correspond to Buffer4 to Buffer6 respectively. The three receivers of selection window 5 correspond to Buffer5 to Buffer7 respectively. The three receivers of selection window 6 correspond to Buffer6 to Buffer8 respectively, and so on.

[0199] During the data reading cycle T3, Buffer0, Buffer1, Buffer2 and Buffer3 are all enabled, while other buffers are disabled.

[0200] During the data reading cycle T3, although Buffers 0 to 2 corresponding to selection window 0 are all enabled, the first enabled target Buffer, namely Buffer 0, is selected from Buffers 0 to 2 corresponding to selection window 0, and the data M1 to be read in Buffer 0 (i.e., one implementation of the target Buffer) is output.

[0201] During the data reading cycle T3, although Buffers 1 to 3 corresponding to selection window 1 are all enabled, the first target Buffer that is enabled, namely Buffer 1, is selected from Buffers 1 to 3 corresponding to selection window 1, and the data M2 to be read in Buffer 1 (i.e., one implementation of the target Buffer) is output.

[0202] During the data reading cycle T3, although Buffer2 and Buffer3 are both enabled in Buffer2 to 4 corresponding to selection window 2, the first enabled target Buffer, namely Buffer2, is selected from Buffer2 to 4 corresponding to selection window 2, and the data M2 to be read in Buffer2 (i.e., one implementation of the target Buffer) is output.

[0203] During the data reading cycle T3, select the first enabled target buffer, i.e., Buffer3, from Buffer3 to 5 corresponding to selection window 3, and output the data M4 to be read in Buffer3 (i.e., one implementation of the target buffer).

[0204] In this embodiment, based on the data reading cycle, multiple receivers in each of the several selection windows correspond to several consecutive storage queues within the column to be read. There is partial overlap between the consecutive storage queues corresponding to any two adjacent selection windows. Selecting the first enabled target storage queue from these consecutive storage queues ensures that data reuse matches the computational needs of the neural network. Then, the data to be read from the target storage queue is output, allowing the computing array to process the input data according to the preset processing method.

[0205] Furthermore, by combining multiple receivers with several selection windows, it can adapt to various neural networks that require delayed output, such as convolutional neural networks and Transformers, as well as neural networks that require synchronous output, such as sparse Transformers, making it convenient to control the output of data arrays of different shapes and reducing the complexity of circuit design.

[0206] As another optional embodiment of this application, a data processing method is provided in embodiment 7 of this application. This embodiment is mainly an implementation of step S1033 described in embodiment 1 above. In this embodiment, the data queue may include: data in several depth directions;

[0207] For example, such as Figure 3 As shown, data queues C0, etc., can each include data in several C directions (i.e., one implementation of the depth direction).

[0208] Step S1033 may include, but is not limited to:

[0209] Step S31: Locate the data to be read in the storage queue corresponding to the target control bit using the position index of the storage queue corresponding to the target control bit.

[0210] The location index can be used to point to the position of the next piece of data to be read in the storage queue.

[0211] The initial position index of the storage queue corresponding to the target control bit can point to the first data in the storage queue corresponding to the target control bit. For example, the data stored in storage queue Buffer0 may include 32 data items in the depth direction contained in data queue C0, namely C0-0 to C0-31. The initial position index can point to C0-0, which is the next data to be read.

[0212] Step S32: Output the data to be read and increment the position index.

[0213] In this embodiment, step S31 is explained using the input data of a convolutional neural network with a 3×3 kernel as an example. For example, data queue C0 contains 32 data points in the depth direction, i.e., C0-0 to C0-31; data queue C3 contains 32 data points in the depth direction, i.e., C3-0 to C3-31; data queue C6 contains 32 data points in the depth direction, i.e., C6-0 to C6-31; data queue C9 contains 32 data points in the depth direction, i.e., C9-0 to C9-31…

[0214] During the data reading cycle T0, the control bit 0 corresponding to window 0 is selected as the target control bit. The position index of the storage queue Buffer0 corresponding to control bit 0 points to C0-0. C0-0 is output at T0, and the position index corresponding to Buffer0 is incremented, with the new position index pointing to C0-1.

[0215] During the data reading cycle T1, the control bit 1 corresponding to selection window 0 and the control bit 1 corresponding to selection window 1 are the target control bits. The position index of the storage queue Buffer1 corresponding to control bit 1 points to C3-0. In T1, both selection windows 0 and 1 output C3-0, and the position index corresponding to Buffer1 is incremented, with the new position index pointing to C3-1.

[0216] During the data reading cycle T2, the control bit 2 corresponding to selection window 0, the control bit 2 corresponding to selection window 1, and the control bit 2 corresponding to selection window 2 are the target control bits. The position index of the storage queue Buffer2 corresponding to control bit 2 points to C6-0. In T2, selection windows 0 to 2 all output C6-0, and the position index corresponding to Buffer2 is incremented, with the new position index pointing to C6-1.

[0217] During data reading cycle T3, control bit 1 corresponding to selection window 0, control bit 3 corresponding to selection window 1, control bit 3 corresponding to selection window 2, and control bit 3 corresponding to selection window 3 are all target control bits. The position index of the storage queue Buffer0 corresponding to control bit 0 points to C0-1. During data reading cycle T3, selection window 0 outputs C0-1 and increments the position index of Buffer0, with the new position index being C0-2. The position index of the storage queue Buffer3 corresponding to control bit 3 points to C9-0. During data reading cycle T3, selection windows 1 to 3 all output C9-0 and increment the position index of Buffer3, with the new position index pointing to C9-1.

[0218] For each subsequent data reading cycle, the data to be read will be located in the storage queue corresponding to the target control bit through the position index of the storage queue corresponding to the target control bit, the data to be read will be output, and the position index will be incremented. This will not be elaborated further here.

[0219] Understandably, for each storage queue's corresponding position index, its update (i.e., increment processing) cycle can match the insertion cycle. To illustrate this more intuitively, consider a concrete example: For instance, a neural network could include a convolutional neural network with 7×7 kernels. In the data reading cycle T0, the target control data could be 10000000…, and the control bit 0 corresponding to selection window 0 (i.e., the selection window corresponding to buffer0-buffer6) is the target control bit. The position index of the storage queue buffer0 corresponding to control bit 0 points to buffer0-0, i.e., C0-0. At T0, C0-0 is output, and the position index corresponding to buffer0 is incremented, with the new position index pointing to buffer0-1, i.e., C0-1.

[0220] like Figure 16 As shown, as time progresses, when the data reading cycle T7 is reached, the control bit 0 corresponding to the selection window 0 becomes the target control bit again, and the position index corresponding to the storage queue buffer0 corresponding to the control bit 0 points to buffer0-1, i.e., C0-1. At T7, C0-1 is output, and the position index corresponding to buffer0 is incremented, and the new position index points to buffer0-2, i.e., C0-2.

[0221] However, it should be noted that in this example, the detailed process of selecting and outputting some data has been omitted for the sake of simplicity and clarity.

[0222] In this embodiment, by using the position index in the storage queue corresponding to the target control bit, the position of the next data to be read in the storage queue can be quickly and accurately located, avoiding the need to traverse the entire storage queue to find the data, thereby significantly improving the efficiency of data reading.

[0223] The neural network accelerator provided in this application will be described below. The neural network accelerator described below and the data processing methods described above can be referred to in correspondence.

[0224] Reference Figure 17 The neural network accelerator may include: a storage module 100, a control module 200, and a reading module 300.

[0225] The storage module 100 is used to cache the input data of the neural network. The storage module includes several storage queues organized in an array. The input data includes several data queues arranged in an array. Each data queue with the same row attribute is stored in a corresponding storage queue with the corresponding column attribute.

[0226] The control module 200 is used to control the read enable state of each storage queue in the column to be read according to a preset rule based on the data reading cycle; the preset rule matches the preset processing method of the input data.

[0227] The reading module 300 is used to select and output the data to be read in the storage queue that is in the enabled state through several partially overlapping selection windows based on the data reading cycle, and output it to the processing module. The processing module is used to process the input data according to the preset processing method, and each selection window corresponds to multiple storage queues.

[0228] The control module 200 may specifically include:

[0229] A shift register is used to store initial control data and, based on the data reading cycle, controls the first data in each control bit of the initial control data to shift bit by bit; the initial control data includes several control bits, each control bit is used to control the read enable state of each storage queue in the corresponding column to be read, each control bit includes the first data or the second data, the first data corresponds to entering the enable state, and the second data corresponds to exiting the enable state.

[0230] A period counter is used to generate a trigger signal according to the insertion period corresponding to each of the control bits, and send the trigger signal to the shift register;

[0231] The shift register is also used to insert the first data into the start control bit of each of the control bits in response to the trigger signal.

[0232] The reading module 300 may specifically include:

[0233] Multiple selection units and output units.

[0234] Each of the selection units is used for:

[0235] Based on the data reading cycle, each corresponds to a plurality of consecutive control bits in the target control data; there is partial overlap between the plurality of consecutive control bits corresponding to every two adjacent selection windows; and,

[0236] Select the first target control bit corresponding to the first data from the plurality of consecutive control bits.

[0237] The output unit is used to output the data to be read in the storage queue corresponding to the target control bit.

[0238] In this embodiment, the selection unit may include multiple receiving ends, and the multiple receiving ends of each selection unit correspond to several consecutive storage queues within the column to be read; there is partial overlap between the several consecutive storage queues corresponding to every two adjacent selection windows; and,

[0239] Select the first target storage queue that is enabled from the plurality of consecutive storage queues.

[0240] Neural network accelerators may also include:

[0241] A position register is used to store the position index of the storage queue corresponding to the target control bit; the position index is used to point to the position of the next piece of data to be read in the storage queue.

[0242] The output unit is used to read the position index from the position register, locate the data to be read in the storage queue corresponding to the target control bit based on the position index, output the data to be read, and increment the position index.

[0243] In another embodiment of this application, an electronic device is provided that may include the aforementioned neural network accelerator. The neural network accelerator can be used to improve the computational efficiency of neural networks.

[0244] In this embodiment, an NPU (Neural Processing Unit) or other types of AI chips can be used as a neural network accelerator.

[0245] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0246] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0247] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0248] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A data processing method, comprising: The input data of the neural network is cached in the storage module, which includes several storage queues organized in an array. The input data includes several data queues arranged in an array. Each data queue with the same row attribute is stored in a corresponding storage queue with the corresponding column attribute. Based on the data reading cycle, the read enable status of each storage queue in the column to be read is controlled according to preset rules; The preset rule matches the preset processing method of the input data; Based on the data reading cycle, the data to be read in the storage queue that is in the enabled state is selected and output through several partially overlapping selection windows, and output to the processing module. The processing module is used to process the input data according to the preset processing method. Each selection window corresponds to multiple storage queues. The step of selecting and outputting the data to be read from the enabled storage queue based on the data reading cycle through several partially overlapping selection windows includes: Based on the data reading cycle, each of the multiple receiving ends of a selection window corresponds to a number of consecutive storage queues within the column to be read; there is partial overlap between the number of consecutive storage queues corresponding to any two adjacent selection windows. Select the first target storage queue that is enabled from the plurality of consecutive storage queues; Output the data to be read from the target storage queue.

2. The data processing method according to claim 1, based on the data reading cycle, controls the read enable state of each storage queue in the column to be read according to a preset rule, including: Obtain initial control data; The initial control data includes several control bits corresponding to the arrangement order of each storage queue in the corresponding column to be read. Each control bit is used to control the read enable state of each storage queue in the corresponding column to be read. Each control bit includes first data or second data. The first data corresponds to entering the enable state, and the second data corresponds to exiting the enable state. Based on the initial control data, and according to the data reading cycle, the first data or the second data of each control bit is adjusted according to a preset rule to obtain the target control data; The initial control data is matched with the preset processing method; Based on the target control data, control the read enable state of each storage queue in the column to be read.

3. The data processing method according to claim 2, wherein the processing module comprises: Computational array; The step of adjusting the first data or the second data of each of the control bits according to a preset rule includes: Based on the data arrangement of the input data corresponding to the computing array, the insertion period corresponding to each control bit is determined; the data arrangement corresponds to the preset processing method; the data arrangement is such that the end position of the data is delayed row by row as the number of rows increases, and the end positions of the data in different rows remain aligned after the set number of rows is reached. Based on the data reading cycle, the first data in each of the control bits is controlled to move bit by bit in the direction from the start control bit to the end control bit. When the data reading cycle reaches the insertion timing based on the insertion cycle, the first data is inserted into the start control bit of each of the control bits.

4. The data processing method according to claim 2, wherein the processing module comprises: Computational array; The step of adjusting the first data or the second data of each of the control bits according to a preset rule includes: Based on the data arrangement of the input data corresponding to the computing array, when the adjustment time is reached, each of the control bits is simultaneously adjusted from the first data to the second data; the data arrangement means that the starting positions of data in different rows are always aligned and the ending positions of data are always aligned.

5. The data processing method according to claim 2, wherein selecting and outputting the data to be read in the enabled storage queue through a plurality of partially overlapping selection windows based on the data reading cycle includes: Based on the data reading cycle, each of the several selection windows corresponds to several consecutive control bits in the target control data; There is partial overlap between the several consecutive control bits corresponding to each two adjacent selection windows; Select the target control bit corresponding to the first data from the plurality of consecutive control bits; Output the data to be read from the storage queue corresponding to the target control bit.

6. The data processing method according to claim 5, wherein the data queue comprises: Data in several depth directions; The step of outputting the data to be read in the storage queue corresponding to the target control bit includes: The data to be read is located within the storage queue corresponding to the target control bit using the position index of the storage queue corresponding to the target control bit; the position index is used to point to the position of the next data to be read in the storage queue. Output the data to be read and increment the position index.

7. A neural network accelerator, comprising: A storage module is used to cache the input data of the neural network; the storage module includes several storage queues organized in an array, and the input data includes several data queues arranged in an array, with each data queue having the same row attribute being stored in a corresponding storage queue with the corresponding column attribute. The control module is used to control the read enable status of each storage queue in the column to be read according to preset rules based on the data reading cycle. The preset rule matches the preset processing method of the input data; The reading module is used to select and output the data to be read in the storage queue that is in the enabled state through several partially overlapping selection windows based on the data reading cycle, and output it to the processing module. The processing module is used to process the input data according to the preset processing method. Each selection window corresponds to multiple storage queues. The step of selecting and outputting the data to be read in the enabled storage queue through several partially overlapping selection windows based on the data reading cycle includes: based on the data reading cycle, having multiple receiving ends of each selection window correspond to several consecutive storage queues in the column to be read. There is partial overlap between the several consecutive storage queues corresponding to each two adjacent selection windows; the first target storage queue that is in an enabled state is selected from the several consecutive storage queues; the data to be read in the target storage queue is output.

8. The neural network accelerator according to claim 7, wherein the control module comprises: A shift register is used to store initial control data and, based on the data reading cycle, controls the first data in each control bit of the initial control data to shift bit by bit; the initial control data includes several control bits, each control bit is used to control the read enable state of each storage queue in the corresponding column to be read, each control bit includes the first data or the second data, the first data corresponds to entering the enable state, and the second data corresponds to exiting the enable state. A period counter is used to generate a trigger signal according to the insertion period corresponding to each of the control bits, and send the trigger signal to the shift register; The shift register is also used to insert the first data into the start control bit of each of the control bits in response to the trigger signal.

9. An electronic device, comprising: The neural network accelerator as described in any one of claims 7-8.

Images