Data processing method and device
By employing a method in pulsating array data processing where each channel writes the same number of rows with non-contiguous row numbers, the efficiency problem of writing to storage units across multiple channels is solved, achieving efficient data writing and output.
Patent Information
- Application Number
- CN202511203165.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-26
- Publication Date
- 2025-12-19
AI Technical Summary
In the prior art, when a direct memory access module writes data to a static random access memory, the writing efficiency is reduced because multiple channels may write to the same memory cell at the same time. An arbitrator is needed to control the writing order, which increases the time delay.
When writing data to the static random access memory through multiple channels, each channel writes the same number of rows, but the row numbers are not consecutive. The row numbers stored in the same memory cell are also not consecutive. Multiple channels can simultaneously write multiple rows of data with consecutive row numbers, and the data is periodically output to the pulse array for processing.
It improves data writing efficiency, avoids write conflicts, reduces the number of arbitrator control steps, shortens write time, and enhances the data output efficiency of the storage unit to the pulsating array.
Smart Images
Figure CN121166599A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing, and in particular to a data processing method and apparatus. Background Technology
[0002] A systolic array is a computing architecture consisting of multiple identical processing units arranged in a matrix. When processing data using a systolic array, the data to be processed is written to the storage cells of static random access memory (SRAM) using direct memory access modules, and then the storage cells input the data into the systolic array for processing.
[0003] In related technologies, direct memory access modules typically write data to storage units through multiple channels. Due to the inherent latency in data reading and writing, multiple channels may simultaneously write data to a single storage unit during the writing process. Therefore, an arbitrator is required for each storage unit to control the sequential writing of data by multiple channels in such situations. However, the arbitrator's control takes time, thus reducing write efficiency. Summary of the Invention
[0004] Therefore, this application discloses the following technical solution:
[0005] The first aspect of this application provides a data processing method, including:
[0006] Multiple rows of data for inputting the pulsating array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module.
[0007] In this configuration, each channel writes the same number of rows, the row numbers of multiple rows of data written in the same channel are not consecutive, the row numbers of multiple rows of data stored in the same storage unit are not consecutive, and the multiple channels can simultaneously write multiple rows of data with consecutive row numbers to the multiple storage units.
[0008] The data stored in the multiple storage units is periodically output to the pulsating array, so that the pulsating array processes the input data.
[0009] Optionally, each of the channels is used to write multiple rows of data corresponding to the row number of the channel into two storage units corresponding to the channel;
[0010] The row number corresponding to a channel is related to the channel number, the total number of channels, and the number of rows in the pulse array;
[0011] Any two rows of data written to the same channel satisfy either the first condition or the second condition;
[0012] The first condition includes that the two rows of data are respectively stored in two storage units corresponding to the channel;
[0013] The second condition includes that the two rows of data are output to the pulsation array at different times.
[0014] Optionally, each of the storage units stores multiple rows of data;
[0015] The step of periodically outputting the data stored in the plurality of storage units to the pulsating array, so that the pulsating array processes the input data, includes:
[0016] In each clock cycle, at least one row of data with non-contiguous row numbers in at least one of the memory cells is output to the systolic array, so that the systolic array processes the input data.
[0017] Within the same clock cycle, the difference between the row numbers corresponding to the data output by any two storage units is an integer multiple of the number of processing cycles of the systolic array, where the number of processing cycles represents the number of clock cycles used by the systolic array to process the data.
[0018] Optionally, the step of writing multiple rows of data for inputting the pulsating array to multiple storage cells of the static random access memory through multiple channels of the direct memory access module includes:
[0019] Based on the target conditions, multiple rows of data for inputting the pulsating array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module.
[0020] In this context, any two rows of data whose corresponding row numbers satisfy the target condition are written to two different storage units, and the target condition includes the difference between the corresponding row numbers being an integer multiple of the number of processing cycles.
[0021] Optionally, the time when the Nth row of data is output from its storage unit to the systolic array is delayed by one clock cycle relative to the time when the (N-1)th row of data is output from its storage unit to the systolic array, where N is any positive integer.
[0022] Optionally, when performing convolution operations using the pulsating array, each row of the multiple rows of data, except for the last row, includes S data blocks required for one convolution operation, and the last row of data includes L data blocks required for one convolution operation.
[0023] S is the stride of the convolution operation, and L is the size of the convolution kernel.
[0024] Optionally, when S is less than L, the step of periodically outputting the data stored in the plurality of storage units to the pulsating array includes:
[0025] If the row identifier corresponding to a row of data is the first identifier, after all the data blocks of that row of data are output to the pulse array, the data blocks of the next row will continue to be output to the pulse array.
[0026] If the row identifier corresponding to a row of data is the second identifier, the output of data blocks to the pulsating array will stop after all data blocks of that row of data have been output to the pulsating array.
[0027] Optionally, when S equals 1, the step of periodically outputting the data stored in the plurality of storage units to the pulsating array includes:
[0028] Read multiple rows of data from the multiple storage units and store the read data into the cache area corresponding to each row;
[0029] After the systolic array finishes processing the data in the current buffer, data corresponding to the target row number is moved from other buffers into the current buffer, so that the systolic array processes the data corresponding to the moved-in target row number.
[0030] Wherein, the current cache area is any one of multiple cache areas, the other cache areas refer to the cache areas other than the current cache area, and the target line number refers to the line number corresponding to the current cache area.
[0031] A second aspect of this application provides a data processing apparatus, comprising:
[0032] The direct memory access module includes multiple channels;
[0033] Static random access memory (SRAM) consists of multiple memory cells;
[0034] A pulsating array composed of multiple processing units is used to acquire and process the data stored in the storage unit;
[0035] The multiple channels are used to write multiple rows of data to be processed by the multiple processing units of the pulsating array to the multiple storage units, with each row of data corresponding to one processing unit.
[0036] After the writing is complete, the data stored in the multiple storage units is periodically output to the pulsating array for processing;
[0037] Each of the channels writes the same number of rows, the row numbers of multiple rows of data written in the same channel are not consecutive, and the multiple channels can simultaneously write multiple rows of data with consecutive row numbers to the multiple storage units.
[0038] Optionally, each of the channels is used to write multiple rows of data corresponding to the row number of the channel into two storage units corresponding to the channel;
[0039] The row number corresponding to a channel is related to the channel number, the total number of channels, the number of rows in the pulsating array, and the number of storage units corresponding to the channel.
[0040] Any two rows of data written to the same channel satisfy either the first condition or the second condition;
[0041] The first condition is that the two rows of data are stored in two storage units corresponding to the channel, respectively;
[0042] The second condition is that the two rows of data are output to the pulsation array at different times. Attached Figure Description
[0043] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0044] Figure 1 This is a flowchart of a data processing method provided in an embodiment of this application;
[0045] Figure 2 This is a schematic diagram of a pulsating array provided in an embodiment of this application;
[0046] Figure 3 This is a schematic diagram illustrating a data storage method provided in an embodiment of this application;
[0047] Figure 4 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Detailed Implementation
[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0049] This embodiment provides a data processing method; please refer to [link / reference]. Figure 1 The method may include the following steps.
[0050] S101, multiple rows of data for inputting the pulsating array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module; each channel writes the same number of rows, the row numbers of the multiple rows of data written in the same channel are not consecutive, the row numbers of the multiple rows of data stored in the same storage cell are not consecutive, and multiple channels can simultaneously write multiple rows of data with consecutive row numbers to multiple storage cells.
[0051] The data in this embodiment can be any data that needs to be processed by a systolic array. Generally, systolic arrays can be used to process matrix multiplication operations, convolution operations, etc. The aforementioned multiple rows of data can be matrix data used for matrix multiplication operations or convolution data used for convolution operations.
[0052] The data processing method of this embodiment can be applied to various scenarios that require the use of systolic arrays. As an example, this data processing method can be applied to various neural network models and deep learning models. When applied to these scenarios, the data that needs to be processed by systolic arrays includes, but is not limited to, feature data such as text, speech, and images.
[0053] The pulsating array consists of multiple computing units arranged and connected in a matrix. For example, the pulsating array used in this embodiment may have the following characteristics: Figure 2 The structure shown.
[0054] Direct memory access modules can write multiple rows of data from memory or storage to multiple storage units in static random access memory. The computational units of a systolic array can be configured to... Figure 2 The data is divided into multiple rows as shown. Each row of computing units is used to process a specific part of the data. Correspondingly, in memory or storage, the data can be assigned a corresponding row number according to the row to which the computing unit in the systolic array belongs, thereby dividing the data that needs to be input into the systolic array into multiple rows.
[0055] A storage unit refers to an independent partition (also called a bank) in static random access memory (SRAM). Each bank contains a complete data storage structure, address decoder, and read / write circuitry, and can therefore be considered a fully functional storage unit in SRAM. From the perspective of read / write logic, different storage units can perform read / write operations simultaneously without interference. For example, when the central processing unit (CPU) accesses storage unit 1, storage unit 2 can simultaneously transmit data to the pulse array.
[0056] The number of memory cells contained in a static random access memory (SRAM) is determined by its inherent structure. Generally, an SRAM can contain 2, 4, or 8 memory cells. In this embodiment, an example containing 8 memory cells will be used for illustration.
[0057] A systolic array can be a computing unit integrated into a specific processor. For example, a systolic array can be a computing unit integrated into an embedded neural network processor (NPU) for processing neural network-related operations.
[0058] Static random access memory (SRAM) can be integrated with the systolic array in the same processor; for example, SRAM can be memory integrated inside the NPU.
[0059] A processor integrating a systolic array and static random access memory can directly read data from the system memory of its electronic device via a direct memory access module for processing according to the method of this embodiment, such as directly reading data from double data rate synchronous dynamic random access memory (DDR SDRAM) for processing.
[0060] Combination Figure 2 For example, the row number corresponding to the data processed by the 0th row computing unit of the systolic array is denoted as TR0, meaning that this part of the data belongs to the 0th row of data; the row number corresponding to the data processed by the 1st row computing unit of the systolic array is denoted as TR1, meaning that this part of the data belongs to the 1st row of data, and so on.
[0061] The direct memory access module can write data to multiple storage units simultaneously through multiple channels. To improve writing efficiency, each channel can be configured to write the same number of rows, allowing multiple channels to start writing data to the storage unit at the same time and complete the writing process simultaneously.
[0062] As an example, the direct memory access module can have 4 channels, and the pulsating array can include 32 rows of computing units. The corresponding data to be written can be divided into 32 rows, which are represented by TR0 to TR31 respectively. Then each channel can be responsible for writing 8 rows of data into the storage unit.
[0063] In this embodiment, the specific rows of data to be written to the storage unit for each channel can be specified by the configuration information corresponding to that channel. The configuration information can be pre-written into the configuration queue corresponding to the channel. After the direct memory access module is started, each channel can read the configuration information from the configuration queue and write the corresponding rows of data to the storage unit based on the configuration information until all the configuration information in the configuration queue is read out and the storage unit is full.
[0064] The configuration queue for each channel can be a first-in-first-out (FIFO) queue.
[0065] The row numbers of multiple rows of data written to the same channel are not consecutive, and the row numbers of multiple rows of data stored in the same storage unit are not consecutive.
[0066] In other words, for any given channel, the difference between the row numbers of any two rows of data written by that channel is greater than 1. For example, if channel 1 is responsible for writing the row of data corresponding to TR1 to the storage unit, then the data corresponding to TR0 and TR2 will be written by other channels, not by channel 1.
[0067] Furthermore, in the multiple rows of data stored in the same storage unit, the difference between the row numbers of any two rows of data is greater than 1. For example, if storage unit 1 stores the row of data corresponding to TR1, then the data corresponding to TR0 and TR2 are stored in other storage units, not in storage unit 1.
[0068] Furthermore, multiple channels can simultaneously write multiple rows of data with consecutive row numbers to multiple storage units. In other words, multiple rows of data with consecutive row numbers can be configured to be written to multiple channels. For example, assuming the direct memory access module has four channels, when configuring the data to be written by each channel, the four rows of data corresponding to TR0 to TR3 can be written by the four channels respectively, with each channel responsible for writing one row, so that the four channels can simultaneously write the four rows of data corresponding to TR0 to TR3 to the storage unit; the four rows of data corresponding to TR4 to TR7 can be written by the four channels respectively, with each channel responsible for writing one row, so that the four channels can simultaneously write the four rows of data corresponding to TR4 to TR7 to the storage unit, and so on.
[0069] S102 periodically outputs the data stored in multiple storage units to the pulsating array, enabling the pulsating array to process the input data.
[0070] After all the storage cells in the static random access memory used to store data are full, the direct memory access module can pause writing data and write new configuration information to the configuration queue. At the same time, the storage cells can output the written data to the systolic array for processing. After all the data in the storage cells has been output to the systolic array, the direct memory access module can continue to write data to the storage cells in the manner of step S101. This cycle continues until all the data that needs to be processed by the systolic array has been processed.
[0071] The beneficial effects of this embodiment are as follows:
[0072] On the one hand, since each channel is responsible for writing the same number of rows, this solution can minimize the time required to write data to the storage unit. For example, when writing 32 rows of data using 4 channels, each channel is responsible for writing 8 rows of data. In this way, the 4 channels only need to spend the time of writing 8 rows of data to write 32 rows of data in parallel.
[0073] On the other hand, since the row numbers of the data written by each channel are not consecutive when writing data, and the row numbers stored in each storage unit are also not consecutive, there will be no write conflict problem when multiple channels write data to a storage unit at the same time. Therefore, when writing data to the storage unit based on this solution, it is not necessary to configure an arbitrator for each storage unit to resolve the write conflict problem. This eliminates the step of the arbitrator controlling the write order during writing, effectively shortening the write time and improving the write efficiency.
[0074] Furthermore, when the storage unit outputs data to the pulsating array, it will output multiple consecutive rows of data in sequence according to the row number. In this embodiment, multiple channels can simultaneously write multiple consecutive rows of data to multiple storage units, which can write the multiple consecutive rows of data to be output by the storage unit to the storage unit at the same time, meet the needs of the storage unit to output data to the pulsating array, and improve the writing efficiency.
[0075] Optionally, each channel is used to write multiple rows of data corresponding to the row number of the channel into two storage units corresponding to the channel;
[0076] The row number corresponding to a channel is related to the channel number, the total number of channels, and the number of rows in the pulsation array;
[0077] Any two rows of data written to the same channel satisfy either the first condition or the second condition;
[0078] The first condition includes: the two rows of data are stored in two storage units corresponding to the channel;
[0079] The second condition includes: the two rows of data are output to the pulsating array at different times.
[0080] To ensure that the first or second condition is met, the row number of the data to be written by each channel can be determined according to the following rule: Channelid = (i / b + i%a)%a. Here, i is the row number of the data. That is, during the writing process, the i-th row of data, i.e., the data corresponding to TRi, should be written by the channel whose Channelid conforms to the above rule.
[0081] Where i / b is not an integer, it can be rounded down; a is the total number of channels in the direct memory access module. For example, if the direct memory access module has 4 channels, then a equals 4. b is the multiple of the number of rows of the systolic array relative to a. For example, if a equals 4 and the systolic array includes 32 rows of computational units, then b equals 8.
[0082] Taking a = 4 and b = 8 as an example, Channelid = (i / 8 + i%4)%4 means that for the i-th row of data, the ratio obtained by dividing i by 8 is rounded down and recorded as the first parameter. The remainder obtained by dividing i by 4 is recorded as the second parameter. The sum of the first parameter and the second parameter is divided by 4. The remainder of the sum is determined as the Channelid number corresponding to the i-th row of data. That is, the channel with this Channelid is responsible for writing the i-th row of data into the static random access memory. i is any integer from 0 to 31.
[0083] Based on this pattern, with 8 storage units, 4 channels, and a systolic array containing 32 rows of computing units, if i equals 0, 4, 11, 15, 18, 22, 25, or 29, according to the aforementioned pattern, the Channelid corresponding to these i is 0. Therefore, channel 0 corresponds to the 8 rows of data TR0, TR4, TR11, TR15, TR18, TR22, TR25, and TR29, meaning channel 0 is responsible for writing these 8 rows of data into the storage unit. Similarly, channel 1 corresponds to TR1, TR5, TR8, TR12, TR19, TR23, TR26, and TR30; channel 2 corresponds to TR2, TR6, TR9, TR13, TR16, TR20, TR27, and TR31; and channel 3 corresponds to TR3, TR7, TR10, TR14, TR17, TR21, TR24, and TR28.
[0084] The above rule for determining the row number of data corresponding to each channel is only one example. In other embodiments, the row number of data corresponding to each channel can also be determined by other calculation methods different from the above rule. We will not exhaustively list them here. As long as we can ensure that the number of rows written in each channel is the same, the row numbers of multiple rows of data written in the same channel are not consecutive, and multiple channels can write multiple rows of data with consecutive row numbers to multiple storage units at the same time, thereby achieving the effect of shortening the writing time and improving the writing efficiency in the aforementioned embodiments.
[0085] Optionally, multiple rows of data for inputting the systolic array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module, including:
[0086] Based on the target conditions, multiple rows of data for inputting the pulsating array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module.
[0087] In this context, any two rows of data whose corresponding row numbers satisfy the target condition are written to two different storage units. The target condition includes the difference between the corresponding row numbers being an integer multiple of the number of processing cycles.
[0088] The number of processing cycles represents the number of clock cycles used by the systolic array to process data, or in other words, the number of clock cycles used by one computing unit in the systolic array to process data. In some examples, the data processing process of each computing unit in the systolic array may include four multiply-accumulate (MAC) operations, each of which requires one clock cycle, so the number of processing cycles can be four. Each computing unit supports various data formats; for example, it can support operations on int8 and int16 data.
[0089] To implement the above writing method, before writing data to the storage unit, the row number corresponding to each channel can be assigned to the two storage units corresponding to that channel according to the target conditions, and the assignment result, that is, the correspondence between the row number and the storage unit, is recorded in the configuration information corresponding to the channel. After starting to write data to the storage unit, each channel can write multiple rows of data to the storage unit according to the correspondence recorded in the configuration information based on the target conditions.
[0090] For any given channel, the storage unit where data needs to be written during data writing is called the corresponding storage unit for that channel. For example, if channel 0 needs to write data to storage units 0 and 4, then storage units 0 and 4 are the two storage units corresponding to channel 0.
[0091] The following rules can be used to determine which storage units each channel is responsible for writing data to: Bankid = Channelid, or Bankid = (Channelid + a) % b. % represents modulo operation, which is calculating the remainder when the number on the left side of % is divided by b, where a is the total number of channels in the direct memory access module, and b is the total number of channels in the static random access memory.
[0092] Here, Bankid is the storage unit number. For example, with 8 storage units, the Bankids for the 8 storage units can be 0 to 7 respectively. Channelid is the channel number in the direct memory access module. For example, with 4 channels, the Channelids for the 4 channels can be 0 to 4 respectively. For example, (Channelid+4)%8 represents the remainder obtained by dividing Channelid+4 by 8. The storage unit number is used to uniquely identify each storage unit, and the storage unit number is pre-set in the static random access memory.
[0093] The above pattern indicates that for any Channelid, there are two storage units with Bankid that conform to the above pattern, and the channel is responsible for writing data to the two storage units with Bankid that conform to the above pattern.
[0094] There are no restrictions on how to determine the correspondence between channels and storage units. The above rule is only an example, as long as the following condition is met:
[0095] The Channelid of a channel is equal to the number of the corresponding storage unit, or the difference between the two is an integer multiple of the total number of channels.
[0096] Determining the correspondence between channels and storage units based on this condition ensures that the number of storage units allocated to each channel is equal, which can improve write efficiency and avoid uneven write loads between different channels.
[0097] Based on the above example, with 8 storage units, 4 channels, and a pulsating array containing 32 rows of computing units, the channel with Channelid 0 (denoted as channel 0, the same below) corresponds to the storage unit with Bankid 0 (denoted as storage unit 0, the same below) and the storage unit 4 with Bankid 4. Channel 1 corresponds to storage unit 1 and storage unit 5, channel 2 corresponds to storage unit 2 and storage unit 6, and channel 3 corresponds to storage unit 3 and storage unit 7.
[0098] Based on the above correspondence between channels and storage units, when the number of processing cycles is equal to 4, and combined with the row number corresponding to each channel in the previous example, the correspondence between storage units and row numbers as shown in Table 1 can be determined based on the target conditions.
[0099] Table 1
[0100] Storage unit 0 Storage unit 1 Storage unit 2 Storage unit 3 Storage unit 4 Storage unit 5 Storage unit 6 Storage unit 7 TR0 TR1 TR2 TR3 TR4 TR5 TR6 TR7 TR15 TR8 TR9 TR10 TR11 TR12 TR13 TR14 TR22 TR23 TR16 TR17 TR18 TR19 TR20 TR21 TR29 TR30 TR31 TR24 TR25 TR26 TR27 TR28
[0101] In Table 1, each column corresponds to a storage unit. The row number in each column indicates the row number of the data stored in that storage unit. For example, storage unit 0 is used to store data for rows TR0, TR15, TR22, and TR29, and storage unit 4 is used to store data for rows TR4, TR11, TR18, and TR29.
[0102] Based on the row number corresponding to channel 0 in the previous example, it can be seen that when channel 0 writes data to storage unit 0 and storage unit 4, the data of row TR0 is written to storage unit 0, and the data of row TR4, whose row number differs from TR0 by 4, is written to another storage unit, namely storage unit 4.
[0103] Since the difference in row number between the data written by channel 0 to the two storage units is consistent with the total number of channels, multiple channels can write data to the storage units simultaneously without causing write conflicts.
[0104] Write the data of row TR15 to storage unit 0, and store the data of row TR11, which has a row number difference of 4 from TR15, in storage unit 4.
[0105] Write the data of row TR22 to storage unit 0, and store the data of row TR22, whose row number differs from TR22 by 4, in storage unit 4.
[0106] Write the data of row TR29 to storage unit 0, and store the data of row TR29, whose row number difference from TR29 is 4, in storage unit 4.
[0107] The purpose of writing data to the storage unit in the above manner is:
[0108] On the one hand, when the data of the storage unit is output to the pulse array for processing, a storage unit can only output one unit of data in one clock cycle. However, a row of data stored in the storage unit can contain multiple units of data. Therefore, after the storage unit outputs one unit of data in a row to the pulse array, it needs to output the next unit of data in the same row again after an integer multiple of M clock cycles, where M is the number of processing cycles and the unit of data can be, for example, 256 bits or other values.
[0109] On the other hand, in some applications of systolic arrays, each row of data is output one clock cycle later than the previous row. That is, the time it takes for the Nth row of data to be output from its storage unit to the systolic array is one clock cycle later than the time it takes for the (N-1)th row of data to be output from its storage unit to the systolic array, where N is any positive integer. For example, data from row TR0 is input first, then data from TR1 is input one clock cycle later, then data from TR2 is input one clock cycle later, and so on.
[0110] When a systolic array processes data, due to the characteristics mentioned above, if the difference between the row numbers of two rows of data stored in a memory cell is equal to an integer multiple of the number of processing cycles, it may be necessary to output both rows of data from that memory cell to the systolic array simultaneously in a certain clock cycle. This will cause output conflicts and affect the efficiency of the memory cell outputting data to the systolic array.
[0111] For example, suppose storage cell 0 stores data from both row TR0 and row TR4. In clock cycle 0, T0, the first unit of data from TR0 is output to the systolic array. After 4 clock cycles, when clock cycle T4 arrives, the second unit of data from TR0 needs to be output to the systolic array. At the same time, the first unit of data from TR4 also needs to be output to the systolic array in T4. That is, in cycle T4, storage cell 0 needs to output a unit of data from TR0 and a unit of data from TR4, resulting in an output conflict.
[0112] Conversely, when data is written to the storage unit in the manner described above, as can be seen from Table 1, the difference between the row numbers of any two rows of data stored in the same storage unit is not equal to an integer multiple of the number of processing cycles. This avoids the output conflict of needing to output two rows of data from one storage unit within the same clock cycle, thereby improving the efficiency of output data.
[0113] The correspondence between storage units and row numbers shown in Table 1 above is only one example. In other embodiments, the row number of each storage unit can also be allocated in other ways. As long as the condition that the difference between the row numbers of any two rows of data stored in the same storage unit is not equal to an integer multiple of the number of processing cycles is met, the effect of avoiding output conflicts and improving output efficiency can be achieved.
[0114] In another example, assume there are 3 processing cycles, 3 channels (channel 0 to channel 2), 6 storage units (storage units 0 to 5), and 18 rows of data (TR0 to TR17) to be stored in the storage units.
[0115] In conjunction with the methods of the foregoing embodiments, channel 0 can be used to write data of TR0, TR3, TR8, TR11, TR14, and TR17 into storage unit 0 and storage unit 3, respectively; channel 1 can be used to write data of TR1, TR4, TR6, TR9, TR14, and TR17 into storage unit 1 and storage unit 4, respectively; and channel 2 can be used to write data of TR2, TR5, TR7, TR10, TR12, and TR15 into storage unit 2 and storage unit 5, respectively.
[0116] Among them, storage unit 0 is used to store data of TR0, TR11 and TR16, storage unit 1 is used to store data of TR1, TR6 and TR17, storage unit 2 is used to store data of TR2, TR7 and TR12, storage unit 3 is used to store data of TR3, TR8 and TR13, storage unit 4 is used to store data of TR4, TR9 and TR14, and storage unit 5 is used to store data of TR5, TR10 and TR15.
[0117] Based on the above correspondence, the three channels can simultaneously write multiple rows of data to six storage units without writing conflicts, and the data from the six storage units can be continuously output to the pulse array for processing without output conflicts.
[0118] Optionally, based on the above method of writing data to the storage units, in step S102, the data stored in multiple storage units is periodically output to the pulsating array, so that the pulsating array processes the input data in the following way:
[0119] In each clock cycle, at least one row of data with non-contiguous row numbers from at least one memory cell is output to the systolic array, so that the systolic array can process the input data.
[0120] Within the same clock cycle, the difference between the row numbers corresponding to the data output by any two memory cells is an integer multiple of the number of processing cycles of the systolic array, where the number of processing cycles represents the number of clock cycles used by the systolic array to process data.
[0121] Each row of data is output one clock cycle later than the previous row. The method for outputting data to the systolic array according to the above embodiment can be:
[0122] After all the memory cells in the static random access memory (SRAM) are filled, the data in the TRi row can be output from the memory cell to the systolic array in the (i+Pn)-th clock cycle, i.e., T(i+Pn). One unit of data from that row is output each time until all the data from all the memory cells are output to the systolic array. Here, P is the number of the aforementioned processing cycles, n is an integer greater than or equal to 0, and the value of i ranges from 0 to the number of rows in the systolic array minus 1. The 0th clock cycle T(0) can be understood as the first clock cycle after all the memory cells are filled.
[0123] Referring to the example in Table 1, when the number of processing cycles is equal to 4 and the number of rows of the systolic array is 32, storage unit 0 first outputs a unit of data from row TR0 to the systolic array in clock cycle T(0). After that, every clock cycle T(4n), storage unit 0 outputs a unit of data from row TR0 to the systolic array. For row 15 data TR15, storage unit 0 will output a unit of data from TR15 to the systolic array for the first time in clock cycle T(15). After that, every clock cycle T(15+4n) will output a unit of data from row TR15. For row 22 data TR22, storage unit 0 will output a unit of data from TR22 to the systolic array for the first time in clock cycle T(22). After that, every clock cycle T(22+4n) will output a unit of data from row TR22. For the 29th row of data TR29, storage unit 0 will output a unit of data of TR29 to the systolic array for the first time in clock cycle T(29), and thereafter output a unit of data of row TR29 every T(29+4n) clock cycle. Starting from clock cycle T(28), storage unit 0 will output a unit of data to the systolic array every clock cycle, that is, T(28) outputs the data of TR0, T(29) outputs the data of TR29, T(30) outputs the data of TR22, T(31) reads the data of TR15, and then the cycle continues.
[0124] Similarly, storage unit 1 can output TR1 data in each T(1+4n) clock cycle, TR8 data in each T(8+4n) clock cycle, TR23 data in each T(23+4n) clock cycle, and TR30 data in each T(30+4n) clock cycle.
[0125] The same applies to storage unit 2 and storage unit 3, so I will not go into details.
[0126] Therefore, for any two rows of data whose row number difference is an integer multiple of 4, they can be simultaneously output from two different memory units to the systolic array for computation within the same clock cycle, effectively avoiding the problem of output conflict.
[0127] In some optional embodiments, the above processing method can be used to perform convolution operations, and the aforementioned data can be the data required for the convolution operation.
[0128] The data required for convolution operations can be stored in memory or storage in a configurable manner. Figure 3 The data is stored in the form of multiple data tiles, each containing the amount of data that can be stored in static random access memory at one time.
[0129] Convolution operations are generally performed on matrix data. The H and W directions can be regarded as the column and row directions of the matrix data. Since the size of a data slice in the H and W directions is smaller than the size of the column and row directions of the matrix data, the matrix data can be stacked along the C direction. Each element in the matrix data can be represented by 32 bits, 64 bits, or 128 bits of data in the C direction, which can be determined according to the data precision requirements.
[0130] For example, suppose a data slice has a size of 34 in both the H and W directions, and the matrix data contains 68 rows and 68 columns of elements, with both rows and columns numbered starting from 0, meaning it contains elements from rows 0 to 67 and columns 0 to 67. Then this matrix data... Figure 3 The storage space shown can be used to store data in the following manner:
[0131] The data storage space from 0 to 31 bits in direction C is used to store the elements located in rows 0 to 33 and columns 0 to 33 in the matrix data. Specifically, bits 0 to 31 of H0W0 correspond to the element in row 0 and column 0, and bits 0 to 31 of H0W1 correspond to the element in row 0 and column 1.
[0132] The data storage space of 32 to 63 bits in direction C is used to store the elements in the matrix data located in rows 0 to 33 and columns 34 to 67. The 32 to 63 bits of H0W0 correspond to the element in row 0 and column 34, and the 32 to 63 bits of H0W1 correspond to the element in row 0 and column 35.
[0133] The data storage space of 64 to 95 bits in the C direction is used to store the elements located in rows 34 to 687 and columns 0 to 33 in the matrix data. H0W0's 64 to 95 bits correspond to the element in row 34 and column 0, and H0W1 corresponds to the element in row 34 and column 1.
[0134] The data storage space in direction C, from 96 to 127 bits, is used to store the elements located in rows 34 to 67 and columns 34 to 67 of the matrix data. H0W0 corresponds to the element in row 34 and column 34, and H0W1 corresponds to the element in row 34 and column 35.
[0135] When performing convolution operations using a systolic array, the above data structure can be divided into multiple data blocks of size kh*tilec. One or more data blocks can then be allocated to multiple rows of computation units in the systolic array for convolution operations.
[0136] Where kh is the size of the convolution kernel used in the row direction. For example, when using a 3*3 convolution kernel for convolution, kh equals 3.
[0137] `tilec` can be determined based on the size of the convolution kernel, the size of the data slice in the W direction (`tilew`), and the number of bits occupied by each element in the convolution matrix in the C direction. `tilew` can be set as needed, with no specific value limit. For example, if the convolution kernel size `kw` in the row direction and `kh` in the column direction are both equal to 3, `tilew` is 4096, and each element occupies 32 bits in the C direction, then `tilec = floor(tilew / 32 / kw / kh) * 32`, which equals `tilec = 448`. `floor()` represents rounding down.
[0138] Optionally, after dividing the data into blocks as described above, each of the multiple rows of data stored in the storage unit, except for the last row, includes S data blocks required for a single convolution operation, and the last row of data includes L data blocks required for a single convolution operation.
[0139] S is the stride of the convolution operation, and L is the size of the convolution kernel.
[0140] Except for the last row, each row of data can contain S data blocks required by the computational unit of the corresponding row in the systolic array when performing convolution operations.
[0141] During convolution operations using a systolic array, the convolution kernel can slide along the W direction of the data slice with a stride of S. After sliding to the end of the W direction, it slides down once along the H direction, and then continues to slide along the W direction starting from W0. Each slide requires one convolution operation. The i-th row of computational units in the systolic array is used to perform the i-th convolution operation, so the data in row TRi can include the data required for performing TRi convolution operations.
[0142] For example, when S equals 1, kh equals 3, and L equals 3, the 0th row computation unit of the systolic array is used to perform convolution operations on the data in columns 0 to 2 (i.e., W0 to W2) of the data slice. Therefore, the data in row TR0 can include a data block of size kh*tilec in column 0, denoted as H0-H2W0C. The 1st row computation unit is used to perform convolution operations on the data in columns 1 to 3 (i.e., W1 to W3) of the data slice. Therefore, the data in row TR1 can include a data block of size kh*tilec in column 1. The data in row TR30 can include a data block of size kh*tilec in column 30, denoted as H0-H2W30C. The calculation unit in row 31, which is the last calculation unit, is used to perform convolution operation on the data in columns 31 to 33 (i.e., W31 to W33) in the data slice. Therefore, the data in row TR31 can include three data blocks of size kh*tilec in column 31, denoted as H0-H2W31C, H0-H2W32C, and H0-H2W33C.
[0143] When S equals 2, kh equals 3, and L equals 3, the 0th row computation unit of the systolic array is used to perform convolution operations on the data in columns 0 to 2 (i.e., W0 to W2) of the data slice. Therefore, the data in row TR0 can include two data blocks of size kh*tilec in columns 0 and 1, namely H0-H2W0C and H0-H2W1C. The 1st row computation unit is used to perform convolution operations on the data in columns 2 to 4 (i.e., W2 to W4) of the data slice. Therefore, the data in row TR1 can include two data blocks of size kh*tilec in columns 2 and 3. The data blocks are denoted as H0-H2W2C and H0-H2W3C. That is, the i-th row of calculation units is used to perform convolution operations on the data in columns 2i to 2i+2 of the data slice. Correspondingly, the 31st row of calculation units, which is the last row of calculation units, is used to perform convolution operations on the data in columns 62 to 64 (i.e., W62 to W64) of the data slice. Therefore, the data in row TR31 can include three data blocks of size kh*tilec in columns 62 to 64, denoted as H0-H2W62C, H0-H2W63C, and H0-H2W64C.
[0144] When S equals 3, kh equals 3, and L equals 3, the computational unit in the i-th row of the pulsating array is used to perform convolution operations on the data in columns 3i to 3i+2 of the data slice. Therefore, the data in row TRi can include three data blocks of size kh*tilec in columns 3i to 3i+2, namely H0-H2W3iC, H0-H2W3i+1C, and H0-H2W3i+2C, respectively. For example, the data in row TR0 can include three data blocks, namely H0-H2W0C, H0-H2W1C, and H0-H2W2C, respectively; the data in row TR1 can include three data blocks, namely H0-H2W3C, H0-H2W4C, and H0-H2W5C, respectively; and the data in row TR31 can include three data blocks, namely H0-H2W93C, H0-H2W94C, and H0-H2W95C, respectively.
[0145] Each row of data contains a data block that can be adjacent to the data block in the row direction of the previous row of data.
[0146] The advantages of storing data in the storage unit in the above manner are:
[0147] When S is less than L, each row of data stored in the storage unit only includes the data block required for the convolution operation of the corresponding row of computing units in the systolic array, thus saving storage space.
[0148] As can be seen from the storage method of the data blocks in the storage unit, when S is less than L, for the i-th row of the calculation unit, the data required for the convolution operation may be distributed in the TRi row and one or more rows after the TRi row.
[0149] For example, when S equals 1 and L equals 3, the data Wi to Wi+2 required for the i-th row of computation units to perform convolution operations are distributed in rows TRi, TRi+1, and TRi+2.
[0150] When S equals 2 and L equals 3, the data W2i to W2i+2 required for the convolution operation of the i-th row of computation units are distributed in rows TRi and TRi+1.
[0151] To address the above situation, a corresponding row identifier can be set for each row of data in the following way to reuse multiple rows of data in the storage unit:
[0152] If the row identifier corresponding to a row of data is the first identifier, after all the data blocks of that row of data are output to the systolic array, continue to output the data blocks of the next row to the systolic array;
[0153] If the row identifier corresponding to a row of data is the second identifier, the output of data blocks to the systolic array will stop after all data blocks of that row of data have been output to the systolic array.
[0154] The first identifier and the second identifier are different; for example, the first identifier is 0 and the second identifier is 1.
[0155] For each row of data, its row identifier is used to distinguish whether the row contains all the data blocks required for a single convolution operation. For example, when the convolution kernel size is equal to 3, a single convolution operation requires 3 data blocks from 3 consecutive columns. Therefore, if a row of data contains 3 data blocks from 3 consecutive columns, the row identifier for that row is the second identifier. If a row of data contains only 1 or 2 data blocks from one or two columns, the row identifier for that row is the first identifier.
[0156] In conjunction with the previous example, when S equals 1 or 2 and L equals 3, the row identifiers of rows TR0 to TR30 are all the first identifier, and the row identifier of row TR31 is the second identifier.
[0157] If S equals L, then the row identifier corresponding to each row of data can be the second identifier. Therefore, after the storage unit outputs all the data blocks of any row of data to the pulsating array, it can stop outputting the data of that row and not continue to output the data blocks of the next row.
[0158] When the storage unit outputs data to the systolic array in the manner described above, it first outputs the data of the TRi row to the i-th row computing unit of the systolic array, so that the i-th row computing unit performs convolution operation.
[0159] After the data for the TRi row has been output, the storage unit determines the row identifier of the TRi row;
[0160] If the row identifier of row TRi is the second identifier, the storage unit determines that all the data required by the i-th row computing unit to perform the convolution operation has been output, and stops outputting data to the i-th row computing unit.
[0161] If the row identifier of row TRi is the first identifier, the storage unit determines that the data required by the i-th row computing unit for the convolution operation has not been completely output. At this time, the data of row TRi+1 continues to be output to the i-th row computing unit.
[0162] After all the data in row TRi+1 is output to the calculation unit of row i, if the row identifier of row TRi+1 is still the first identifier, the storage unit determines that the data required for the convolution operation of the calculation unit of row i+1 has not been output. At this time, the data of the next row, namely row TRi+2, is output to the calculation unit of row i.
[0163] During the output of data from TRi+1 and / or TRi+2 to the i-th row computation unit, the storage unit can determine the cumulative number of data blocks output to the i-th row computation unit. When the cumulative number of data blocks output to the i-th row computation unit is equal to the size of the convolution kernel, the storage unit can determine that all the data required for the i-th row computation unit to perform the convolution operation has been output and stop outputting data to the i-th row computation unit.
[0164] Following the previous example, when S equals 1, kh equals 3, and L equals 3, after all the data from TR0 is output to the 0th row computation unit of the systolic array, the row identifier of TR0 is determined to be the first identifier. Therefore, the data from TR1 is then output to the 0th row computation unit of the systolic array for convolution. After all the data from TR1 is output to the 0th row computation unit of the systolic array, the row identifier of TR1 is determined to be the first identifier. Therefore, the data from TR2 is then output to the 0th row computation unit of the systolic array for convolution. After all the data from TR2 is output to the 0th row computation unit of the systolic array, the cumulative number of data blocks output to the 0th row computation unit equals the size of the convolution kernel, 3. Therefore, it is determined that all the data required for the convolution operation of the 0th row computation unit has been output, and data output to the 0th row computation unit stops. The method of outputting data to the computation units of other rows is the same and will not be elaborated further.
[0165] When S equals 2, kh equals 3, and L equals 3, after all the data in TR0 is output to the 0th row computation unit of the systolic array, the row identifier of TR0 is determined to be the first identifier. Then, the data in TR1 is output to the 0th row computation unit of the systolic array for convolution operation. After the first data block in TR1, i.e., the H0-H2W2C data block, is output to the 0th row computation unit, the cumulative number of data blocks output to the 0th row computation unit equals the size of the convolution kernel, 3. Therefore, it is determined that all the data required for the convolution operation of the 0th row computation unit has been output, and data output to the 0th row computation unit stops. The method of outputting data to the computation units of other rows is the same and will not be described in detail.
[0166] The above output method can reuse multiple rows of data stored in the storage unit, thereby meeting the data requirements of each row of computing units in the pulsating array for convolution operations while saving storage space in the storage unit.
[0167] Optionally, when the storage unit outputs data to the systolic array, the data of the storage unit is first output to the buffer corresponding to each row of computing units in the systolic array, and then output from the buffer to the corresponding row of computing units.
[0168] Based on this, when S equals 1, during the periodic output of data stored in multiple storage units to the pulsating array, the data in each buffer can be reused in the following way to reduce the number of times data is read from the storage unit and reduce the power consumption of the storage unit:
[0169] Read multiple rows of data from multiple storage units and store the read data into the cache area corresponding to each row;
[0170] After the systolic array finishes processing the data in the current buffer, data corresponding to the target row number is moved from other buffers into the current buffer, so that the systolic array processes the data corresponding to the moved-in target row number.
[0171] Here, the current cache area refers to any one of the multiple cache areas, the other cache areas refer to the cache areas other than the current cache area, and the target line number refers to the line number corresponding to the current cache area.
[0172] Multiple buffers are set between the storage unit and the systolic array, each buffer being called a buffer. Each buffer corresponds one-to-one with a row of computation units in the systolic array. For example, if the systolic array has 32 rows of computation units, there are 32 buffers, denoted as buffer(0) to buffer(31), corresponding to the 0th to 31st rows of computation units in the systolic array. One buffer can store one unit of data.
[0173] When the storage unit outputs data to the i-th row of the systolic array, it first outputs a unit of data to buffer(i). After M clock cycles, buffer(i) is completely output to the i-th row of the systolic array. Then the storage unit outputs another unit of data to buffer(i), and so on, until all the data required for the i-th row of the systolic array is output.
[0174] If data is output according to the method of the previous embodiment, then when S equals 1 and L equals 3, the storage unit needs to output the data of TRi row, TRi+1 row and TRi+2 row to buffer(i) in sequence. The i-th row is then output by buffer(i) for calculation. Data needs to be output to buffer(i) once every M clock cycles. The number of outputs is large and the frequency is high, so the storage unit will generate high power consumption.
[0175] Therefore, the method of reusing the data in the buffer can be used to output data to the systolic array.
[0176] Specifically, in the above method of reusing data in the buffer, except for the last buffer, after all the data in each buffer is output to the corresponding row calculation unit, the data of the next buffer in the buffer can be copied to the buffer, and then the copied data can be output to the corresponding row calculation unit. This operation is repeated L-1 times, and then the data of the corresponding row is output from the buffer unit to the buffer.
[0177] The following example, using buffer(0), illustrates the method of reusing data in the buffer with L equal to 3 and M equal to 4.
[0178] In the first clock cycle T0, the storage unit first outputs one unit of data from the H0-H2W0C data block in row TR0 to buffer (0). Then, in clock cycle T1, the storage unit outputs one unit of data from the H0-H2W1C data block in row TR1 to buffer (1). In clock cycle T2, the storage unit outputs one unit of data from the H0-H2W2C data block in row TR2 to buffer (2).
[0179] After M=4 clock cycles starting from T0, that is, at clock cycle T4, all the data belonging to H0-H2W0C in buffer(0) has been output to the 0th row of the systolic array. At this time, the data belonging to H0-H2W1C in the next buffer area of buffer(0), that is, buffer(1), can be copied to buffer(0), so that the data of H0-H2W1C can be output from buffer(0) to the 0th row of the systolic array for convolution operation in the next 4 clock cycles.
[0180] After M=4 clock cycles starting from T1, that is, at clock cycle T5, all the data belonging to H0-H2W1C in buffer(1) has been output to the first row of the systolic array calculation unit. At this time, the data belonging to H0-H2W2C in buffer(2) can be copied to buffer(1), so that the data of H0-H2W2C can be output from buffer(1) to the first row of the calculation unit for convolution operation in the next 4 clock cycles.
[0181] After M=4 clock cycles starting from T4, that is, at clock cycle T8, all the data belonging to H0-H2W1C in buffer(0) has been output to the 0th row of the systolic array. At this time, the data belonging to H0-H2W2C in buffer(1) can be copied to buffer(0) again, so that the data of H0-H2W2C can be output from buffer(0) to the 0th row of the systolic array for convolution operation in the next 4 clock cycles.
[0182] After M=4 clock cycles starting from T8, that is, at clock cycle T12, all the data belonging to H0-H2W2C in buffer(0) has been output to the 0th row calculation unit of the pulsating array. Since the copy operation of buffer(0) has been performed twice (i.e., L-1 times), it is necessary to continue to output one unit of data of the H0-H2W0C data block in TR0 to buffer(0) from the storage unit. In the following 12 clock cycles, the above copy operation can be repeated to copy the data belonging to H0-H2W1C and the data belonging to H0-H2W2C from buffer(1) to buffer(0) in turn, and so on.
[0183] For the last buffer, every M clock cycles, one unit of data from the last row of data can be output to this buffer from the memory unit. Furthermore, each time the memory unit outputs the last row of data, L data blocks of the last row can be output cyclically; that is, each data block of the last row is output once in sequence, and after one cycle, each data block of the last row is output once again, and so on.
[0184] Combining the previous example, when L equals 3, the data in TR31 is the last data, which contains three data blocks: H0-H2W31C, H0-H2W32C, and H0-H2W33C. When the storage unit outputs the data in TR31 for the first time, it outputs one unit of data from H0-H2W31C to the corresponding buffer (31). When it outputs the data in TR31 for the second time, it outputs one unit of data from H0-H2W32C to the corresponding buffer (31). When it outputs the data in TR31 for the third time, it outputs one unit of data from H0-H2W33C to the corresponding buffer (31). Then, the data in H0-H2W31C, H0-H2W32C, and H0-H2W33C are output in sequence for the fourth, fifth, and sixth times.
[0185] If data is output to the systolic array in the manner described above using the data from the buffer, then after the storage unit outputs any row of data, it can disregard the row identifier of that row and directly stop providing data to the corresponding row's computation unit in the systolic array. Subsequent data is then provided to the systolic array by using the data from the buffer.
[0186] Furthermore, if data is output to the systolic array in the manner described above for reusing the data in the buffer, then in the storage unit, except for the last row of data, each row of data can be output once every L*M clock cycles, while the last row of data will still be output once every M clock cycles.
[0187] By reusing data from the buffer to output data to the systolic array, the number and frequency of reading data from the storage unit to the systolic array can be effectively reduced, thereby reducing the power consumption of the storage unit.
[0188] This embodiment also provides a data processing device; please refer to [link / reference]. Figure 4 :
[0189] Direct memory access module 401 includes multiple channels;
[0190] The static random access memory 402 includes multiple memory cells;
[0191] A pulsating array 403 composed of multiple processing units is used to acquire and process data stored in the storage unit;
[0192] Multiple channels are used to write multiple rows of data to be processed by the multi-row processing unit of the pulsating array to multiple storage units, with each row of data corresponding to one row of processing unit;
[0193] After the writing is complete, the data stored in multiple storage units is periodically output to the pulse array for processing;
[0194] Each channel writes the same number of rows, the row numbers of multiple rows of data written in the same channel are not consecutive, and multiple channels can simultaneously write multiple rows of data with consecutive row numbers to multiple storage units.
[0195] Optionally, each channel is used to write multiple rows of data corresponding to the row number of the channel into two storage units corresponding to the channel;
[0196] The row number corresponding to a channel is related to the channel number, the total number of channels, the number of rows in the systolic array, and the number of storage cells corresponding to the channel;
[0197] Any two rows of data written to the same channel satisfy either the first condition or the second condition;
[0198] The first condition is that the two rows of data are stored in two storage units corresponding to the channel;
[0199] The second condition is that the two rows of data are output to the pulsating array at different times.
[0200] In some alternative embodiments, the data processing apparatus may further include multiple buffers for caching data output from the storage cells of the static random access memory 402 and transmitting the cached data to the pulsating array 403.
[0201] When the device is used to process convolution operations, and the stride S of the convolution operation is equal to 1, the static random access memory 402 periodically outputs the data stored in multiple memory cells to the pulsating array, including:
[0202] Read multiple rows of data from multiple storage units and store the read data into the cache area corresponding to each row;
[0203] After the systolic array finishes processing the data in the current buffer, data corresponding to the target row number is moved from other buffers into the current buffer, so that the systolic array processes the data corresponding to the moved-in target row number.
[0204] Here, the current cache area refers to any one of the multiple cache areas, the other cache areas refer to the cache areas other than the current cache area, and the target line number refers to the line number corresponding to the current cache area.
[0205] The working principle of the above data processing device can be found in the relevant steps of the data processing method in the foregoing embodiments, and will not be repeated here.
[0206] The data processing device in this embodiment may be an electronic device that integrates a processor for implementing specific data processing functions. For example, the processor may be an embedded neural network processor (NPU) for running neural network models or deep learning models.
[0207] The data processing device can be a standalone device, such as a standalone NPU device. In this case, the data processing device can be connected to any electronic device, such as a desktop computer via a data cable. Thus, the data processing device can write the data to be processed from the connected electronic device into a static random access memory (SRAM) based on the data processing method of the above embodiments, and then output the data from the SRAM to the pulse array for processing.
[0208] The data processing device can also be a processing system integrated into an electronic device, such as an NPU processor module integrated into a system-on-chip (SoC), or an NPU chip integrated into an electronic device but located outside the SoC chip. Thus, the NPU can load the data to be processed from the system memory of the electronic device into a static random access memory based on the data processing method of the above embodiments, and then output the data from the static random access memory to the pulse array for processing.
[0209] On the one hand, when the data processing device writes data from system memory to static random access memory, it can write multiple lines of data at the same time and avoid write conflicts. Therefore, the data that needs to be processed in the system memory of the electronic device can be written to the static random access memory of the data processing device more quickly, thereby improving the data transfer rate between system memory and static random access memory.
[0210] On the other hand, when the data output from the static random access memory is processed by the systolic array, output conflicts can be avoided, and multiple lines of data can be continuously output to the systolic array according to the number of processing cycles of the systolic array, which improves the efficiency of the data output from the static random access memory. Thus, without changing the inherent processing efficiency of the systolic array, the data processing results can be obtained faster.
[0211] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0212] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.
[0213] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0214] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0215] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A data processing method, comprising: Multiple rows of data for inputting the pulsating array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module. In this configuration, each channel writes the same number of rows, the row numbers of multiple rows of data written in the same channel are not consecutive, the row numbers of multiple rows of data stored in the same storage unit are not consecutive, and the multiple channels can simultaneously write multiple rows of data with consecutive row numbers to the multiple storage units. The data stored in the multiple storage units is periodically output to the pulsating array, so that the pulsating array processes the input data.
2. The method according to claim 1, wherein each channel is used to write multiple rows of data corresponding to the row number of the channel into two storage units corresponding to the channel; The row number corresponding to a channel is related to the channel number, the total number of channels, and the number of rows in the pulse array; Any two rows of data written to the same channel satisfy either the first condition or the second condition; The first condition includes that the two rows of data are respectively stored in two storage units corresponding to the channel; The second condition includes that the two rows of data are output to the pulsation array at different times.
3. The method according to claim 1, wherein each of the storage units stores multiple rows of data; The step of periodically outputting the data stored in the plurality of storage units to the pulsating array, so that the pulsating array processes the input data, includes: In each clock cycle, at least one row of data with non-contiguous row numbers in at least one of the memory cells is output to the systolic array, so that the systolic array processes the input data. Within the same clock cycle, the difference between the row numbers corresponding to the data output by any two storage units is an integer multiple of the number of processing cycles of the systolic array, where the number of processing cycles represents the number of clock cycles used by the systolic array to process the data.
4. The method according to claim 3, wherein writing multiple rows of data for inputting the pulsating array to multiple storage cells of the static random access memory through multiple channels of the direct memory access module comprises: Based on the target conditions, multiple rows of data for inputting the pulsating array are written to multiple storage cells of the static random access memory through multiple channels of the direct memory access module. In this context, any two rows of data whose corresponding row numbers satisfy the target condition are written to two different storage units, and the target condition includes the difference between the corresponding row numbers being an integer multiple of the number of processing cycles.
5. The method according to claim 3, wherein, The time it takes for the Nth row of data to be output from its storage unit to the systolic array is delayed by one clock cycle relative to the time it takes for the (N-1)th row of data to be output from its storage unit to the systolic array, where N is any positive integer.
6. The method according to claim 1, wherein when performing convolution operation using the pulsating array, each row of the multiple rows of data, except for the last row, includes S data blocks required for one convolution operation, and the last row of data includes L data blocks required for one convolution operation; S is the stride of the convolution operation, and L is the size of the convolution kernel.
7. The method according to claim 6, wherein when S is less than L, the step of periodically outputting the data stored in the plurality of storage units to the pulsating array comprises: If the row identifier corresponding to a row of data is the first identifier, after all the data blocks of that row of data are output to the pulse array, the data blocks of the next row will continue to be output to the pulse array. If the row identifier corresponding to a row of data is the second identifier, the output of data blocks to the pulsating array will stop after all data blocks of that row of data have been output to the pulsating array.
8. The method according to claim 6, wherein when S equals 1, the step of periodically outputting the data stored in the plurality of storage units to the pulsating array comprises: Read multiple rows of data from the multiple storage units and store the read data into the cache area corresponding to each row; After the systolic array finishes processing the data in the current buffer, data corresponding to the target row number is moved from other buffers into the current buffer, so that the systolic array processes the data corresponding to the moved-in target row number. Wherein, the current cache area is any one of multiple cache areas, the other cache areas refer to the cache areas other than the current cache area, and the target line number refers to the line number corresponding to the current cache area.
9. A data processing apparatus, comprising: The direct memory access module includes multiple channels; Static random access memory (SRAM) consists of multiple memory cells; A pulsating array composed of multiple processing units is used to acquire and process the data stored in the storage unit; The multiple channels are used to write multiple rows of data to be processed by the multiple processing units of the pulsating array to the multiple storage units, with each row of data corresponding to one processing unit. After the writing is complete, the data stored in the multiple storage units is periodically output to the pulsating array for processing; Each of the channels writes the same number of rows, the row numbers of multiple rows of data written in the same channel are not consecutive, and the multiple channels can simultaneously write multiple rows of data with consecutive row numbers to the multiple storage units.
10. The apparatus according to claim 9, wherein each channel is configured to write multiple rows of data corresponding to the row number of the channel into two storage units corresponding to the channel; The row number corresponding to a channel is related to the channel number, the total number of channels, the number of rows in the pulsating array, and the number of storage units corresponding to the channel. Any two rows of data written to the same channel satisfy either the first condition or the second condition; The first condition is that the two rows of data are stored in two storage units corresponding to the channel, respectively; The second condition is that the two rows of data are output to the pulsation array at different times.