Scalable parallel convolution data output device and output method thereof

Abstract

The application relates to the field of FPGA and ASIC related to artificial intelligence, and discloses an extensible parallel convolution data output device and an output method thereof, which can improve the data throughput rate of convolution operation. The device comprises: a data storage configured to store data subblocks required to participate in convolution; a data reading module configured to send a data reading instruction to the data storage, read a data block with a size of K*BW from the data subblock, and output the data block to a line buffer module, wherein BW=K+DN-1, DN=WS / DS, K is a convolution kernel size, WS is a word length of each address of the data storage, and DS is a bit number of each data; the line buffer module comprises K rows of BW columns of caches and DN channels for output; and the line buffer module is configured to write data from the data reading module into the caches and output BW data of a row of the caches through the DN channels when the row of the caches is written.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence deep learning and the technical field of FPGA and ASIC, in particular to a product-level implementation of the design technology of a convolution operation array, which is a bottom component for FPGA hardware design and artificial intelligence ASIC design. BACKGROUND

[0002] This section is intended to provide background or context to the embodiments of the application recited in the claims. The description herein does not constitute admission of prior art.

[0003] In the implementation process of CNN neural network in the field of artificial intelligence, a large amount of data is operated. In the CNN neural network, the main demand for computing power is multiplication and accumulation operation. In order to improve the computing power, MAC ARRAY is used as a general design method. The scale and use efficiency of MAC ARRAY represent the upper limit of computing power, and the efficiency of data transmission to MAC ARRAY limits the effective computing power of MAC ARRAY, so how to quickly and efficiently transmit data to the MAC calculation unit becomes the design focus of the convolution calculation network.

[0004] For example, the Chinese patent application No. CN202210156894.8 “Data loading method and device for convolution operation” forms multiple texture subgraphs and cache modes by splitting the texture graph, and outputs the MAC calculation module to realize the acceleration process.

[0005] In the prior art, the design of receiving and block allocation of convolution data is relatively mature, but in the multi-path parallel sending of convolution data, due to the fixed delay of the internal SRAM (data RAM) itself and the combined effect of the write-back waiting of the MAC ARRAY, there are many window periods in the data sending process, and the data transmission efficiency is not sufficient. In addition, in order to improve the bus transmission efficiency, the RAM bit width of data storage is wide (generally more than 32 bits), and the bit width of output data is narrow (commonly used in CNN network is 8 bits), if each sending module only outputs one way of data, the readout efficiency of the cache will be reduced. SUMMARY

[0006] The purpose of the present application is to provide an extensible parallel convolution data output device and its output method, which can improve the data throughput rate of convolution operation.

[0007] The present application discloses an extensible parallel convolution data output device, comprising:

[0008] a data storage configured to store data subblocks required for convolution;

[0009] a data reading module configured to send data reading instructions to the data memory, read a data block of size K*BW from the data sub-block and output to a line buffer module, wherein BW=K+DN-1, DN=WS / DS, K is the size of the convolution kernel, WS is the word length of each address of the data memory, and DS is the number of bits of each data;

[0010] the line buffer module comprises a cache of K lines and BW columns and DN channels for output; the line buffer module is configured to write data from the data reading module to the cache, and when a line of the cache is written, output BW data of the line through the DN channels.

[0011] In a preferred embodiment, a virtual computing module is further included;

[0012] the virtual computing module is configured to maintain the virtual state of each line in the cache of the line buffer module, update the virtual state of each line in the cache according to the write line number, write column number and write length from the data reading module and feed back to the data reading module, calculate the read line number and read column number according to the virtual state of each line in the cache, and send the write line number, write column number, write length, read line number and read column number to the line buffer module;

[0013] the line buffer module is further configured to write data from the data memory to the corresponding line in the cache according to the write line number, write column number and write length from the virtual computing module, and output data in the corresponding line in the cache to the corresponding channel according to the read line number and write line number from the virtual computing module;

[0014] the data reading module is further configured to generate the data reading instructions, write line number, write column number and write length according to the virtual state of each line in the cache from the virtual computing module, and send the write line number, write column number and write length to the virtual computing module.

[0015] In a preferred embodiment, the virtual computing module is further configured to delay the write line number, write column number, write length, read line number and read column number sent to the line buffer module, so that the time when the write line number, write column number, write length, read line number and read column number arrive at the line buffer module matches the time when the corresponding data arrives at the line buffer module from the data memory.

[0016] In a preferred embodiment, the data storage and the row buffer module have N groups corresponding to the same data reading module and the same virtual computing module, one data reading module simultaneously outputs the same data reading instruction to N data storages, and one virtual computing module simultaneously outputs the same write row number, write column number, write length, read row number and read column number to N row buffer modules, wherein N is an integer greater than 1.

[0017] In a preferred embodiment, the row buffer module is further configured to output BW data of a row in the cache through the DN channels in K clock cycles after the row is written, wherein the jth channel outputs the i+jth data in the row in the ith clock cycle, 0≤i

[0018] In a preferred embodiment, the virtual state of each row in the cache includes a current write row number, a current write column number, a current output starting row number and a current output starting column number.

[0019] The application also discloses an output method of an extensible parallel convolution data output device, the device including a data storage and a row buffer module, the data storage storing data subblocks to be involved in convolution, the row buffer module including a cache of K rows and BW columns and DN channels for output, the method including:

[0020] sending a data reading instruction to the data storage to read a data block of K*BW from the data subblocks to the row buffer module, wherein BW=K+DN-1, DN=WS / DS, K is a convolution kernel size, WS is a word length of each address of the data storage, and DS is a bit number of each data;

[0021] the row buffer module writes data from the data reading module to the cache and outputs BW data of a row in the cache through the DN channels after the row is written.

[0022] In a preferred embodiment, the sending of the data reading instruction to the data storage further includes generating the data reading instruction, a write row number, a write column number and a write length according to the virtual state of each row in the cache;

[0023] The sending of the data reading instruction to the data storage further includes updating the virtual state of each row in the cache according to the write row number, the write column number and the write length corresponding to the data reading instruction, calculating a read row number and a read column number according to the virtual state of each row in the cache, and sending the write row number, the write column number, the write length, the read row number and the read column number to the row buffer module.

[0024] The line buffer module receives data from the data memory, and writes the data from the data memory into corresponding lines in the cache according to the write row number, write column number and write length, and outputs data in corresponding lines in the cache to corresponding channels according to the read row number and write row number.

[0025] In a preferred embodiment, the outputting of the BW data of the line through the DN channels further comprises:

[0026] The BW data of the line are outputted through the DN channels within K clock cycles, wherein the jth channel outputs the i+jth data in the line in the ith clock cycle, 0≤i

[0027] The application also discloses a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the steps in the method described above.

[0028] In the embodiment of the application, according to the difference between the data storage RAM bit width and the actual use data bit width, more data is read at one time, and a multi-channel parallel output is formed through a buffering mechanism, so that the data multiplexing ratio is improved, and the data throughput rate of the convolution operation is ultimately improved.

[0029] Further, the use of virtual computing reduces the transmission cycle of state feedback on the loop and reduces the data transmission window period, thereby improving the effective data bandwidth. The virtual computing only advances the calculation part of the buffering module, and does not increase the function, so as not to increase the resource consumption.

[0030] Each of the technical features disclosed in the above summary, each of the technical features disclosed in the following embodiments and examples, and each of the technical features disclosed in the drawings can be freely combined with each other to form various new technical solutions (these technical solutions should be regarded as having been described in the present specification), unless such combination of technical features is technically infeasible. For example, features A+B+C are disclosed in one example, features A+B+D+E are disclosed in another example, features C and D are equivalent technical means that play the same role, and only one of them can be used in technology, and feature E can be combined with feature C in technology. Therefore, the scheme of A+B+C+D should not be regarded as having been described because it is technically infeasible, and the scheme of A+B+C+E should be regarded as having been described. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a scalable parallel convolution data output device structure schematic diagram according to the first embodiment of the application;

[0032] Figure 2 is a schematic diagram of data storage state in data RAM according to an embodiment of the present application;

[0033] Figure 3 is a data multiplexing reading state in 3X3 convolution state according to an embodiment of the present application;

[0034] Figure 4 is a schematic diagram of row buffer reading process according to an embodiment of the present application;

[0035] Figure 5 is a schematic diagram of scalable parallel convolution data output method process according to a second embodiment of the present application. DETAILED DESCRIPTION

[0036] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without such specific details and that numerous implementation variations and modifications can be possible.

[0037] Explanation of some concepts:

[0038] FPGA: Field Programmable Gate Array.

[0039] ASIC: Application Specific Integrated Circuit.

[0040] MAC ARRAY: Multiply Accumulate Unit Array.

[0041] CNN: Convolutional Neural Network.

[0042] RAM: Random Access Memory.

[0043] Convolution Kernel: In image processing, given an input image, the weighted average of the pixels in a small region in the input image becomes each corresponding pixel in the output image, where the weight is defined by a function, which is called convolution kernel.

[0044] To make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0045] The first embodiment of the present application relates to a scalable parallel convolution data output device, the structure of which is shown in Figure 1 The device comprises:

[0046] Data storage, used for storing data sub-blocks that need to participate in convolution.

[0047] a data reading module configured to send data reading instructions to the data storage, read data blocks of size K*BW from the data sub-blocks and output to the row buffer module, wherein BW=K+DN-1, DN=WS / DS, K is the size of the convolution kernel, WS is the word length of each address of the data storage, and DS is the number of bits of each data. In one embodiment, the data reading instructions include address parameters of data reading.

[0048] a row buffer module including K rows of BW columns of cache and DN channels for output. The row buffer module is configured to write data from the data reading module to the cache, and when a row of the cache is written, output BW data of the row through the DN channels.

[0049] According to the characteristics of convolution calculation, the data multiplexing of reading is designed, the single module parallel output of multiple data is realized, the reading efficiency of the data cache RAM can be improved, and the data bandwidth of input and output tends to be balanced.

[0050] Optionally, in one embodiment, the device further includes a virtual calculation module.

[0051] The virtual calculation module is configured to maintain the virtual state of each row in the cache of the row buffer module, update the virtual state of each row in the cache according to the write row number, write column number and write length from the data reading module and feed back to the data reading module, calculate the read row number and read column number according to the virtual state of each row in the cache, and send the write row number, write column number, write length, read row number and read column number to the row buffer module.

[0052] The row buffer module is further configured to write data from the data storage into the corresponding row in the cache according to the write row number, write column number and write length from the virtual calculation module, and output data in the corresponding row in the cache to the corresponding channel according to the read row number and write row number from the virtual calculation module.

[0053] The data reading module is further configured to generate data reading instructions, write row number, write column number and write length according to the virtual state of each row in the cache from the virtual calculation module, and send the write row number, write column number and write length to the virtual calculation module.

[0054] Using virtual calculation, the state of the buffer row is calculated in advance before the data reaches the buffer module and is fed back to the data reading address module, which can reduce the number of cycles of the data reading module waiting for the state of the buffer module, so as to achieve the purpose of reducing the data sending empty window period.

[0055] Optionally, in one embodiment, the virtual computing module is further configured to delay the write row number, write column number, write length, read row number, and read column number sent to the row buffer module, such that the time when the write row number, write column number, write length, read row number, and read column number arrive at the row buffer module matches the time when the corresponding data arrives at the row buffer module from the data storage.

[0056] Optionally, in one embodiment, there are N groups of data storage and line buffer modules, each corresponding to the same data reading module and the same virtual computing module. That is... Figure 1 There is one data reading module and one virtual computing module. Figure 1 The dashed box contains multiple sets of data storage and row buffer modules, each set consisting of a coupled data storage unit and a row buffer module. The data read module simultaneously outputs the same data read instruction to N data storage units, and the virtual computation module simultaneously outputs the same write row number, write column number, write length, read row number, and read column number to N row buffer modules, where N is an integer greater than 1. Through the separate design of the data read module and buffer modules, one data read module can control multiple data storage units and buffer modules, resulting in lightweight buffer resources. The number of buffer modules can be expanded according to the different CNN networks to be implemented.

[0057] Optionally, in one embodiment, the row buffer module is further configured to output BW data points of that row through DN channels within K clock cycles after a row has been written, wherein the j-th channel outputs the (i+j)-th data point of that row in the i-th clock cycle, 0 ≤ i < K, 0 ≤ j < DN. Those skilled in the art will understand that, according to convention in the art, when it comes to sequence numbering, such as row numbers, clock cycle numbers, and output channel numbers, they are mostly numbered starting from 0. For example, the first channel is numbered 0, the second channel is numbered 1, and so on. Of course, numbering can also start from 1 or other specified values; the essence is the same.

[0058] Optionally, in one embodiment, the virtual state of each row in the cache includes the current write row number, the current write column number, the current output first point row number, and the current output first point column number.

[0059] To better understand the technical solution of this application, a specific example is provided below. The details listed in this example are mainly for ease of understanding and are not intended to limit the scope of protection of this application.

[0060] In this specific example, the scalable parallel convolutional data output device includes a data RAM, a line buffer module, a data reading module, and a virtual computing module.

[0061] Data RAM is a data storage realized by RAM. Convolution data is temporarily stored in the RAM array in a block manner, and one RAM block provides data to one buffer module.

[0062] Line buffer module: contains several lines of data buffer, and reads and writes the line buffer according to the current write line number, the current write column number, the current output starting line number, the current output starting column number from the virtual calculation module and the data from the RAM module.

[0063] Data reading module: generates address parameters, alignment parameters and effective length parameters of data reading according to the convolution configuration and the state parameters of the virtual calculation module, sends the address parameters to the data RAM, and sends the alignment parameters and the read effective length parameters to the virtual calculation module.

[0064] Virtual calculation module: calculates the subsequent state of the virtual line buffer according to the state of the virtual line buffer, the alignment parameters and the read effective length parameters, including the current write line number, the current write column number, the current output starting point line number, the current output starting point column number and other key information, and sends these information to the buffer module after the same delay as the RAM module.

[0065] Data RAM and line buffer module can be instantiated in multiple groups according to the size of MAC ARRAY.

[0066] The following is the detailed design of each module:

[0067] First, initialize the storage parameters.

[0068] The data storage state in the data RAM is as shown in Figure 2 .

[0069] In CNN convolution calculation, the data area of the whole layer has three dimensions, width, height and layer. In order to achieve parallel calculation, the data area is generally divided into blocks, and each sub-block after division is stored in a separate data RAM.

[0070] The width of each data RAM sub-block is WIDTH, the height is HEIGHT, and the number of layers is M. The address word length of the RAM is word_size, and the data width of each data is data_size. Then byte_num = word_size / data_size is stored in each address bit of the RAM.

[0071] According to the convolution kernel size Ksize, the row buffer window size is initialized, and BUFF_W = Ksize + byte_num - 1.

[0072] Second, the design of the data reading module is as follows.

[0073] The conventional data reading module reads Ksize*Ksize size data at the same position of each layer of the data block and sends the data to the MAC for calculation, but for the convolution behavior of adjacent data, a request is sent to the data RAM every time, and the data reuse is not fully achieved. In order to improve the frequency of data reuse, the conventional mode of the data reading module needs to be modified.

[0074] The main function of the data reading module is to generate the RAM reading address req_addr of each cycle, which sequentially reads the sub-data block in the RAM according to the size of the Ksize*BUFF_W plane at each layer, and then the process is repeated for the next layer until the same sub-plane (Ksize*BUFF_W) of all layers is completed. The starting point byte_num is right shifted by one position (if the first data position exceeds the current row range after right shifting, the first data is moved to the first position of the second row), and the starting position of the next round of convolution data is repeated. The above behavior is repeated until the starting point moves out of the entire data plane, so that all convolution data is read out to the row buffer module.

[0075] Figure 3 The data reuse reading state under the condition of 3*3 convolution is shown. In order to read the data of layer 0 of the convolution data group, for the case of using only one channel in the prior art, the reading of each row of 3*3 data needs 1 cycle, 3 rows need 3 cycles, and the reading of 4 3*3 data needs 12 cycles. According to the scheme of the present application, using 4 channels, reading 6 data of each row of 3*6 data needs 2 cycles, and 3 rows need 6 cycles. That is, the same data is read, and the technical scheme of the present application only needs half the time of the prior art. For other sizes of convolution kernel, the data reading time of the technical scheme of the present application is also much lower than that of the prior art.

[0076] In order to cooperate with the behavior of reading multiple groups of convolution data at a time by the data reading module, a corresponding row buffer module needs to be designed, which is composed of 4 row buffers, each row has a length of BUFF_W, and the buffer row has two states, write state and read state. When initialized, the 4 rows are in write state, and when data comes, the writing starts from the first row buffer. After writing one row of data, the current row buffer is flipped to read state, and the new data is written to the subsequent row buffer. The output of the data starts from the first row buffer when it is in read state.

[0077] The req_addr is incremented every cycle, and when the number of data taken out of the current row >= BUFF_W, the req_addr is transferred to the starting point of the second row data, and req_buff_id=req_buff_id+1. Then the above process is repeated. The write row number, write column number and write length generated by the process are delayed and then transmitted to the row buffer module for actual operation.

[0078] Meanwhile, the read process of the row buffer is designed. Note that the process is also implemented in the virtual computing module. The first read row is identified as R0 in initialization, and then the state of the current R_id is checked every cycle. When the R_id turns to the read state, the data in the read buffer is output to the corresponding channel. When the data in the current row buffer is completely output, the id of the row buffer is incremented, and the process of checking the row state, outputting the data, and incrementing the row number is repeated until the data in the entire data block is output.

[0079] For example, first, when R0 turns to the read state, it indicates that the data reading module has written R0 row full. At this time, R0C0, R0C1, R0C2 are output to channel 0 according to the cycle, R0C1, R0C2, R0C3 are output to channel 1, R0C2, R0C3, R0C4 are output to channel 2, and R0C3, R0C4, R0C5 are output to channel 3. When all the data in R0 row is output according to the shape of the convolution group data, R0 is marked as the write state, and the state of R1 is checked, and the above behavior is repeated until all the convolution data is output. The read row number, read column number and delay are generated and transmitted to the row buffer module for actual operation.

[0080] The row buffer module is the module that actually performs the behavior of writing data to the buffer and outputting data to the channel. The behavior is divided into two parts. One is to write the data from the RAM to the corresponding row buff according to the write row number, write column number and write length transmitted from the virtual computing module. Two, output the data in the corresponding row buffer to the corresponding channel according to the read row number and read column number generated by the virtual computing module to complete the output behavior.

[0081] Accordingly, the design of byte_num channel parallel output is completed.

[0082] The row buffer module and the data RAM module are instantiated together in multiple groups to match the size of the MAC array, and the final design is completed.

[0083] Design example:

[0084] The storage bit width of the data RAM is selected as 64 bits, the delay is 1 clock cycle (the delay from the address to the stable output of the data), the single data bit width is 16 bits, the convolution size is 3X3, and the size of the convolution data block to be processed is 64*64*16. Therefore, the mode of 64 / 16=4 channel parallel output can be designed for acceleration.

[0085] Design data reading module, calculate BUFF_W = 3 + 4 - 1 = 6, send the address of the current 3X6 size data block of the current layer to the data RAM in order, and send the row number, column number and quantity information of the current reading data subplane to the virtual calculation module. When the current data subblock is sent, move 4 data to the right and send the data block again. After 16 moves, the data sending of the row is completed, and the data subblock should be moved to the first column of the next row as the starting position of the subblock.

[0086] Design virtual calculation module, calculate the state of the row buffer according to the write row number, write column number and write length generated by the data reading module, and feed back to the data reading module. Delay the write row number, write column number and write length by 1 clock cycle and pass them to the row buffer module for actual write operation. At the same time, calculate the read row number and read column number according to the virtual row state and the state of the multi-channel parallel output channel, and delay them by 1 clock cycle and pass them to the row buffer module for actual read operation.

[0087] Design row buffer module, the length of each row should be no less than BUFF_W, that is, 6 data. According to the write row number, write column number and write length from the virtual calculation module, write the data from the RAM into the corresponding row buff. According to the read row number and read column number generated by the virtual calculation module, output the data in the corresponding row buffer to the corresponding channel to complete the output behavior.

[0088] Instance 4 of the row buffer module and the data RAM module, and make good overlap (edge overlap) when writing convolution data into multiple RAMs by the top layer, so as to realize parallel output of 16 channels of data to the MAC_ARRAY.

[0089] The second embodiment of the application relates to an output method of an extensible parallel convolution data output device, and the flowchart is as shown in Figure 5 The device includes a data storage and a row buffer module, the data storage stores data subblocks that need to participate in convolution, and the row buffer module includes K rows of BW columns of cache and DN channels for output. The method includes:

[0090] Step 102, send a data reading instruction to the data storage,

[0091] Step 104, the data storage reads a data block with a size of K*BW from the data subblock according to the data reading instruction and outputs it to the row buffer module, wherein BW = K + DN - 1, DN = WS / DS, K is the convolution kernel size, WS is the word length of each address of the data storage, and DS is the number of bits of each data

[0092] In step 106, the row buffer module writes the data from the data reading module into the cache, and when the writing of a row of the cache is completed, outputs BW data of the row through DN channels.

[0093] Optionally, in an embodiment, step 106 can further include: when the writing of a row of the cache is completed, outputting BW data of the row through DN channels within K clock cycles, wherein the jth channel outputs the i+jth data of the row in the ith clock cycle, 0≤i

[0094] Optionally, in an embodiment, before step 102, it can further include: generating a data reading instruction, a write row number, a write column number and a write length according to the virtual state of each row of the cache.

[0095] After step 102, it can further include: updating the virtual state of each row of the cache according to the write row number, the write column number and the write length corresponding to the data reading instruction; calculating a read row number and a read column number according to the virtual state of each row of the cache; and after delaying a predetermined clock cycle, sending the write row number, the write column number, the write length, the read row number and the read column number to the row buffer module. The processing of delaying the predetermined clock cycle makes the time when the write row number, the write column number, the write length, the read row number and the read column number arrive at the row buffer module match the time when the corresponding data arrives at the row buffer module from the data storage.

[0096] Step 106 can further include: after the row buffer module receives the data from the data storage, writing the data from the data storage into the corresponding row of the cache according to the write row number, the write column number and the write length, and outputting the data in the corresponding row of the cache to the corresponding channel according to the read row number and the write row number.

[0097] Optionally, in an embodiment, the data storage and the row buffer module have N groups, N being an integer greater than 1. The same data reading instruction is output to N data storages at the same time. The same write row number, write column number, write length, read row number and read column number are output to multiple row buffer modules at the same time.

[0098] The first embodiment is a method embodiment corresponding to the present embodiment, and the technical details in the first embodiment can be applied to the present embodiment, and the technical details in the present embodiment can also be applied to the first embodiment.

[0099] Accordingly, the embodiments of the present application also provide a computer readable storage medium, in which computer executable instructions are stored, and the computer executable instructions are executed by a processor to implement the method embodiments of the present application. The computer readable storage medium includes permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, the computer readable storage medium does not include transitory computer readable media, such as modulated data signals and carriers.

[0100] It should be noted that, in the present application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or sequence between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including one" does not exclude the presence of another identical element in the process, method, article or device including the element. In the present application, if it is mentioned that a certain behavior is performed according to a certain element, it means that the behavior is performed at least according to the element, which includes two cases: the behavior is performed only according to the element, and the behavior is performed according to the element and other elements. The expressions of multiple, multiple times, multiple varieties, etc. include 2, 2 times, 2 varieties and more than 2, more than 2 times, more than 2 varieties.

[0101] The serial numbers used in describing the steps of the method do not constitute any limitation on the order of the steps. For example, the step with a larger serial number does not necessarily have to be executed after the step with a smaller serial number, but can also be executed before the step with a smaller serial number, or can be executed in parallel, as long as such execution order is reasonable for a person skilled in the art.

[0102] This specification includes combinations of the various embodiments described herein. References to an embodiment, e.g., "one embodiment" or "some embodiments" or "an exemplary embodiment" do not necessarily refer to the same embodiment; however, such embodiments are not mutually exclusive, unless otherwise indicated. It should be noted that, unless otherwise specified, the use of "or" herein is used in the inclusive sense, i.e., an inclusive "or" will be interpreted to mean "and / or" to the extent that such interpretation is consistent with the context and intended meaning of the words used.

[0103] All documents mentioned in this specification are hereby incorporated by reference in their entirety to disclose and describe the products and / or methods in conjunction with embodiments described herein. In addition, the disclosure shows and describes preferred embodiments by way of illustration only; it is therefore not intended that the disclosure be limited to the preferred embodiments. Modifications of the disclosure in its spirit and principles can occur to those skilled in the art to which the disclosure pertains and such occurrences should be considered within the scope of the disclosure as defined by the request.

[0104] In some cases, the actions recited in the claims can be performed in a different order and still accomplish the desired results. Also, the process depicted in the accompanying figures does not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

Claims

1. A scalable parallel convolution data output apparatus, characterized by, The application comprises: a data storage configured to store data sub-blocks requiring convolution; a data reading module configured to send a data reading instruction to the data storage, read a data block of size K*BW from the data sub-blocks and output to a line buffer module, wherein BW=K+DN-1, DN=WS / DS, K is the size of a convolution kernel, WS is the word length of each address of the data storage, and DS is the number of bits of each data; the line buffer module comprises a cache of K lines and BW columns and DN channels for output; the line buffer module is configured to write data from the data reading module to the cache, and when the writing of a line of the cache is completed, output BW data of the line through the DN channels; a virtual computing module configured to maintain the virtual state of each line in the cache of the line buffer module, update the virtual state of each line in the cache according to the write line number, write column number and write length from the data reading module and feed back to the data reading module, and calculate the read line number and read column number according to the virtual state of each line in the cache, and send the write line number, write column number, write length, read line number and read column number to the line buffer module; the line buffer module is further configured to write data from the data storage to the corresponding line in the cache according to the write line number, write column number and write length from the virtual computing module, and output data in the corresponding line in the cache to the corresponding channel according to the read line number and write line number from the virtual computing module; the data reading module is further configured to generate the data reading instruction, write line number, write column number and write length according to the virtual state of each line in the cache from the virtual computing module, and send the write line number, write column number and write length to the virtual computing module.

2. The scalable parallel convolution data output device of claim 1, wherein the virtual computing module is further configured to delay the write line number, write column number, write length, read line number and read column number sent to the line buffer module, so that the time when the write line number, write column number, write length, read line number and read column number arrive at the line buffer module matches the time when the corresponding data arrives at the line buffer module from the data storage.

3. The scalable parallel convolution data output device of claim 1, wherein, The data storage and the line buffer module have N groups, corresponding to the same data reading module and the same virtual computing module, one data reading module simultaneously outputs the same data reading instruction to N data storages, and one virtual computing module simultaneously outputs the same write line number, write column number, write length, read line number and read column number to N line buffer modules, wherein N is an integer greater than 1.

4. The scalable parallel convolution data output device of claim 1, wherein, The line buffer module is further configured to output BW data of a line of the cache through the DN channels within K clock cycles when the writing of the line is completed, wherein the jth channel outputs the i+jth data in the line in the ith clock cycle, 0≤i 5. The scalable parallel convolution data output device of claim 1, wherein, The virtual state of each row in the cache includes a current write row number, a current write column number, a current output starting row number, and a current output starting column number.

6. An output method of an extensible parallel convolution data output apparatus, characterized by, The device includes a data storage storing data sub-blocks to be convolved and a row buffer module including a cache of K rows and BW columns and DN channels for output, and the method includes: The data reading module sends a data reading instruction to the data storage to read a data block of K*BW from the data sub-blocks to the row buffer module, where BW=K+DN-1, DN=WS / DS, K is a convolution kernel size, WS is a word length of each address of the data storage, and DS is a bit number of each data; The row buffer module writes data from the data reading module to the cache, and when a row of the cache is written, BW data of the row are output through the DN channels; Before the sending of the data reading instruction to the data storage, the method further includes generating the data reading instruction, a write row number, a write column number, and a write length according to the virtual state of each row in the cache; After the sending of the data reading instruction to the data storage, the method further includes updating the virtual state of each row in the cache according to the write row number, the write column number, and the write length corresponding to the data reading instruction, calculating a read row number and a read column number according to the virtual state of each row in the cache, and sending the write row number, the write column number, the write length, the read row number, and the read column number to the row buffer module; After the row buffer module receives data from the data storage, the row buffer module writes the data from the data storage to a corresponding row in the cache according to the write row number, the write column number, and the write length, and outputs data in the corresponding row in the cache to a corresponding channel according to the read row number and the write row number.

7. The scalable parallel convolution data output apparatus output method of claim 6, wherein, The outputting of the BW data of the row through the DN channels further includes: The BW data of the row are output through the DN channels within K clock cycles, where the jth channel outputs the i+jth data in the row in the ith clock cycle, 0≤i 8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, and the computer executable instructions are executed by a processor to implement the steps in the method of claim 6 or 7.

Images