Pipelined architecture based computing engine acceleration apparatus, method, device and medium

Abstract

The application provides a pipeline structure-based computing engine acceleration device, method, equipment and medium in the technical field of neural network hardware acceleration, the device comprises: a circular row buffer, comprising: a first multiplexer, used for determining the storage position of n feature map pixels of an input feature map, wherein n is a positive integer greater than 0; m random access memories, used for caching n feature map pixels according to the storage position, wherein m is a positive integer greater than 0; a second multiplexer, used for outputting the feature map pixels written into the m random access memories; a stream driving filler, used for performing boundary filling on the feature map pixels output by the second multiplexer according to the position of the feature map pixels and the boundary parameter of the convolution layer in the neural network; a computing unit array, used for performing convolution calculation on the filled feature map pixels; and a weight memory, used for caching the weight data of the neural network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural network hardware acceleration, in particular to a computing engine acceleration device and method based on a pipeline structure, equipment and medium. BACKGROUND

[0002] In the existing accelerator structure, the pipeline structure is a commonly used accelerator structure, which is composed of multiple computing engines, each of which corresponds to a computing layer in a lightweight neural network, including ordinary convolution computing engines, point convolution computing engines, deep convolution computing engines, pooling engines, fully connected computing engines and element-level operation engines. All computing engines work in a pipeline manner, and each layer in the neural network corresponds to a stage in the pipeline. This structure fully utilizes the inter-layer parallelism of the neural network algorithm, and can allocate appropriate computing resources to the corresponding computing engines according to the computing power requirements of each layer when deployed, thereby maximizing the computing efficiency.

[0003] However, the computing engine structure in the existing accelerator has two problems of low computing efficiency and serious resource waste, which leads to low computing efficiency and large resource consumption of the pipeline structure neural network accelerator when deployed and implemented. Therefore, there is an urgent need for an efficient computing engine structure with high computing efficiency and low resource occupation. SUMMARY

[0004] (I) Technical problems to be solved

[0005] To solve the above problems, the present application provides a computing engine acceleration device and method based on a pipeline structure, equipment and medium, which solves the technical problems of low computing efficiency and large resource consumption of the computing engine in the existing accelerator by designing a circular row buffer with a circular cache mode, a stream driving filler capable of automatically filling according to the original input feature map position and boundary parameters, and a computing unit array with a multiplexing mechanism.

[0006] (II) Technical solutions

[0007] The first aspect of the embodiment of the present application provides a computing engine acceleration device based on a pipeline structure, comprising: a cycle line buffer, comprising: a first multiplexer, configured to determine storage positions of n feature map pixels of an input feature map, wherein n is a positive integer greater than 0; m random access memories, configured to cache the n feature map pixels according to the storage positions, wherein when the jth feature map pixel is written into the last position of the ith random access memory, the j+1th feature map pixel is written into the i+1th random access memory, and when the jth feature map pixel is written into the last random access memory, the j+1th feature map pixel is cyclically written into the first random access memory, wherein 0

[0008] In an embodiment of the present application, the input line buffer is configured to be capable of adjusting the number of the m random access memories according to the size k of the convolution kernel and the step length s of the convolution kernel in the neural network, wherein in the case that the size of the convolution kernel is 1*1 and the step length of the convolution kernel is 1, the number of the random access memories m = 1; in the case that the size of the convolution kernel is 3*3 and the step length of the convolution kernel is 1, the number of the random access memories m = 3; in the case that the size of the convolution kernel is 3*3 and the step length of the convolution kernel is not 1, the number of the random access memories m = k + s - 1.

[0009] In an embodiment of the present application, the stream-driven filler comprises: a controller, configured to determine whether the filling condition is met according to the position of the feature map pixel and the boundary parameter of the convolution layer in the neural network, and obtain a determination result; a selector, configured to selectively fill according to the determination result, wherein in the case that the filling condition is met, a control signal 0 is sent to fill the feature map pixel, and in the case that the filling condition is not met, the feature map pixel is directly sent to the computing unit array.

[0010] In an embodiment of the present application, the computing unit array comprises: a computing unit, configured to perform convolution calculation on the filled feature map pixel according to a computing unit multiplexing mechanism, wherein the computing unit multiplexing mechanism is configured to perform two multiplication operations in one clock cycle.

[0011] In an embodiment of the present application, the neural network comprises a general neural network and a lightweight neural network.

[0012] In an embodiment of the present application, further comprising: a weight memory, configured to cache weight data of the neural network.

[0013] The second aspect of the embodiment of the present application provides a computing engine acceleration method based on a pipeline structure, comprising: determining input positions of n feature map pixels of an input feature map, wherein n is a positive integer greater than 0; caching the n feature map pixels according to the input positions, wherein when the jth feature map pixel is written into the last position of the ith random access memory, the j+1th feature map pixel is written into the i+1th random access memory, and when the jth feature map pixel is written into the last random access memory, the j+1th feature map pixel is cyclically written into the first random access memory, wherein 0

[0014] The third aspect of the embodiment of the present application provides an electronic device, comprising: one or more processors; a storage device configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the computing engine acceleration method based on the pipeline structure.

[0015] The fourth aspect of the embodiment of the present application provides a computer readable storage medium having stored executable instructions, which, when executed by a processor, cause the processor to perform the computing engine acceleration method based on the pipeline structure.

[0016] (Three) beneficial effects

[0017] The embodiment of the present application provides a computing engine acceleration device based on a pipeline structure, which has at least the following beneficial effects:

[0018] (1) The computing engine acceleration device based on the pipeline structure provided by the embodiment of the present application can effectively save BRAM (block random access memory) storage resources on an FPGA (field programmable gate array) chip through the cache mode of the circular row buffer caching feature maps, and avoids the use of a Window buffer.

[0019] (2) The computing engine acceleration device based on the pipeline structure provided by the embodiment of the present application can flexibly adjust the number of random access memories according to the size of a convolution kernel in a neural network and the step length of the convolution kernel, to adapt to convolution operations of different step lengths, and avoids the generation of invalid windows.

[0020] (3) The pipeline structure-based computing engine acceleration device provided by the embodiment of the application multiplexes the DSPs (computing units), so that each DSP can provide higher performance, and one DSP can complete two multiplication operations in one clock cycle, which is twice the amount of computation of the non-multiplexed DSP computing method, greatly improving the utilization rate of the DSP. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0022] Figure 1 The structure diagram of the circular row buffer in the pipeline structure-based computing engine acceleration device provided by the embodiment of the application is schematically shown.

[0023] Figure 2 The structure diagram of the stream-driven filler in the pipeline structure-based computing engine acceleration device provided by the embodiment of the application is schematically shown.

[0024] Figure 3 The flowchart of the computing unit multiplexing method in the pipeline structure-based computing engine acceleration device provided by the embodiment of the application is schematically shown.

[0025] Figure 4 The convolution window movement schematic diagram of the pipeline structure-based computing engine acceleration device provided by the embodiment of the application in the step size S = 2 is schematically shown.

[0026] Figure 5 The flowchart of the pipeline structure-based computing engine acceleration method provided by the embodiment of the application is schematically shown. DETAILED DESCRIPTION

[0027] To make the objectives, technical solutions and advantages of the application clearer, the following further describes the application in detail with reference to the embodiments and the accompanying drawings. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative effort fall within the protection scope of the application. The terms used herein are only for describing the specific embodiments, and are not intended to limit the application. The terms "include", "contain" and the like used herein indicate the existence of the described features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0028] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting", "fixing" and the like should be understood in a broad sense, for example, can be fixed connection, can be detachable connection, or can be integrated; can be mechanical connection, or can be electrical connection or can be in communication with each other; can be direct connection, or can be indirect connection through an intermediate medium, or can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0029] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "longitudinal", "length", "circumferential", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the subsystems or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0030] Throughout the drawings, the same elements are denoted by the same or similar reference numerals. When it may cause confusion in understanding the present application, conventional structures or configurations will be omitted. Also, the shape, size, positional relationship of the components in the drawings do not reflect the true size, scale and actual positional relationship. In addition, in the claims, any reference symbol located between parentheses should not be construed as a limitation on the claims.

[0031] Similarly, in order to simplify the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, various features of the present application are sometimes grouped together in a single embodiment, figure or description thereof. The description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the description, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0032] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise explicitly specified and limited.

[0033] The pipeline structure-based computing engine acceleration device provided by the embodiment of the present application can include a cycle line buffer, a stream driving filler and a computing unit array.

[0034] Figure 1 The structure diagram of the cycle line buffer in the pipeline structure-based computing engine acceleration device provided by the embodiment of the present application is schematically shown.

[0035] As shown in the figure, Figure 1 the cycle line buffer in the pipeline structure-based computing engine acceleration device provided by the embodiment of the present application can include a first multiplexer, m random access memories and a second multiplexer.

[0036] The first multiplexer is used to determine the storage position of n feature map pixels of an input feature map, wherein n is a positive integer greater than 0.

[0037] The m random access memories are used to cache the n feature map pixels according to the storage positions, wherein when the jth feature map pixel is written into the last position of the ith random access memory, the j+1th feature map pixel is written into the (i+1)th random access memory, and when the jth feature map pixel is written into the last random access memory, the j+1th feature map pixel is cyclically written into the first random access memory, wherein 0

[0038] The second multiplexer is used to output the feature map pixels written into the m random access memories.

[0039] The pipeline structure-based computing engine acceleration device provided by the embodiment of the present application can effectively save the BRAM storage resources on the FPGA (field programmable gate array) chip through the cache mode of the cycle line buffer, and avoid the use of the Window buffer.

[0040] Figure 2 The structure diagram of the stream driving filler in the pipeline structure-based computing engine acceleration device provided by the embodiment of the present application is schematically shown.

[0041] As shown in the figure, Figure 2 in the pipeline structure-based computing engine acceleration device provided by the embodiment of the present application, the stream driving filler is used to fill the feature map pixels output by the second multiplexer according to the position of the feature map pixels and the boundary parameter of the convolution layer in the neural network.

[0042] A stream-driven filler may include a controller and a selector. The controller determines whether the fill condition is met based on the position of the feature map pixels and the boundary parameters of the convolutional layers in the neural network, and obtains a judgment result. The selector performs selective fill based on the judgment result. If the fill condition is met, a control signal of 0 is sent to fill the feature map pixels; otherwise, the feature map pixels are directly sent to the computational unit array.

[0043] In the pipelined computing engine acceleration device provided in this invention, the stream-driven filler is a filler that can automatically complete the filling based on the position and boundary parameters of the original input feature map. The row buffer only stores the data output from the previous layer, so the filling operation does not affect the writing of data to the previous layer. The controller determines whether filling is needed based on the position of the feature map; if filling is enabled, it outputs 0; otherwise, it maps the address in the feature map to the RAM (Random Access Memory) address in the row buffer and reads the data from it. The stream-driven filler proposed in this invention avoids blocking the output of the upper layer when writing padding data. Combined with RAM, it can effectively reduce the data preprocessing time before convolution calculation and has good scalability and applicability, making it suitable for use in pipelined dataflow accelerators.

[0044] Figure 3 The flowchart illustrates a method for reusing computing units in a pipelined computing engine acceleration device provided by an embodiment of the present invention.

[0045] like Figure 3 As shown, in the pipelined computing engine acceleration device provided in this embodiment of the invention, the computing unit array is used to perform convolution calculation on the filled feature map pixels. The computing unit array includes computing units, which are used to perform convolution calculation on the filled feature map pixels according to the computing unit reuse mechanism. The computing unit reuse mechanism is configured to perform two multiplication operations within one clock cycle.

[0046] First, the pre-adder in the DSP splices the 8-bit data a and b into the 25-bit port, a and b should be separated as much as possible to ensure that the output result will not interfere with each other due to bit width overlap, and the multiplier b is sent into the 18-bit port after symbol expansion, and finally the calculation results ac and bc are intercepted in the 43-bit output, due to the smaller bit width in the DSP48E1, the lowest bit of the bc calculation result will be affected by the symbol of ac, therefore the calculation output of the high bit needs to be selected whether to perform a plus one operation according to the positive and negative of the low bit operation result. From the above analysis, one DSP completes two multiplication operations in one clock cycle, the calculation amount is 2 times that of the non-DSP multiplexing calculation method, which greatly improves the utilization rate of the DSP. The DSP multiplexing can effectively improve the calculation performance of the calculation engine when performing ordinary convolution and point convolution operations. In the calculation process of deep convolution, since only the convolution kernel level parallel mode is adopted, and there is no weight reuse in deep convolution, therefore the method cannot be used for optimization, at this time the DSP is used as a multiply-accumulator to realize the functions of multiplication, addition and storage of data, and the use of logic resources is saved.

[0047] On the basis of the above embodiment, the input line buffer is configured to be able to adjust the number of m random access memories according to the size k of the convolution kernel and the step length s of the convolution kernel in the neural network, wherein in the case that the size of the convolution kernel is 1*1 and the step length of the convolution kernel is 1, the number of random access memories m = 1; in the case that the size of the convolution kernel is 3*3 and the step length of the convolution kernel is 1, the number of random access memories m = 3; in the case that the size of the convolution kernel is 3*3 and the step length of the convolution kernel is not 1, the number of random access memories m = k + s - 1.

[0048] The calculation engine acceleration device based on the pipeline structure provided by the embodiment of the application can flexibly adjust the number of random access memories according to the size of the convolution kernel and the step length of the convolution kernel in the neural network, so as to adapt to convolution operations of different step length sizes and avoid the generation of invalid windows.

[0049] On the basis of the above-mentioned embodiments, when the step length S of the convolution kernel is 1, the number of RAMs in the circular row buffer is 3, and the size of the convolution kernel is the same, the input feature map is sequentially written into Row0, Row1 and Row2 first, at this time, the third row in the input feature map is in Row2, after Row2 is filled, the new input is circularly written from Row0, at this time, Row0 buffers the fourth row of the input feature map, therefore, when calculating the convolution, the local feature map needs to circularly read the buffer in the order of "012", "120" and "201". When the step length S of the convolution kernel is 2, in order to ensure the continuity of reading the feature map and avoid the bubble phenomenon of the feature map across rows, an additional row buffer is added on the basis of the circular row buffer, so that the calculation unit array can start the calculation of the next row immediately after completing the calculation of the current row.

[0050] On the basis of the above-mentioned embodiments, the pipeline structure-based computing engine acceleration device provided by the embodiments of the present application further comprises a weight memory for buffering weight data of a neural network.

[0051] Figure 4 The pipeline structure-based computing engine acceleration device provided by the embodiments of the present application is schematically shown in a convolution window moving diagram in a circular row buffer mode when the step length S is 2.

[0052] As Figure 4 shown, when the step length S is 2, in order to ensure the continuity of reading the feature map and avoid the bubble phenomenon of the feature map across rows, an additional row buffer is added on the basis of the circular row buffer, so that the calculation unit array can start the calculation of the next row immediately after completing the calculation of the current row. The moving process of the convolution window in the circular row buffer mode is shown in the figure, where the step length S of the convolution kernel is 2, the size of the convolution kernel K is 3*3, and the number of random access memories m is 4.

[0053] (a) The system is initialized to enter a calculation state, and the pixel points are sequentially written into the row buffers in the figure, the four row buffers are Row0 to Row3 from bottom to top, after the third feature map pixel point in the third row is written into Row2, the control logic starts reading the data in the convolution window in the red frame and sends the data into the calculation unit array for convolution operation, and the system initialization is completed.

[0054] (b) The convolution window moves gradually to the left. Since the data is read in a jump address mode, the switching of the window position is completed immediately, without the delay of sliding window operation. When the convolution window moves to the leftmost side, the calculation of the first feature map pixel point in the first row of the output feature map is performed. In terms of writing, since an additional row buffer is used, the feature map data of the previous layer input is fully buffered, and the write pointer has traversed the 4-row buffer and returned to Row0 to start writing the data in the 5th row of the feature map. The first row of data initially written in Row0 has been used up and can be directly overwritten.

[0055] (c) The calculation of all pixel points in the first row of the output feature map is completed, and the calculation of the second row is started. The convolution window with a step size of 2 directly jumps to the 3rd, 4th and 5th rows to read the data in Row2, Row3 and Row0, respectively. Since the data has been buffered in advance, the convolution operation of the second row of the output feature map can be directly started, avoiding the generation of row bubbles. The write pointer continues to write in Row0 and Row1 to pre-buffer the data.

[0056] (d) The convolution calculation of the second row is completed, and the sliding window will jump to the position in figure (d) in the next convolution. At this time, the input feature maps of the 6th and 7th rows have been written in Row1 and Row2, and the convolution calculation can be directly performed. Then the calculation process shown in the four pictures will be repeated until the calculation of the entire picture is completed. A new feature map can be directly written in the idle row position when it is input, without waiting for the data in Row0 to be used up, thereby saving the buffering time of the first few rows of the new feature map and realizing the full utilization of the computing resources.

[0057] The feature map buffering method based on the circular row RAM can effectively save the BRAM storage resources on the FPGA chip and avoid the use of Window buffer. By adjusting the number of row buffer RAMs, different convolution operations with different step sizes can be flexibly adapted, and the generation of invalid windows is avoided. The larger the step size, the higher the efficiency of window generation. When the step size S=2, the present scheme can achieve a performance improvement of about 50% compared to the traditional scheme for a single calculation engine.

[0058] The pipeline structure-based computing engine acceleration device provided by the embodiment of the present application is based on the typical lightweight neural network shufflenet v2x0.5 to realize and deploy the accelerator. The pipeline structure-based computing engine acceleration device provided by the embodiment of the present application is based on the network after quantization to deploy, about 60 computing engines are used in total, and the resource consumption of the accelerator after the layout and wiring of the KC705 development board is shown in the following table. In terms of resource consumption, the entire accelerator consumes less than half of the total amount of FPGA resources such as LUTBRAM, and the resource occupation is less. In terms of computing resource utilization, the accelerator runs at a main frequency of 200MHZ, a total of 587 DSPs are used, the single-frame recognition time based on Modelsim simulation is 188860ns, the entire accelerator can realize an ultra-high frame rate of 5200fps, and the single-DSP throughput rate can reach 0.363GOPS, which significantly improves the computing efficiency.

[0059] Resource Utilization Available Utilization (%) LUT 100045 203800 49.09 LUTRAM 12323 64000 19.25 FF 123611 407600 30.33 BRAM 127.50 445 28.65 DSP 587 840 69.88

[0060] Table I (TCP / IP protocol stack device resource consumption data)

[0061] Figure 5 The flowchart of the pipeline structure-based computing engine acceleration method provided by the embodiment of the present application is schematically shown.

[0062] As shown in Figure 5 The pipeline structure-based computing engine acceleration method provided by the embodiment of the present application can include:

[0063] S1, determine the storage position of n feature map pixels of an input feature map, wherein n is a positive integer greater than 0.

[0064] S2, cache n feature map pixels according to the storage position, wherein when the jth feature map pixel is written to the last position of the ith random access memory, the j+1th feature map pixel is written from the i+1th random access memory, and when the jth feature map pixel is written to the last random access memory, the j+1th feature map pixel is cyclically written from the first random access memory, wherein 0

[0065] S3, output the feature map pixels written to the m random access memories.

[0066] S4, according to the position of the feature map pixels and the boundary parameters of the convolution layer in the neural network, perform boundary padding on the feature map pixels output by the second multiplexer.

[0067] S5, perform convolution calculation on the padded feature map pixels.

[0068] It should be noted that the pipeline structure based computing engine acceleration method in the embodiments of the present application corresponds to the pipeline structure based computing engine acceleration device part in the embodiments of the present application, and the specific implementation details and the resulting technical effects are the same, which will not be repeated here.

[0069] The embodiments of the present application also provide an electronic device, comprising: one or more processors; a storage device configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the pipeline structure based computing engine acceleration method.

[0070] The embodiments of the present application also provide a computer readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the pipeline structure based computing engine acceleration method.

[0071] Although the present application has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description are to be considered illustrative or exemplary only and not restrictive in character.

[0072] Those skilled in the art can understand that the features described in various embodiments of the present application and / or claims can be combined in various ranges and / or combined, even if such combinations are not explicitly described in the present application. In particular, the features described in various embodiments of the present application and / or claims can be combined in various combinations and / or combined without departing from the spirit and teachings of the present application. All such combinations and / or combinations fall within the scope of the present application.

[0073] Although the present application has been shown and described with reference to certain exemplary embodiments thereof, it should be understood by those skilled in the art that various changes in form and detail can be made therein without departing from the spirit and scope of the present application as defined by the appended claims and their equivalents. Therefore, the scope of the present application should not be limited to the above-described embodiments, but should be determined only by the appended claims, and should be defined by the equivalents of the appended claims.

Claims

1. A pipelined computing engine acceleration device, applied to neural networks, characterized in that, include: Circular line buffer, including: The first multiplexer is used to determine the storage locations of n feature map pixels of the input feature map, where n is a positive integer greater than 0; m random access memories are used to cache the n feature map pixels according to the storage location. When the j-th feature map pixel is written to the last location of the i-th random access memory, the (j+1)-th feature map pixel is written starting from the (i+1)-th random access memory. When the j-th feature map pixel is written to the last random access memory, the (j+1)-th feature map pixel is written cyclically from the first random access memory. Where 0 < j < n, 0 < i < m, and m is a positive integer greater than 0. The second multiplexer is used to output the feature map pixels written to the m random access memories; A stream-driven filler is used to fill the boundaries of the feature map pixels output by the second multiplexer according to the position of the feature map pixels and the boundary parameters of the convolutional layer in the neural network. The computational unit array is used to perform convolution calculations on the padded feature map pixels.

2. The computing engine acceleration device based on a pipelined structure according to claim 1, characterized in that, The circular row buffer is configured to adjust the number of m random access memories based on the kernel size k and the kernel stride s in the neural network, wherein, When the size of the convolution kernel is 1*1 and the stride of the convolution kernel is 1, the number of random access memories is m=1; When the size of the convolution kernel is 3*3 and the stride of the convolution kernel is 1, the number of random access memories is m=3; When the size of the convolution kernel is 3*3 and the stride of the convolution kernel is not 1, the number of random access memories is m=k+s-1.

3. The computing engine acceleration device based on pipeline structure according to claim 1, characterized in that, The flow-driven filler includes: The controller is used to determine whether the padding condition is met based on the position of the pixel in the feature map and the boundary parameters of the convolutional layer in the neural network, and to obtain the determination result. A selector is used to selectively fill based on the judgment result. If the filling condition is met, a control signal 0 is sent to fill the feature map pixels. If the filling condition is not met, the feature map pixels are sent directly to the computing unit array.

4. The pipelined computing engine acceleration device according to claim 1, characterized in that, The computing unit array includes: A computing unit is used to perform convolution calculations on the filled feature map pixels according to a computing unit reuse mechanism, wherein the computing unit reuse mechanism is configured to perform two multiplication operations within one clock cycle.

5. The pipelined computing engine acceleration device according to claim 1, characterized in that, The neural networks include ordinary neural networks and lightweight neural networks.

6. The pipelined computing engine acceleration device according to claim 1, characterized in that, Also includes: A weight memory is used to cache the weight data of the neural network.

7. A method for accelerating a computing engine based on a pipelined architecture, characterized in that, include: Determine the storage locations of n feature map pixels of the input feature map, where n is a positive integer greater than 0; The n feature map pixels are cached according to the storage location, wherein when the j-th feature map pixel is written to the last location of the i-th random access memory, the (j+1)-th feature map pixel is written starting from the (i+1)-th random access memory; when the j-th feature map pixel is written to the last random access memory, the (j+1)-th feature map pixel is written cyclically from the first random access memory, wherein 0 < j < n, 0 < i < m, and m is a positive integer greater than 0; Output the feature map pixels written to m random access memories; Based on the position of the feature map pixels and the boundary parameters of the convolutional layer in the neural network, the feature map pixels output by the second multiplexer are filled with boundaries. Convolution calculations are performed on the pixels of the padded feature map.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to claim 7.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method according to claim 7.

Images