General neural network accelerator optimization method and system for edge computing scene

Abstract

The invention provides a general neural network accelerator optimization method and system for an edge computing scene, and solves the technical problem of time delay caused by low computing power of a device terminal in an existing edge scene. The method comprises the following steps: data access: reading input feature map data of an edge computing scene, performing convolution operation, sorting the input data according to requirements of a convolution engine to obtain the input data of the edge computing scene, and then caching the input data; carrying out convolution calculation, loading input data of the edge calculation scene, carrying out rearrangement, obtaining a weight and image data of a convolution kernel of the rearranged edge calculation scene, carrying out multiplication and addition operation to obtain a multiplication and addition result of the edge calculation scene, carrying out accumulation calculation to output a convolution result of the edge calculation scene, and carrying out calculation by utilizing an activation function to obtain an activated convolution result. And finally, obtaining a target identification result through target identification. The method can be widely applied to the technical field of neural network accelerators.

Description

Technical Field

[0001] The present application belongs to the field of neural network accelerator technology, and more specifically, to a general neural network accelerator optimization method and system for edge computing scenarios. Background Art

[0002] In recent years, with the continuous development of artificial intelligence technology, convolutional neural network algorithms, as a basic algorithm for deep learning, have been used in more and more scenarios. Compared with traditional algorithms, convolutional neural network algorithms have higher accuracy and versatility, and have been widely used in many scenarios such as face recognition, unmanned driving and ship detection.

[0003] With the continuous development of integrated circuit technology, the research on neural network accelerators is also deepening. In 2014, Chen Tianshi's team proposed the DianNao architecture, which can initially complete the neural network acceleration function on ASIC devices; in 2017, Liang Yun's team used the Xilinx ZC706 platform to adopt Winograd convolution acceleration technology to further accelerate the calculation of the YOLO model, greatly improving the calculation speed; in recent years, Microsoft has successfully designed a neural network acceleration platform composed of FPGA in the Project Catapult project, which can accelerate various applications such as Bing search and Azure cloud services. It is superior to traditional systems in terms of processing speed, operating cost and energy consumption.

[0004] The use of edge computing technology can apply convolutional neural network algorithms in actual application scenarios, allowing edge device terminals to perform data processing operations independently, independent of the control of the host, and maintaining low latency and high reliability.

[0005] Despite this, the computing power of edge device terminals is generally low. When processing complex convolution operations of convolutional neural networks, they will face delay problems caused by insufficient algorithms. Therefore, how to accelerate the large number of convolution calculations of neural networks in edge computing scenarios and solve the delay problem caused by insufficient algorithms has become a hot topic of research today. Summary of the invention

[0006] The purpose of the embodiments of the present application is to provide a general neural network accelerator optimization method for edge computing scenarios to solve the technical problem of low computing power of device terminals in edge scenarios in the prior art, resulting in delays.

[0007] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a general neural network accelerator optimization method for edge computing scenarios, comprising the following steps: Data access: read the input feature map data of the edge computing scenario and perform convolution operations. Sort the input data according to the requirements of the convolution engine to obtain the input data of the edge computing scenario, and then cache it. Convolution calculation: load the input data of the edge computing scene and rearrange it according to the requirements of the multiplication and addition queue arrangement to obtain the weights and image data of the convolution kernel of the edge computing scene after rearrangement, perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scene, perform cumulative calculations to output the convolution results of the edge computing scene, use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition; The formula for the multiply-add operation is: ; ; ; , ; In the formula, is the multiplication and addition result of the edge computing scenario, that is, the sum of the weight of the convolution kernel of the edge computing scenario and the element-by-element product of the image data. To rearrange the image data of the edge computing scene, Calculate the weights of the scene convolution kernel for the rearranged edges, is the most significant bit of the image data of the edge computing scene after rearrangement, The most significant digit of the weight of the scene convolution kernel for edge calculation after rearrangement, and are all offset weight coefficients.

[0008] Preferably, the input data of the edge computing scenario is loaded and rearranged according to the requirements of the multiplication and addition queue arrangement. The specific implementation method is as follows: Poll the input data of the edge computing scene, extract the weight and image data of the edge computing scene convolution kernel, generate two data loading packages, and then dequeue them in the order in which they enter the queue to obtain the weight and image data of the edge computing scene convolution kernel after rearrangement; The image data of the edge computing scene after rearrangement includes model parameter data and input image data.

[0009] Preferably, the input feature graph data of the edge computing scenario is read, and the specific implementation method is as follows: The input feature map data of the edge computing scenario is stored in an external storage device, i.e., a shared cache; Reorder the input data according to the requirements of the convolution engine; Write the generated convolution data cache to the corresponding shared storage address; Write feature data into the convolution data cache, update the convolution data cache, and obtain the input data for the edge computing scenario.

[0010] Preferably, a convolution operation includes direct convolution, Winograd convolution or image convolution: Direct convolution refers to grouping convolution, that is, grouping the multiplication operations from each convolution kernel so that each group contains the same number of multiplications; Winograd convolution refers to convolution through matrix transformation, that is, linearly transforming the weights of the convolution kernel of the edge computing scene and the image data, performing a small number of multiplication operations in the transformation domain to reduce the number of multiplications in each area, and then restoring the result through inverse transformation.

[0011] Preferably, the convolution engine's requirement for input data is that it is input in the form of image pixel data in an edge computing scenario.

[0012] Preferably, cumulative calculation, the specific steps are: Initialize the accumulator to zero, take out each value of the sum of the element-by-element product of the convolution kernel of the edge computing scene and the image data one by one, and add them to the accumulator in turn to obtain the convolution result of the edge computing scene.

[0013] Preferably, the convolution result after activation is as follows: ; In the formula, is the convolution result after activation, This is the convolution result of the edge computing scenario.

[0014] Preferably, the input feature map data of the edge computing scenario includes feature map data, weight data, pictures and feature values.

[0015] Preferably, the input data of the edge computing scenario includes input feature data, pixel data and weight data of the edge computing scenario.

[0016] A second aspect of an embodiment of the present application provides a general neural network accelerator optimization system for edge computing scenarios, including: The data access module is used to read the input feature map data of the edge computing scenario and perform convolution operations, sort the input data according to the requirements of the convolution engine to obtain the input data of the edge computing scenario, and then cache it; The convolution calculation module is used to load the input data of the edge computing scenario and rearrange it according to the requirements of the multiplication and addition queue arrangement, obtain the weights and image data of the convolution kernel of the edge computing scenario after rearrangement, perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scenario, perform cumulative calculations to output the convolution results of the edge computing scenario, use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition.

[0017] Beneficial effects of the present invention: The present invention designs a general neural network accelerator optimization method and system for edge computing scenarios. In the process of data access, the input feature map data is sorted and cached, which improves the efficiency of data access and reduces the delay of data processing in edge computing scenarios. In the process of convolution calculation, the weights and image data of the convolution kernel of the edge computing scenario are rearranged to make the convolution calculation more efficient, reduce the consumption of computing resources, and improve the computing performance of edge computing devices. The multiplication and addition operation designed by the present invention is used to perform multiplication and addition calculations, and the calculation steps are simplified by removing the partial product other than the most significant bit, thereby improving the speed of calculation, ensuring the efficiency of calculation, and reducing the delay. Moreover, the multiplication and addition operation designed by the present invention has been verified by simulation tests, and the test results confirm that the multiplication and addition operation designed by the present invention significantly reduces the error while ensuring the stability of power consumption without obvious changes. Based on the traditional convolution pipeline structure, the present invention constructs a two-layer pipeline structure to achieve higher computing power, faster computing speed and lower power consumption, significantly reduce delays, and improve the overall performance of the convolution pipeline structure.

[0018] In summary, the general neural network accelerator optimization method and system designed by the present invention for edge computing scenarios have greater computing power, faster computing speed, and more significant effect in reducing latency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0020] Figure 1 A schematic diagram of the overall process provided for an embodiment of the present application; Figure 2 A flowchart of a DMA controller provided in one embodiment of the present application; Figure 3 A schematic diagram of input feature graph data for an edge computing scenario provided in one embodiment of the present application; Figure 4A schematic diagram of image data of an edge computing scenario provided in one embodiment of the present application; Figure 5 A weight diagram of a convolution kernel for an edge computing scenario provided in one embodiment of the present application; Figure 6 A diagram showing the output result of a sequence controller provided in an embodiment of the present application; Figure 7 A multiplication-addition result diagram of a convolution multiplier and adder provided in one embodiment of the present application; Figure 8 A convolution result diagram of an accumulator provided in an embodiment of the present application; Fig. 9 This is a target recognition result diagram of the handwritten number “9” provided in one embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] See also Figure 1 , is a schematic diagram of the overall process of a general neural network accelerator optimization method for edge computing scenarios provided by an embodiment of the present application, including: The first aspect of the present application provides a general neural network accelerator optimization method for edge computing scenarios, including: data access, reading input feature map data of the edge computing scenario from a connected storage device and performing a convolution operation, sorting the input data according to the requirements of the convolution engine to obtain the input data of the edge computing scenario, and then caching; Convolution calculation, load the input data of the edge computing scene and rearrange it according to the requirements of the multiplication and addition queue arrangement, obtain the weight and image data of the rearranged edge computing scene convolution kernel, and then perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scene, and then perform cumulative calculations to output the convolution results of the edge computing scene, and use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition. Among them, the use of the designed multiplication and addition operations speeds up the calculation of the accelerator and reduces latency.

[0023] S1: Data access, reads the input feature map data of the edge computing scenario from the connected storage device and performs convolution operations, then sorts the input data according to the requirements of the convolution engine, and then caches the input data of the edge computing scenario.

[0024] The specific steps include: 1. The DMA (direct access memory bridge) controller batch reads the input feature map data of the edge computing scenario from the connected storage device and performs convolution operations, and then sorts the input data according to the requirements of the convolution engine to obtain the input data of the edge computing scenario.

[0025] Specifically, in an optional embodiment, the DMA controller is responsible for batch reading the input feature map data of the edge computing scenario from the connected RAM and DDR and other storage devices, including feature map data, weight data, image information and feature value parameters. In order to meet the requirements of the convolution engine for input data, namely data type, value range, size, etc., it is input in the edge computing scenario as image pixel data. The DMA controller supports data input in multiple formats, such as pixel data, feature data, weight data before and after compression, weight mask bits, weight group size, etc.

[0026] Among them, the DMA controller has two AXI interfaces, which respectively complete the reading of feature map data and neural network model weights. Inside the DMA controller, there are three sub-modules for acquiring and reorganizing pixel data and model parameter data, namely direct convolution, Winograd convolution and image convolution. Their internal structures are different. Except that the feature data formats generated by direct convolution and Winograd convolution are different, and the former will simultaneously perform Winograd channel expansion when reading, the workflow is basically the same.

[0027] The specific workflow is as follows: (1) Determine whether there is enough space in the DMA controller to cache the input feature map data of the edge computing scenario after convolution; (2) Generate a read transaction; (3) Storing the input feature map data of the edge computing scenario into an external storage device, i.e., a shared cache; (4) Reordering the feature map data; (5) Write the generated convolution data cache to the address of the corresponding shared cache; (6) Writing feature data into the convolution data cache; (7) Update the convolution data cache in the submodule to obtain the input data for the edge computing scenario.

[0028] Specifically, in an optional embodiment, the convolution operation steps of direct convolution and Winograd convolution are as follows: 1.1 Direct Convolution: See also Figure 2 , based on the direct convolution calculation of the eigenvalue, corresponding to the calculation of each convolution layer except the first layer in the neural network mox1, deploys multiple channels, and adopts the multiplication and accumulation pipeline to support efficient and parallel convolution operations. The formula for direct convolution is: ; ; ; ; ; Where S' is the scale parameter, S is the input step size, DX is the horizontal expansion coefficient, R' is the height transformation result, R is the number of input rows, DY is the vertical expansion coefficient, and W' is the width transformation result. is the left border, W is the input width, is the right boundary, SX is the horizontal step size, H' is the output height, is the top boundary, H is the height of the feature map data cube, is the bottom boundary, SY is the vertical step size, C is the number of input channels, and K is the total number of cores.

[0029] In the output data of the edge computing scenario, the input feature data cube kernel element wt of each element y, x in the weight kernel is: ; In the formula, is the value of the output feature map at position (w,h)(w,h)(w,h) and channel kkk after the convolution operation, R is the vertical size of the convolution kernel, S is the number of neurons in the hidden layer, C is the number of neurons in the output layer, SX is the horizontal step size of the convolution kernel, SY is the vertical step size of the convolution kernel, is the left border, is the top boundary, r is the vertical index of the convolution kernel, s is the horizontal index of the convolution kernel, c is the channel index of the input image or feature map, w is the horizontal position index of the convolution kernel in the output feature map, h is the vertical position index of the convolution kernel in the output feature map, and k is the channel index of the output feature map. Everything starts from scratch.

[0030] Based on the above formula, the direct convolution mode rule designed by the present invention is as follows, and its core idea is to group the multiplication operations from each convolution kernel so that the number of multiplications contained in each group is the same.

[0031] The specific steps include: (1) All MAC hardware is evenly distributed among the 16 subunits. A single subunit is called a MAC Cell, and each sub-cell has 64 INT16 MAC hardware, or 128 INT8 units; (2) Arrange the MAC Cells into a MAC cell array; (3) Divide the input feature map data cube of all edge computing scenarios into small cube elements of 1x1x64 for int16 and int8 calculations; (4) Divide the weight data cubes of all edge computing scenarios into small cube elements of 1x1x64 for int16 and int8 calculations; (5) Multiply and add the input feature map data cube and the weight data cube of the edge computing scenario obtained in (3) and (4); (6) These computing operations are combined into four levels: atomic operation, stripe operation, block operation, and channel operation.

[0032] 1.2Winograd convolution: Winograd convolution can reduce the need for multipliers in the multiplication-addition array by changing the calculation method. It performs convolution through matrix transformation, that is, linearly transforms the weights of the convolution kernel and image data in the edge computing scene, performs fewer multiplication operations in the transform domain, reduces the number of multiplications in each area, and then restores the result through inverse transformation, thereby improving performance and energy efficiency.

[0033] In a Winograd atomic operation, the 64 cells in a MAC Cell are not simply accumulated, but require three stages of operation: (1) In the channel accumulation unit, the four units in the channel direction of each MAC Cell are added together. Since there are 16 such channel groups, the sum of 16 parts will be output. Each sum represents a 4x4 matrix for dot multiplication calculation, which can meet the needs of 4, 8, 16, and 32 bits.

[0034] (2) Each 4x4 matrix is ​​multiplied by a predefined transformation matrix A , output a 4x2 matrix, represented as a transposed matrix .

[0035] (3) Each transposed matrix and A -1 (the inverse matrix of the pre-defined transformation matrix A) is multiplied.

[0036] 2. The cache area caches the input data of the edge computing scenario.

[0037] Specifically, each cache contains 512KB of static random access memory (SRAM), each 512KB of static random access memory is composed of 16 32KB BANK memories, and each BANK memory is composed of 2 dual-port static random access memories with 512-bit output and 256-bit input. The cache composed of the above devices is used to cache the input data of the edge computing scene, including input pixel data, input feature data, weight data, and weight mark bit data.

[0038] For weight data, if the weight is compressed, BANK memory 15 is allocated to the weight storage buffer for caching, and BANK memories 0-14 are allocated to two other types of caches, namely input activation cache and output activation cache; otherwise, it will not be allocated to any BANK memory.

[0039] S2: Convolution calculation, rearrange the input data of the edge computing scene according to the requirements of the multiplication and addition queue arrangement to obtain the weights and image data of the convolution kernel of the rearranged edge computing scene, and then perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scene, and then perform cumulative calculations to output the convolution results of the edge computing scene, use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition.

[0040] The specific steps include: 1. The sequence controller loads the input data of the edge computing scene from the cache, rearranges it according to the requirements of data arrangement during multiplication and addition queue calculation, and obtains the weights and image data of the convolution kernel of the edge computing scene after rearrangement.

[0041] Specifically, the sequence controller is a key step in calculating and controlling the decomposition of the convolution in the convolution pipeline. It is responsible for loading the input data of the edge computing scene from the cache and rearranging it according to the requirements of the data arrangement during the multiplication and addition queue calculation to obtain the weights and image data of the convolution kernel of the edge computing scene after rearrangement. Among them, the image data of the edge computing scene after rearrangement includes model parameter data and input image data. By rearranging, the weights and image data of the convolution kernel of the edge computing scene meet the requirements of the multiplication and addition array data arrangement to improve the utilization rate of the multiplication and addition queue.

[0042] Among them, the sequence controller consists of three sub-modules: convolution generator, weight loader and data loader.

[0043] The convolution generator is responsible for generating the sequence that controls the convolution operation, that is, the reordering, and its workflow is as follows: (1) Perform polling operations in the cache until sufficient edge computing scene convolution kernel weights and image data are obtained; (2) The weight of the convolution kernel of the edge computing scene is converted into a weight data loading package, and the image data of the edge computing scene is converted into a data loading package. Thus, two loading packages are generated, which constitute a pair of corresponding sequence data packets, representing a strip operation; (3) Push a pair of corresponding sequence data packets into the corresponding two FIFO queues, so that they can be dequeued in the order in which they entered the queue; (4) At the same time, the weight counter and the feature counter continue to count down; (5) When the weight counter reaches 0, check the signal from the volume totalizer to determine whether the task has been completed; (6) If loading is completed, the weights and image data of the rearranged edge computing scene convolution kernel are sent to the weight loader and data loader.

[0044] The data loader workflow is as follows: Load the image data of the edge computing scene from the convolution generator and send it to the convolution multiplier to determine whether the data layout in the image data of the edge computing scene is in the order and format required by the convolution multiplier. If so, send it directly; otherwise, rearrange it again and send it, and communicate with the DMA controller to keep the latest status of the image data of the edge computing scene.

[0045] The weight loader workflow is as follows: Receive a pair of corresponding sequence data packets from the convolution generator, load the weights of the edge computing scenario convolution kernel, and determine whether the data layout of the edge computing scenario convolution kernel weights is in the order and format required by the convolution accumulator. If so, send it directly; otherwise, rearrange it again and send it, then decompress it and send it to the convolution multiplier.

[0046] 2. The convolution multiplier performs multiplication and addition operations on the rearranged image data of the edge computing scene and the weight of the convolution kernel to obtain the multiplication and addition result of the edge computing scene.

[0047] Specifically, the convolution multiplier implements the multiplication-addition queue and is the core of the convolution calculation. It implements the multiplication-addition operation of the rearranged image data of the edge computing scene and the weight of the convolution kernel, and obtains the multiplication-addition result of the edge computing scene, that is, the sum of the element-by-element product of the weight of the convolution kernel of the edge computing scene and the image data.

[0048] The convolution multiplier unit integrates 16 MAC cells, each of which includes 64 16-bit accumulators and 72 INT16 / INT8 adders.

[0049] In edge computing scenarios, neural network models are generally compressed and quantized to INT8 for calculation, which can effectively speed up the calculation. The present invention mainly optimizes the INT8 calculation. For the INT8 calculation, each component accumulator module can be split into two.

[0050] In the design of traditional convolutional neural network accelerators, the choice of multipliers is also quite diverse. There are implementations based on the logarithmic multiplier proposed by JOHNN.MITCHELL, the dynamic range unbiased multiplier proposed by Soheil Hashemi, and the approximate multiplier proposed by Z.Babic et al.: Iterative Logarithmic Multiplier (ILM).

[0051] Based on the ILM multiplier, the present invention is optimized for edge computing scenarios.

[0052] Specifically, the convolution multiplier designed in the present invention does not directly use multiplication as the result in order to meet the requirements of high accuracy and low power consumption in edge computing scenarios. Instead, it processes the multiplication and then uses addition to obtain an approximate product. This can reduce the number of calculation steps and reduce the delay problem caused by cumbersome calculations.

[0053] Assume that there are two rearranged edge computing scenes with image data and convolution kernel weights: and , whose most significant digits are , , then we have the following formula: ; ; ; , ; In the formula, and are manually set offset weight coefficients, and are decimals, so that To complete the approximate multiplication, we only need to implement it by shifting and And add them together.

[0054] In the traditional floating-point multiplication process, if both multipliers contain decimal points, the decimal point position of the product is determined by the sum of the decimal places of the two multipliers. However, if the decimal part of the multiplier is very small, the precision loss of the decimal part of the product may be very large. In this case, ignoring the decimal part of the product can improve the accuracy of the calculation. Therefore, by ignoring the decimal part of the product, the decimal point position of the product is determined by the sum of the decimal places of the two multipliers. To complete the approximate multiplication work, it not only improves the calculation speed of the adder, but also improves the calculation accuracy.

[0055] The convolution multiplier and adder before and after improvement were used for simulation test. The simulation was carried out under SMIC's 40nmLVT (Low Voltage Threshold) process. The two inputs of the approximate convolution multiplier and adder were 10,000 16-bit binary numbers randomly obtained by the LFSR function. The frequency was 100MHZ, the input voltage was 1.1V, the temperature was 25 degrees Celsius, and the simulation running time was 20,000 cycles.

[0056] In the original convolution multiplier scenario, the average relative error is 9.71% and the power consumption is 32.7uw.

[0057] The average relative error in the improved convolution multiplier scenario is 7.38%, and the power consumption is 27.1uw.

[0058] It can be seen that before and after the convolution multiplier was improved, the average relative error was reduced by 2.33% and the power consumption was reduced by 17.12%. It can be obtained that under the premise of reducing the error, the power consumption did not change significantly.

[0059] 3. The accumulator receives the multiplication and addition results of the edge computing scenario and performs accumulation calculation to obtain the convolution result of the edge computing scenario, and then calculates the activation function. After obtaining the activated convolution result, the target is recognized to obtain the target recognition result.

[0060] In an optional embodiment, the accumulator receives the multiplication and addition results of the edge computing scenario and performs the accumulation operation according to the following formula: ; In the formula, is the convolution result, * is the multiplication and addition result of the edge computing scenario, where Image data for edge computing scenarios elements; is the weight of the convolution kernel in the edge computing scenario, and are the sizes of the convolution kernels respectively.

[0061] Specifically, in an optional embodiment, the specific steps of cumulative calculation are: Initialize the accumulator to zero to store the convolution result; For the multiplication and addition results of the edge computing scenario, each value of the sum of the element-by-element products of the image data of the edge computing scenario and the weight of the convolution kernel is taken out one by one and added to the accumulator.

[0062] The consistency of data types needs to be ensured during the accumulation process, especially in edge computing scenarios, where the multiplication and addition results may be quantized integer data. Therefore, the multiplication and addition results of edge computing scenarios are converted to the data type of the accumulator to ensure the accuracy of the accumulation.

[0063] After the accumulation operation is completed, the convolution result retains the effective number of digits as the input of the single data point processor, and then the activation function is calculated to obtain the activated convolution result, that is, the result of the feature map data after the convolution calculation after the nonlinear operation of the activation function. The present invention can adapt to different precision ranges of neural network models in different scenarios: if the precision is INT16, 48 bits are output, and if the precision is INT8, 34 bits are output. The specific rounding algorithm can be selected by the corresponding bit CLIP_TRUNCATE of register D_CLIP_CFG.

[0064] Specifically, the calculation of a single data point processor is usually divided into the following steps: Receive the convolution result of the edge computing scenario and use the activation function for calculation, where the activation function includes ReLU function, PReLU function, etc. There is no restriction on the selection of activation function, which can be selected according to the specific application scenario.

[0065] This embodiment selects the ReLU activation function for calculation, and the specific formula is: ; In the formula, is the convolution result after activation, is the convolution result.

[0066] The target recognition result can be obtained by using the recognition model to perform target recognition on the activated convolution result. The recognition model here is not required and can be added according to the actual application scenario.

[0067] In addition, the accumulator also includes an assembly static random access memory SRAM and a transmission static random access memory SRAM group, an adder array, a flow control and detection module, etc.

[0068] Specifically, in order to cache high-precision accumulated numbers, the assembled static random access memory SRAM includes 4 groups of 96Bx32 (96-bit width, 32-bit depth) static random access memory SRAMs, and 4 groups of 64Bx32 (62-bit width, 32-bit depth) static random access memory SRAMs; the transmission static random access memory SRAM includes 8 64Bx32 (62-bit width, 32-bit depth) static random access memory SRAMs.

[0069] Through the above configuration, the convolution results of the edge computing scenario are cached and then output to the single data point processor, which can be matched with a pooling module or an activation function module as needed.

[0070] A second aspect of an embodiment of the present invention also provides a general neural network accelerator system for edge computing scenarios.

[0071] The data access module is used to read the input feature map data of the edge computing scenario and perform convolution operations, sort the input data according to the requirements of the convolution engine, and then cache it; The convolution calculation module is used to load the input data of the edge computing scenario and rearrange it according to the requirements of the multiplication and addition queue arrangement, obtain the weights and image data of the convolution kernel of the edge computing scenario after rearrangement, perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scenario, perform cumulative calculations to output the convolution results of the edge computing scenario, and use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition.

[0072] It should be noted that the general neural network accelerator optimization system for edge computing scenarios in this embodiment is an embodiment of the system corresponding to the general neural network accelerator optimization method for edge computing scenarios described above. Therefore, for the specific implementation of the software method in each module of the system, please refer to Figure 1-Figure 2 The embodiments of the present invention will not be described in detail here.

[0073] Specific embodiment: a target recognition is realized by using the present invention in an edge scenario.

[0074] Read the input feature map data of the edge computing scenario and get Figure 3 , a picture of a white handwritten number “9” against a black background.

[0075] See also Figure 4-Figure 5 , select direct convolution to perform convolution operation, and obtain the image data of the edge computing scene and the weight of the edge computing scene convolution kernel that are not sorted according to the convolution engine's requirements for input data. Among them, the image data of the edge computing scene is in hexadecimal, 8 rows and 4 columns. For details, please refer to Figure 4 The weight of the convolution kernel in the edge computing scenario is “01 0203 00 00 00 00 00 00”.

[0076] The unsorted image data of the edge computing scene and the weights of the convolution kernel of the edge computing scene are sorted according to the requirements of the convolution engine for input data to obtain the input data of the edge computing scene, and then cached.

[0077] See also Figure 6, load the input data of the edge computing scene, and rearrange it according to the requirements of the multiplication and addition queue arrangement to obtain the weights and image data of the convolution kernel of the edge computing scene after rearrangement. In the figure, it is specifically shown as follows: (1) The upper part is the image data of the edge computing scene after reordering. The first line "1 2 3 5" corresponds to the image data of the edge computing scene that is not sorted according to the requirements of the convolution engine for input data, such as Figure 4 The first column "0100 02 00 03 00 05 00" shown in the figure; the second row "3 4 6 8" corresponds to the image data of the edge computing scene that is not sorted according to the convolution engine's requirements for input data, such as Figure 4 The second column "03 00 04 00 06 00 08 00" shown in FIG; the third row "5 7 9 a" corresponds to the image data of the edge computing scenario that is not sorted according to the convolution engine's requirements for input data, such as Figure 4 The third column shown is "05 00 07 00 09 00 0a 00"; because the image data of the edge computing scene is not sorted according to the convolution engine's requirements for input data, such as Figure 4 The fourth column shown is all "00", so only three rows appear here.

[0078] (2) The lower part is the weight of the edge computing scenario convolution kernel after reordering. The first row "1" corresponds to the weight of the edge computing scenario convolution kernel that is not sorted according to the convolution engine's requirements for input data, such as Figure 5 The first column is "0100" as shown; the second row is "2" corresponding to the weights of the convolution kernels in the edge computing scenario that are not sorted according to the convolution engine's requirements for input data, such as Figure 5 The second column is "02 00" as shown; the third line "3" corresponds to the weight of the convolution kernel in the edge computing scenario that is not sorted according to the requirements of the convolution engine for the input data, such as Figure 5 The second column "03 00" shown in the figure is because the weights of the edge computing scene convolution kernels are not sorted according to the convolution engine's requirements for input data, such as Figure 5 The fourth column shown is all "00", so only three rows appear here.

[0079] Then, the weights of the rearranged edge computing scene convolution kernel and the image data are multiplied and added to obtain the multiplication and addition result of the edge computing scene, such as Figure 7 As shown, the first line is "5_0301 7_0402 9_0603 A_0805" and the second line is "40_0201".

[0080] See also Figure 8, the multiplication and addition results of the edge computing scene are accumulated and the convolution result of the edge computing scene is output, which is specifically expressed as: "16 1f 2a 33". The activation function is used to calculate the activated convolution result, and finally the target recognition result is obtained through target recognition.

[0081] Specifically, the recognition model Lenet is used to perform target recognition on the activated convolution result, and the target recognition result obtained is as follows: Fig. 9 As shown. The gray area "0123456789" corresponds to the "0000000001" below, and the "1" here corresponds to the "9" in the gray area, that is, "9" is correctly identified, and a target recognition in an edge scene is completed accurately and quickly. It represents that the convolution module of the present invention can be matched with different neural network accelerator modules to complete the convolution calculation acceleration task and realize accurate target recognition.

[0082] The present invention designs a general neural network accelerator optimization method and system for edge computing scenarios. In the process of data access, the input feature map data is sorted and cached, which improves the efficiency of data access and reduces the delay of data processing in edge computing scenarios. In the process of convolution calculation, the weights and image data of the convolution kernel of the edge computing scenario are rearranged to make the convolution calculation more efficient, reduce the consumption of computing resources, and improve the computing performance of the edge computing device. The multiplication and addition operation designed by the present invention is used to perform multiplication and addition calculations, and the calculation steps are simplified by removing the partial product other than the most significant bit, thereby improving the speed of calculation, ensuring the efficiency of calculation, and reducing the delay. Moreover, the multiplication and addition operation adopted by the present invention has been verified by simulation tests, and the test results confirm that the multiplication and addition operation designed by the present invention significantly reduces the error while ensuring the stability of power consumption without obvious changes. Based on the traditional convolution pipeline structure, the present invention constructs a two-layer pipeline structure to achieve higher computing power, faster computing speed and lower power consumption, significantly reduce delays, and improve the overall performance of the convolution pipeline structure.

[0083] In summary, the general neural network accelerator optimization method and system designed by the present invention for edge computing scenarios have greater computing power, faster computing speed, and more significant effect in reducing latency.

[0084] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0085] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A general neural network accelerator optimization method for edge computing scenarios, characterized in that: The following steps are involved: Data access: read the input feature map data of the edge computing scenario and perform convolution operations. Sort the input data according to the requirements of the convolution engine to obtain the input data of the edge computing scenario, and then cache it. Convolution calculation: load the input data of the edge computing scene and rearrange it according to the requirements of the multiplication and addition queue arrangement to obtain the weights and image data of the convolution kernel of the edge computing scene after rearrangement, perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scene, perform cumulative calculations to output the convolution results of the edge computing scene, use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition; The formula for the multiplication and addition operation is: ； ； ； ， ； In the formula, is the multiplication and addition result of the edge computing scenario, that is, the sum of the weight of the convolution kernel of the edge computing scenario and the element-by-element product of the image data. To rearrange the image data of the edge computing scene, Calculate the weights of the scene convolution kernel for the rearranged edges, is the most significant bit of the image data of the edge computing scene after rearrangement, The most significant digit of the weight of the scene convolution kernel for edge calculation after rearrangement, and are all offset weight coefficients.

2. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The input data of the edge computing scenario is loaded and rearranged according to the requirements of the multiplication and addition queue arrangement. The specific implementation method is as follows: Poll the input data of the edge computing scene, extract the weight and image data of the edge computing scene convolution kernel, generate two data loading packages, and dequeue them in the order in which they enter the queue to obtain the weight and image data of the edge computing scene convolution kernel after rearrangement; The rearranged image data of the edge computing scene includes model parameter data and input image data.

3. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The specific implementation method of reading the input feature map data of the edge computing scenario is as follows: The input feature map data of the edge computing scenario is stored in an external storage device, i.e., a shared cache; Reorder the input data according to the requirements of the convolution engine; Writing the generated convolution data cache into the address corresponding to the shared storage; Write feature data into the convolution data cache, update the convolution data cache, and obtain the input data for the edge computing scenario.

4. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The convolution operation includes direct convolution, Winograd convolution or image convolution: The direct convolution refers to performing convolution in groups, that is, grouping the multiplication operations from each convolution kernel so that the number of multiplications contained in each group is the same; The Winograd convolution refers to convolution through matrix transformation, that is, linearly transforming the weights of the convolution kernel of the edge computing scene and the image data, performing a small number of multiplication operations in the transformation domain to reduce the number of multiplications in each area, and then restoring the result through inverse transformation.

5. A general neural network accelerator optimization method for edge computing scenarios as described in claim 1 or 3, characterized in that: The convolution engine's requirement for input data refers to the input data being transmitted in the form of picture pixel data in edge computing scenarios.

6. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The cumulative calculation comprises the following specific steps: The accumulator is initialized to zero, and each value of the sum of the element-by-element product of the weight of the convolution kernel of the edge computing scene and the image data is taken out one by one, and added to the accumulator in sequence to obtain the convolution result of the edge computing scene.

7. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The specific formula of the convolution result after activation is: ； In the formula, is the convolution result after activation, This is the convolution result of the edge computing scenario.

8. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The input feature map data of the edge computing scenario includes feature map data, weight data, pictures and feature values.

9. A general neural network accelerator optimization method for edge computing scenarios as claimed in claim 1, characterized in that: The input data of the edge computing scene includes input feature data, pixel data and weight data of the edge computing scene.

10. A general neural network accelerator optimization system for edge computing scenarios, characterized in that: include: The data access module is used to read the input feature map data of the edge computing scenario and perform convolution operations, sort the input data according to the requirements of the convolution engine to obtain the input data of the edge computing scenario, and then cache it; The convolution calculation module is used to load the input data of the edge computing scenario and rearrange it according to the requirements of the multiplication and addition queue arrangement, obtain the weights and image data of the convolution kernel of the edge computing scenario after rearrangement, perform multiplication and addition operations to obtain the multiplication and addition results of the edge computing scenario, perform cumulative calculations to output the convolution results of the edge computing scenario, use the activation function to calculate the activated convolution results, and finally obtain the target recognition results through target recognition.

Images