A system based on deep learning neural network hardware accelerator and a control method thereof

Abstract

The application belongs to the field of computer system science and artificial intelligence, and particularly relates to a system based on a deep learning neural network hardware accelerator and a control method thereof, which comprises a power supply, a computer system and a neural network hardware accelerator, an on-chip input buffer unit provides internal storage of input feature maps for a PE array, an on-chip weight buffer unit provides internal data storage of weights for the PE array, and an on-chip partial accumulation value buffer unit is responsible for storing partial accumulation values or final results calculated from the PE array. By introducing the on-chip buffer unit and the PE array, the system can efficiently process deep learning computing tasks. The on-chip buffer unit reduces frequent access to external memory, thereby improving computing speed and throughput. At the same time, the parallel execution capability of the PE array further accelerates the processing speed, so that the system can quickly process large-scale neural network models.

Description

Technical Field

[0001] This invention belongs to the fields of computer systems science and artificial intelligence, and particularly relates to a system and its control method based on a deep learning neural network hardware accelerator. Background Technology

[0002] In recent years, convolutional neural networks have made significant progress in the promotion and popularity of artificial intelligence, achieving remarkable results in tasks such as image recognition, image classification, and image reconstruction. However, with the continuous deepening of neural networks and the increase in input parameters, the system faces enormous challenges in terms of operating speed, throughput, memory access, and energy consumption. Therefore, hardware accelerators for deep learning neural networks that improve computing speed and reduce operating energy consumption have become crucial.

[0003] Currently, systems based on deep learning neural network hardware accelerators have emerged. These systems include a power supply, a computer system, and the neural network hardware accelerator, such as a Field Programmable Gate Array (FPGA) or an Application-Specific Integrated Circuit (ASIC). The power supply provides a stable power supply to the computer system and the neural network hardware accelerator, while the neural network hardware accelerator efficiently processes convolution operations and off-chip data read / write operations within the neural network running on the computer system.

[0004] By employing deep learning neural network hardware accelerators, the system can significantly improve computational speed, thereby accelerating the training and inference processes of neural networks. Simultaneously, the optimized design of the neural network hardware accelerators can effectively reduce energy consumption and improve system energy efficiency. This provides more powerful computational support for the application of deep learning in various fields, promoting the further development of artificial intelligence technology.

[0005] Existing related patents include the invention patent with patent number CN202010128655.2, entitled "A Hardware Accelerator for Training Neural Networks Using Sparsity in Deep Learning". The invention discloses a training accelerator and accelerated training method using a sparsity deep learning algorithm, which decomposes the computational process of model training into multiple one-dimensional discrete sparse convolution operations and the summation of their results.

[0006] For example, patent CN201910310573.7, entitled "Configurable Precision Convolutional Hardware Structure for Deep Learning Neural Network Hardware Accelerators," describes a configurable precision convolutional hardware structure for deep learning neural network hardware accelerators. The structure includes: the outputs of a first and second remapping module connected to the inputs of a first and second adder array, respectively; the outputs of the first and second adder arrays connected to the inputs of multiple unsigned multiplication units; the outputs of the multiple unsigned multiplication units connected to the input of a partial product shift-adder array; the output of the partial product shift-adder array connected to the input of a third remapping module; and the output of the third remapping module connected to the input of a third adder array. The first, second, and third remapping modules select the data path based on a precision selection signal. This invention allows for dynamic switching of computational precision, and the computational power of the corresponding neural network can meet the needs of most applications.

[0007] For example, the invention patent with patent number CN202111499692.5, entitled "A Reconfigurable Hardware Acceleration Architecture for Convolutional Neural Networks and Graph Convolutional Neural Networks," describes itself as follows: This invention belongs to the field of deep learning neural networks, specifically involving a reconfigurable hardware acceleration architecture for convolutional neural networks and graph convolutional neural networks. This invention utilizes configurable registers to configure and reconfigure the operations of both convolutional neural networks and graph convolutional neural networks; for convolutional neural network data, it uses a method of rearranging image blocks into matrix columns to convert three-dimensional convolution into two-dimensional matrix operations, allowing different neural network operations to reuse processing element array units, as well as data static storage control units and weight static storage control units, achieving resource reuse.

[0008] Among the aforementioned existing technologies, "A Hardware Accelerator for Training Neural Networks Using Sparsity in Deep Learning" proposes a neural network hardware accelerator and accelerated training method, but it has low data utilization and high energy consumption. While "A Convolutional Hardware Structure with Configurable Precision Suitable for Deep Learning Neural Network Hardware Accelerators" proposes an accelerator that can reuse arrays, it lacks specific analysis of data movement for various reuse methods, making it unable to make optimal judgments and choices for various needs. "A Reconfigurable Hardware Acceleration Architecture for Convolutional Neural Networks - Graph Convolutional Neural Networks" only considers the optimization method for image convolution, without considering the overall data processing of convolution operations. Summary of the Invention

[0009] To reduce data movement involving memory read / write operations during convolution operations, avoid limitations imposed by buffer size, and reduce power consumption of chips such as FPGAs or ASICs during operation, this invention proposes a system and its control method based on a deep learning neural network hardware accelerator.

[0010] To achieve the above effects, the technical solution of this application is as follows:

[0011] A system based on a deep learning neural network hardware accelerator includes a power supply, a computer system, and a neural network hardware accelerator. The power supply powers the computer system, which in turn powers the neural network hardware accelerator, which interacts with the computer system. The neural network hardware accelerator includes off-chip memory, an on-chip input buffer unit, an on-chip weight buffer unit, an on-chip partial accumulation buffer unit, a computation unit, and a PE array. The on-chip input buffer unit provides internal storage for the input feature maps of the PE array; the on-chip weight buffer unit provides internal data storage for the weights of the PE array; and the on-chip partial accumulation buffer unit stores partial accumulation values ​​or the final result calculated by the PE array.

[0012] Furthermore, the power supply is a PC power supply.

[0013] Furthermore, the neural network hardware accelerator is an image processing unit, an FPGA, or an ASIC.

[0014] Furthermore, the PE array includes multiple arithmetic logic units or vector processing units.

[0015] Furthermore, the arithmetic logic unit includes a multiplier and an adder.

[0016] Furthermore, neural network hardware accelerators can execute multiple data stream modes, including: fixed output, fixed feature map, and fixed weights.

[0017] In the fixed data stream output mode, the calculation results of the processing unit are accumulated in the output buffer unit, and some of the accumulated values ​​are stored in the output buffer unit. The final output result is obtained by accumulating these partially accumulated values.

[0018] In the feature map data stream mode, the input feature map and weights are divided into multiple rows, and the processing unit independently computes the result for each row.

[0019] In a fixed-weight data stream model, weights are stored statically, and each processing unit independently uses the same weights for computation.

[0020] A control method for a hardware accelerator system based on deep learning neural networks includes the following steps:

[0021] Step 1) Obtain computer hardware resource conditions;

[0022] Step 2) Input the convolution operation parameters;

[0023] The parameters include the size and number of convolutional images, the size and number of filters, the size of the output feature image, the size of the buffer unit, the size of one convolution operation, the number of operations, and the convolution stride;

[0024] Step 3) Calculate the total amount of data movement under the three strategies of fixed output, fixed feature map, and fixed weight;

[0025] Step 4) Compare the total amount of data movement for the three strategies;

[0026] Step 5) If the total amount of data movement is the lowest under the fixed output strategy, then the control calculation strategy is fixed output; if the total amount of data movement is the lowest under the fixed weight strategy, then the control calculation strategy is fixed weight; if the total amount of data movement is the lowest under the fixed feature map strategy, then the control calculation strategy is fixed feature map.

[0027] Further, step 2) specifically involves:

[0028] Obtain feature data of the input image, including the height H of the input image. i Width W i and the number of channels C i ;

[0029] Obtain feature data of the output image, including the height H of the output image. o Width W o and the number of channels C o .

[0030] Obtain the feature data of the filter, including the height H of the weights. f and width W f ;

[0031] Obtain the input buffer size and weight buffer size S. i S w

[0032] Get the height H of a single convolution C Width W C The number of input channels N in one operation i And the number of filters N for a single input f To obtain the number of operations N allowed due to the size limitation of the buffer unit. h ,in:

[0033] Si ≥H C ×W C ×N h S w ≥H f ×W f ×N f

[0034] Get the stride S of the convolution.

[0035] Further, step 3) specifically involves:

[0036] Calculate the data shifts for the feature map and weights, Fmap data and Weight data, respectively;

[0037] In a computational strategy with a fixed output, the data shift calculation formulas for feature map (Fmap data) and weight data are as follows:

[0038]

[0039]

[0040] Among them, H o W o These are the height and width of the output image, H. C W C These are the height and width of a single convolution operation, H. f W f These are the height and width of the weights, respectively; S is the stride of the convolution operation; and C... i and C o These are the number of channels in the input and output images, respectively, N. i N f and N h These represent the number of input channels, the number of input filters, and the number of operations allowed in a single convolution operation, respectively. The Roundup function rounds up, and the Rounddown function rounds down. When the convolution stride S ≥ 2, the Roundup function in the calculation formula should be replaced with the Rounddown function.

[0041] In a computation strategy with a fixed feature map, the data movement calculation formulas for the feature map (Fmap data), weights (Weight data), and partial accumulated values ​​(Psum data) are as follows:

[0042]

[0043]

[0044]

[0045] In the fixed-weight calculation strategy, the data movement calculation formula for the feature map is the same as formula (1), and the calculation formula for the partial accumulated value is the same as formula (5). The data movement calculation formula for the weights is as follows:

[0046] Weight data = C i ×C o ×H f ×W f (6)

[0047] Among them, H o W o These are the height and width of the output image, H. C W C These are the height and width of a single convolution operation, H. f W f These are the height and width of the weights, respectively; S is the stride of the convolution operation; and C... i and C o These are the number of channels in the input and output images, respectively, N. i N f and represent the number of input channels and the number of input filters for a single convolution operation, respectively;

[0048] Under normal operating conditions, the convolution stride, the height and width S and H of the input feature image and the filter are... i W i and H f W f The following conditions must be met:

[0049] H i ≥H f ≥S,W i ≥W f ≥S

[0050] The height and width H of the input and output feature images i W i and H o W o The following conditions must be met:

[0051] H i >H o W i >W o .

[0052] Furthermore, in step 5)

[0053] When some accumulated values ​​are set to zero, the computation strategy of the control neural network hardware accelerator is to fix the output.

[0054] When the size of the input buffer unit is limited by physical resources, the computation strategy for controlling the neural network hardware accelerator is to fix the feature map.

[0055] When the size of the weight buffer unit is limited by physical resources, the computing strategy for controlling the neural network hardware accelerator is to fix the weights.

[0056] The advantages of this application are:

[0057] 1. This application focuses on analyzing the impact of data access strategies on convolutional neural networks, and realizes low-energy convolutional neural network operations for deep learning, significantly reducing energy consumption during convolution calculation.

[0058] 2. By introducing on-chip buffer units and PE arrays, the system can efficiently handle deep learning computation tasks. On-chip buffer units reduce frequent access to external memory, thereby improving computation speed and throughput. Simultaneously, the parallel execution capability of the PE array further accelerates processing speed, enabling the system to quickly process large-scale neural network models.

[0059] 3. The system of this invention has flexible configuration capabilities and can adapt to neural network models of different sizes and complexities. By adjusting input parameters and processing hardware parameters such as the size and capacity of the PE array and buffer units, the system can meet the needs of different application scenarios, such as high parallelism or low memory requirements. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of a system structure based on a neural network hardware accelerator according to the present invention.

[0061] Figure 2 This is a schematic diagram of a neural network hardware accelerator according to an embodiment of the present invention.

[0062] Figure 3 This is a schematic diagram illustrating the implementation process of a control method according to an embodiment of the present invention. Detailed Implementation

[0063] The technical solution of the present invention will be further described below with reference to the accompanying drawings in the embodiments.

[0064] Example 1

[0065] A system based on a deep learning neural network hardware accelerator includes a power supply, a computer system, and a neural network hardware accelerator. The power supply powers the computer system, which in turn powers the neural network hardware accelerator, which interacts with the computer system. The neural network hardware accelerator includes off-chip memory, an on-chip input buffer, an on-chip weight buffer, an on-chip partial summation buffer, a computation unit, and a PE array (Processing Element, PE Array). The on-chip input buffer provides internal storage for the input feature maps of the PE array; the on-chip weight buffer provides internal data storage for the weights (filters) of the PE array; and the on-chip partial summation buffer stores partial summations or the final result calculated by the PE array.

[0066] The power supply is a PC power supply.

[0067] Neural network hardware accelerators are image processing units (GPUs), FPGAs, or ASICs.

[0068] The PE array comprises multiple arithmetic logic units (ALUs) or vector processing units. The arithmetic logic units include multipliers and adders.

[0069] Convolutional neural networks consist of multiple convolutional layers, and the dimensions of various parameters required for convolution operations are determined by the input image, output image, and physical hardware resources. Different computational strategies for neural network hardware accelerators result in variations in memory read / write operations, data flow, and ultimately, energy consumption.

[0070] Neural network hardware accelerators can execute multiple dataflow modes, including but not limited to: Output Stationary, Feature Map Stationary, and Weight Stationary. The application of these dataflow modes in deep learning neural network hardware accelerators helps improve computational efficiency, reduce power consumption, and accelerate the execution of deep learning tasks.

[0071] In fixed-data-flow output mode, the computation results of the processing unit are accumulated in the output buffer unit instead of being output immediately. Partial sums are stored in the output buffer unit, and the final output result is obtained by summing these partial sums. Static-data-flow output mode can reduce data transmission and output latency, improving the system's computational efficiency.

[0072] In the feature map dataflow model, the input feature map and weights are divided into multiple rows, and the processing unit independently computes the result for each row. This dataflow model leverages the locality of data, reduces communication and computational overhead between processing units, and improves the efficiency of parallel computing.

[0073] In a fixed-weight dataflow model, weights are stored statically, and each processing unit independently calculates using the same weights. Weight updates only require modification at a single location, eliminating the need for partial summation. This dataflow model reduces communication and computational overhead between processing units, thereby improving computational efficiency.

[0074] Example 2

[0075] A control method for a hardware accelerator system based on deep learning neural networks includes the following steps:

[0076] Step 1) Obtain computer hardware resource conditions;

[0077] Step 2) Input the convolution operation parameters;

[0078] The parameters include the size and number of convolutional images, the size and number of filters, the size of the output feature image, the size of the buffer unit, the size of one convolution operation, the number of operations, and the convolution stride;

[0079] Step 3) Calculate the total amount of data movement under the three strategies of fixed output, fixed feature map, and fixed weight;

[0080] Step 4) Compare the total amount of data movement for the three strategies;

[0081] Step 5) If the total amount of data movement is the lowest under the fixed output strategy, then the control calculation strategy is fixed output; if the total amount of data movement is the lowest under the fixed weight strategy, then the control calculation strategy is fixed weight; if the total amount of data movement is the lowest under the fixed feature map strategy, then the control calculation strategy is fixed feature map.

[0082] Step 2) specifically involves:

[0083] Obtain feature data of the input image, including the height H of the input image. i Width W i and the number of channels Ci ;

[0084] Obtain feature data of the output image, including the height H of the output image. o Width W o and the number of channels C o .

[0085] Obtain the feature data of the filter, including the height H of the weights. f and width W f ;

[0086] Obtain the input buffer size and weight buffer size S. i S w

[0087] Get the height H of a single convolution C Width W C The number of input channels N in one operation i And the number of filters N for a single input f To obtain the number of operations N allowed due to the size limitation of the buffer unit. h ,in:

[0088] S i ≥H C ×W C ×N h S w ≥H f ×W f ×N f

[0089] Get the stride S of the convolution.

[0090] Step 3) specifically involves:

[0091] Calculate the data shifts for the feature map and weights, Fmap data and Weight data, respectively;

[0092] In a computational strategy with a fixed output, the data shift calculation formulas for feature map (Fmap data) and weight data are as follows:

[0093]

[0094]

[0095] Among them, H o W o These are the height and width of the output image, H. C W C These are the height and width of a single convolution operation, H. f W fThese are the height and width of the weights, respectively; S is the stride of the convolution operation; and C... i and C o These are the number of channels in the input and output images, respectively, N. i N f and N h These represent the number of input channels, the number of input filters, and the number of operations allowed in a single convolution operation, respectively. The Roundup function rounds up, and the Rounddown function rounds down. When the convolution stride S ≥ 2, the Roundup function in the calculation formula should be replaced with the Rounddown function.

[0096] In a computation strategy with a fixed feature map, the data movement calculation formulas for the feature map (Fmap data), weights (Weight data), and partial accumulated values ​​(Psum data) are as follows:

[0097]

[0098]

[0099]

[0100] In the fixed-weight calculation strategy, the data movement calculation formula for the feature map is the same as formula (1), and the calculation formula for the partial accumulated value is the same as formula (5). The data movement calculation formula for the weights is as follows:

[0101] Weight data = C i ×C o ×H f ×W f (6)

[0102] Among them, H o W o These are the height and width of the output image, H. C W C These are the height and width of a single convolution operation, H. f W f These are the height and width of the weights, respectively; S is the stride of the convolution operation; and C... i and C o These are the number of channels in the input and output images, respectively, N. i N f and represent the number of input channels and the number of input filters for a single convolution operation, respectively;

[0103] Under normal operating conditions, the convolution stride, the height and width S and H of the input feature image and the filter are... iW i and H f W f The following conditions must be met:

[0104] H i ≥H f ≥S,W i ≥W f ≥S

[0105] The height and width H of the input and output feature images i W i and H o W o The following conditions must be met:

[0106] H i >H o W i >W o .

[0107] In step 5)

[0108] When some accumulated values ​​are set to zero, the computation strategy of the control neural network hardware accelerator is to fix the output.

[0109] When the size of the input buffer unit is limited by physical resources, the computation strategy for controlling the neural network hardware accelerator is to fix the feature map.

[0110] When the size of the weight buffer unit is limited by physical resources, the computing strategy for controlling the neural network hardware accelerator is to fix the weights.

[0111] Example 3

[0112] like Figure 1 As shown, a system based on a deep neural network hardware accelerator includes a central processing unit (CPU), memory, a neural network hardware accelerator, and off-chip storage. This system achieves efficient data transfer and collaborative operation to improve the speed and efficiency of deep learning computation. During data transfer, the CPU communicates with memory through a dedicated interface, enabling rapid data read and write operations. The CPU loads parameters, input data, and instructions of the deep learning model from memory and writes the computation results back to memory. The neural network hardware accelerator communicates with the CPU through an interface, receiving instructions and data from the CPU. The neural network hardware accelerator executes computation tasks according to specified algorithms and parameters and returns the computation results to the CPU. The neural network hardware accelerator interacts with off-chip storage through a dedicated interface. Off-chip storage is used to store large-scale data, such as training datasets and model weights. The neural network hardware accelerator can read the required data from off-chip storage and write the computation results back to it.

[0113] A control method for a system based on a deep neural network hardware accelerator includes the following steps:

[0114] Input image feature data, including the height H of the input image i Width W i and the number of channels C i The total number of pixels in the image is H. i ×W i ×C i ;

[0115] Feature data of input and output images, including the height H of the output image. o Width W o and the number of channels C o The total number of pixels in the image is H. o ×W o ×C o ;

[0116] Input the feature data of the weights, including the height H of the weights. f and width W f The weight is H f ×W f ;

[0117] Input the height H of a single convolution C Width W C The number of input channels N in one operation i And the number of filters N for a single input f Due to the size limitation of the input buffer, the number of operations N allowed is limited. h ;

[0118] Get the stride S of the convolution;

[0119] The value of S is generally [1, 3], and S is an integer;

[0120] Based on formulas (1)-(6), the data movement involved in the convolution process under different computation strategies is calculated, and the total data movement is obtained:

[0121] Total data=fmap data+weight data+Psum

[0122] Among them, the total amount of data involved in the fixed output strategy is d1, the total amount of data involved in the fixed feature map strategy is d2, and the total amount of data involved in the fixed weight strategy is d3.

[0123] If the requirement for partial sum is zero, the control calculation strategy is to keep the output fixed.

[0124] This patent only considers the optimal energy consumption for convolutional computation in deep learning. In actual operation, the control strategy is adjusted according to different requirements for hardware and software resources, computational efficiency, response time, and energy consumption.

[0125] The implementation flow of a system control method based on a deep learning neural network hardware accelerator provided in this embodiment is as follows: Figure 3 As shown:

[0126] When the total data flow satisfies d1≤min(d2,d3), the computation strategy for controlling the neural network hardware accelerator is to fix the output, at which point the data transmission cost of convolution computation is the lowest.

[0127] When the total data flow satisfies d2≤min(d1,d3), the computation strategy for controlling the neural network hardware accelerator is to fix the feature map, at which point the data transmission cost of convolution computation is the lowest.

[0128] When the total data flow satisfies d3≤min(d1,d2), the computation strategy for controlling the neural network hardware accelerator is to fix the weights, at which point the data transmission cost of convolution computation is the lowest.

[0129] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A control method for a system based on a deep learning neural network hardware accelerator, characterized in that: The system includes a power supply, a computer system, and a neural network hardware accelerator. The power supply powers the computer system, which in turn powers the neural network hardware accelerator, which interacts with the computer system. The neural network hardware accelerator includes off-chip memory, an on-chip input buffer unit, an on-chip weight buffer unit, an on-chip partial accumulation buffer unit, an arithmetic unit, and a PE array. The on-chip input buffer unit provides internal storage for the input feature maps of the PE array; the on-chip weight buffer unit provides internal data storage for the weights of the PE array; and the on-chip partial accumulation buffer unit stores partial accumulation values ​​or the final result calculated by the PE array. Neural network hardware accelerators can execute multiple data stream modes, including: fixed output, fixed feature map, and fixed weights; In the fixed data stream output mode, the calculation results of the processing unit are accumulated into the on-chip partial accumulation buffer unit. The partial accumulation values ​​are stored in the on-chip partial accumulation buffer unit, and the final output result is obtained by accumulating these partial accumulation values. In the fixed data flow mode of feature maps, the input feature maps and weights are divided into multiple row blocks, and the processing unit independently computes the result of each row block; In a fixed-weight data stream model, weights are stored statically, and each processing unit independently uses the same weights for computation. The method includes the following steps: Step 1) Obtain computer hardware resource conditions; Step 2) Input the convolution operation parameters; The parameters include the size and number of convolutional images, the size and number of filters, the size of the output feature image, the size of the buffer unit, the size of one convolution operation, the number of operations, and the convolution stride; Step 3) Calculate the total amount of data movement under the three strategies of fixed output, fixed feature map, and fixed weight; Step 4) Compare the total amount of data movement for the three strategies; Step 5) If the total amount of data movement is the lowest under the fixed output strategy, then the control calculation strategy is fixed output; if the total amount of data movement is the lowest under the fixed weight strategy, then the control calculation strategy is fixed weight; if the total amount of data movement is the lowest under the fixed feature map strategy, then the control calculation strategy is fixed feature map. Step 2) specifically involves, Obtain feature data of the input image, including the height of the input image. ,width and number of channels ; Obtain the feature data of the output feature image, including the height of the output feature image. ,width and number of channels ; Obtain the feature data of the filter, including the height of the weights. and width ; Get the on-chip input buffer size and the on-chip weight buffer size. ; Get the height of a single convolution ,width Number of channels for one input and the number of filters in one input To obtain the number of operations allowed due to the size limitation of the buffer unit. ,in: Get the stride of the convolution. ; Step 3) specifically involves: Calculate the data shifts for feature maps and weights separately. ; In a computational strategy with a fixed output, the feature map Fmap data and weight Weight data The data movement calculation formulas are as follows: in, , These are the height and width of the output feature image, respectively. , These are the height and width for performing one convolution operation, respectively. , These are the height and width of the weights, respectively. ， It is the stride of this convolution operation. and These are the number of channels in the input image and the output feature image, respectively. , and These represent the number of input channels, the number of input filters, and the number of operations allowed in a single convolution operation, respectively. The Roundup function rounds up, and the Rounddown function rounds down. (The last two characters, "when convolution stride," appear to be unrelated to the preceding text and are likely data points from a different context.) satisfy In step 2, the Roundup function in the calculation formula should be replaced with the Rounddown function; In a computational strategy where the feature map is fixed, the feature map Fmap data Weight Weight data and partial accumulated values Psum data The data movement calculation formulas are as follows: In the fixed-weight calculation strategy, the data movement calculation formula for the feature map is the same as formula (1), and the calculation formula for the partial accumulated value is the same as formula (5). The data movement calculation formula for the weights is as follows: in, , These are the height and width of the output feature image, respectively. , These are the height and width for performing one convolution operation, respectively. , These are the height and width of the weights, respectively. ， It is the stride of this convolution operation. and These are the number of channels in the input image and the output feature image, respectively. , and represent the number of input channels and the number of input filters for a single convolution operation, respectively; Under normal operating conditions, the convolution stride, the height and width of the input image and the filter are... , and The following conditions must be met: Height and width of input image and output feature image and The following conditions must be met: 。 2. The control method for a system based on a deep learning neural network hardware accelerator according to claim 1, characterized in that: The power supply is a PC power supply.

3. The control method for a system based on a deep learning neural network hardware accelerator according to claim 1, characterized in that: The neural network hardware accelerator is an image processing unit, FPGA, or ASIC.

4. The control method for a system based on a deep learning neural network hardware accelerator according to claim 1, characterized in that: The PE array includes multiple arithmetic logic units or vector processing units.

5. The control method for a system based on a deep learning neural network hardware accelerator according to claim 4, characterized in that: The arithmetic logic unit includes a multiplier and an adder.

6. The control method for a system based on a deep learning neural network hardware accelerator according to claim 1, characterized in that: Step 5) When some accumulated values ​​are set to zero, the computation strategy for controlling the neural network hardware accelerator is to fix the output. When the size of the on-chip input buffer unit is limited by physical resources, the computation strategy for controlling the neural network hardware accelerator is to fix the feature map. When the size of the on-chip weight buffer unit is limited by physical resources, the computation strategy for controlling the neural network hardware accelerator is to fix the weights.

Images