FPGA-based adaptive cnn inference acceleration system with collaborative pruning and quantization

Abstract

The embodiment of the application relates to the technical field of computer systems based on specific calculation models, and discloses a FPGA-based adaptive CNNs inference acceleration system utilizing pruning and quantization in cooperation, which comprises a user input setting module and a convolution acceleration calculation module, wherein the user input setting module accepts user input control parameters; the convolution acceleration calculation module sets a filter pruning rate, fixed-point parameter quantization and parallel computing unit number of a CNNs model according to the control parameters, is realized based on FPGA, and comprises a hybrid pruning module, a parameter quantization optimization module and a double-buffering calculation module. Compared with the prior art, the application can select different pruning rate thresholds according to the accuracy and real-time requirements of actual neural network applications based on the configurability and pipeline parallelism characteristics of FPGA, prune part of filters in the CNNs model, thereby compressing the CNNs model, and reducing the convolution calculation of the CNNs model as a whole.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer system technology based on specific computational models, and particularly to computer systems based on biological models. Background Technology

[0002] With the convergence of the Internet of Things (IoT) and mobile edge computing, the scale and types of data have exploded, forming massive real-time data. In intelligent applications of this data, deep convolutional networks can adaptively learn and abstract data features in an end-to-end manner for specific identification or detection tasks, thereby making decisions. Therefore, deep neural networks have achieved significant application results in fields such as image classification, speech recognition, object detection, video analysis, and natural language processing. To improve the accuracy of neural network models, deep neural networks are increasingly complex, with more and more layers, parameters, and computational demands. This places higher challenges and requirements on hardware computing power, memory bandwidth, and data storage. Especially with the development of IoT and mobile edge computing applications, excessive parameters and overly complex models in deep neural networks can reduce the efficiency of intelligent applications, hindering the practical application of neural networks in edge computing. Therefore, there is a greater demand for real-time performance in the inference stage of neural network applications, making the acceleration of the neural network inference stage increasingly important.

[0003] In neural network intelligence applications with limited hardware computing resources, building more real-time neural network applications and effectively balancing inference speed and accuracy remains a significant challenge. Especially with the development of System-on-Chip (SoC) and the emergence of AI chips, mobile edge devices have gradually acquired strong computing and storage capabilities. Neural network inference tasks that previously required cloud computing can now be performed on mobile edge devices. Performing neural network inference computations directly on mobile edge devices (such as drones and autonomous vehicles) avoids the network latency required for large-scale data transmission to the cloud and mitigates the privacy risks associated with cloud-based image recognition applications. Therefore, exploring ways to accelerate the inference speed of deep neural networks in edge computing neural network intelligence applications with limited hardware computing resources has significant theoretical and practical application value.

[0004] Currently, hardware acceleration research includes CPUs, GPUs, FPGAs, and ASICs. CPUs are resource-constrained sequential execution systems, resulting in low execution efficiency for computationally intensive applications such as convolutional operations in deep neural networks, thus limiting their use for accelerating deep neural networks. GPUs, processors composed of massively parallel, smaller, and more specialized cores, are widely used as hardware accelerators in the training phase of deep neural networks. However, the low energy efficiency of GPUs limits their use in accelerating inference tasks for deep neural network applications in edge computing systems.

[0005] Convolutional Neural Networks (CNNs) are a type of deep neural network that incorporates convolutional computations and represents a typical algorithm in deep learning. A CNN model consists of many neural network layers. Due to the unique design of its structure, CNNs have surpassed many advanced algorithms in fields ranging from text and video to speech processing, far exceeding their initial applications in image processing. Therefore, CNNs are widely used in computer vision, natural language processing, and other fields.

[0006] The computation of convolutional layers in neural networks requires numerous convolution operations, which are computationally intensive tasks that consume significant computing resources. To accelerate the overall performance of convolutional neural networks, several methods have been explored to reduce computation, parameter and memory transfers in convolution operations, as well as to parallelize these operations. However, current accelerators are typically based on a fixed computational pattern or a single-layer optimization method, failing to adequately address the trade-off between inference speed and accuracy in convolutional neural networks when dealing with networks of varying sizes and application requirements.

[0007] The inventors have discovered that the prior art has at least the following problems: it cannot adequately adapt to the trade-off between inference speed and accuracy of convolutional neural networks when dealing with the needs of networks and applications of different sizes. Summary of the Invention

[0008] The purpose of this invention is to provide an adaptive CNNs inference acceleration system based on FPGA that utilizes pruning and quantization in a collaborative manner, which can effectively match the trade-off between accuracy and speed of convolutional neural networks, so as to better meet the needs of intelligent applications in edge computing.

[0009] To address the aforementioned technical problems, this invention provides an FPGA-based adaptive CNN inference acceleration system that collaboratively utilizes pruning and quantization, comprising a user input setting module and a convolution acceleration computation module, wherein...

[0010] The user input settings module accepts user input control parameters;

[0011] The convolution acceleration computation module sets the filter pruning rate, fixed-point parameter quantization, and number of parallel computing units of the CNN model according to the control parameters. It is implemented based on FPGA and includes a hybrid pruning module, a parameter quantization optimization module, and a double-buffered computation module.

[0012] Therefore, based on the configurability of FPGA, the inference process of CNNs can be effectively and adaptively accelerated, achieving an adaptive trade-off between accuracy, speed and resources. It is easy to set up multiple parallel computing units to increase the parallelism of convolution calculations. At the same time, the number of computing units can be set to achieve scalable performance, improve the inference speed of CNN models, and meet the real-time requirements of neural network applications on edge devices.

[0013] The control parameters specifically include: accuracy threshold level, real-time configuration parameters, and CNN model parameters.

[0014] The hybrid pruning module employs the L1-norm and ApoZ algorithms.

[0015] like Figure 2 As shown, the L1-norm and ApoZ algorithm first calculates the sum of the absolute values ​​of the filters using the L1-norm method, then sorts them using the ApoZ method and prunes the filters according to the set pruning rate.

[0016] This allows the pruning method to be effectively combined with the FPGA platform for application in intelligent edge computing applications.

[0017] The parameter quantization optimization module changes the convolution calculation in the CNN model to a fixed-point calculation method based on fixed-point quantization bit values; the fixed-point quantization bit values ​​are 8, 16, 32 or 64 bits.

[0018] Therefore, fixed-point computation can effectively reduce the storage space and number of floating-point operations in the CNN model operation process, thereby reducing hardware overhead and memory bandwidth requirements.

[0019] The fixed-point quantization bit value is configured by setting parameters.

[0020] The double-buffered computation module uses multiple parallel computation units to compute the CNN model, and sets buffers at both the input and output of the computation units to achieve double buffering of input and output. This increases the parallelism of convolution computation, improves the overall execution performance of the system, and accelerates convolution operations by utilizing FPGA optimization mechanisms such as data flow, pipeline, and loop unrolling.

[0021] The buffer employs a ping-pong data transmission mechanism. This allows the time spent on data computation to mask the time spent on data transmission, significantly reducing the time spent solely on loading data and improving overall speed.

[0022] The product acceleration module consists of a convolution model accelerator connected to the CPU. The convolution model accelerator is based on an FPGA and consists of cascaded multiple modules. The cascaded multiple modules include a memory read module, a core multiply-accumulate module, a pooling module, a write-back module, and a normalization module, wherein:

[0023] The memory read module reads data from the global memory of the convolution acceleration calculation module;

[0024] The core multiply-accumulate module performs the multiply-accumulate operations of convolution;

[0025] The pooling module performs the pooling operation for the convolution;

[0026] The write-back module writes the calculation results back to the global memory of the convolution acceleration calculation module;

[0027] The normalization module performs the normalization operation on CNN models.

[0028] This also facilitates the acceleration of kernel functions for convolution operations in CNN models using optimization mechanisms such as FPGA's Loop pipeline, Loop Unrolling, and dataflow.

[0029] The convolutional model accelerator and the CPU are connected via a PCIe interface for data transmission and communication.

[0030] Compared with existing technologies, this invention leverages the configurability and pipelined parallelism of FPGAs to select different pruning rate thresholds based on the accuracy and real-time requirements of actual neural network applications. This allows for the removal of some filters from CNN models, thereby compressing the CNN models and reducing the overall convolutional computation. Furthermore, it reduces storage space and floating-point operations during CNN model computation and increases convolutional parallelism, thus effectively and significantly improving the inference speed of CNN models.

[0031] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0032] One or more embodiments are illustrated by way of example with the corresponding pictures in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings do not constitute a limitation on scale.

[0033] Figure 1 This is a schematic diagram of the module arrangement in at least one embodiment of the present invention;

[0034] Figure 2 yes Figure 1 A schematic diagram illustrating the computational principle of the L1-norm and ApoZ algorithms;

[0035] Figure 3 This is a comparison chart of the evaluation test of the pruning method in one embodiment of the present invention;

[0036] Figure 4 This is a comparison chart of the accuracy of different pruning rates for each layer of VGG16 in one embodiment of the present invention;

[0037] Figure 5 This is a comparison chart of the accuracy of different pruning rates of each layer of ResNet50 in one embodiment of the present invention;

[0038] Figure 6 This is a graph showing the performance evaluation of CNN parameter quantization in one embodiment of the present invention;

[0039] Figure 7 This is a performance comparison chart of CNNs models under multiple computational units in one embodiment of the present invention;

[0040] Figure 8 This is a graph showing the comprehensive performance evaluation results of three CNNs accelerators in one embodiment of the present invention;

[0041] Figure 9 This is a schematic diagram of the module setup of the convolution acceleration calculation module in at least one embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the various embodiments of this invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this invention to facilitate a better understanding of this application. However, the technical solutions claimed in this application can be implemented even without these technical details and with various variations and modifications based on the following embodiments. The division of the various embodiments below is for ease of description and should not constitute any limitation on the specific implementation of this invention. The various embodiments can be combined with and referenced by each other without contradiction.

[0043] The first embodiment of the present invention relates to an adaptive CNN inference acceleration system based on FPGA that collaboratively utilizes pruning and quantization, including a user input setting module and a convolution acceleration computation module, wherein,

[0044] The user input settings module accepts user input control parameters;

[0045] The convolution acceleration computation module sets the filter pruning rate, fixed-point parameter quantization, and number of parallel computing units of the CNN model according to the control parameters. It is implemented based on FPGA and includes a hybrid pruning module, a parameter quantization optimization module, and a double-buffered computation module.

[0046] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0047] Generally, CNN models consist of a combination of convolutional layers, pooling layers, and fully connected layers. The principles behind these techniques are well-described in existing technologies and will not be repeated here. The complexity of CNN models can be categorized into the number of parameters and computational cost. The number of parameters refers to the sum of the number of parameters that need to be learned, including the weights of the convolutional kernels, the weights of the fully connected layers, and other weights. The computational cost typically refers to the number of multiplication and addition operations performed during the forward inference of the CNN model. Since multiplication and addition operations are implemented using floating-point arithmetic, floating-point operations (FLOPs) are generally used to measure computational cost. Based on a detailed analysis of CNN models, the relationship between the input layer, convolutional kernels, and output layer, as well as the relationship between the number of parameters and computational cost, can be easily derived. Analysis of the computational cost of the convolutional and fully connected layers in CNN models reveals that the computational cost is mainly concentrated in the convolutional layers. Furthermore, it is known that CNN models with more convolutional layers have more parameters and higher computational cost. Therefore, accelerating the convolutional operations of the convolutional layers is a key consideration.

[0048] Furthermore, the control parameters specifically include: accuracy threshold level, real-time configuration parameters, and CNN model parameters.

[0049] The second embodiment of the present invention is largely the same as the first embodiment, mainly in the preferred scheme of the hybrid pruning module, which adopts the L1-norm and ApoZ algorithm.

[0050] Furthermore, the L1-norm and ApoZ algorithm first uses the L1-norm method to calculate the sum of the absolute values ​​of the filters, then uses the ApoZ method to sort them and prune the filters according to the set pruning rate.

[0051] Deep CNN models contain a large number of redundant parameters from convolutional layers to fully connected layers, with many neurons having activation values ​​close to 0. Removing these neurons can achieve the same model expressive power; this technique is called model pruning, as described above. Model pruning techniques can be broadly categorized into four granularity levels: element-level, vector-level, kernel-level, and filter-level. The technical solution in this invention employs filter-level pruning, which essentially prunes the entire convolutional kernel group, resulting in a change in the number of output feature channels during inference.

[0052] The core of filter pruning is determining the importance of filters. L1-norm is when p=1, i.e. This represents the sum of the absolute values ​​of all elements in a vector (matrix). The smaller the absolute value of a filter's weight, the less impact it has. Therefore, filters with small absolute weights can be pruned based on a pruning threshold. Considering the uneven distribution of weights in each layer of a neural network, the average percentage of zero values ​​(APoZ) is used to further measure the importance of filters. APoZ is determined by the number of zeros in the filter layer; a higher APoZ indicates higher redundancy and can be pruned. Generally, APoZ is calculated as follows: Here, f(i,j)==0 is used to determine whether the value at position (i,j) in the filter is 0; if it is 0, it is incremented by 1. Based on this, after obtaining the importance of filters in each convolutional layer, the filter with the smallest weight (r%) is removed according to the pruning rate r of each convolutional layer. This effectively removes some filters without affecting the final result, greatly reducing the amount of computation and improving the computation speed.

[0053] The essence of this implementation is to use the L1-norm method as a criterion to determine the importance of the filter, and then remove the less important ones.

[0054] The third embodiment of the present invention is largely the same as the first embodiment, mainly in the preferred scheme of the parameter quantization optimization module. The parameter quantization optimization module changes the convolution calculation in the CNN model to a fixed-point calculation method based on the fixed-point quantization bit value; the fixed-point quantization bit value is 8, 16, 32 or 64 bits.

[0055] Therefore, replacing the float type multiplication and addition calculations in CNNs models with int type allows for more calculations to be performed in batches, thereby effectively improving the computational performance of CNNs inference. In other words, implementing a low-bit fixed-point computing architecture instead of a floating-point computing architecture reduces the storage space and number of floating-point operations in the CNNs model operation process, thereby reducing hardware overhead and memory bandwidth requirements.

[0056] Furthermore, the fixed-point quantization bit width is configurable through parameter settings. The design of the fixed-point quantization bit width achieves a trade-off between performance and speed for neural network applications by sacrificing only a small portion of model accuracy and selecting configurable fixed-point quantization parameters. Specifically, it optimizes both CNN model parameters and intermediate computation results with low-bit fixed-point quantization, while allowing the configurable low-bit width of the fixed-point quantization to be set for each convolutional layer of the CNN model.

[0057] The fourth embodiment of the present invention is largely the same as the first embodiment, mainly differing in the preferred scheme of the double-buffered computing module. The double-buffered computing module uses multiple parallel computing units to compute the CNN model, and sets buffers at both the input and output of the computing units to achieve double buffering of input and output. Parallel computing based on FPGA has inherent advantages, and the double-buffered approach effectively improves the overall efficiency of parallel computing.

[0058] Furthermore, the buffer employs a ping-pong data transmission mechanism. Among double-buffered methods, the ping-pong data transmission mechanism is most suitable for parallel computation of CNN models. This mechanism, implemented using FPGA, is a unique data transmission mechanism that uses a rhythmic, coordinated switching between the "input data selection unit" and the "output data selection unit" to continuously send the buffered data stream to the "data stream processing module" for computation and processing. In the computation of CNN models, parallel convolution calculations are performed using data stream processing, maximizing computational efficiency through double buffering.

[0059] The fourth embodiment of the present invention is largely the same as the first embodiment, mainly differing in the preferred scheme for accelerating convolution computation, such as... Figure 9 As shown, the convolution acceleration computation module consists of a convolution model accelerator connected to the CPU. The convolution model accelerator is based on an FPGA and consists of cascaded multiple modules. The cascaded multiple modules include a memory read module, a core multiply-accumulate module, a pooling module, a write-back module, and a normalization module, wherein:

[0060] The memory read module reads data from the global memory of the convolution acceleration calculation module;

[0061] The core multiply-accumulate module performs the multiply-accumulate operations of convolution;

[0062] The pooling module performs the pooling operation for the convolution;

[0063] The write-back module writes the calculation results back to the global memory of the convolution acceleration calculation module;

[0064] The normalization module performs the normalization operation on CNN models.

[0065] The deep pipeline and dataflow characteristics of FPGAs are key to parallel acceleration of computationally intensive operations. CNN models, including convolutional, pooling, and fully connected layers, have a deep pipeline structure, which is suitable for acceleration using the pipeline and dataflow mechanisms of FPGAs. However, the multiply-accumulate operations of the convolutional layers in CNN models are computationally intensive tasks. Therefore, using loop unrolling optimization based on CPU kernel mechanisms can achieve better acceleration results compared to simply using FPGA kernels.

[0066] Furthermore, the convolutional model accelerator and the CPU are connected via a PCIe interface for data transmission and communication. Generally, PCIe interface devices are directly connected to the CPU processor and are physically located close to the CPU, effectively ensuring high-speed data transmission and communication between the CPU and the FPGA.

[0067] Therefore, this invention uses a hybrid algorithm of L1-norm and APoZ to determine the importance of filters and remove corresponding filters according to the set pruning threshold. It designs configurable fixed-point parameter quantization bit values, implements a fixed-point computing architecture to replace the floating-point computing architecture, and optimizes hardware overhead or memory bandwidth requirements. Utilizing the configurability and pipeline parallelism of FPGAs, multiple parallel computing units are designed to increase the parallelism of convolution calculations, and the number of computing units is configurable to achieve scalable performance.

[0068] In this embodiment, since a CPU-FPGA hybrid platform is used, the CNNs inference acceleration framework can be implemented using a hybrid filter algorithm based on L1-norm and APoZ, configurable fixed-point parameter quantization, and parallel implementation of multiple computing units. Furthermore, the FPGA's data flow, pipeline, and loop unrolling optimization mechanisms are utilized to accelerate convolution calculations. In other words, the aforementioned convolution acceleration calculation module is implemented based on FPGA combined with CPU.

[0069] The following performance comparison experiments, using multiple real and synthetic datasets and based on multiple implementation examples, are used to evaluate the test.

[0070] Example 1

[0071] Based on the above implementation method, FPGA hardware resource utilization test was conducted: The CNNs accelerator framework includes two original models, VGG16 and ResNet50. The parameters and intermediate results of the CNNs model are quantized to 8 bits, and the number of computation units is 1.

[0072] Table 1 shows the hardware resource usage of the FPGA integrated on the Xilinx U50 accelerator card.

[0073] Table 1 Hardware Resource Usage of Xilinx U50 FPGA

[0074]

[0075]

[0076] Example 2

[0077] Based on the above implementation methods, the performance of the pruning methods was evaluated: the performance of L1-norm, APoZ, and hybrid L1-norm and APoZ pruning algorithms was tested in the VGG16 model. The basic parameters were set as follows: 8-bit parameter quantization for the CNN model, 1 computational unit, and a single test image size of 224*224*3. The filter pruning rate of each convolutional layer in the CNN model was set from 0 to 80%, and accuracy experiments were conducted using the hybrid L1-norm, APoZ, and L1-norm pruning algorithms. The results are as follows. Figure 3 As shown in the figure, "original" represents the baseline accuracy of the CNN model without pruning. The test results show that when pruning 20%-30% of the filters, the model's accuracy performance loss is small. This is because some filters in the CNN model are redundant. However, as the pruning rate increases, the accuracy loss becomes more significant. This is because pruning too many filters causes changes in the model, thus reducing the accuracy of the CNN model. Regarding filter importance identification, three pruning algorithms were tested: L1-norm, APoZ, and a combination of L1-norm and APoZ. The experimental results show that the hybrid pruning algorithm based on L1-norm and APoZ performs better because it can identify not only the overall importance of filters but also distinguish local importance.

[0078] Example 3

[0079] Based on the above implementation method, the accuracy of pruning rates was compared: A hybrid pruning algorithm based on L1-norm and APoZ was used in VGG16 to evaluate the accuracy performance of each filter layer with different pruning rates. The basic parameters in the experiment were set as follows: 8-bit parameter quantization was used for the CNN models, and the pruning rate of each filter layer was set from 0% to 80%. The test image data was 224*224*3. The hybrid pruning algorithm based on L1-norm and APoZ was used to calculate the filter with the lowest relative weight in each layer of the CNN models. The accuracy comparison of the VGG16 model based on different pruning rates is as follows: Figure 4As shown, further verification can be made that there is a certain degree of redundancy in the filters of each layer of the CNN model. When pruning 20%-30% of the filters with the smallest weights, the accuracy loss of the CNN model is small. The experimental results also show that the distribution of filter weights in each layer of the CNN model is uneven, and the recognition of some features is determined by filters with some important weights. In addition, it was found that the accuracy of the intermediate convolutional layers with the same convolutional kernel in VGG16 decreases less when the pruning ratio increases, while the accuracy of convolutional layers Conv1, Conv2, Conv3, Conv4, and Conv5 drops sharply when the pruning ratio exceeds 40%. Therefore, an effective pruning criterion can improve the accuracy of filter pruning in each convolutional layer of the CNN model. Furthermore, under the condition of a computational unit of 1 and exploring the minimum accuracy loss, the pruning rate and execution time of each layer of the VGG16 model are shown in Table 2.

[0080] Table 2. Parameters, pruning, and execution time for each layer of the VGG16 model.

[0081] Con1 224*224 64 89M 64 0 6.33 Con2 224*224 64 1853M 48 25 23.63 Con3 112*112 128 926M 128 25 11.01 Con4 112*112 128 1851M 96 25 22.03 Con5 56*56 256 926M 256 25 10.41 Con6 56*56 256 1850M 192 25 20.57 Con7 56*56 256 1850M 192 43.75 15.69 Con8 28*28 512 925M 512 25 10.03 Con9 28*28 512 1850M 256 50 13.29 Con10 28*28 512 1850M 256 75 6.71 Con11 14*14 512 462M 512 50 3.06 Con12 14*14 512 462M 256 50 3.07 Con13 14*14 512 462M 384 63.5 2.32 FC1 1*1 4096 102M 4096 25 2.31 FC2 1*1 4096 16M 4096 0 0.53 FC3 1*1 1000 4M 1000 0 0.13 Total 15478M 38.68 151.

[0082] In Table 2, wi and hi represent the width and height of the feature map; MAPS represents the original output feature map; FLOP represents the computational cost of the original convolution; #MAPS represents the pruned output feature map; #FLOP represents the percentage reduction in computational cost of the layer after pruning; and #TIME represents the execution time of the layer after pruning.

[0083] Example 4

[0084] Based on the above implementation method, the accuracy performance of pruning rate was tested: the accuracy performance of each layer of filters was tested with different pruning rates using a hybrid pruning algorithm based on L1-norm and APoZ. The basic parameters in the experiment were set as follows: the parameter quantization of the ResNet50 model was 8-bit, the pruning rate of each layer of filters in the model was set to 0% to 80%, the test image data was 224*224*3, and the hybrid pruning algorithm based on L1-norm and APoZ was used to calculate and remove the filter with the lowest relative weight in each layer of the ResNet50 model. The impact of different filter pruning rates on the accuracy of each layer of the ResNet50 model is as follows. Figure 5 As shown, this further verifies that each layer of the ResNet50 model has a certain degree of redundancy. When pruning 20%-30% of the filters with less influence, the accuracy loss of the ResNet50 model is almost negligible. At the same time, we can find that the earlier layers and the layers near the changes in the number of feature maps in the ResNet50 model are more sensitive to pruning.

[0085] Furthermore, with almost no loss of accuracy, the ResNet50 model pruned its filters by 25% in each layer except the first layer, resulting in a 41.7% reduction in computational cost. The test results for the parameters, filter pruning rate, computational cost, and execution time of each layer of the ResNet50 model are shown in Table 3.

[0086] Table 3. Parameters, pruning, and execution time of each layer of the ResNet50 model.

[0087]

[0088]

[0089] Example 5

[0090] Based on the above implementation method, the performance of fixed-point quantization of parameters was evaluated: the execution time and accuracy performance of CNN model parameters were tested at different quantization bit widths. The basic environment settings in the experiment were: VGG16 CNN model, 1 computational unit, and a single test image data of 224*224*3. The parameters of each layer in the CNN model and the intermediate calculation results were quantized using variable fixed-point bit widths (8, 16, 32, and 64 bits), and the execution time and accuracy performance of the CNN model were tested. The performance of the CNN model in variable fixed-point bit width parameter quantization is as follows: Figure 6 As shown, using low-bit fixed-width quantization parameters can improve the execution time of CNN models, while the accuracy of CNN models is almost unaffected. This is because using low-bit fixed-width quantization for the parameters and intermediate calculation results of CNN models requires fewer computational resources, thereby reducing hardware overhead and memory bandwidth requirements, thus improving the execution speed of CNN models.

[0091] Example 6

[0092] Based on the above implementation method, a performance evaluation of multiple computational units (CUs) was conducted: the performance of CNN models under multiple CUs was tested. The basic parameters in the experimental environment were set as follows: the CNN models included VGG16 and ResNet50; the CNN models were original and unpruned; model parameters and intermediate results were quantized using 8-bit fixed-point numbers; and the test image data was 224*224*3. We tested the execution time of the CNN models when the number of CUs was set to 1, 2, 4, 6, 8, 12, 16, 20, 24, 28, and 32. The performance comparison results of the CNN models under multiple CUs are as follows: Figure 7As shown, when FPGA resources are sufficient, increasing the number of computational units to improve parallelism can reduce the execution time of CNN models and improve their speed performance. In the VGG16 and ResNet50 models, the execution time is lowest when the number of computational units is between 8 and 16. However, as the number of computational units continues to increase, the execution time of CNN models actually increases due to the increased demand for system resources. Furthermore, the ResNet50 model can have a slightly larger number of parallel computational units than the V166 model because the ResNet50 model has fewer parameters and requires fewer FPGA resources. Experimental results verify that designing multiple computational units can increase the parallelism of convolutional computation, improve the inference speed of CNN models, and provide a configurable number of computational units, thus better achieving the scalability of CNN accelerators.

[0093] Example 7

[0094] Based on the above implementation method, a comprehensive performance evaluation was conducted: the overall performance of the AFPQ-CNN CNN accelerator was evaluated and compared with that of the current state-of-the-art FPGA-based CNN accelerators (PipeCNN, OctCNN). The basic parameters in the experiment were set as follows: the CNN models included VGG16 and ResNet50; the CNNs used a hybrid pruning algorithm based on L1-norm and APoZ with a pruning rate of 25%; the CNN model parameters and intermediate computation results used 8-bit fixed-point quantization; the number of computation units was set to 8; and the single image data was 224*224*3. The experiment tested the execution time and Top-5 accuracy of the three accelerators, AFPQ-CNN, PipeCNN, and OctCNN, respectively. The comprehensive performance evaluation results of the three CNN accelerators are as follows: Figure 8 As shown, with similar Top-5 accuracy, the AFPQ-CNN accelerator based on the VGG16 model is 2.15 times faster than the PipeCNN accelerator, and based on the ResNet50 model, it is 1.91 times faster than the OctCNN accelerator. Furthermore, the AFPQ-CNN accelerator can improve execution speed by adjusting the filter pruning rate and compressing the CNN model to reduce computational load, based on the actual accuracy and speed requirements of edge computing applications. The AFPQ-CNN accelerator can achieve a balance between execution speed and accuracy by setting the filter pruning rate, parameter fixed-point quantization bits, and the number of parallel computing units, ultimately achieving good scalability for a CNN accelerator.

[0095] In summary, this invention uses a hybrid algorithm of L1-norm and APoZ to identify and prune filters with relatively low weights, thereby reducing the convolutional computation overhead of CNN models without generating irregular sparsity. The CNN model parameters and intermediate computation results use low-bit fixed-point quantization instead of floating-point operations, reducing hardware overhead and memory bandwidth requirements. At the same time, the configurable fixed-point quantization bit width parameter and multiple computation units provide better scalability for CNN model accelerators to meet the practical application needs in edge computing.

[0096] Those skilled in the art will understand that the above embodiments can be modified in form and detail in practical applications without departing from the spirit and scope of the invention.

Claims

1. An adaptive CNNs inference acceleration system based on FPGA that collaboratively utilizes pruning and quantization, characterized in that, This system is used for image processing. It includes a user input settings module and a convolution acceleration calculation module. The user input settings module accepts user input control parameters; The convolution acceleration computation module sets the filter pruning rate, fixed-point parameter quantization, and number of parallel computing units of the CNN model according to the control parameters. It is implemented based on FPGA and includes a hybrid pruning module, a parameter quantization optimization module, and a double-buffered computation module. The hybrid pruning module employs the L1-norm and ApoZ algorithms. The L1-norm and ApoZ algorithm first uses the L1-norm method to calculate the sum of the absolute values ​​of the filters, then uses the ApoZ method to sort them and prune the filters according to the set pruning rate. The double-buffered computing module uses multiple parallel computing units to compute CNNs models, and sets buffers at both the input and output of the computing units to achieve double buffering of input and output.

2. The FPGA-based adaptive CNNs inference acceleration system that collaboratively utilizes pruning and quantization as described in claim 1, characterized in that, The control parameters specifically include: accuracy threshold level, real-time configuration parameters, and CNN model parameters.

3. The FPGA-based adaptive CNNs inference acceleration system that collaboratively utilizes pruning and quantization as described in claim 1, characterized in that, The parameter quantization optimization module changes the convolution calculation in the CNN model to a fixed-point calculation method based on fixed-point quantization bit values; the fixed-point quantization bit values ​​are 8, 16, 32 or 64 bits.

4. The FPGA-based adaptive CNNs inference acceleration system that collaboratively utilizes pruning and quantization as described in claim 3, characterized in that, The fixed-point quantization bit value is configured by setting parameters.

5. The FPGA-based adaptive CNNs inference acceleration system that collaboratively utilizes pruning and quantization as described in claim 1, characterized in that, The buffer uses a ping-pong data transmission mechanism.

6. The FPGA-based adaptive CNNs inference acceleration system that collaboratively utilizes pruning and quantization as described in claim 1, characterized in that, The convolution acceleration computation module consists of a convolution model accelerator connected to the CPU. The convolution model accelerator is based on an FPGA and consists of cascaded multiple modules. The cascaded multiple modules include a memory read module, a core multiply-accumulate module, a pooling module, a write-back module, and a normalization module, wherein: The memory read module reads data from the global memory of the convolution acceleration calculation module; The core multiply-accumulate module performs the multiply-accumulate operations of convolution; The pooling module performs the pooling operation for the convolution; The write-back module writes the calculation results back to the global memory of the convolution acceleration calculation module; The normalization module performs the normalization operation on CNN models.

7. The FPGA-based adaptive CNNs inference acceleration system that collaboratively utilizes pruning and quantization according to claim 6, characterized in that, The convolutional model accelerator and the CPU are connected via a PCIe interface for data transmission and communication.

Images