Dynamic reconfigurable convolutional neural network accelerator based on annealing method and parameter optimization method thereof

Abstract

The application discloses an annealing method-based dynamic reconfigurable convolutional neural network accelerator and a parameter optimization method thereof, and the accelerator comprises a data buffer, a controller, an address generator, a convolution calculation engine and a dimension rearrangement module; the data buffer is used for buffering input and weight data of the convolutional neural network read from outside the chip; the controller controls input of input data and weight data of different addresses to the convolution calculation engine; the convolution calculation engine carries out convolution calculation on different convolution layers according to a parallel calculation mode and a corresponding parallel degree value pair obtained by the parameter optimization method based on the annealing method; if the parallel calculation mode of the next convolution layer changes, the result data of the current convolution layer is rearranged and stored to the data buffer, otherwise, the result data is directly stored to the data buffer. The application can improve resource utilization, calculation efficiency and throughput of the accelerator and reduce the delay.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of neural network accelerator, and particularly relates to a dynamic reconfigurable convolutional neural network accelerator based on an annealing method and a parameter optimization method thereof. BACKGROUND

[0002] Convolutional Neural Networks (CNNs) is an important branch of deep learning algorithm. It is evolved from Artificial Neural Networks (ANN). Compared with fully connected networks, CNN has simple network structure, high computing efficiency, good nonlinear fitting characteristics, and obvious image feature extraction, so CNN has a crucial position in the field of image processing. These advantages make it achieve brilliant achievements in image classification 【1】, target recognition 【2】, face detection 【3】, semantic segmentation 【4】 and other fields. With the continuous development and progress of CNNs, the fields involving CNNs in real applications are becoming more and more extensive, and the performance requirements of the application are becoming higher and higher. Through the analysis of the excellent neural networks in the ImageNet competition in recent years, the network is more deep and complex, which has become a trend. Its memory and calculation have become the main bottleneck of performance limitation. Therefore, the acceleration of CNNs has become an indispensable work. Compared with GPU and ASIC, FPGA has a prior advantage in flexibility, power consumption and development cycle, so it has become an important field that cannot be ignored in accelerating CNNs.

[0003] The CNN structure is complex and diverse. The convolution calculation process and its parameter representation are as follows Figure 1As shown in the figure, the required amount of calculation of different layers is quite different, so for different networks, how to reasonably allocate the calculation, storage and bandwidth resources on the FPGA chip for each layer becomes the key factor limiting the overall performance of the hardware accelerator. Many research teams have proposed the idea of design space exploration for FPGA-based convolutional neural network hardware accelerators. In 2015, Zhang et al. proposed a design space exploration method tailored for ConvNets. The proposed method is based on the adjustment of the Roofline model to balance the calculation time and communication time for hardware parameter optimization. For example, work 【8】proposed a DSE tool for Systolic accelerator, but its exploration method is to use the traditional open-loop exhaustive iteration method, including 【9】also uses the open-loop exhaustive iteration method, they do not have a model that can quickly evaluate the performance of the Systolic accelerator, work

[10] adopts a new approach to explore the design space of hardware accelerators, transforming the accelerator design problem into a multi-dimensional variable optimization programming model, and pioneering the use of genetic algorithms to iterate the optimal accelerator configuration. The design space exploration methods in the above mentioned articles are all for fixed parallelism accelerators. Genetic algorithm is a global optimization algorithm suitable for problems with large search space, and the search starts from the group, which has potential parallelism. However, the programming implementation of genetic algorithm is complex, first the problem needs to be coded, and after finding the optimal solution the problem needs to be decoded, in addition, genetic algorithm cannot use the feedback information of the network in time, so the search speed of the algorithm is slow. The traditional brute force exhaustive iteration method has a wide range of applications, the search space is comprehensive, and the reconfigurability is strong. However, the advantage of the traditional brute force exhaustive iteration method can only be realized when the problem model is simple. When the problem size is large, the algorithm running time will be much longer than the evolutionary algorithm, so the exhaustive method is not suitable for large problem size.

[0004] For different CNN structures, as the application scenarios become more complex, the network structure also becomes more diverse, accompanied by an order of magnitude increase in network parameters, and the design complexity of hardware accelerators is increasing. Therefore, how to automatically explore the optimal design space of hardware accelerators is an indispensable part of reducing the design difficulty of hardware accelerators.

[0005] References:

[0006] [1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, 2012, pp. 1097-1105.

[0007] [2] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv: 1409.1556, 2014.

[0008] [3] J. Dai, Y. Li, K. He, and J. Sun, “R-fcn: Object detection via region based fully convolutional networks,” in Advances in neural information processing systems, 2016, pp. 379-387.

[0009] [4] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence., vol. 39, no. 12, pp. 2481-2495, 2017.

[0010] [5] ZL202111171756.9 Method for constructing convolutional neural network data flow design space analysis tool and device

[0011] [6] ZL202210982043.9 Spatial accelerator performance evaluation method, electronic device, and computer storage medium

[0012] [7] C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, and J. Cong, "Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks," in Proceedings of the 2015 ACM / SIGDA International Symposium on Field-Programmable Gate Arrays, ser. FPGA '15. New York, NY, USA: ACM, 2015, pp. 161-170.

[0013] [8] Peemen M, Setio A A A, Mesman B, et al. Memory-centric accelerator design for Convolutional Neural Networks [C] / / IEEE, International Conference on Computer Design. 2013: 13-19.

[0014] [9] ZL202211245567.6 Hardware architecture for accelerating multi-channel convolution and design space exploration method

[0015]

[10] Cadambi S, Majumdar A, Becchi M, et al. A programmable parallel accelerator for learning and classification. [C] / / International Conference on Parallel Architecture and Compilation Techniques. 2010: 273-284. SUMMARY

[0016] In view of the problem of unbalanced resource utilization and non-maximal use of resources in the prior art hardware architecture, the application provides a dynamic reconfigurable convolutional neural network accelerator based on an annealing method and a parameter optimization method, which can improve the resource utilization, computing efficiency and throughput of the accelerator and reduce the delay.

[0017] To achieve the above technical purposes, the application adopts the following technical solutions:

[0018] The application discloses a parameter optimization method of a dynamic reconfigurable convolutional neural network accelerator based on an annealing method.

[0019] Step 1, setting initial parameters and constraint conditions of the annealing method according to different convolutional neural network models and hardware resources; wherein, unknown solutions of the annealing method are expressed by using parallel computing modes and corresponding parallel degree values, and optional parallel computing modes are Pv+Pf or Pc+Pf, wherein Pv, Pc and Pf respectively represent input data parallel degree, channel parallel degree and convolution kernel parallel degree; the initial set parameters include initial temperature, Markov chain, temperature reduction parameter, annealing termination temperature, maximum number of network layers and initial solutions of two parallel modes;

[0020] Calculating a target function value of the initial solution; wherein, the target function is constructed according to running time and resource utilization of the current convolutional layer;

[0021] Step 2, randomly obtaining a neighborhood solution of the current solution at the current temperature T, and calculating a target function value of the neighborhood solution;

[0022] If the current convolutional layer is the first convolutional layer, only neighborhood solutions of the same parallel computing mode as the current solution are obtained; otherwise, one neighborhood solution is obtained for each of the two parallel computing modes;

[0023] Step 3, calculating a difference value AL of the target function value L(X0) of the current solution X0 and the target function value X1 of the neighborhood solution X1 of each parallel computing mode; L() represents the target function;

[0024] If AL is less than or equal to 0, the neighborhood solution is accepted as the current solution of the corresponding parallel computing mode;

[0025] If AL is greater than 0, the neighborhood solution is accepted as the current solution of the corresponding parallel computing mode according to a preset probability;

[0026] Step 4, repeating steps 2 and 3 to perform inner loop iteration until the number of inner loop iterations reaches the set length of the Markov chain, and then continuing step 5;

[0027] Step 5, updating the current temperature based on the temperature reduction parameter;

[0028] Step 6, judging whether the current temperature reaches the annealing termination temperature;

[0029] If the current temperature reaches the annealing termination temperature, comparing the target function values of the current solutions of the two parallel computing modes, and selecting the parallel computing mode and the current solution corresponding to the optimal target function value as the optimal parallel degree value of the current convolutional layer; if the current temperature does not reach the annealing termination temperature, returning to step 2.

[0030] Further, the parallel computing mode of the initial solution of the first convolutional layer is set as Pv+Pf.

[0031] Further, the constraint condition set in step 1 is:

[0032]

[0033] DSP layer_i ≤DSP MAX

[0034]

[0035] wherein Runtime represents the total running time of the accelerator, i.e., the sum of the running time of all convolutional layers of the convolutional neural network model, L layer_i represents the running time of the i-th convolutional layer of the convolutional neural network, DSP layer_i represents the number of DSPs used in the i-th layer, DSP MAX represents the DSP resource in the FPGA board card selected by the accelerator, BRAMi represents the BRAM resource usage in the i-th layer, BRAM MAX represents the BRAM resource in the FPGA board card selected by the accelerator, and n represents the total number of layers of the convolutional neural network model.

[0036] Further, the objective function expression is:

[0037]

[0038] In the formula, means rounding up; H, W and C respectively represent the height, width and channel number of the input image of the current convolutional layer, K represents the kernel size of the current convolutional layer, F represents the number of convolution kernels of the current convolutional layer, and L represents the running time of the current convolutional layer.

[0039] Further, the preset probability calculation formula of the accepted neighborhood solution in step 3 is:

[0040]

[0041] wherein γ is the Boltzmann constant and T is the current temperature.

[0042] Further, if the parallel computing mode of a convolutional layer is Pv+Pf, the convolutional calculation result is stored in the H-W-C dimension; if the parallel computing mode of a convolutional layer is Pc+Pf, the convolutional calculation result is stored in the C-H-W dimension.

[0043] An annealing method-based dynamic reconfigurable convolutional neural network accelerator comprises a data buffer, a controller, an address generator, a convolution calculation engine and a dimension rearrangement module.

[0044] The data buffer is used for buffering input data and weight data of a convolutional neural network read from outside the chip.

[0045] The controller controls the input of input data and weight data of different addresses to the convolution calculation engine.

[0046] The convolution calculation engine performs convolution calculation on different convolution layers by using the parallel computing mode and the corresponding parallel degree value obtained by the parameter optimization method according to any one of the above.

[0047] If the parallel computing mode of the next convolution layer changes relative to the current convolution layer, the result data of the current convolution layer calculated by the convolution calculation engine is input to the dimension rearrangement module, the dimension rearrangement module rearranges the result data of the current convolution layer according to the dimension corresponding to the parallel computing mode of the next convolution layer, and outputs and stores the result data to the data buffer.

[0048] If the parallel computing mode of the next convolution layer does not change relative to the current convolution layer, the result data of the current convolution layer calculated by the convolution calculation engine is directly output and stored to the data buffer according to the corresponding dimension.

[0049] Advantages

[0050] The present application can solve the problem of unbalanced resource utilization and the problem that resources cannot be maximally used in the hardware architecture of a convolutional neural network, and can also flexibly explore the design space and give the most ideal parallel computing mode and parallel degree parameters according to the requirements of different convolutional neural networks, different hardware resource conditions and different application scenarios.

[0051] In the FPGA application scenario with limited computing resources and storage resources, the proposed dynamic reconfigurable hardware architecture is designed to explore the design space according to resource constraints and running time constraints, to determine the best design scheme for each layer of the network, i.e., to determine the parallel computing mode and the corresponding parallel degree, and to deploy it on the FPGA to realize the acceleration processing of the dynamic reconfigurable convolution scheme. Compared with other traditional fixed parallel accelerators, the resource efficiency of the accelerator of the present application is higher, the throughput is larger, and the acceleration effect on the convolutional neural network is better. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A schematic diagram of a convolution calculation process and its parameters.

[0053] Figure 2The flowchart for adopting annealing method to preferentially select parallel computing mode and corresponding parallel degree value in the embodiment of the present application.

[0054] Figure 3 The schematic diagram of the accelerator hardware architecture module in the embodiment of the present application.

[0055] Figure 4 The schematic diagram of the hierarchical flow-water multiplication and addition calculation used in the embodiment of the present application.

[0056] In this multiplication and addition calculation schematic diagram, it can be roughly divided into three levels, the data input level, this level is divided into Pf small modules, how many small modules are determined by the parallel degree obtained by design space exploration calculation, if Pv+Pf parallel mode is more optimal at this time, then the data at this time is calculated by Pv data of the single channel of input data and weight data, if it is Pc+Pf at this time, then the data at this time is calculated by Pc channel data of the input data and the weight data, and the specific parallel computing mode is determined by the controller using annealing method to optimize the results, and then the controller controls the input data module and the calculation module. The second level is the multiplication and addition array level, this level also corresponds to Pf small modules, the corresponding number of small modules is also determined by Pf parallel degree, each small module includes Pc or Pv multipliers, in the case of Pc parallel design, each calculation small module adds up the results and outputs a result, Pf small modules output Pf results. In the case of Pv parallel design, each multiplier calculates the data of the convolution kernel size, and then the accumulator is added, and then a selector is used to judge whether the convolution kernel data is calculated, if not, the result is selected and returned to the multiplier for calculation, if yes, the result is selected and output, each small module outputs Pv results, and Pf small modules output Pv*Pf results. Finally, the results are input to the data cache level for use in the next layer.

[0057] Figure 5 The schematic diagram of the dimension conversion module in the embodiment of the present application.

[0058] In the schematic diagram, the results stored in H-W-C dimension in Pv+Pf parallel mode are converted into results stored in C-H-W dimension suitable for Pc+Pf parallel mode.

[0059] Figure 6 The calculation efficiency comparison diagram of the present application method and the traditional method fixedly adopting parallel computing mode for the convolutional neural network ResNet50.

[0060] Figure 7 The calculation efficiency comparison diagram of the present application method and the traditional method fixedly adopting parallel computing mode for the convolutional neural network VGG-16. DETAILED DESCRIPTION

[0061] The embodiments of the present application are described in detail below, which are based on the technical solutions of the present application, and give detailed implementation manners and specific operation processes, and further explain the technical solutions of the present application.

[0062] Embodiment 1

[0063] The embodiment provides a parameter optimization method of a dynamic reconfigurable convolutional neural network accelerator based on an annealing method, which determines a parallel computing mode and a corresponding parallel degree value for all convolutional layers by using an analog annealing method; and the parameter optimization method is used for reference Figure 2 As shown in the figure, the determination process of any i-th convolutional layer is as follows:

[0064] Step 1, according to different convolutional neural networks and resource utilization, the initial parameters and constraint conditions of the annealing method are set;

[0065] Wherein, the unknown solution of the annealing method is expressed by using a parallel computing mode and a corresponding parallel degree value, and the optional parallel computing mode is Pv+Pf or Pc+Pf, Pv, Pc and Pf represent input data parallel degree, channel parallel degree and convolution kernel parallel degree respectively. For the parallel computing mode of Pv+Pf, the unknown solution is expressed as X(Pv, Pf); for the parallel computing mode of Pc+Pf, the unknown solution is expressed as X(Pc, Pf).

[0066] The initial set parameters include:

[0067] 1) Initial temperature; that is, the temperature at the beginning of the algorithm of the analog annealing method, which is the length of the outer loop in the algorithm, and the higher the temperature, the higher the algorithm accuracy, but the corresponding model running time also rises.

[0068] 2) Markov chain M arkov ; that is, the number of inner loop operations in the algorithm, in the case of a certain outer loop temperature, different solutions are found by random disturbance, and the higher the temperature, the larger the solution range, the Markov chain determines the model accuracy, and also has a great influence on the running time.

[0069] 3) Cooling parameter After the end of the inner loop, the temperature is reduced according to the cooling parameter.

[0070] 4) Annealing termination temperature T final ; when the temperature is reduced to less than or equal to T final , the optimal solution of the current convolutional layer is output.

[0071] 5) Maximum number of layers; that is, the maximum number of layers of the selected convolutional neural network.

[0072] 6) initial solution X0_1 of Pv+Pf parallel computing mode and initial solution X0_2 of Pc+Pf parallel computing mode; can be taken arbitrarily in the value range.

[0073] According to the existing hardware architecture, after comprehensively considering the model running time and resource utilization, the delay and resource utilization are modeled to obtain a resource model and a performance model.

[0074] The performance model is a mathematical model, which in this embodiment refers to the hardware running time of a certain convolutional layer, and the performance model is as follows:

[0075]

[0076] In the above formula, the means rounding up the result.

[0077] For the Pv+Pf parallel computing mode, the Pc in the performance model above is 1 by default; for the Pc+Pf parallel computing mode, the Pv in the performance model above is 1 by default.

[0078] The FPGA dynamic reconfigurable accelerator of the convolutional neural network mainly has two limited resources, namely logic resources such as DSP and storage resources such as BRAM and the like memory. After determining the convolutional neural network model, the resource model is represented as follows:

[0079]

[0080] The meanings of the letters in the above performance model and resource model are shown in the following table:

[0081] letter definition H The height of the input image W Input image width C Number of channels of the input image K Convolution kernel size F Number of convolution kernels L The time it takes to run a certain layer Dw Data bit width Pv Input data parallelism PC Channel parallelism Pf Convolution kernel parallelism DSP Digital Signal Processor BRAM Block storage

[0082] The constraint conditions are preset according to the performance model and the resource model, and the algorithm needs to satisfy the following constraints:

[0083]

[0084] DSP layer_i ≤DSP MAX

[0085]

[0086] Wherein, Runtime represents the total running time of the accelerator, that is, the total running time of all convolutional layers of the convolutional neural network model, L layer_i represents the running time of the i-th convolutional layer of the convolutional neural network, DSP layer_i represents the number of DSPs used in the i-th layer, DSP MAX represents the DSP resource in the FPGA board selected by the accelerator, BRAM iL(X0) represents the BRAM resource usage in the i-th layer, BRAM MAX L(X0) represents the BRAM resource usage in the accelerator selected FPGA board card, n represents the total number of convolutional neural network layers.

[0087] Before starting the loop iteration, the initial solution is calculated as the current solution to obtain the target function value L(X0_1) and L(X0_2); wherein the target function is constructed according to the running time and resource utilization of the current convolutional layer.

[0088] The embodiment directly uses the performance model as the target function to calculate the target function value.

[0089] Step 2, at the current temperature T, randomly obtain the neighborhood solutions X1_1 and X2_1 of the current solutions X0_1 and X0_2 of the two parallel computing modes respectively, and calculate the target function values L(X1_1) and L(X1_2) of the neighborhood solutions;

[0090] According to the characteristics that the input images of the initial convolutional layers of the convolutional neural network are relatively large, and considering the delay and resource utilization, the parallel computing mode of the first convolutional layer is generally fixed as Pv+Pf, and in order to optimize the calculation efficiency, in the step 2, the neighborhood solution of the other parallel computing mode Pc+Pf and the target function value thereof do not need to be obtained, that is, only the neighborhood solution of the parallel computing mode Pc+Pf and the target function value thereof are obtained, so as to obtain the parallel degree value under the parallel computing mode Pv+Pf, that is, the Pv and Pf values.

[0091] Step 3, calculate the difference ΔL=L(X1)-L(X0) between the target function value L(X0) of the current solution X0 and the target function value X1 of the neighborhood solution X1; L() represents the target function;

[0092] If ΔL≤0, it means that the neighborhood solution X1 is better than the current solution X0, and the neighborhood solution is directly accepted as the current solution of the corresponding parallel computing mode;

[0093] If ΔL>0, it means that the neighborhood solution X1 is not as good as the current solution X0, and the neighborhood solution is accepted as the current solution of the corresponding parallel computing mode according to the preset probability. The preset probability calculation formula is:

[0094]

[0095] In the formula, γ is the Boltzmann constant, which is between 0 and 1; T is the current temperature, and the higher the current temperature, the higher the probability of the algorithm accepting the new solution. This method of selecting whether to accept the difference solution according to a certain probability is called Metropolos criterion. The three principles of acceptance state are:

[0096] (1) In a fixed temperature, the probability of accepting the solution that makes the objective function decrease is greater than the probability of accepting the candidate solution that makes the objective function increase;

[0097] (2) As the temperature decreases, the probability of accepting the solution that makes the objective function increase gradually decreases;

[0098] (3) When the temperature tends to zero, only the solution that makes the objective function decrease can be accepted.

[0099] Step 4, repeat steps 2 and 3 to perform an inner loop iteration, until the number of inner loop iterations reaches the set length of the Markov chain, and then continue to step 5;

[0100] Step 5, update the current temperature based on the cooling parameter;

[0101] Step 6, judge whether the current temperature reaches the annealing termination temperature;

[0102] (1) If the current temperature reaches the annealing termination temperature, compare the objective function values L(X0_1) and L(X0_2) of the current solutions of the two parallel computing modes, and select the parallel computing mode and the current solution corresponding to the better objective function value as the preferred parallel degree value of the current convolution layer;

[0103] (2) If the current temperature does not reach the annealing termination temperature, return to step 2.

[0104] Example 2

[0105] The application further provides a dynamic reconfigurable convolutional neural network accelerator based on the annealing method, which comprises a data buffer, a controller, an address generator, a convolution calculation engine and a dimension rearrangement module; the above several modules work together to realize dynamic parallel processing of three levels of feature map data (Pv), feature map channel (Pc) and convolution kernel (Pf), as shown in Figure 3 .

[0106] The data buffer is used to buffer the input data and weight data of the convolutional neural network read from outside the chip;

[0107] The controller controls the input of input data and weight data of different addresses to the convolution calculation engine;

[0108] The convolution calculation engine performs convolution calculation on different convolution layers using the parallel computing mode and the corresponding parallel degree value obtained by the parameter optimization method in the above-mentioned embodiment 1; it is worth mentioning that the convolution calculation engine of the application can perfectly compatible with two different parallel designs, as shown in Figure 4 ;

[0109] If the parallel computing mode of the next convolution layer changes relative to the current convolution layer, the result data of the current convolution layer calculated by the convolution calculation engine is input to the dimension rearrangement module, the result data of the current convolution layer is rearranged by the dimension rearrangement module according to the dimension corresponding to the parallel computing mode of the next convolution layer, and the result data is output and stored in the data buffer;

[0110] If the parallel computing mode of the next convolution layer does not change relative to the current convolution layer, the result data of the current convolution layer calculated by the convolution calculation engine is directly output and stored in the data buffer according to the corresponding dimension.

[0111] In a specific running environment, an application program running on a host communicates with an accelerator through a PCIe line. The trained weight and input image are sent by the upper computer, converted into an 8-bit fixed-point format, and then the data is stored in the on-chip data buffer. Then, the data is input into the convolution calculation module after the controller gives a signal. After the calculation, if the parallel degree conversion is required for the next layer (i.e., the parallel computing mode of the next convolution layer is different from that of the current convolution layer), the dimension rearrangement module stores the result from the dimension suitable for the parallel computing mode of the current convolution layer, and converts the result into a result suitable for another parallel computing mode (for example, the current Pv+Pf parallel computing mode is stored according to the H-W-C dimension, and after the dimension rearrangement, it is converted into a result suitable for the Pc+Pf parallel mode and stored according to the C-H-W dimension). As shown in FIG. 6. Figure 5

[0112] Figure 6 , Figure 7 The calculation efficiency of two convolutional neural network models is provided, and it can be obviously seen that the accelerator of the application can make the calculation efficiency much higher than that of the traditional parallel design. This advantage becomes more and more obvious with the increase of board resources. Although the calculation efficiency of the accelerator of the application decreases with the increase of board resources, the calculation efficiency is still much higher than that of the traditional design.

[0113] The above embodiments are preferred embodiments of the present application, and those skilled in the art can make various transformations or improvements on the basis of the above embodiments. Without departing from the general concept of the present application, these transformations or improvements should be within the scope of the present application.​

Claims

1. A parameter optimization method for a dynamically reconfigurable convolutional neural network accelerator based on annealing method, characterized in that: The simulated annealing method is used to determine the parallel computing mode and the corresponding parallelism value for all convolutional layers. The determination process for any i-th convolutional layer is: Step 1: Set the initial parameters and constraints of the annealing method according to different convolutional neural network models and hardware resources. The unknown solution of the annealing method is represented by the parallel computing mode and the corresponding parallelism value. The optional parallel computing modes are Pv+Pf or Pc+Pf, where Pv, Pc, and Pf represent the input data parallelism, channel parallelism, and convolution kernel parallelism, respectively. The initial parameters include: initial temperature, Markov chain, cooling parameter, annealing termination temperature, maximum number of network layers, and the initial solutions of the two parallel modes. Calculate the objective function value of the initial solution; where the objective function is constructed based on the running time and resource utilization of the current convolutional layer; Step 2: At the current temperature T, randomly obtain the neighboring solution of the current solution and calculate the objective function value of the neighboring solution; If the current convolutional layer is the first convolutional layer, only the neighborhood solution with the same parallel computing mode as the current solution is obtained; otherwise, one neighborhood solution is obtained for each of the two parallel computing modes. Step 3: Calculate the difference ΔL = L(X1) - L(X0) between the objective function value L(X0) of the current solution X0 and the objective function value X1 of the neighboring solution X1 in each parallel computing mode; L() represents the objective function; If ΔL≤0, accept the neighborhood solution as the current solution of the corresponding parallel computing mode; If ΔL>0, the neighborhood solution is accepted as the current solution of the corresponding parallel computing mode according to the preset probability; Step 4: Repeat steps 2 and 3 to iterate the inner loop until the number of inner loop iterations reaches the set length of the Markov chain, and then proceed to step 5; Step 5: Update the current temperature based on the cooling parameter; Step 6, determining whether the current temperature reaches the annealing termination temperature; If the current temperature reaches the annealing termination temperature, compare the objective function values ​​of the current solutions of the two parallel computing modes, and select the parallel computing mode and current solution corresponding to the better objective function value as the preferred parallelism value of the current convolutional layer; if the current temperature does not reach the annealing termination temperature, return to step 2.

2. The parameter optimization method according to claim 1, characterized in that The parallel computing mode of the initial solution of the first convolutional layer is set to Pv+Pf.

3. The parameter optimization method according to claim 1, characterized in that The constraints set in step 1 are: DSP layer_i ≤DSP MAX Among them, Runtime represents the total running time of the accelerator, that is, the sum of the running time of all convolutional layers of the convolutional neural network model, L layer_i Represents the running time of the i-th convolutional layer of the convolutional neural network, DSP layer_i Indicates the number of DSPs used in layer i, DSP MAX Indicates the DSP resources in the FPGA board selected by the accelerator, BRAM i Indicates the BRAM resource usage in layer i, BRAM MAX It represents the BRAM resources in the FPGA board selected by the accelerator, and n represents the total number of layers of the convolutional neural network model.

4. The parameter optimization method according to claim 1, characterized in that The objective function expression is: Where, It means rounding the result up; H, W, and C represent the height, width, and number of channels of the input image of the current convolutional layer, respectively; K represents the size of the convolution kernel of the current convolutional layer; F represents the number of convolution kernels of the current convolutional layer; and L represents the running time of the current convolutional layer.

5. The parameter optimization method according to claim 1, characterized in that: Step 3 is calculated based on the preset probability of accepting the neighborhood solution: Where γ is the Boltzmann constant and T is the current temperature.

6. The parameter optimization method according to claim 1, characterized in that: If the parallel computing mode of a convolutional layer is Pv+Pf, the convolution calculation result is stored in the HWC dimension; if the parallel computing mode of a convolutional layer is Pc+Pf, the convolution calculation result is stored in the CHW dimension.

7. A dynamically reconfigurable convolutional neural network accelerator based on annealing method, characterized in that: Includes data buffer, controller, address generator, convolution calculation engine and dimension rearrangement module; The data buffer is used to cache the input data and weight data of the convolutional neural network read from outside the chip; The controller controls input data and weight data of different addresses to be input into the convolution calculation engine; The convolution computing engine performs convolution computing on different convolution layers using a parallel computing mode and a corresponding parallelism value obtained by the parameter optimization method according to any one of claims 1 to 6; If the parallel computing mode used by the next convolutional layer changes relative to the current convolutional layer, the result data of the convolutional computing engine for the current convolutional layer is input to the dimension rearrangement module, which rearranges the result data of the current convolutional layer according to the dimension corresponding to the parallel computing mode of the next convolutional layer and outputs it to the data buffer; If the parallel computing mode adopted by the next convolutional layer does not change relative to the current convolutional layer, the result data of the convolutional computing engine for the current convolutional layer is directly output and stored in the data buffer according to the corresponding dimension.

Images