A neural network reconfigurable configuration mapping method for FPGA resources

Abstract

This invention discloses a neural network reconfigurable configuration mapping method for FPGA resources, relating to the field of embedded artificial intelligence systems. The method includes the following steps: adjusting the FPGA platform according to different task requirements to construct multiple mapping schemes oriented towards resource constraints; establishing a mapping evaluation model by analyzing the resource characteristics of different FPGA platforms, evaluating multiple mapping schemes, and analyzing the differences between different mapping schemes and their performance differences on different FPGA platforms; and constructing a partially reconfigurable mapping adjustment scheme based on the deployment of different neural network algorithms according to the application domain. This invention analyzes the resource characteristics of the FPGA platform and effectively reduces the mapping space by imposing resource constraints. When a certain resource is limited, the accelerator performance boundary is further explored through the replacement of different resources with the same logic. It supports configuration mapping methods under the partially reconfigurable FPGA approach, and by analyzing the resource changes and parameter adjustments after partial reconfiguration of the target platform, a new mapping scheme can be quickly completed.

Description

Technical Field

[0001] This invention relates to the field of embedded artificial intelligence system technology, and in particular to a neural network reconfigurable configuration mapping method for FPGA resources. Background Technology

[0002] Due to the limited bandwidth of FPGA platforms, there is a mismatch between the high data throughput of neural network algorithms and the limited bandwidth available. Existing methods cannot fully utilize existing resources to achieve optimal performance because FPGA resources and bandwidth are not fully utilized during the algorithm-to-hardware mapping process. Neural network algorithms iterate rapidly, leading to high complexity and unique characteristics. The large data volume of neural networks results in high storage requirements, necessitating local mapping from algorithm to hardware combined with data flow scheduling methods. Furthermore, due to the complexity and variability of neural network structures and the diversity of deployment environments, rapid design space exploration should be conducted, using the resources of the target FPGA platform as a constraint, and combining the adaptability between the algorithm and hardware to find the optimal mapping method.

[0003] On a single design instance, the choice of different mappings for two different convolutional layers in ResNet-50 has a significant impact on performance. For the same convolutional layer and a fixed accelerator architecture, different mapping methods can result in a performance difference of up to 10 times. Therefore, it is necessary to find the optimal mapping method specific to a particular FPGA platform and a particular neural network algorithm through algorithmic analysis.

[0004] Using a specific FPGA platform provides limited on-chip memory and limited off-chip bandwidth. Furthermore, different FPGA platforms have different resource and bandwidth constraints, meaning the available parallelism for the same deep neural network (DNN) model deployed on different FPGA platforms varies, preventing full utilization of the FPGA's on-chip resources. On the other hand, due to the large memory footprint of DNNs and the significant variability in the number of operations and model size for different DNN models, fixed mapping methods cannot fully utilize the FPGA's limited resources for each DNN model. Therefore, it is necessary to optimize the accelerator hardware-software mapping method to overcome the impact of the FPGA's limited on-chip memory on each DNN model. Summary of the Invention

[0005] This invention provides a neural network reconfigurable configuration mapping method for FPGA resources, which can solve the problems existing in the prior art.

[0006] This invention provides a neural network reconfigurable configuration mapping method for FPGA resources, comprising the following steps:

[0007] Adjust the FPGA platform according to different task requirements and build multiple mapping schemes oriented towards resource constraints;

[0008] By analyzing the resource characteristics of different FPGA platforms, a mapping evaluation model is established to evaluate various mapping schemes, analyze the differences between different mapping schemes and their performance differences on different FPGA platforms.

[0009] Based on the application domain, different neural network algorithms are deployed to construct a partially reconfigurable mapping adjustment scheme.

[0010] Compared with the prior art, the beneficial effects of the present invention are:

[0011] (1) Analyze the resource characteristics of the FPGA platform and effectively reduce the mapping space by imposing resource constraints.

[0012] (2) When a certain resource is limited, the performance boundary of the accelerator can be further explored by replacing different resources with the same logic.

[0013] (3) Supports configuration mapping method under FPGA partial reconfigurable method. By analyzing the resource changes and parameter adjustments after partial reconfiguration of the target platform, a new mapping scheme can be quickly completed. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart of a neural network reconfigurable configuration mapping method for FPGA resources according to the present invention;

[0016] Figure 2 A schematic diagram illustrating the difference between the size of the NN model and the size of the memory unit on an FPGA;

[0017] Figure 3 A graph showing the inference delay performance error of the performance prediction method;

[0018] Figure 4 Error plot of accelerator power consumption performance for performance prediction method;

[0019] Figure 5 A schematic diagram of the algorithm for mapping multidimensional arrays to linear storage space;

[0020] Figure 6 A schematic diagram of the search algorithm for reconfigurable configuration mapping schemes. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Reference Figure 1-6 This invention provides a neural network reconfigurable configuration mapping method for FPGA resources, comprising the following steps:

[0023] Step 1: Adjust the FPGA platform according to different task requirements and build multiple mapping schemes oriented towards resource constraints. This includes the following steps:

[0024] (1) Combining FPGA resource characteristics, extract accelerator structure parameters and runtime parameters with common characteristics to provide support for the construction of resource constraint evaluation methods and performance prediction models;

[0025] (2) Combining computational data dependencies and neural network structure parameters, we predict the performance of the computing system and evaluate the adaptability of the hardware structure for the accelerator system architecture.

[0026] (3) Select and determine the optimal intelligent computing system architecture and its hardware mapping scheme based on the evaluation results.

[0027] Step 2: By analyzing the resource characteristics of different FPGA platforms, establish a mapping evaluation model, evaluate multiple mapping schemes, and analyze the differences between different mapping schemes and their performance differences on different FPGA platforms.

[0028] Step 3: Deploy different neural network algorithms according to the application domain and construct a partially reconfigurable mapping adjustment scheme.

[0029] In the process of resource-adaptive architecture design, FPGA resources are used as constraints to reduce the design space, and the mapping method is optimized through a performance prediction evaluation model.

[0030] Based on the determined size of the FPGA's internal computing array and the data multiplexing method, the on-chip data bandwidth requirements can be determined, thereby constraining the memory output bandwidth. Meanwhile, because parallel computing methods further restrict the mapping method, the overhead of storage resources due to data caching is constrained by the BRAM resources in the FPGA, and the overhead of computing resources due to the computational tasks to be executed is constrained by the DSP resources in the FPGA.

[0031] When storage resources are limited, the task allocation strategy at each organizational level of the computing component needs to be adjusted to meet the storage resource constraints. If adjusting the computing task allocation strategy still fails to meet the constraints, consider reducing the size of the computing component and restarting the architecture search.

[0032] When computing resources are constrained, additional computing components need to be built according to the computing task, and efficient multiply-adder built using LUT resources in advance can replace the necessary DSP resource gaps.

[0033] In addition to the aforementioned computing and storage resources, the mapping scheme is also limited by FPGA interface resources. When the computing array is large, the demand for data supply is high, resulting in frequent updates of data in the on-chip buffer. In this case, off-chip memory access is required via the bus interface. When the bus transmission rate cannot meet the computing demands, the mapping method needs to be adjusted to transmit large amounts of data in a single burst. This requires adjusting the mapping scheme and simultaneously using off-chip bandwidth optimization and on-chip buffer access optimization schemes to further reduce communication bandwidth pressure.

[0034] The performance prediction model accepts parameters such as DNN structure (number of layers, layer structure, accuracy, etc.), accelerator hardware architecture (memory hierarchy, number of PEs, etc.), hardware mapping, unit energy / latency cost of MAC operations, and energy consumption for memory accesses across various memory hierarchies. When executing a DNN on a target accelerator given the neural network topology, hardware architecture, and hardware mapping definition, the performance prediction model outputs estimated energy consumption, latency, and resource consumption. It provides a coarse-grained, fast evaluation, making performance predictions without considering the data paths of individual modules.

[0035] The specific steps for constructing a partially reconfigurable mapping adjustment scheme are as follows:

[0036] Determine the internal adjustments to the neural network accelerator, including loading, unloading, and modifying neural network operator acceleration units, adjusting the size of the computing array, and adjusting the size of the storage array; determine resource constraints on the adjusted accelerator; and redetermine the mapping scheme in conjunction with the neural network algorithm.

[0037] The accelerator adjustment content is used to update the mapping model parameter list, including algorithm parameters such as convolution kernel size, feature map size, input and output, stride, activation method, etc., as well as architecture parameters such as computing array size, storage array size, interface bandwidth, etc. The updated parameter information is used as new constraints to redetermine the mapping scheme. According to the various resource-constrained mapping schemes, after local FPGA reconstruction, the mapping scheme is determined within the updated mapping space boundary.

[0038] Example 1

[0039] The mapping method of the present invention determines the parameter information used for adjustment.

[0040] 1. Determine the parameters of the neural network. The basic operators of the neural network include convolution (conv), fully connected network (fc), pooling (pool), batch normalization (bn), nonlinear activation (act), and residuals (res). Each operator has its own structural information.

[0041] (1)conv, the basic structural information of the convolutional layer includes the number of input channels (Chin), the number of output channels (Chout), the feature map size (H / W), the boundary size (pad), the stride, and the kernel size (K).

[0042] (2) fc, the basic structural information of the fully connected layer includes the number of input channels (Chin), the number of output channels (Chout), and the feature map size (H / W).

[0043] (3) pool: The basic structural information of the pooling layer includes pooling type (pool_type), pooling window size (pool_size), feature map size (H / W), and number of channels (Ch).

[0044] (4) bn, the basic structural information of the batch processing normalization layer includes feature map size (H / W) and number of channels (Ch).

[0045] (5)act, the basic structural information of the nonlinear activation layer includes activation type (act_type), feature map size (H / W), and number of channels (Ch).

[0046] (6)res, the residual layer includes the splicing data source (res_src) information.

[0047] 2. Determine Accelerator Architecture Parameters. To determine the accelerator architecture parameters, the hardware organization must be specified, namely the topology of the PE interconnect computing and storage units, and the mapping space constraints that limit the set of mappings allowed by the hardware. An abstract template of the architecture is constructed, possessing sufficient parameterization capabilities to model various architectures of interest. For each storage level, the number of memory banks MN, the number of entries per memory bank I, the number of bits per entry W, the memory bandwidth B, and various other microarchitectural attributes can be specified. The interconnect network topology is automatically inferred from the storage hierarchy specification, and other microarchitectural attributes can be explicitly specified. For computing units, the number of coarse processing units (RCUs) RN, the number of fine processing units (FCUs) in each RCU FN, the number of processing elements (PEs) within each FCU PN, and the computation data quantization bit width QW need to be determined. For the data interface, the data bit width DW and the burst length BL need to be determined.

[0048] 3. Once the neural network and accelerator structure parameters are determined, runtime parameters need to be determined based on the mapping and scheduling methods. Memory access counts (MA), average resource utilization (RA), and on-chip buffer throughput (TR) must all be considered as runtime parameters in the evaluation model. Furthermore, since the neural network structure is segmented during the mapping process, the four additional dimensions (TI, TO, TW, TH) resulting from the channel and feature map segmentation are also used as runtime parameters to determine the computational parallelism scale and data scheduling method.

[0049] Example 2

[0050] The mapping method of the present invention uses FPGA resources as constraints to confirm the boundaries of the mapping space.

[0051] First, 70% of the DSP usage is used as the base value for the convolutional computation array, with the remainder reserved for units used for accelerating computations by other operators. The parallel computation scale is organized according to three dimensions, corresponding to RN, FN, and PN in the structural parameters, and TI, TO, and K in the runtime and network parameters. Once the computation array size is determined, adjustments to the computational parallelism and data reuse will lead to changes in data requirements. Without data reuse, the total data requirement is RN*FN*PN*2, resulting in significant on-chip memory access and data transfer overhead. Using data reuse can reduce the total on-chip data transfer, requiring only partial data access and sending data to the input of the computation unit via broadcast and multicast. Therefore, based on the determined computation array size and data reuse method, the on-chip data bandwidth requirement can be determined, thus constraining the memory output bandwidth. Simultaneously, because the parallel computation method further restricts the mapping method, the overhead of storage resources due to data caching is constrained by the BRAM resources in the FPGA. When storage resources are limited, the task allocation strategy at each organizational level of the computation unit needs to be adjusted to meet the storage resource constraints. If the constraints still cannot be met after adjusting the computing task allocation strategy, consider reducing the size of the computing components and restarting the architecture search.

[0052] The network construction process requires three sets of parameters: the network splitting method, accelerator parameters, and the layer tasks assigned to the accelerators. To determine the optimal parameter configuration under given resource constraints, an accurate resource and performance model needs to be established for each layer of the accelerator.

[0053] Adjusting accelerator parameters has varying impacts on resource consumption. Precise formulas for resource costs and parameter settings are crucial for system performance optimization. In neural network accelerator design, LUTs and FFs are not bottlenecks in accelerator system generation, while DSPs and on-chip BRAMs are significant limiting factors, thus requiring careful evaluation during modeling. The main purpose of the DSP module in a convolutional accelerator is to construct multipliers and adders; DSP consumption is related to the type of data being processed. Therefore, the DSP consumption in a convolutional accelerator can be expressed using formula (1).

[0054] N dsp =DSP data_type *TI*TO (1)

[0055] Table 1 shows the corresponding DSP usage for multipliers and adders when dealing with different data types. Furthermore, the max-pooling accelerator does not consume DSP resources, while the average pooling accelerator requires a DSP module to calculate the average output.

[0056] Table 1 Data Types and DSP Consumption

[0057] Data types float fixed32 fixed16 fixed8 DSP consumption 5 4 1 0.5

[0058] Considering three resources (DSP, BRAM, and Logic) in an FPGA platform, resource utilization is compared to reveal the design's preference for hardware resources. The comparison between DSP and BRAM shows that accelerator designs tend to use more DSP resources, and a similar preference is observed when comparing DSP and Logic. This suggests that current FPGA designs are more likely constrained by computational resources. Therefore, for FPGA platforms with relatively limited DSP resources, both logic and DSP can be used simultaneously to complete computational tasks to maintain high parallelism; for FPGA platforms with ample DSP resources, only DSP can be used to ensure a higher operating frequency.

[0059] Table 2 shows the resource consumption of building multipliers or adders of different precisions using different FPGA resources. When a certain FPGA resource is insufficient, the resource allocation can be adjusted appropriately to maximize resource utilization and achieve higher computational efficiency.

[0060] Table 2 Comparison of FPGA resource consumption for multipliers and adders of different data types

[0061]

[0062] Accelerators require BRAM resources for data backup between and within accelerators. Although current large FPGAs offer alternative on-chip storage such as Ultra RAM (URAM), the RAM consumption model is similar to that of BRAM. Therefore, the RAM usage of an accelerator system can be estimated using a BRAM model. Output data between computation blocks is stored in on-chip BRAM whenever possible, except for raw input data and weight data. BRAM consumption is calculated in two ways: 1) BRAM consumed by caching input data within the accelerator; 2) BRAM consumed by caching data between different modules of the accelerator. The internal BRAM used by the accelerator is determined by the tile size after slicing and the architectural information of the stride and padding of layers in the input DNN model. The weight buffer size depends on the maximum kernel size of the layer allocated to the accelerator. A single BRAM block is limited to one read port and one write port. RAM in the FPGA is organized into BRAM blocks with fixed memory capacity, each BRAM block having a capacity of 18Kb (URAM is 288Kb). Therefore, in the BRAM usage approximation, each partition's buffer occupies at least one BRAM block. The approximate RAM consumption of the accelerator system is shown in Equation (2).

[0063]

[0064] BRAM depth This refers to a 1K-deep BRAM block in the platform. When the buffer size is small (less than 16), such as the weight buffer of most network models, they are not counted as BRAM and can be implemented using LUTRAM resources in the FPGA chip.

[0065] Example 3

[0066] The mapping method of this invention uses an analytical model to complete the performance evaluation of the early mapping schemes.

[0067] The performance prediction model accepts parameters such as DNN structure (number of layers, layer structure, accuracy, etc.), accelerator hardware architecture (memory hierarchy, number of PEs, etc.), hardware mapping, unit energy / latency cost of MAC operations, and energy consumption for memory accesses across various memory hierarchies. When executing a DNN on a target accelerator given the neural network topology, hardware architecture, and hardware mapping definition, the performance prediction model outputs estimated energy consumption, latency, and resource consumption. It provides a coarse-grained, fast evaluation, making performance predictions without considering the data paths of individual modules.

[0068] Performance evaluation uses an analytical model, employing equations to describe the accelerator's energy, latency, and resource consumption based on the DNN model and hardware design description. A system-level modeling approach is used, employing analytical equations and the built-in properties of each module to evaluate the total energy and latency of all modules. This invention adopts a layer-wise sequential computation method; therefore, computation and energy consumption prediction are based on layer-by-layer modeling and then accumulating all layers. Computational latency is primarily affected by parallel computing methods and computational scale. Since each granularity of the computing unit in the computing array is responsible for computation in one dimension, poor mapping methods or extreme mismatches between algorithm size parameters and computing array size parameters will lead to some computing units being idle; therefore, the execution cycle is rounded up. For each layer, regardless of the layer type, energy consumption can be mainly divided into three parts: computational energy consumption, on-chip memory access energy consumption, and off-chip memory access energy consumption. Among these, off-chip memory access accounts for the largest proportion of energy consumption.

[0069] Latency performance is primarily affected by mapping methods, computational parallelism methods, data flow scheduling methods, and off-chip memory access bandwidth. Performance evaluation can also help explore optimal network slicing schemes. 4D slicing is divided into two types: channel slicing, constrained by computational resources, and feature map slicing, constrained by on-chip memory resources. Assuming a convolutional layer is divided into p computational blocks based on channel parameters, the internal structural parameters of each block are [TI, TO, TW, TH]. The computational layer latency can be obtained by summing the latency of the computational blocks. During computation, latency mainly needs to consider memory access latency and computation latency. Memory access latency can be expressed by the following formula.

[0070]

[0071]

[0072] Where TW and TH represent the width and height of the computation block, and BandWidth represents the off-chip memory access bandwidth. When multiple data interfaces are used for off-chip memory access, feature map data and weight data can be moved to the on-chip buffer simultaneously. Therefore, only the larger delay value needs to be taken as the memory access delay, and formula (3) can be used in this case. When only one data interface can complete the off-chip memory access, the input feature map, weight, and output feature map data need to be time-division multiplexed to complete the data transmission, so formula (4) is used.

[0073] Computation latency is mainly affected by parallel computing methods and computing scale. Since each granularity of computing unit in the computing array is responsible for the computation of one dimension, when the mapping method is poor or the algorithm size parameters are extremely mismatched with the computing array size parameters, some computing units will be idle. Therefore, the execution cycle is rounded up.

[0074]

[0075] Furthermore, since off-chip memory access and data-parallel computation can be performed simultaneously, considering the more time-consuming part as the critical path can mask the time consumption of another type of operation.

[0076]

[0077]

[0078] Therefore, for the optimization objective as shown in formula (7), under the constraints of on-chip storage resources BRAM and computing resources MAC, the optimal slicing scheme for each layer is found, and the minimum latency after layer-by-layer accumulation is the latency performance optimization index.

[0079] Because a layer-wise sequential computation method is used, energy consumption prediction is based on modeling layer by layer and then accumulating all layers. For each layer, regardless of the layer type, the main energy consumption can be divided into three parts: computation energy consumption, on-chip memory access energy consumption, and off-chip memory access energy consumption. Among these, off-chip memory access accounts for the largest proportion of energy consumption.

[0080] Based on the above premises, the total network energy consumption is:

[0081]

[0082] The energy consumption of the i-th layer can be modeled as follows:

[0083] Energy i =E C *C i +E BRAM *N buffer +E DRAM *N DRAM (9)

[0084] Among them, E C C is the energy coefficient for a single calculation. i N represents the computational cost of this layer. buffer N represents the number of on-chip storage accesses for this layer. DRAM E represents the number of off-chip memory accesses for this layer. BRAM and E DRAM These are the energy coefficients for accessing on-chip BRAM and off-chip DRAM, respectively.

[0085] In convolutional layers,

[0086] C conv =Co*Ci*H*W*K 2 (MUL)+Co*H*W*(Ci*K 2 -1)(ADD) (10)

[0087] Formula (10) can be approximated as:

[0088] C conv =2*Co / Ci*H*W*K 2 (11)

[0089] Since the number of multiplication operations is roughly the same as the number of addition operations, the energy coefficient is at this point.

[0090] The total energy consumption can be calculated by multiplying and summing M measurements and then dividing by M.

[0091] In the activation layer,

[0092] C act =Co*H*W (12)

[0093] At this time, the energy coefficient E C =E COM .

[0094] In the pooling layer (calculated using average pooling),

[0095]

[0096] At this time, the energy coefficient E C =E ADD .

[0097] In the bn layer,

[0098] C bn =Co*H*W (14)

[0099] At this time, the energy coefficient E C =2*(E MUL +E ADD ).

[0100] Memory access costs are categorized into on-chip memory access and off-chip memory access, namely, the energy consumption incurred by accessing on-chip memory and the energy consumption in accessing off-chip DRAM. During convolution calculations, the total number of memory accesses is...

[0101] NB conv =2*Co*Ci*H*W*K 2 +Co*H*W (15)

[0102] In the activation layer,

[0103] NB act =2*Co*H*W (16)

[0104] Since the data is pre-stored in the on-chip buffer, this data is obtained entirely from on-chip memory access.

[0105] The number of off-site memory accesses was

[0106] ND conv = a*Ci*H*W+b*Co*Ci*K 2 +CO*H*W (17)

[0107] Due to the different computational order caused by different data flow scheduling methods, the input data and weight data are not necessarily read from off-chip only once. Therefore, the corresponding scaling coefficients a and b are obtained according to different mapping and scheduling methods. Since the computational fusion method is used, multiple operators are organized in a pipeline manner within a computational layer. For the on-chip memory access part, the number of memory accesses can still be obtained by adding the results in formulas (15) and (16).

[0108] NB = NB conv +NB act =2*Co*Ci*H*W*K 2 +3*Co*H*W (18)

[0109] For off-chip memory access, the calculation is still performed according to formula (17).

[0110] Furthermore, a Roofline model is used to further constrain the evaluation. Since the input / output / weight data has different amounts in each tile, different burst lengths and access patterns will result in different effective bandwidths. Therefore, different designs will have different final Rooflines, making the original Roofline-based approach highly inaccurate for bandwidth-intensive applications such as fully connected layers. Therefore, a normalization factor is needed to normalize the DRAM traffic of input / output / weight accesses to the maximum effective bandwidth.

[0111] Example 4

[0112] The mapping method of this invention, combined with an optimization method, determines the optimal mapping scheme.

[0113] Before performing the mapping, the following principles are defined: (1) The loop boundary of each tiling level determines the tiling size of each data space at that level, and the size of the data space shards is constrained by the buffer size of each level. (2) Parallel loops represent the partitioning of shard spaces between instances of a level, and this specific mapping results in the copying of some input data between adjacent shards. (3) The order of loops within a shard level determines the order in which sub-shards are passed from that level to inner levels during execution. This representation produces a strictly contained sharding hierarchy, but this may not be optimal. When a data space is less reused at a certain level, it is allowed to extend its capacity to other data spaces through that level, thereby achieving a larger shard size and potentially leading to a more optimized mapping. The mapping specification is used to specify which data spaces are allowed to reside at each level. This unified mapping representation allows for reasoning about the possible mapping spaces of the architecture in a structured, programmatic way, ultimately finding the optimal mapping method.

[0114] The mapping space is the set of efficient mappings from neural network layers to the architecture. Efficient mappings must satisfy three requirements. First, it must be compatible with the hardware dataflow. For example, a mapping with a fixed weight dataflow will not work correctly on hardware with a fixed output. Second, the mapping needs to match available computational resources. For example, a mapping requiring eight PEs will not run if there are only four PEs in the system. Finally, it should satisfy memory constraints at different levels of the hierarchy. The buffer size chosen at design time sets an upper limit on the size of the tiles that can be executed on the hardware. Given the architecture specification and network layer dimensions, the mapper generates efficient mappings by enumerating all possible factorizations that satisfy the above constraints and are compatible with the cyclic order. The size of the mapping space can be very large, so thoroughly scanning all possible mappings is usually impractical. To address this issue, a mapping space pruning method needs to be designed.

[0115] Design space pruning requires architectural parameters, which must be determined within the resource constraints of the FPGA platform. The mapper prunes the space by applying two user-defined constraints to optimize energy efficiency or performance. The first constraint is the reuse factor, which determines the minimum amount of time reuse for different data types. Lower data reuse will lead to more off-chip memory accesses. The reuse factor provides a threshold for pruning smaller slice sizes, which are less energy efficient due to lower memory reuse. The second constraint is resource utilization, which determines the number of physical exponents (PEs) used in the system. Higher utilization means a larger PE array size and less idle resources, which generally leads to better computational efficiency within the PE, but may reduce overall performance if few PEs are used. Higher utilization helps improve system performance but may not be energy-optimal. The constraints are only used to prune easily identifiable lower optimization points in the design space, such as mapping methods that are obviously too small, resulting in insufficient resource utilization or poor data reuse, and large-size accelerator mappings that exceed FPGA resource constraints.

[0116] Example 5

[0117] The steps for partially reconfigurable mapping adjustment are as follows: (1) Determine the internal adjustment content of the neural network accelerator, including loading, unloading, and modification of neural network operator acceleration units; adjustment of the computing array size; and adjustment of the storage array size. (2) Determine the resource constraints on the adjusted accelerator, i.e., the boundary of the mapping. Re-determine the mapping scheme in conjunction with the neural network algorithm.

[0118] The accelerator adjustment is used to update the mapping model parameter list, including algorithm parameters such as kernel size, feature map size, input / output, stride, and activation method, as well as architecture parameters such as compute array size, storage array size, and interface bandwidth. The updated parameter information serves as new constraints for redetermining the mapping scheme. Following the resource-constrained mapping scheme, after local FPGA reconstruction, the mapping scheme is determined within the updated mapping space boundaries.

[0119] This invention's mapping method is primarily aimed at neural network acceleration applications, completing the mapping process from algorithm software to accelerator hardware. This invention is applicable to various FPGA platform types and a variety of neural network algorithms. It mainly explores the mapping space under the resource constraints of the FPGA platform, predicting the performance of mapping schemes and determining the optimal mapping scheme while satisfying the resource constraints of the target FPGA platform. By analyzing different resource characteristics and combining resource preferences in the accelerator design process, suitable resource substitution methods are constructed to achieve better acceleration performance.

[0120] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0121] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A neural network reconfigurable configuration mapping method for FPGA resources, characterized in that, Includes the following steps: Adjust the FPGA platform according to different task requirements and build multiple mapping schemes oriented towards resource constraints; By analyzing the resource characteristics of different FPGA platforms, a mapping evaluation model is established to evaluate various mapping schemes, analyze the differences between different mapping schemes and their performance differences on different FPGA platforms. Based on the application domain, different neural network algorithms are deployed, and a partially reconfigurable mapping adjustment scheme is constructed. Constructing the aforementioned resource-constrained multiple mapping schemes specifically includes the following steps: By combining the resource characteristics of the FPGA platform, we extract the structural parameters and operating parameters of accelerators with common characteristics, providing support for the construction of resource constraint evaluation methods and performance prediction models. By combining computational data dependencies and neural network structure parameters, we can conduct resource-adaptive architecture design, predict the computing system performance and evaluate the hardware structure adaptability for the accelerator system architecture. Based on the evaluation results, select and determine the optimal intelligent computing system architecture and its hardware mapping scheme; In the process of resource-adaptive architecture design, FPGA resources are used as constraints to reduce the design space, and the mapping method is optimized through a performance prediction evaluation model; reuse factor and resource utilization are used as constraints to reduce the design space. The on-chip data bandwidth requirements are determined based on the defined FPGA computing array size and data multiplexing method, thereby constraining the memory output bandwidth. The overhead of storage resources caused by data caching is constrained by the BRAM resources in the FPGA, and the overhead of computing resources caused by the computing tasks to be executed is constrained by the DSP resources in the FPGA. When storage resources are limited, the task allocation strategy of each organizational level of the computing unit needs to be adjusted to meet the storage resource constraints; if the constraints still cannot be met after adjusting the computing task allocation strategy, the size of the computing unit is reduced and the architecture search is restarted. When the computing resources are constrained, additional computing components need to be built according to the computing task, and the necessary DSP resource gaps are replaced by efficient multiply-adders built in advance using LUT resources. In addition to the computing and storage resources mentioned above, the mapping scheme is also limited by the FPGA interface resources. When the computing array is large, it is necessary to access off-chip memory through the bus interface. When the bus transmission rate is difficult to meet the computing requirements, both off-chip bandwidth optimization and on-chip buffer access optimization schemes are used. The specific steps for constructing a partially reconfigurable mapping adjustment scheme are as follows: Determine the internal adjustments of the neural network accelerator, including the loading, unloading, and modification of neural network operator acceleration units, as well as adjustments to the computing array size and storage array size. Determine resource constraints on the adjusted accelerator; The mapping scheme was redefined by combining neural network algorithms; The accelerator adjustment content is used to update the mapping model parameter list, including algorithm parameters and architecture parameters; the algorithm parameters include convolution kernel size, feature map size, input and output, sliding stride and activation method, and the architecture parameters calculate array size, storage array size and interface bandwidth; The updated parameter information is used as a new constraint to redetermine the mapping scheme. According to the resource-constrained multiple mapping schemes, after the FPGA is partially reconstructed, the mapping scheme is determined within the updated mapping space boundary.

2. The neural network reconfigurable configuration mapping method for FPGA resources as described in claim 1, characterized in that, The performance prediction model accepts various parameters, including DNN structure, accelerator hardware architecture, hardware mapping, unit energy / latency cost of MAC operations, and memory access energy consumption for various memory hierarchies; the performance prediction model outputs estimated energy consumption, latency, and resource consumption.

Images