Optimization method and optimization system for non-multiplication-addition computing operations in FPGA hardware accelerator

Abstract

Provided are an optimization method and an optimization system for non-multiplication-addition type computing operations in an FPGA hardware accelerator. The optimization method comprises: generating a first computing graph by decomposing the non-multiplication-addition type computing operations into a plurality of basic operations; obtaining at least one merged operation by merging consecutive basic operations in the first computing graph; generating a second computing graph based on the first computing graph and the at least one merged operation; and executing the operation in the FPGA hardware accelerator according to the second computing graph; wherein the executing the operation in the FPGA hardware accelerator according to the second computing graph comprises: executing each merged operation of the at least one merged operation by pipeline optimization.

Description

Technical Field

[0001] The present disclosure relates to the field of hardware accelerators, and more specifically, to an optimization method and an optimization system for non-multiplication-addition computing operations in FPGA hardware accelerators. Background Art

[0002] In recent years, due to the rapid growth of data volume and accessibility, the importance of artificial intelligence algorithms has gradually been recognized by the industry. Among them, artificial intelligence algorithms based on deep neural networks have shown high accuracy in image, voice and video processing, and have received great attention from the industry. However, these deep neural networks often have huge parameters and complex structures, and require a lot of computing resources for training and reasoning, which makes the traditional computing power architecture seem stretched everywhere. And with the in-depth implementation of deep neural networks in key fields such as unmanned driving, medical care, and education, the high concurrency and real-time requirements in these diverse scenarios have further put forward higher requirements on the computing power required to calculate deep neural networks.

[0003] Specifically, deep neural networks often contain multiple computing operations, such as multiplication and addition and non-multiplication and addition operations. Multiplication and addition operations can often be converted into matrix multiplications in batch calculations, and are calculated through efficient multiplier arrays in hardware implementations. As for non-multiplication and addition operations, due to their variety and different calculations, it is often necessary to implement a variety of different circuits in hardware acceleration to accelerate the calculations. Existing solutions either cannot support multiple networks or have limited computing performance, and cannot meet the emerging new networks and growing computing power requirements. Summary of the invention

[0004] Exemplary embodiments of the present disclosure may solve at least the above problems, or may not solve the above problems.

[0005] According to one aspect of the present disclosure, a method for optimizing non-multiplication-addition type computing operations in an FPGA hardware accelerator is provided, comprising: generating a first computing graph by decomposing the non-multiplication-addition type computing operations into multiple basic operations; obtaining at least one merged operation by merging consecutive basic operations in the first computing graph; generating a second computing graph based on the first computing graph and the at least one merged operation; and executing operations in the FPGA hardware accelerator according to the second computing graph; wherein the executing operations in the FPGA hardware accelerator according to the second computing graph comprises: executing each merged operation of the at least one merged operation using pipeline optimization.

[0006] Optionally, merging the continuous basic operations in the first computation graph to obtain at least one merged operation may include: based on the total amount of resources of the FPGA hardware accelerator and the first resource usage required for the multiple basic operations, using a greedy algorithm to explore combinations of continuous basic operations that can be merged; and merging the explored combinations of continuous basic operations that can be merged into a merged operation.

[0007] Optionally, the total amount of resources of the FPGA hardware accelerator and the resource usage required for the multiple basic operations, using a greedy algorithm to explore combinations of continuous basic operations that can be merged, may include: looping the following operations until the exploration is completed: (1) searching for candidate combinations of all continuous basic operations in the first calculation graph; (2) calculating acceleration factors for all candidate combinations to reflect the acceleration effect; (3) sorting all candidate combinations according to the calculated acceleration factors; (4) obtaining the second resource usage required for the candidate combination with the largest acceleration factor in the sorted result; (5) determining the sum of the first resource usage and the second resource usage whether it exceeds the total amount of resources; (6) when the sum of the first resource usage and the second resource usage does not exceed the total amount of resources, determine the candidate combination with the largest acceleration factor as a combination that can be merged, and based on the candidate combination, update the first computational graph and the first resource usage; (7) when the sum of the first resource usage and the second resource usage exceeds the total amount of resources, determine whether there are other unexplored candidate combinations. If so, update the candidate combination in the sorting result with the acceleration factor second only to the current candidate combination to the candidate combination with the largest acceleration factor in the sorting result and jump to operation (4). If not, end the exploration.

[0008] Optionally, the calculation of the acceleration factor for all candidate combinations to reflect the acceleration effect may include: for each candidate combination, calculating the average number of required operation time cycles, the average number of calculation data items, and the number of occurrences of the candidate combination in the first calculation graph, and multiplying the three together as the acceleration factor of the candidate combination.

[0009] Optionally, updating the first resource usage may include: updating the first resource usage to the sum of the first resource usage and the second resource usage.

[0010] Optionally, executing each merge operation in the at least one merge operation using pipeline optimization may include: for each merge operation, decomposing the calculation process of the merge operation into different pipeline stages, and executing the different pipeline stages in parallel in each clock cycle.

[0011] Optionally, the FPGA hardware accelerator may be a deep neural network accelerator.

[0012] According to another aspect of the present disclosure, there is provided an optimization system for non-multiplication-addition type computing operations in an FPGA hardware accelerator, comprising: a computing graph generating device; an operation executing device; wherein the computing graph generating device is configured to: generate a first computing graph by decomposing the non-multiplication-addition type computing operations into a plurality of basic operations, obtain at least one merged operation by merging consecutive basic operations in the first computing graph, and generate a second computing graph based on the first computing graph and the at least one merged operation; wherein the operation executing device is configured to: execute operations in the FPGA hardware accelerator according to the second computing graph; wherein the operation executing device executes each merged operation in the at least one merged operation using pipeline optimization.

[0013] Optionally, the computational graph generating device may be configured to: based on the total amount of resources of the FPGA hardware accelerator and the first resource usage required for the multiple basic operations, use a greedy algorithm to explore combinations of continuous basic operations that can be merged; and merge the explored combinations of continuous basic operations that can be merged into a merged operation.

[0014] Optionally, the computation graph generation device may be configured to: loop and perform the following operations until the exploration is terminated: (1) search for candidate combinations of all continuous basic operations in the first computation graph; (2) calculate acceleration factors of all candidate combinations for reflecting the acceleration effect; (3) sort all candidate combinations according to the calculated acceleration factors; (4) obtain the second resource usage required for the first candidate combination in the candidate combination list; (5) determine whether the sum of the first resource usage and the second resource usage exceeds the total resource amount; (6) when the sum of the first resource usage and the second resource usage does not exceed the total resource amount, determine the candidate combination with the largest acceleration factor as a combination that can be merged, and update the first computation graph and the first resource usage based on the candidate combination; (7) when the sum of the first resource usage and the second resource usage exceeds the total resource amount, determine whether there are other unexplored candidate combinations. If so, update the candidate combination in the sorted result whose acceleration factor is second only to the current candidate combination to the candidate combination with the largest acceleration factor in the sorted result and jump to operation (4); if not, terminate the exploration.

[0015] Optionally, the computation graph generating device may be configured to: for each candidate combination, calculate the average number of required operation time cycles, the average number of computational data items, and the number of occurrences of the candidate combination in the first computation graph, and multiply the three together as the acceleration factor of the candidate combination.

[0016] Optionally, the computation graph generating device may be configured to: update the first resource usage to the sum of the first resource usage and the second resource usage.

[0017] Optionally, the operation execution device may be configured to: for each merge operation, decompose the calculation process of the merge operation into different pipeline stages, and execute the different pipeline stages in parallel in each clock cycle.

[0018] Optionally, the FPGA hardware accelerator may be a deep neural network accelerator.

[0019] According to another aspect of the present disclosure, a system is provided comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, cause the at least one computing device to perform operations performed by a computation graph generation device.

[0020] According to another aspect of the present disclosure, a computer-readable storage medium storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform operations performed by a computation graph generating device.

[0021] According to the optimization method and optimization system of non-multiplication-addition type computing operations in the FPGA hardware accelerator of the exemplary embodiment of the present disclosure, the non-multiplication-addition type computing operations are split into multiple basic calculations, and a solution of merging calculations combined with pipeline optimization is adopted on the basis of the multiple basic calculations, which not only improves the computing performance of the non-multiplication-addition type computing, but also increases the versatility and scalability of the hardware accelerator.

[0022] In addition, according to the optimization method and optimization system of non-multiplication-addition computing operations in the FPGA hardware accelerator of the exemplary embodiment of the present disclosure, based on the erasable and rewritable characteristics of FPGA devices, continuous computing combinations that appear repeatedly in the computing graph are effectively explored, thereby improving the utilization rate of hardware resources and enhancing the performance of deep neural network acceleration on FPGA. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] These and / or other aspects and advantages of the present disclosure will become clear and more easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 The flowchart of the method for optimizing non-multiplication-addition computing operations in the FPGA hardware accelerator according to the exemplary embodiment of the present disclosure is shown.

[0025] Figure 2 is a schematic diagram illustrating a merge operation according to an exemplary embodiment of the present disclosure.

[0026] Figure 3 is a schematic diagram illustrating pipeline optimization according to an exemplary embodiment of the present disclosure.

[0027] Figure 4is a flowchart illustrating a method of exploring a merge operation according to an exemplary embodiment of the present disclosure.

[0028] Figure 5 is a block diagram of an optimization system for non-multiply-add computing operations in an FPGA hardware accelerator according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of the embodiments of the present disclosure as defined by the claims and their equivalents. Various specific details are included to assist in understanding, but these details are considered to be exemplary only. Therefore, one of ordinary skill in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. In addition, descriptions of well-known functions and structures are omitted for clarity and brevity.

[0030] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0031] A deep neural network is a type of neural network structure that contains multiple computing operations. The input data needs to be sequentially processed through various computing operations in the neural network to obtain the computing results required by the network. The structures of different layers in the currently popular deep neural networks are often different, and the computing operations they contain are also different, requiring different computing steps and methods to be calculated.

[0032] Most traditional computing architectures are based on central processing units. However, since the central processing unit is a general-purpose hardware, in order to meet its versatility requirements, a large number of hardware resources on the central processing unit chip are used to implement complex scheduling designs and control systems. The occupation of these hardware resources not only reduces the performance of the calculation, but also increases the additional power consumption during calculation. In order to pursue higher performance and lower power consumption, more designs are beginning to adopt solutions based on dedicated hardware such as ASIC and FPGA. By reducing general capabilities, these dedicated hardware chips can optimize and improve the computing power of the bottleneck parts in deep neural network calculations in a targeted manner, providing a smooth path to processing power and high energy efficiency.

[0033] In the hardware implementation of deep neural network accelerators, we often need to implement hardware circuits of different structures on the hardware for acceleration. These deep neural networks often contain a large number of multiplication and addition operations, which use multiplication and addition as the basic operation unit and can often be converted into matrix multiplication for batch calculation. Common deep network hardware accelerators will implement independent matrix multiplier units on the chip, so as to perform these matrix multiplication operations efficiently and quickly.

[0034] At the same time, common deep neural networks also contain many computing operations that are not based on multiplication and addition. Since the basic computing units of these operations are not multipliers, we cannot use the multiplier arrays (such as matrix multiplication calculators) commonly used in deep neural network accelerators to accelerate the operation. However, these non-multiplication and addition computing operations are often interspersed and connected in series between multiplication and addition computing operations, and together constitute the context of the entire neural network. For network computing structures such as deep neural networks, the input data needs to pass through various computing operations in the network in turn to obtain the final required results, which also makes it difficult for us to skip or delay these non-multiplication and addition calculations. Depending on the selected deep neural network, these non-multiplication and addition computing operations often have different numbers, different computing structures, and different computing circuits required to accelerate their calculations, which greatly increases the difficulty of designing general-purpose deep neural network hardware accelerators. In addition, with the in-depth research on deep neural networks and the continuous emergence of various new deep neural network structures, the types and diversity of these non-multiplication and addition computing operations are also increasing. Since these different non-multiplication and addition computing operations often require different hardware acceleration implementations, this also brings more difficulties and challenges to the design and implementation of general-purpose deep neural network hardware accelerators.

[0035] In existing FPGA deep neural network hardware acceleration solutions, the following solutions are often used to solve this problem.

[0036] Existing Solution 1: Implement all possible computing operations in the network using proprietary circuits on hardware accelerators

[0037] This solution needs to consider all possible computing operations in the selected network during design, and support for the selected deep neural network is achieved by designing dedicated hardware acceleration circuits for each possible computing operation. The advantage of this solution is that since each computing operation in the network is accelerated by a custom-designed hardware circuit, ideal performance and power consumption indicators can often be achieved. However, this solution also has the following disadvantages: 1. Due to the limitation of hardware on-chip resources, the types of computing operations that can be implemented in the accelerator are limited. 2. Since it is impossible to exhaustively implement all computing operations, hardware accelerators using this solution can often only support selected networks or common standard networks. 3. This type of implementation often cannot support newly developed networks.

[0038] Existing Solution 2: Allocate non-multiplication-addition operations in the network to the central processor for calculation

[0039] This solution takes advantage of the collaborative work of software and hardware in the integrated hardware and software design, and assigns the computing operations that are not implemented in the hardware accelerator to the central processing unit for calculation. Since the central processing unit is a general-purpose computing device, in theory this solution can support any computing operation. At the same time, since a large number of operations in many deep neural networks are concentrated on multiplication and addition operations, the amount of operations assigned to the central processing unit may not be large, so it does not necessarily lead to excessive performance loss. However, due to the sequential calculation characteristics of deep neural networks, under this solution, we inevitably need to transmit the calculation data between the hardware accelerator and the central processing unit. Depending on the communication architecture, this solution may generate huge additional communication overhead, affecting the overall performance.

[0040] Existing solution 3: Split non-multiplication and addition operations into multiple basic calculations and perform them sequentially

[0041] This solution splits non-multiplication-addition computing operations into basic operations such as addition, subtraction, multiplication, and magnitude on the software side, and then completes the computing functions required for the original computing operations by executing these basic operations in sequence. Since the types of basic operations are limited, the hardware accelerator designed according to this solution can implement all the required basic operations in hardware to support the vast majority of deep neural networks. At the same time, since the implementation of the circuits for these basic operations is relatively simple, this solution does not require a large amount of on-chip hardware resources. The disadvantage of this solution is that since each non-multiplication-addition computing operation often requires multiple basic operations to be executed in sequence, the computing performance of this operation method is relatively weak. Once non-multiplication-addition computing operations account for a high proportion of the entire network, this solution will greatly reduce the acceleration performance of the implemented hardware accelerator.

[0042] In order to improve the computing performance of non-multiplication-addition computing and increase the versatility and scalability of hardware accelerators, the present disclosure proposes an optimization method and optimization system for non-multiplication-addition computing operations in FPGA hardware accelerators by adopting a solution of merging calculations combined with pipeline optimization on the basis of the existing solution of splitting non-multiplication-addition computing operations into multiple basic calculations, optimizing the accelerator performance from the computational graph operation execution level, and can be applied to the acceleration of deep neural networks in one or more FPGA devices. In addition, based on the limitation of hardware resources in FPGA and the repeated occurrence of merge operations in the computational graph, the present disclosure proposes a complete exploration process for merging and optimizing non-multiplication-addition computing operations, and effectively improves the acceleration performance of FPGA-based deep neural network accelerators by balancing resource utilization and optimization effects. In addition, the optimization method and optimization system for non-multiplication-addition computing operations in FPGA hardware accelerators proposed in the present disclosure can be used not only for FPGA hardware accelerators of deep neural networks, but also for FPGA hardware accelerators of any network or model that requires a large number of different types of non-multiplication-addition computing operations while performing multiplication-addition computing operations.

[0043] Next, we will refer to Figures 1 to 5 The optimization method and optimization system of non-multiplication-addition type computing operations in an FPGA hardware accelerator according to an exemplary embodiment of the present disclosure are described in detail.

[0044] Figure 1 The flowchart of the method for optimizing non-multiplication-addition computing operations in the FPGA hardware accelerator according to the exemplary embodiment of the present disclosure is shown.

[0045] Reference Figure 1 In step 101, a first computation graph is generated by decomposing a non-multiplication-addition type computation operation into a plurality of basic operations. In step 102, at least one merged operation is obtained by merging consecutive basic operations in the first computation graph. In step 103, a second computation graph is generated based on the first computation graph and the at least one merged operation. Steps 101 to 103 will be described in detail below.

[0046] In engineering practice, non-multiplication and addition operations can be decomposed into basic operations such as addition, subtraction, multiplication, and magnitude in software analysis to generate the first calculation graph. If the operations are performed sequentially on the hardware accelerator according to the decomposed basic operations, a large number of new basic operations will be generated in the first calculation graph, and these basic operations are often interspersed between matrix multiplication calculations. Since the hardware accelerator needs to execute these basic operations sequentially, its acceleration performance will be affected. Since these basic operations are executed continuously in the calculation graph, you can try to merge them into a set of operations and apply pipeline optimization. Since pipeline optimization can greatly improve the performance of combined operations, this method can effectively improve the performance of the hardware accelerator.

[0047] Since these combined "merge operations" have their own independent operation order and operation logic, these newly generated merge operations often cannot reuse the hardware circuits of the previous basic operations for calculation. In other words, new operation circuits need to be deployed on the hardware accelerator to perform these merge operations. Similarly, merge operations synthesized by basic operations of different orders and types cannot reuse the same calculation circuits. If multiple merge operations are generated after applying merge optimization, such as merge operation A, merge operation B, merge operation C..., it is necessary to generate the corresponding operation circuit for each merge operation in the hardware optimizer.

[0048] For example, Figure 2 is a schematic diagram illustrating a merging operation according to an exemplary embodiment of the present disclosure. Figure 2 (a) in FIG. 1 shows a first computational graph generated by decomposing a non-multiplication-addition computational operation into a plurality of basic operations. Figure 2 (b) in FIG. 1 shows a second computation graph generated by the merging operation.

[0049] According to an exemplary embodiment of the present disclosure, the continuous basic operations that appear repeatedly multiple times in the first computation graph may be merged to obtain a merged operation, and a new computation graph, that is, a second computation graph, may be regenerated based on the first computation graph and the obtained merged operation. Figure 2 As shown in the figure, in the first calculation graph, operations 2 and 3 are continuous basic operations, which are repeated multiple times, and the types and orders of operations are consistent. Therefore, operations 2 and 3 in the first calculation graph can be merged to obtain a merged operation A, thereby generating a second calculation graph. For such repeated merged operations A, the same hardware circuit can be reused for accelerated operations.

[0050] According to another exemplary embodiment of the present disclosure, any continuous basic operations in the first computation graph that can improve acceleration performance based on pipeline optimization after merging can be merged to obtain a merged operation, and a new computation graph, i.e., a second computation graph, is regenerated based on the first computation graph and the obtained merged operation. The exploration process of merge optimization will be described later.

[0051] Return to reference Figure 1 In step 104, operations are performed in the FPGA hardware accelerator according to the second computation graph, wherein each merge operation of the at least one merge operation is performed using pipeline optimization.

[0052] Pipeline optimization is a common optimization method in hardware implementation. Pipeline optimization overlaps operations at different steps so that multiple operations can be executed in parallel, thereby greatly improving the running speed of the overall program and effectively improving the utilization efficiency of hardware resources. For example, in the traditional computing execution mode of finite state machines, it often takes multiple clock cycles for a hardware accelerator to complete a set of operations. Assuming that each operation requires 1 clock cycle, it takes N×M clock cycles to perform a set of M operations on N data items. After applying pipeline optimization, it now only takes N+M-1 clock cycles to perform a set of M operations on N data items. Therefore, in the design of hardware accelerators, pipeline optimization is widely used to improve the performance of calculations. For continuous basic operations, although the clock cycles required for each operation are different, these continuous basic operations can be merged into a set of operations (i.e., merged operations). By overlapping operations in subdivided steps and applying pipeline optimization, the computing performance can be greatly improved.

[0053] According to an exemplary embodiment of the present disclosure, for each merge operation, the calculation process of the merge operation can be decomposed into different pipeline stages, and different pipeline stages are executed in parallel in each clock cycle. For example, Figure 3 is a schematic diagram showing pipeline optimization according to an exemplary embodiment of the present disclosure. Figure 3 (a) in FIG. 1 shows a periodogram of performing successive basic operations on each data item. Figure 3 (b) in FIG. 5 shows a cycle diagram of executing operations after combining consecutive basic operations and applying pipeline optimization.

[0054] Reference Figure 3 , assuming that the continuous basic operations include calculation 1, calculation 2 and calculation 3, calculation 1 requires 6 clock cycles, calculation 2 requires 4 clock cycles, and calculation 3 requires 7 clock cycles. Figure 3 As shown in (a) in FIG. 1 , when the operation is performed according to the original calculation graph (for example, the first calculation graph), a total of (3×(6+4+7)=51) clock cycles are required to complete the operation of 3 data items. Figure 3 As shown in (b), calculation 1, calculation 2 and calculation 3 are combined into a combined operation to generate a new calculation graph (for example, a second calculation graph). When the operation is performed according to the second calculation graph, the combined operation is optimized using pipeline optimization. Only (3+6+4+7-1=19) clock cycles are required to complete the same calculation, which greatly improves the performance of the accelerator.

[0055] The exploration process for obtaining the merge operation will be described in detail below.

[0056] According to the exemplary embodiment of the present disclosure, the exploration process of obtaining the merge operation is based on the rewritable characteristics of the FPGA device, and effectively mines the continuous calculation combinations that appear repeatedly in the calculation graph for merging. Within the scope of the available resources on the FPGA chip, the utilization rate of hardware resources is increased as much as possible to improve the performance of the deep neural network acceleration on the FPGA.

[0057] According to an exemplary embodiment of the present disclosure, based on the total amount of resources S of the FPGA hardware accelerator and the first resource usage U required for the multiple basic operations included in the first calculation graph, a greedy algorithm is used to explore the combination of continuous basic operations that can be merged, and the explored combination of continuous basic operations that can be merged is merged into a merged operation. Here, the total amount of resources S of the FPGA hardware accelerator can be determined based on the total amount of resources indicated in the relevant parameter table of the FPGA hardware. The first resource usage U required for the multiple basic operations included in the first calculation graph can be determined based on the resources occupied by executing the operation according to the first calculation graph in the FPGA hardware accelerator.

[0058] The following will refer to Figure 4 A method of exploring a combination of consecutive basic operations that can be merged according to an exemplary embodiment of the present disclosure is described in detail. Figure 4 is a flowchart illustrating a method of exploring a merge operation according to an exemplary embodiment of the present disclosure.

[0059] Reference Figure 4 In step 401, candidate combinations of all continuous basic operations in the first computation graph may be searched. For example, all possible combined continuous basic operations in the first computation graph may be searched as candidate combinations by exhaustive search. Of course, the present disclosure is not limited thereto, and any feasible search method may be used according to user needs.

[0060] In step 402, acceleration factors for reflecting acceleration effects of all candidate combinations may be calculated.

[0061] According to an exemplary embodiment of the present disclosure, for a group of continuous operation combinations that can be merged in a calculation graph, assuming that the average number of operation time cycles required is T, the average number of calculation data items is N, and the number of occurrences is R, then the number of clock cycles reduced after merging and using pipeline optimization is R(T×NN-T+1), where T, N>>1. Therefore, the acceleration factor can be defined as F=R×T×N. The larger the acceleration factor, the greater the number of clock cycles reduced after optimization, and the better the acceleration effect. That is, for each candidate combination, the average number of operation time cycles required, the average number of calculation data items, and the number of occurrences of the candidate combination in the first calculation graph are calculated, and the three are multiplied together as the acceleration factor of the candidate combination. Of course, the present disclosure is not limited to this, and any feasible solution can also be used to define the acceleration factor used to reflect the acceleration effect.

[0062] In step 403, all candidate combinations may be sorted according to the calculated acceleration factors. For example, all candidate combinations may be sorted in descending order of the acceleration factors to generate a candidate combination list, that is, in the candidate combination list, candidate combinations with larger acceleration factors are ranked higher. Of course, the present disclosure is not limited thereto, and any feasible solution may be used to sort all candidate combinations.

[0063] In step 404, the second resource usage A required by the candidate combination with the largest acceleration factor in the sorting result may be obtained. Here, the second resource usage A may be determined based on the resources occupied by executing the candidate combination using pipeline optimization in the FPGA hardware accelerator. For example, when in the candidate combination list sorted in descending order of acceleration factor, the second resource usage required by the first candidate combination in the candidate combination list may be obtained.

[0064] In step 405, it may be determined whether the sum of the first resource usage U and the second resource usage A exceeds the total resource amount S of the FPGA hardware accelerator. In other words, it is determined whether U+A>S.

[0065] In step 406, when the sum of the first resource usage U and the second resource usage A does not exceed the total resource amount S of the FPGA hardware accelerator, that is, U+A≤S, the candidate combination with the largest acceleration factor (for example, the first candidate combination in the list of candidate combinations arranged in descending order of acceleration factor) can be determined as a combination that can be merged, and based on the candidate combination, the first computation graph and the first resource usage U are updated.

[0066] Here, when U+A≤S, it means that the FPGA hardware accelerator is capable of running the updated calculation graph (i.e., running the original calculation graph with the original hardware circuit + running the candidate combination with the new hardware circuit), then the candidate combination can be determined as a combination that can be merged, and the candidate combination can be introduced as a new merge operation. Whenever a new merge operation is introduced, it is necessary to regenerate a new first calculation graph based on the current first calculation graph and the new merge operation, that is, update the first calculation graph, and it is also necessary to update the first resource usage to the sum of the current first resource usage and the second resource usage, that is, U=U+A. Then, enter the next exploration.

[0067] In step 407, when the sum of the first resource usage U and the second resource usage A exceeds the total resource amount S of the FPGA hardware accelerator, that is, U+A>S, it can be determined whether there are other unexplored candidate combinations. If so, in step 408, the candidate combination with the acceleration factor second only to the current candidate combination in the sorting result can be updated to the candidate combination with the largest acceleration factor in the sorting result (for example, the next candidate combination of the current candidate combination in the candidate combination list sorted in descending order of acceleration factor) and jump to step 404. If not, in step 409, the exploration can be ended.

[0068] Here, when U+A>S, it means that the FPGA hardware accelerator is not capable of running the updated computation graph (i.e., running the original computation graph with the original hardware circuit + running the candidate combination with the new hardware circuit), then the candidate combination can be abandoned and the next candidate combination with the next best acceleration factor can be explored, that is, for the next candidate combination, steps 404 to 406 or steps 404, 405, 407 and 408 or steps 404, 405, 407 and 409 are executed. And so on.

[0069] In addition, when the first computation graph is updated in step 406, it is necessary to search and sort all possible candidate combinations in the updated first computation graph again, and execute the same exploration scheme. In other words, steps 401 to 409 can be executed cyclically until the exploration is completed.

[0070] Figure 5 is a block diagram of an optimization system for non-multiply-add computing operations in an FPGA hardware accelerator according to an exemplary embodiment of the present disclosure.

[0071] Reference Figure 5 According to an exemplary embodiment of the present disclosure, an optimization system 500 for non-multiplication-addition type computing operations in an FPGA hardware accelerator (hereinafter referred to as the optimization system 500 ) includes a computing graph generating device 501 and an operation executing device 502 .

[0072] The calculation graph generation device 501 can generate a first calculation graph by decomposing a non-multiplication-addition type calculation operation into multiple basic operations, obtain at least one merged operation by merging consecutive basic operations in the first calculation graph, and generate a second calculation graph based on the first calculation graph and the at least one merged operation. According to an exemplary embodiment of the present disclosure, the calculation graph generation device 501 may include a first calculation graph generation unit (not shown), a merged operation acquisition unit (not shown), and a second calculation graph generation unit (not shown), wherein the first calculation graph generation unit is used to generate a first calculation graph by decomposing a non-multiplication-addition type calculation operation into multiple basic operations, the merged operation acquisition unit is used to obtain at least one merged operation by merging consecutive basic operations in the first calculation graph, and the second calculation graph generation unit is used to generate a second calculation graph based on the first calculation graph and the at least one merged operation. The operation of the calculation graph generation device 501 (e.g., the first calculation graph generation unit, the merged operation acquisition unit, and the second calculation graph generation unit) will be described in detail below.

[0073] In engineering practice, the calculation graph generation device 501 (for example, the first calculation graph generation unit) can decompose non-multiplication and addition type calculation operations into basic operations such as addition, subtraction, multiplication, and size in software analysis, thereby generating a first calculation graph. If the operations are performed sequentially on the hardware accelerator according to the decomposed basic operations, a large number of new basic operations will be generated in the first calculation graph, and these basic operations are often interspersed between matrix multiplication calculations. Since the hardware accelerator needs to execute these basic operations sequentially, its acceleration performance will be affected. Since these basic operations are executed continuously in the calculation graph, you can try to merge them into a set of operations and apply pipeline optimization. Since pipeline optimization can greatly improve the performance of combined operations, this method can effectively improve the performance of the hardware accelerator.

[0074] Since these combined "merge operations" have their own independent operation order and operation logic, these newly generated merge operations often cannot reuse the hardware circuits of the previous basic operations for calculation. In other words, new operation circuits need to be deployed on the hardware accelerator to perform these merge operations. Similarly, merge operations synthesized by basic operations of different orders and types cannot reuse the same calculation circuits. If multiple merge operations are generated after applying merge optimization, such as merge operation A, merge operation B, merge operation C..., it is necessary to generate the corresponding operation circuit for each merge operation in the hardware optimizer.

[0075] According to an exemplary embodiment of the present disclosure, the computation graph generation device 501 (e.g., the merged operation acquisition unit) may merge the continuous basic operations that appear repeatedly multiple times in the first computation graph to obtain a merged operation, and the computation graph generation device 501 (e.g., the second computation graph generation unit) may regenerate a new computation graph, i.e., the second computation graph, based on the first computation graph and the obtained merged operation. Figure 2 As shown in the figure, in the first calculation graph, operations 2 and 3 are continuous basic operations, which are repeated multiple times, and the types and orders of operations are consistent. Therefore, operations 2 and 3 in the first calculation graph can be merged to obtain a merged operation A, thereby generating a second calculation graph. For such repeated merged operations A, the same hardware circuit can be reused for accelerated operations.

[0076] According to another exemplary embodiment of the present disclosure, the computation graph generation device 501 (e.g., the merged operation acquisition unit) may merge any continuous basic operations in the first computation graph that can improve the acceleration performance based on pipeline optimization after merging to obtain a merged operation, and the computation graph generation device 501 (e.g., the second computation graph generation unit) may regenerate a new computation graph based on the first computation graph and the obtained merged operation, that is, the second computation graph. The exploration process of the merged optimization of the merged operation acquisition unit will be described later.

[0077] The operation execution device 502 can execute the operation in the FPGA hardware accelerator according to the second computation graph, wherein each merge operation of the at least one merge operation is executed using pipeline optimization.

[0078] Pipeline optimization is a common optimization method in hardware implementation. Pipeline optimization overlaps operations at different steps so that multiple operations can be executed in parallel, thereby greatly improving the running speed of the overall program and effectively improving the utilization efficiency of hardware resources. For example, in the traditional computing execution mode of finite state machines, it often takes multiple clock cycles for a hardware accelerator to complete a set of operations. Assuming that each operation requires 1 clock cycle, it takes N×M clock cycles to perform a set of M operations on N data items. After applying pipeline optimization, it now only takes N+M-1 clock cycles to perform a set of M operations on N data items. Therefore, in the design of hardware accelerators, pipeline optimization is widely used to improve the performance of calculations. For continuous basic operations, although the clock cycles required for each operation are different, these continuous basic operations can be merged into a set of operations (i.e., merged operations). By overlapping operations in subdivided steps and applying pipeline optimization, the computing performance can be greatly improved.

[0079] According to an exemplary embodiment of the present disclosure, the operation execution device 502 may decompose the calculation process of each merge operation into different pipeline stages, and execute the different pipeline stages in parallel in each clock cycle.

[0080] The following specifically describes the exploration process of the computation graph generating device 501 (eg, the merge operation acquisition unit) to obtain the merge operation.

[0081] According to the exemplary embodiment of the present disclosure, the computation graph generation device 501 (e.g., the merge operation acquisition unit) obtains the exploration process of the merge operation based on the rewritable characteristics of the FPGA device, and effectively mines the continuous calculation combinations that appear repeatedly in the computation graph for merging. Within the scope of the available resources on the FPGA chip, the utilization rate of hardware resources is increased as much as possible to improve the performance of the deep neural network acceleration on the FPGA.

[0082] According to an exemplary embodiment of the present disclosure, the calculation graph generation device 501 (for example, a merge operation acquisition unit) can use a greedy algorithm to explore the combination of continuous basic operations that can be merged based on the total resource amount S of the FPGA hardware accelerator and the first resource usage U required for the multiple basic operations included in the first calculation graph, and merge the explored combination of continuous basic operations that can be merged into a merge operation. Here, the total resource amount S of the FPGA hardware accelerator can be determined based on the total resource amount indicated in the relevant parameter table of the FPGA hardware. The first resource usage U required for the multiple basic operations included in the first calculation graph can be determined based on the resources occupied by executing operations according to the first calculation graph in the FPGA hardware accelerator.

[0083] The computation graph generating device 501 (e.g., the merge operation acquisition unit) may execute the following steps: Figure 4 The method for exploring the combination of continuous basic operations that can be merged according to an exemplary embodiment of the present disclosure is shown. That is, the calculation graph generation device 501 (for example, the merge operation acquisition unit) can perform the following steps:

[0084] In step 401, candidate combinations of all continuous basic operations in the first computation graph may be searched. For example, all possible combined continuous basic operations in the first computation graph may be searched as candidate combinations by exhaustive search. Of course, the present disclosure is not limited thereto, and any feasible search method may be used according to user needs.

[0085] In step 402, acceleration factors for reflecting acceleration effects of all candidate combinations may be calculated.

[0086] According to an exemplary embodiment of the present disclosure, for a group of continuous operation combinations that can be merged in a calculation graph, assuming that the average number of operation time cycles required is T, the average number of calculation data items is N, and the number of occurrences is R, then the number of clock cycles reduced after merging and using pipeline optimization is R(T×NN-T+1), where T, N>>1. Therefore, the acceleration factor can be defined as F=R×T×N. The larger the acceleration factor, the greater the number of clock cycles reduced after optimization, and the better the acceleration effect. That is, for each candidate combination, the average number of operation time cycles required, the average number of calculation data items, and the number of occurrences of the candidate combination in the first calculation graph are calculated, and the three are multiplied together as the acceleration factor of the candidate combination. Of course, the present disclosure is not limited to this, and any feasible solution can also be used to define the acceleration factor used to reflect the acceleration effect.

[0087] In step 403, all candidate combinations may be sorted according to the calculated acceleration factors. For example, all candidate combinations may be sorted in descending order of the acceleration factors to generate a candidate combination list, that is, in the candidate combination list, candidate combinations with larger acceleration factors are ranked higher. Of course, the present disclosure is not limited thereto, and any feasible solution may be used to sort all candidate combinations.

[0088] In step 404, the second resource usage A required by the candidate combination with the largest acceleration factor in the sorting result may be obtained. Here, the second resource usage A may be determined based on the resources occupied by executing the candidate combination using pipeline optimization in the FPGA hardware accelerator. For example, when in the candidate combination list sorted in descending order of acceleration factor, the second resource usage required by the first candidate combination in the candidate combination list may be obtained.

[0089] In step 405, it may be determined whether the sum of the first resource usage U and the second resource usage A exceeds the total resource amount S of the FPGA hardware accelerator. In other words, it is determined whether U+A>S.

[0090] In step 406, when the sum of the first resource usage U and the second resource usage A does not exceed the total resource amount S of the FPGA hardware accelerator, that is, U+A≤S, the candidate combination with the largest acceleration factor (for example, the first candidate combination in the list of candidate combinations arranged in descending order of acceleration factor) can be determined as a combination that can be merged, and based on the candidate combination, the first computation graph and the first resource usage U are updated.

[0091] Here, when U+A≤S, it means that the FPGA hardware accelerator is capable of running the updated calculation graph (i.e., running the original calculation graph with the original hardware circuit + running the candidate combination with the new hardware circuit), then the candidate combination can be determined as a combination that can be merged, and the candidate combination can be introduced as a new merge operation. Whenever a new merge operation is introduced, it is necessary to regenerate a new first calculation graph based on the current first calculation graph and the new merge operation, that is, update the first calculation graph, and it is also necessary to update the first resource usage to the sum of the current first resource usage and the second resource usage, that is, U=U+A. Then, enter the next exploration.

[0092] In step 407, when the sum of the first resource usage U and the second resource usage A exceeds the total resource amount S of the FPGA hardware accelerator, that is, U+A>S, it can be determined whether there are other unexplored candidate combinations. If so, in step 408, the candidate combination with the acceleration factor second only to the current candidate combination in the sorting result can be updated to the candidate combination with the largest acceleration factor in the sorting result (for example, the next candidate combination of the current candidate combination in the candidate combination list sorted in descending order of acceleration factor) and jump to step 404. If not, in step 409, the exploration can be ended.

[0093] Here, when U+A>S, it means that the FPGA hardware accelerator is not capable of running the updated computation graph (i.e., running the original computation graph with the original hardware circuit + running the candidate combination with the new hardware circuit), then the candidate combination can be abandoned and the next candidate combination with the next best acceleration factor can be explored, that is, for the next candidate combination, steps 404 to 406 or steps 404, 405, 407 and 408 or steps 404, 405, 407 and 409 are executed. And so on.

[0094] In addition, when the first computation graph is updated in step 406, it is necessary to re-search and sort all possible candidate combinations in the updated first computation graph, and execute the same exploration scheme. That is, the computation graph generating device 501 (e.g., the merge operation acquisition unit) can cyclically execute steps 401 to 409 until the exploration is completed.

[0095] According to the exemplary embodiment of the present disclosure, the optimization method and optimization system of non-multiplication-addition type computing operations in the FPGA hardware accelerator can be used for non-multiplication-addition type computing operations in the deep neural network accelerator. However, the present disclosure is not limited to this, and the optimization method and optimization system of non-multiplication-addition type computing operations in the FPGA hardware accelerator according to the exemplary embodiment of the present disclosure can also be used for non-multiplication-addition type computing operations in any network or model that requires a large number of different types of non-multiplication-addition type computing operations at the same time as the multiplication-addition type computing operations.

[0096] According to the optimization method and optimization system of non-multiplication-addition type computing operations in the FPGA hardware accelerator of the exemplary embodiment of the present disclosure, the non-multiplication-addition type computing operations are split into multiple basic calculations, and a solution of merging calculations combined with pipeline optimization is adopted on the basis of the multiple basic calculations, which not only improves the computing performance of the non-multiplication-addition type computing, but also increases the versatility and scalability of the hardware accelerator.

[0097] In addition, according to the optimization method and optimization system of non-multiplication-addition computing operations in the FPGA hardware accelerator of the exemplary embodiment of the present disclosure, based on the erasable and rewritable characteristics of FPGA devices, continuous computing combinations that appear repeatedly in the computing graph are effectively explored, thereby improving the utilization rate of hardware resources and enhancing the performance of deep neural network acceleration on FPGA.

[0098] The above has been referred to Figures 1 to 5 An optimization method and an optimization system for non-multiply-add computing operations in an FPGA hardware accelerator according to an exemplary embodiment of the present disclosure are described.

[0099] Figure 5 The computation graph generation device in the optimization system of non-multiplication-addition computing operations in the FPGA hardware accelerator shown can be configured as software, hardware, firmware, or any combination of the above items to perform specific functions. For example, each unit may correspond to a dedicated integrated circuit, or to a pure software code, or to a module combining software and hardware. In addition, one or more functions implemented by each unit may also be uniformly executed by components in a physical entity device (e.g., a processor, a client, or a server, etc.).

[0100] In addition, refer to Figure 5 The operation method performed by the computation graph generation device described can be implemented by a program (or instruction) recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present disclosure, a computer-readable storage medium storing instructions can be provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the operation performed by the computation graph generation device according to the present disclosure.

[0101] The computer program in the computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the computer program can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been described in reference to Figure 5 The above is mentioned in the description of the related method performed by the calculation graph generation device, so it will not be repeated here to avoid repetition.

[0102] It should be noted that the various units of the computational graph generation device in the optimization system for non-multiplication-addition type computing operations in the FPGA hardware accelerator according to the exemplary embodiment of the present disclosure can completely rely on the operation of the computer program to realize the corresponding functions, that is, each unit corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (for example, lib library) to realize the corresponding function.

[0103] on the other hand, Figure 5 The various units of the computation graph generation device shown may also be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented by software, firmware, middleware, or microcode, the program code or code segment for performing the corresponding operation may be stored in a computer-readable medium such as a storage medium, so that the processor may perform the corresponding operation by reading and running the corresponding program code or code segment.

[0104] For example, an exemplary embodiment of the present disclosure may also be implemented as a computing device, which includes a storage component and a processor, wherein a set of computer-executable instructions is stored in the storage component. When the set of computer-executable instructions is executed by the processor, operations performed by a computational graph generating device according to an exemplary embodiment of the present disclosure are performed.

[0105] Specifically, the computing device can be deployed in a server or client, or can be deployed on a node device in a distributed network environment. In addition, the computing device can be a PC, a tablet device, a personal digital assistant, a smart phone, a web application, or other device capable of executing the above instruction set.

[0106] Here, the computing device is not necessarily a single computing device, but may also be any device or circuit collection that can execute the above instructions (or instruction sets) individually or jointly. The computing device may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0107] In a computing device, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, a processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0108] According to the exemplary embodiments of the present disclosure, some operations described in the operations performed by the computational graph generating device may be implemented by software, some operations may be implemented by hardware, and furthermore, these operations may be implemented by a combination of software and hardware.

[0109] The processor may execute instructions or codes stored in one of the storage components, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0110] The storage component may be integrated with the processor, for example, RAM or flash memory is arranged within an integrated circuit microprocessor, etc. In addition, the storage component may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The storage component and the processor may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor can read files stored in the storage component.

[0111] In addition, the computing device may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the computing device may be connected to each other via a bus and / or a network.

[0112] The operations performed by the computation graph generation device according to the exemplary embodiments of the present disclosure may be described as various interconnected or coupled functional blocks or functional diagrams. However, these functional blocks or functional diagrams may be equally integrated into a single logical device or operated according to non-exact boundaries.

[0113] Therefore, refer to Figure 5 The operations described as being performed by the computation graph generation device may be implemented by a system comprising at least one computing device and at least one storage device storing instructions.

[0114] According to an exemplary embodiment of the present disclosure, at least one computing device is a computing device for executing a method for assisting manual text annotation according to an exemplary embodiment of the present disclosure, and a computer executable instruction set is stored in a storage device. When the computer executable instruction set is executed by the at least one computing device, the operations performed by the computational graph generation device are performed.

[0115] The above describes various exemplary embodiments of the present disclosure, and it should be understood that the above description is only exemplary and not exhaustive, and the present disclosure is not limited to the disclosed exemplary embodiments. Without departing from the scope and spirit of the present disclosure, many modifications and changes are obvious to those of ordinary skill in the art. Therefore, the scope of protection of the present disclosure should be based on the scope of the claims.

Claims

1. A method for optimizing non-multiplication-addition computing operations in an FPGA hardware accelerator, comprising: Generate a first calculation graph by decomposing non-multiplication-addition calculation operations into multiple basic operations; At least one merged operation is obtained by merging consecutive basic operations in the first computation graph; generating a second computation graph based on the first computation graph and the at least one merge operation; Execute operations in the FPGA hardware accelerator according to the second computation graph; The performing of operations in the FPGA hardware accelerator according to the second computation graph includes: Each merge operation of the at least one merge operation is performed using pipeline optimization.

2. The optimization method according to claim 1, wherein: The step of merging the continuous basic operations in the first computation graph to obtain at least one merged operation includes: Based on the total amount of resources of the FPGA hardware accelerator and the first resource usage required for the plurality of basic operations, using a greedy algorithm to explore a combination of continuous basic operations that can be merged; The explored combinations of consecutive basic operations that can be merged are merged into merge operations.

3. The optimization method according to claim 2, wherein: The total amount of resources of the FPGA hardware accelerator and the amount of resources required for the multiple basic operations are used to explore the combination of continuous basic operations that can be merged using a greedy algorithm, including: Loop through the following steps until the exploration is complete: (1) Search for candidate combinations of all consecutive basic operations in the first computation graph; (2) Calculate the acceleration factors of all candidate combinations to reflect the acceleration effect; (3) Sort all candidate combinations according to the calculated acceleration factors; (4) obtaining the second resource usage required for the candidate combination with the largest acceleration factor in the sorting result; (5) determining whether the sum of the first resource usage and the second resource usage exceeds the total resource amount; (6) When the sum of the first resource usage and the second resource usage does not exceed the total resource amount, determine the candidate combination with the largest acceleration factor as a combination that can be merged, and update the first computation graph and the first resource usage based on the candidate combination; (7) When the sum of the first resource usage and the second resource usage exceeds the total resource amount, determine whether there are other unexplored candidate combinations. If so, update the candidate combination in the sorting result whose acceleration factor is second only to the current candidate combination to the candidate combination with the largest acceleration factor in the sorting result and jump to operation (4). If not, end the exploration.

4. The optimization method according to claim 3, wherein: The calculation of the acceleration factors for all candidate combinations to reflect the acceleration effect includes: For each candidate combination, the average required operation time cycles, the average number of calculation data items, and the number of occurrences of the candidate combination in the first calculation graph are calculated, and the three are multiplied together to obtain the acceleration factor of the candidate combination.

5. The optimization method according to claim 3, wherein: The updating of the first resource usage includes: The first resource usage is updated to the sum of the first resource usage and the second resource usage.

6. The optimization method according to claim 1, wherein: Utilizing pipeline optimization to perform each of the at least one merge operation, comprising: For each merge operation, the calculation process of the merge operation is decomposed into different pipeline stages, and the different pipeline stages are executed in parallel in each clock cycle.

7. The optimization method according to claim 1, wherein: The FPGA hardware accelerator is a deep neural network accelerator.

8. An optimization system for non-multiplication-addition computing operations in an FPGA hardware accelerator, comprising: A computational graph generating device; A calculation execution device; Wherein, the computation graph generating device is configured as follows: The first computation graph is generated by decomposing non-multiplication-addition computation operations into multiple basic operations. At least one merged operation is obtained by merging consecutive basic operations in the first computation graph, generating a second computation graph based on the first computation graph and the at least one merge operation; Wherein, the operation execution device is configured as: Execute operations in the FPGA hardware accelerator according to the second computation graph; The operation execution device uses pipeline optimization to execute each merge operation in the at least one merge operation.

9. The optimization system according to claim 8, wherein the computation graph generating device is configured to: Based on the total amount of resources of the FPGA hardware accelerator and the first resource usage required for the plurality of basic operations, using a greedy algorithm to explore a combination of continuous basic operations that can be merged; The explored combinations of consecutive basic operations that can be merged are merged into merge operations.

10. The optimization system of claim 9, wherein: The computation graph generator is configured as follows: Loop through the following steps until the exploration is complete: (1) Search for candidate combinations of all consecutive basic operations in the first computation graph; (2) Calculate the acceleration factors of all candidate combinations to reflect the acceleration effect; (3) Sort all candidate combinations according to the calculated acceleration factors; (4) obtaining a second resource usage required by a first candidate combination in the candidate combination list; (5) determining whether the sum of the first resource usage and the second resource usage exceeds the total resource amount; (6) When the first resource usage and the second resource usage do not exceed the total resource amount, determine the candidate combination with the largest acceleration factor as a combination that can be merged, and update the first computation graph and the first resource usage based on the candidate combination; (7) When the sum of the first resource usage and the second resource usage exceeds the total resource amount, determine whether there are other unexplored candidate combinations. If so, update the candidate combination in the sorting result whose acceleration factor is second only to the current candidate combination to the candidate combination with the largest acceleration factor in the sorting result and jump to operation (4). If not, end the exploration.

11. The optimization system of claim 10, wherein: The computation graph generator is configured as follows: For each candidate combination, the average required operation time cycles, the average number of calculation data items, and the number of occurrences of the candidate combination in the first calculation graph are calculated, and the three are multiplied together to obtain the acceleration factor of the candidate combination.

12. The optimization system of claim 10, wherein: The computation graph generator is configured as follows: The first resource usage is updated to the sum of the first resource usage and the second resource usage.

13. The optimization system of claim 8, wherein: The operation execution device is configured as follows: For each merge operation, the calculation process of the merge operation is decomposed into different pipeline stages, and the different pipeline stages are executed in parallel in each clock cycle.

14. The optimization system of claim 8, wherein: The FPGA hardware accelerator is a deep neural network accelerator.

15. A system comprising at least one computing device and at least one storage device storing instructions, wherein: When the instructions are executed by the at least one computing device, the at least one computing device is prompted to perform the operations performed by the computation graph generating device as claimed in any one of claims 8 to 12.

16. A computer-readable storage medium storing instructions, wherein: When the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the operations performed by the computation graph generating device as claimed in any one of claims 8 to 12.

Images