A hybrid accelerator for convolutional neural networks and its load balancing method

Abstract

The present invention discloses a hybrid accelerator for convolutional neural networks and a load balancing method thereof, which belongs to the technical field of deep learning hardware acceleration, and is used to solve the technical problem that the two current dedicated accelerator architectures for convolutional neural networks cannot realize the flexible allocation and efficient utilization of computing resources of the computing platform, and the improvement of the computing performance of the convolutional neural network is limited. The hybrid accelerator includes: a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component; the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected through a double buffer; the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to process different batches of input data at the same time; wherein the input data is the data to be calculated of each convolutional layer of the convolutional neural network; the double buffer is used to store the output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning hardware acceleration, and in particular to a hybrid accelerator for a convolutional neural network and a load balancing method thereof. Background Art

[0002] With the rapid development of deep learning technology, convolutional neural networks have become a key tool for processing complex data such as images, videos, and speech. However, the computationally intensive and memory-access intensive characteristics of convolutional neural networks place extremely high demands on computing hardware, especially when dealing with large-scale data sets and deep network structures. Although traditional computing platforms, such as CPUs and GPUs, can support the operation of convolutional neural networks, they often cannot meet the needs of practical applications in terms of energy efficiency, latency, and throughput.

[0003] To overcome these challenges, researchers have proposed a variety of dedicated accelerator architectures designed to optimize the characteristics of convolutional neural networks. Among them, single-layer multi-engine accelerators and single-layer single-engine accelerators are the two main architectural directions. Single-layer multi-engine accelerators process multiple layers by reusing computing resources, which has the advantage of high resource utilization, but ignores the possible computing heterogeneity between and within layers. In contrast, single-layer single-engine accelerators allocate independent computing engines to each layer of the convolutional neural network, which can better adapt to the computing needs of different layers, but consumes a lot of resources and is not easy to expand.

[0004] Therefore, the two current convolutional neural network dedicated accelerator architectures cannot achieve flexible allocation and efficient utilization of computing platform computing resources, and have limited effect on improving the computing performance of convolutional neural networks. Summary of the invention

[0005] The embodiment of the present invention provides a hybrid accelerator for convolutional neural networks and a load balancing method thereof, which are used to solve the following technical problems: the two current convolutional neural network dedicated accelerator architectures cannot achieve flexible allocation and efficient utilization of computing platform computing resources, and have limited improvement on the computing performance of convolutional neural networks.

[0006] The embodiment of the present invention adopts the following technical solution:

[0007] On the one hand, an embodiment of the present invention provides a hybrid accelerator for a convolutional neural network, the hybrid accelerator comprising: a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component;

[0008] The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected via a double buffer;

[0009] The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to simultaneously process different batches of input data; wherein the input data is the data to be calculated for each convolutional layer of the convolutional neural network;

[0010] The double buffer is used to store output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0011] In a feasible implementation, the single-layer single-engine architecture acceleration component is composed of a plurality of first-connected single-layer single engines, and elastic buffers are inserted between the single-layer single engines according to actual needs to form a pipeline structure.

[0012] In a feasible implementation, each of the single-layer single engines includes an independent weight buffer area for storing weight parameters of the convolutional neural network.

[0013] In a feasible implementation manner, the input data is sequentially calculated by each single-layer single engine in the single-layer single-engine architecture acceleration component in a pipeline manner;

[0014] After the last single-layer single engine in the single-layer single-engine architecture acceleration component completes the calculation of a batch of input data, the calculation result is stored in the double buffer;

[0015] While the current single-layer single engine is processing the current batch of input data, the previous single-layer single engine is processing the next batch of input data.

[0016] In a feasible implementation, the single-layer multi-engine architecture acceleration component includes a channel-by-channel convolution engine and a point-by-point convolution engine;

[0017] The channel-by-channel convolution engine is used to complete the channel-by-channel convolution calculation task of the convolutional neural network;

[0018] The point-by-point convolution engine is used to complete the point-by-point convolution calculation task of the convolutional neural network.

[0019] On the other hand, an embodiment of the present invention further provides a hybrid accelerator load balancing method for a convolutional neural network, which is applied to a hybrid accelerator for a convolutional neural network, and the method includes:

[0020] Initialize the hybrid accelerator and obtain the computing unit data of the current computing platform;

[0021] Traverse each convolutional layer of the convolutional neural network and determine the segmentation points one by one;

[0022] For each partition point, determining the number of computing units required for the single-layer single-engine architecture acceleration component;

[0023] An optimal computing unit distribution strategy is determined according to the number of computing units corresponding to each segmentation point.

[0024] In a feasible implementation, initializing the hybrid accelerator and obtaining computing unit data of the current computing platform specifically includes:

[0025] Initialize the hybrid accelerator into the current computing platform, and obtain the computing unit data of the current computing platform; wherein the computing unit data includes at least: the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the initial number of computing units allocated to the single-layer single-engine architecture acceleration component, the initial distribution of computing units between the two parts of the acceleration components, and the overall throughput of the hybrid accelerator.

[0026] In a feasible implementation, traversing each convolution layer of the convolutional neural network and determining the segmentation points in sequence specifically includes:

[0027] Each layer of the convolutional neural network is traversed to determine each position in the convolutional neural network that can be segmented as the segmentation point.

[0028] In a feasible implementation manner, for each segmentation point, determining the number of computing units required for the single-layer single-engine architecture acceleration component specifically includes:

[0029] Based on the segmentation point, the convolutional neural network is divided into two parts, the first part is processed by the single-layer single-engine architecture acceleration component, and the second part is processed by the single-layer multi-engine architecture acceleration component;

[0030] The number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each split point is calculated cyclically, and the target split points whose number of computing units is within the constraint range are selected.

[0031] In a feasible implementation, according to the number of computing units corresponding to each segmentation point, an optimal computing unit distribution strategy is determined, specifically including:

[0032] Calculate the accelerator throughput corresponding to each target segmentation point according to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target segmentation point;

[0033] Determine the target split point with the minimum accelerator throughput as the optimal split point;

[0034] The convolutional neural network is segmented by the optimal segmentation point, and according to the final segmentation, the number of first computing units required by the single-layer single-engine architecture acceleration component and the number of second computing units required by the single-layer multi-engine architecture acceleration component are respectively calculated;

[0035] The optimal computing unit distribution strategy is determined according to the first computing unit quantity and the second computing unit quantity.

[0036] Compared with the prior art, the hybrid accelerator for convolutional neural network and the load balancing method thereof provided by the embodiment of the present invention have the following beneficial effects:

[0037] The architecture proposed in the present invention combines the advantages of single-layer multi-engine and single-layer single-engine, and realizes flexible allocation and efficient utilization of computing resources through a hybrid mapping strategy. In addition, the present invention also proposes a load balancing method for determining the split point and resource allocation scheme of a hybrid accelerator. The method intelligently selects the split point and optimizes the number of engines and resource configuration of the single-layer multi-engine and single-layer single-engine parts by analyzing the structural characteristics and resource budget constraints of the convolutional neural network. It can improve the utilization efficiency of limited resources and enhance computing flexibility, overcome the limitations of traditional accelerator architectures, improve the computing performance of convolutional neural networks and reduce energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0039] Figure 1 A schematic diagram of the architecture of a hybrid accelerator for a convolutional neural network provided by an embodiment of the present invention;

[0040] Figure 2 A flow chart of a hybrid accelerator load balancing method for a convolutional neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] First, the embodiment of the present invention provides a Figure 1The hybrid accelerator for convolutional neural networks shown in FIG. 1 mainly includes two parts: a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component.

[0043] The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected via a double buffer. The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to simultaneously process different batches of input data; wherein the input data is the data to be calculated for each convolution layer of the convolutional neural network.

[0044] The double buffer is used to store output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0045] As a feasible implementation method, Figure 1 As shown in the figure, there are two dotted boxes in the hybrid accelerator. The first dotted box is a single-layer single-engine architecture acceleration component, and the second dotted box is a single-layer multi-engine architecture acceleration component. The two acceleration components are connected through a double buffer ( Figure 1 not shown).

[0046] In one embodiment, assuming that a single-layer multi-engine architecture acceleration component is calculating a batch of input data batch(n), then at the same time, a single-layer single-engine architecture acceleration component is calculating the next batch of input data batch(n+1), where batch is a batch.

[0047] Furthermore, if Figure 1 As shown in the figure, the single-layer single-engine architecture acceleration component consists of several single-layer single engines connected in the first place. Elastic buffers are inserted between each single-layer single engine according to actual needs to form a pipeline structure. Each single-layer single engine contains an independent weight buffer area for storing the weight parameters of the convolutional neural network.

[0048] Furthermore, the input data is calculated by each single-layer single engine in the single-layer single engine architecture acceleration component in a pipeline manner. After the last single-layer single engine in the single-layer single engine architecture acceleration component completes the calculation of a batch of input data, the calculation result is stored in the double buffer. While the current single-layer single engine is processing the current batch of input data, the previous single-layer single engine is processing the next batch of input data.

[0049] As a feasible implementation method, in a single-layer single-engine architecture acceleration component, each engine corresponds to a convolutional layer of a convolutional neural network model and is used to process the data to be calculated in the corresponding convolutional layer.

[0050] In one embodiment, Figure 1As shown in the figure, assuming that engine 0 is calculating input data batch(n), engine 1 is calculating input data batch(n-1) at the same time. Assuming the length of the pipeline is m, when engine m-1 (i.e. the last engine) completes the calculation of the input data, it will store the calculation results in the double buffer connecting the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0051] Furthermore, the single-layer multi-engine architecture acceleration component includes a channel-by-channel convolution engine and a point-by-point convolution engine; the channel-by-channel convolution engine is used to complete the channel-by-channel convolution calculation task of the convolutional neural network; the point-by-point convolution engine is used to complete the point-by-point convolution calculation task of the convolutional neural network.

[0052] The above architecture combines the advantages of single-layer multi-engine and single-layer single-engine, which can improve computing flexibility and overcome the limitations of traditional accelerator architecture. However, the number of computing units in the computing platform is fixed. If the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are to run simultaneously, it is necessary to allocate specific computing units to the two components based on the computing load, so as to achieve optimal load balancing between the engines and achieve that the time for processing input data between the engines is basically the same, thereby improving the computing efficiency of the pipeline.

[0053] Based on this purpose, an embodiment of the present invention further provides a hybrid accelerator load balancing method for a convolutional neural network, such as Figure 2 As shown, the hybrid accelerator load balancing method for convolutional neural networks specifically includes steps S201-S204:

[0054] S201, initialize the hybrid accelerator and obtain computing unit data of the current computing platform.

[0055] Specifically, the hybrid accelerator is initialized into the current computing platform, and computing unit data of the current computing platform is obtained.

[0056] Among them, the computing unit data includes at least: the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the initial number of computing units allocated to the single-layer single-engine architecture acceleration component, the initial distribution of computing units between the two parts of the acceleration components, and the overall throughput of the hybrid accelerator.

[0057] S202, traverse each convolution layer of the convolutional neural network and determine the segmentation points in sequence.

[0058] Specifically, each layer of the convolutional neural network is traversed, and each position in the convolutional neural network that can be segmented is determined as a segmentation point.

[0059] S203: For each partition point, determine the number of computing units required for the single-layer single-engine architecture acceleration component.

[0060] Specifically, based on the split point, the convolutional neural network is divided into two parts, the first part is processed by a single-layer single-engine architecture acceleration component, and the second part is processed by a single-layer multi-engine architecture acceleration component.

[0061] Furthermore, the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each split point is cyclically calculated, and the target split points whose number of computing units is within the constraint range are screened out.

[0062] S204: Determine an optimal computing unit distribution strategy according to the number of computing units corresponding to each segmentation point.

[0063] Specifically, according to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target segmentation point, the accelerator throughput corresponding to each target segmentation point is calculated, and the target segmentation point with the minimum accelerator throughput is determined as the optimal segmentation point.

[0064] Furthermore, the convolutional neural network is segmented by the optimal segmentation point, and according to the final segmentation, the number of first computing units required for the single-layer single-engine architecture acceleration component and the number of second computing units required for the single-layer multi-engine architecture acceleration component are calculated respectively. According to the number of first computing units and the number of second computing units, the optimal computing unit distribution strategy is determined.

[0065] The load balancing method is used to reasonably divide the limited computing unit resources into the single-layer single-engine architecture part and the single-layer multi-engine architecture part of the hybrid accelerator architecture for convolutional neural networks based on the convolutional neural network model, and to balance the execution time of each engine in the single-layer single-engine part.

[0066] The present invention adjusts the allocation of computing units and the division of convolutional neural network layers through iterative optimization to ensure that the execution time of all single-layer and single-engines is similar, thereby maximizing resource utilization efficiency.

[0067] In addition, the method is also used to minimize the execution time of the bottleneck part. Since the overall throughput is limited by the part with the longest execution time, that is, the bottleneck part, the construction system makes a trade-off between the single-layer single-engine part and the single-layer multi-engine part so that the execution time of the two parts is as close as possible and both are as short as possible under a given computing unit budget.

[0068] As a feasible implementation method, the above load balancing optimization concept is implemented by the following method:

[0069] 1. Initialize and obtain the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the number of computing units allocated to the single-layer single-engine component, the distribution of computing units between the two components, and the overall throughput of the accelerator.

[0070] 2. Traverse each layer of the convolutional neural network and try to split the convolutional neural network at each possible segmentation point.

[0071] 3. For each split point, calculate the number of computing units required for the single-layer single-engine part and ensure that the number of these computing units is within the constraint.

[0072] 4. Cycle through different numbers of single-layer, single-engine computing units, calculate the throughput of the accelerator under each configuration, and update the computing unit distribution corresponding to the optimal throughput.

[0073] 5. Output the optimal accelerator throughput and computing unit distribution.

[0074] Furthermore, in order to clearly and in detail describe the above technical solution, a hybrid accelerator for convolutional neural networks is implemented by taking FPGA as an example. Note that this hybrid accelerator is not limited to being implemented using FPGA, but can also be implemented by Chiplet IC and other methods.

[0075] In this embodiment, the implementation mainly includes two parts: the first half is an acceleration component based on a single-layer single-engine architecture, and the second half is an acceleration component based on a single-layer multi-engine architecture; the two parts are connected through double buffering to ensure that the two parts can run simultaneously.

[0076] By writing the computing unit code, the computing units and buffer units suitable for each computing task in the convolutional neural network are generated. In addition, the corresponding interface units, scheduling units, etc. should also be included.

[0077] Assume that the acceleration component based on the single-layer multi-engine architecture is calculating a batch of input data batch(n), and at the same time, the acceleration component based on the single-layer single-engine architecture is calculating the next batch of input data batch(n+1).

[0078] The acceleration component based on the single-layer single-engine architecture adopts a pipeline design, and elastic buffers are inserted between the engines in the pipeline as needed. In addition, each single-layer single engine contains an independent weight cache to store the weight parameters of the convolutional neural network.

[0079] The input data is calculated in batches in a pipeline manner in the acceleration component of the single-layer single-engine architecture. That is, if engine 0 calculates the input number batch(n), then engine 1 is calculating the input data batch(n-1) at the same time. Assuming the length of the pipeline is m, when engine m-1 completes the calculation of the input data, it will store the calculation results in the double buffer connecting the acceleration component of the single-layer single-engine architecture and the acceleration component of the single-layer multi-engine architecture.

[0080] Preferably, each engine achieves optimal load balancing among the engines by allocating specific computing units based on the computing load through the construction system, so that the time for processing input data among the engines is basically the same, thereby improving the computing efficiency of the pipeline.

[0081] The m layers of the convolutional neural network until the last layer are calculated in the acceleration component of the single-layer multi-engine architecture. The acceleration component of the single-layer multi-engine architecture includes the depthwise convolution engine and the pointwise convolution engine. The channel-wise convolution engine completes the channel-wise convolution calculation task of the convolutional neural network, and the pointwise convolution engine completes the pointwise convolution calculation task of the convolutional neural network.

[0082] Furthermore, a construction system of a hybrid accelerator for convolutional neural networks is constructed, and the construction system includes a load balancing method for reasonably dividing limited computing unit resources into a single-layer single-engine architecture part and a single-layer multi-engine architecture part of the hybrid accelerator architecture for convolutional neural networks based on a convolutional neural network model.

[0083] This construction system is used to balance the execution time of each engine in the single-layer single-engine part. The construction system adjusts the allocation of computing units and the division of convolutional neural network layers through iterative optimization to ensure that the execution time of all single-layer single engines is similar, thereby maximizing resource utilization efficiency.

[0084] In addition, the construction system is used to minimize the execution time of the bottleneck part. Since the overall throughput is limited by the part with the longest execution time, that is, the bottleneck part, the construction system makes a trade-off between the single-layer single-engine part and the single-layer multi-engine part so that the execution time of the two parts is as close as possible and both are as short as possible under a given computing unit budget.

[0085] The optimization concept of the above construction system is realized by the following methods:

[0086] 1. Initialize and obtain the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the number of computing units allocated to the single-layer single-engine component, the distribution of computing units between the two components, and the overall throughput of the accelerator.

[0087] 2. Traverse each layer of the convolutional neural network and try to split the convolutional neural network at each possible segmentation point.

[0088] 3. For each split point, calculate the number of computing units required for the single-layer single-engine part and ensure that the number of these computing units is within the constraint.

[0089] 4. Cycle through different numbers of single-layer, single-engine computing units, calculate the throughput of the accelerator under each configuration, and update the computing unit distribution corresponding to the optimal throughput.

[0090] 5. Output the optimal accelerator throughput and computing unit distribution.

[0091] The above-mentioned construction system is realized by writing corresponding software programs, and the basic unit components obtained by writing the computing unit code are used as component libraries to construct FPGA engineering codes, which are then integrated into corresponding FPGA resources to realize a hybrid accelerator for convolutional neural networks.

[0092] It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0093] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0094] Each embodiment of the present invention is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0095] The above describes specific embodiments of the present invention. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0096] The above description is only an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A hybrid accelerator for convolutional neural networks, characterized in that: The hybrid accelerator includes: a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component; The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected via a double buffer; The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to simultaneously process different batches of input data; wherein the input data is the data to be calculated for each convolutional layer of the convolutional neural network; The double buffer is used to store output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component The hybrid accelerator applies the following load balancing method: Initialize the hybrid accelerator and obtain the computing unit data of the current computing platform; Traverse each convolutional layer of the convolutional neural network and determine the segmentation points one by one; For each split point, the number of computing units required for the single-layer single-engine architecture acceleration component is determined, specifically including: Based on the segmentation point, the convolutional neural network is segmented into two parts, the first part is processed by the single-layer single-engine architecture acceleration component, and the second part is processed by the single-layer multi-engine architecture acceleration component; the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each segmentation point is cyclically calculated, and the target segmentation point whose number of computing units is within the constraint range is screened out; According to the number of computing units corresponding to each segmentation point, an optimal computing unit distribution strategy is determined, specifically including: Calculate the accelerator throughput corresponding to each target segmentation point according to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target segmentation point; Determine the target split point with the minimum accelerator throughput as the optimal split point; The convolutional neural network is segmented by the optimal segmentation point, and according to the final segmentation, the number of first computing units required by the single-layer single-engine architecture acceleration component and the number of second computing units required by the single-layer multi-engine architecture acceleration component are respectively calculated; The optimal computing unit distribution strategy is determined according to the first computing unit quantity and the second computing unit quantity.

2. A hybrid accelerator for convolutional neural networks according to claim 1, characterized in that: The single-layer single-engine architecture acceleration component is composed of a plurality of first-connected single-layer single engines, and elastic buffers are inserted between the single-layer single engines according to actual needs to form a pipeline structure.

3. A hybrid accelerator for convolutional neural networks according to claim 2, characterized in that: Each of the single-layer single engines includes an independent weight buffer area for storing weight parameters of the convolutional neural network.

4. A hybrid accelerator for convolutional neural networks according to claim 1, characterized in that: The input data is calculated by each single-layer single engine in the single-layer single-engine architecture acceleration component in a pipeline manner in sequence; After the last single-layer single engine in the single-layer single-engine architecture acceleration component completes the calculation of a batch of input data, the calculation result is stored in the double buffer; While the current single-layer single engine is processing the current batch of input data, the previous single-layer single engine is processing the next batch of input data.

5. The hybrid accelerator for convolutional neural network according to claim 1, characterized in that: The single-layer multi-engine architecture acceleration component includes a channel-by-channel convolution engine and a point-by-point convolution engine; The channel-by-channel convolution engine is used to complete the channel-by-channel convolution calculation task of the convolutional neural network; The point-by-point convolution engine is used to complete the point-by-point convolution calculation task of the convolutional neural network.

6. A hybrid accelerator load balancing method for convolutional neural networks, applied to a hybrid accelerator for convolutional neural networks as claimed in claim 1, characterized in that: The method comprises: Initialize the hybrid accelerator and obtain the computing unit data of the current computing platform; Traverse each convolutional layer of the convolutional neural network and determine the segmentation points one by one; For each split point, the number of computing units required for the single-layer single-engine architecture acceleration component is determined, specifically including: Based on the segmentation point, the convolutional neural network is segmented into two parts, the first part is processed by the single-layer single-engine architecture acceleration component, and the second part is processed by the single-layer multi-engine architecture acceleration component; the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each segmentation point is cyclically calculated, and the target segmentation point whose number of computing units is within the constraint range is screened out; According to the number of computing units corresponding to each segmentation point, an optimal computing unit distribution strategy is determined, specifically including: Calculate the accelerator throughput corresponding to each target segmentation point according to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target segmentation point; Determine the target split point with the minimum accelerator throughput as the optimal split point; The convolutional neural network is segmented by the optimal segmentation point, and according to the final segmentation, the number of first computing units required by the single-layer single-engine architecture acceleration component and the number of second computing units required by the single-layer multi-engine architecture acceleration component are respectively calculated; The optimal computing unit distribution strategy is determined according to the first computing unit quantity and the second computing unit quantity.

7. A hybrid accelerator load balancing method for convolutional neural networks according to claim 6, characterized in that: Initialize the hybrid accelerator and obtain the computing unit data of the current computing platform, including: Initialize the hybrid accelerator into the current computing platform, and obtain the computing unit data of the current computing platform; wherein the computing unit data includes at least: the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the initial number of computing units allocated to the single-layer single-engine architecture acceleration component, the initial distribution of computing units between the two parts of the acceleration components, and the overall throughput of the hybrid accelerator.

8. The hybrid accelerator load balancing method for convolutional neural networks according to claim 6, characterized in that: Traverse each convolutional layer of the convolutional neural network and determine the segmentation points in turn, including: Each layer of the convolutional neural network is traversed to determine each position in the convolutional neural network that can be segmented as the segmentation point.

Images