Hybrid accelerator for convolutional neural network and load balancing method therefor

Abstract

A hybrid accelerator for a convolutional neural network and a load balancing method therefor, which relate to the technical field of deep learning hardware acceleration, and are configured for addressing the technical problem where increases in convolutional neural network computing performance are limited due to the inability of two existing types of dedicated convolutional neural network accelerator architecture to flexibly allocate and efficiently utilize computing resources of computing platforms. The hybrid accelerator comprises: a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component; the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected by means of a double buffer; the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are configured for simultaneously processing input data from different batches, wherein the input data is data to be calculated of each convolutional layer of a convolutional neural network; and the double buffer is configured for storing output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

Description

Hybrid accelerator for convolutional neural network and load balancing method thereof

[0001] The present application claims priority to the Chinese patent application No. 202411288516.0, filed on September 14, 2024, and entitled "Hybrid accelerator for convolutional neural network and load balancing method thereof", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of deep learning hardware acceleration, and particularly relates to a hybrid accelerator for convolutional neural network and a load balancing method thereof. BACKGROUND

[0003] With the rapid development of deep learning technology, convolutional neural networks have become a key tool for processing complex data such as images, videos and speech. However, the compute-intensive and memory access-intensive nature of convolutional neural networks puts extremely high demands on computing hardware, especially when dealing with large-scale data sets and deep network structures. Traditional computing platforms such as CPUs and GPUs, while able to support the operation of convolutional neural networks, often cannot meet the needs of practical applications in terms of energy efficiency, latency and throughput.

[0004] To overcome these challenges, researchers have proposed a variety of specialized accelerator architectures aimed at optimizing for the characteristics of convolutional neural networks. Among them, single-layer multi-engine accelerators and single-layer single-engine accelerators are two main architectural directions. Single-layer multi-engine accelerators process multiple layers by reusing computing resources, with the advantage of high resource utilization, but ignore the computational heterogeneity that may exist between and within layers. On the contrary, single-layer single-engine accelerators allocate independent computing engines for each layer of the convolutional neural network, which can better adapt to the computational needs of different layers, but consume a lot of resources and are not easy to expand.

[0005] Therefore, the current two specialized accelerator architectures for convolutional neural networks cannot achieve flexible allocation and efficient utilization of computing resources on the computing platform, and have limited improvement in the computational performance of convolutional neural networks. SUMMARY

[0006] The embodiments of the present application provide a hybrid accelerator for convolutional neural network and a load balancing method thereof, which are used to solve the technical problem that the current two specialized accelerator architectures for convolutional neural networks cannot achieve flexible allocation and efficient utilization of computing resources on the computing platform, and have limited improvement in the computational performance of convolutional neural networks.

[0007] The embodiments of the present application adopt the following technical solutions:

[0008] In one aspect, the embodiment of the present application provides a hybrid accelerator for a convolutional neural network, which comprises a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component.

[0009] The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected through a double buffer.

[0010] The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to process different batches of input data at the same time, wherein the input data is to-be-calculated data of each convolutional layer of the convolutional neural network.

[0011] The double buffer is used to store output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0012] In one possible implementation, the single-layer single-engine architecture acceleration component is composed of a plurality of single-layer single-engines connected in sequence, and a flexible buffer is inserted between each single-layer single-engine according to actual requirements to form a pipeline structure.

[0013] In one possible implementation, each single-layer single-engine comprises an independent weight buffer used to store weight parameters of the convolutional neural network.

[0014] In one possible implementation, the input data sequentially passes through each single-layer single-engine in the single-layer single-engine architecture acceleration component in a pipeline manner.

[0015] After the last single-layer single-engine in the single-layer single-engine architecture acceleration component completes the calculation of a batch of input data, the calculation result is stored in the double buffer.

[0016] While the current single-layer single-engine is processing the current batch of input data, the previous single-layer single-engine processes the next batch of input data.

[0017] In one possible implementation, the single-layer multi-engine architecture acceleration component comprises a channel-wise convolution engine and a point-wise convolution engine.

[0018] The channel-wise convolution engine is used to complete a channel-wise convolution calculation task of the convolutional neural network.

[0019] The point-wise convolution engine is used to complete a point-wise convolution calculation task of the convolutional neural network.

[0020] In another aspect, the embodiment of the present application further provides a load balancing method for a hybrid accelerator for a convolutional neural network, which is applied to the hybrid accelerator for the convolutional neural network, and the method comprises the following steps.

[0021] Initialize the hybrid accelerator, and obtain the computing unit data of the current computing platform;

[0022] Traverse each convolution layer of the convolutional neural network, and determine the segmentation points in sequence;

[0023] For each segmentation point, determine the number of computing units required by the single-layer single-engine architecture acceleration component;

[0024] According to the number of computing units corresponding to each segmentation point, determine the optimal computing unit distribution strategy.

[0025] In a feasible implementation, the hybrid accelerator is initialized, and the computing unit data of the current computing platform is obtained, specifically including:

[0026] The hybrid accelerator is initialized to the current computing platform, and the computing unit data of the current computing platform is obtained; wherein the computing unit data at least includes: the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the initial number of computing units allocated to the single-layer single-engine architecture acceleration component, the initial distribution of computing units between the two-part acceleration components, and the overall throughput of the hybrid accelerator.

[0027] In a feasible implementation, each convolution layer of the convolutional neural network is traversed, and the segmentation points are determined in sequence, specifically including:

[0028] Traverse each layer of the convolutional neural network, determine each position in the convolutional neural network that can be segmented, and determine the segmentation points.

[0029] In a feasible implementation, for each segmentation point, the number of computing units required by the single-layer single-engine architecture acceleration component is determined, specifically including:

[0030] Based on the segmentation point, the convolutional neural network is divided into two parts, the former part is processed by the single-layer single-engine architecture acceleration component, and the latter part is processed by the single-layer multi-engine architecture acceleration component;

[0031] Loop the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each segmentation point, and filter out the target segmentation point within the constraint range of the number of computing units.

[0032] In a feasible implementation, according to the number of computing units corresponding to each segmentation point, the optimal computing unit distribution strategy is determined, specifically including:

[0033] According to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target segmentation point, calculate the accelerator throughput corresponding to each target segmentation point;

[0034] determine the target segmentation point with the minimum accelerator throughput as the optimal segmentation point;

[0035] segment the convolutional neural network through the optimal segmentation point, and calculate the first number of computing units required by the single-layer single-engine architecture acceleration component and the second number of computing units required by the single-layer multi-engine architecture acceleration component according to the final segmentation condition;

[0036] determine the optimal computing unit distribution strategy according to the first number of computing units and the second number of computing units.

[0037] Compared with the prior art, the hybrid accelerator for a convolutional neural network and the load balancing method thereof provided by the embodiment of the present application have the following beneficial effects:

[0038] The architecture provided by the present application combines the advantages of single-layer multi-engine and single-layer single-engine, and realizes flexible allocation and efficient utilization of computing resources through a hybrid mapping strategy. In addition, the present application also provides a load balancing method for determining the segmentation point and resource allocation scheme of the hybrid accelerator. The method intelligently selects the segmentation point by analyzing the structural characteristics and resource budget constraints of the convolutional neural network, and optimizes the number of engines and resource configuration of the single-layer multi-engine and single-layer single-engine parts. The method can improve the utilization efficiency of limited resources and improve the computing flexibility, overcome the limitations of traditional accelerator architectures, improve the computing performance of the convolutional neural network, and reduce the energy consumption. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained according to these drawings without creative labor. In the drawings:

[0040] Fig. 1 is a schematic diagram of the architecture of a hybrid accelerator for a convolutional neural network provided by an embodiment of the present application;

[0041] Fig. 2 is a flowchart of a load balancing method for a hybrid accelerator for a convolutional neural network provided by an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order for those skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor should fall within the protection scope of the present application.

[0043] Firstly, the embodiment of the present application provides a hybrid accelerator for convolutional neural network as shown in FIG. 1, which mainly comprises two parts: single-layer single-engine architecture acceleration component and single-layer multi-engine architecture acceleration component.

[0044] The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected through double buffers. The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to process different batches of input data at the same time; wherein the input data is the data to be calculated of each convolutional layer of the convolutional neural network.

[0045] The double buffers are used to store the output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0046] As a feasible implementation, as shown in FIG. 1, the two dashed boxes in the hybrid accelerator, the first dashed box is the single-layer single-engine architecture acceleration component, and the second dashed box is the single-layer multi-engine architecture acceleration component. The two acceleration components are connected through double buffers (not shown in FIG. 1).

[0047] In one embodiment, assuming that the single-layer multi-engine architecture acceleration component is calculating a batch of input data batch(n), at the same time, the single-layer single-engine architecture acceleration component is calculating the next batch of input data batch(n+1), batch is the batch.

[0048] Further, as shown in FIG. 1, the single-layer single-engine architecture acceleration component is composed of a plurality of single-layer single-engines connected in the first place, and a flexible buffer is inserted between each single-layer single-engine according to actual needs, forming a pipeline structure. Each single-layer single-engine contains an independent weight buffer for storing the weight parameters of the convolutional neural network.

[0049] Further, the input data passes through each single-layer single-engine in the single-layer single-engine architecture acceleration component in a pipeline manner. After the last single-layer single-engine in the single-layer single-engine architecture acceleration component completes the calculation of a batch of input data, the calculation result is stored in the double buffer. While the current single-layer single-engine is processing the current batch of input data, the previous single-layer single-engine is processing the next batch of input data.

[0050] As a feasible implementation, in the single-layer single-engine architecture acceleration component, each engine corresponds to a convolution layer of the convolutional neural network model, and is used for processing to-be-calculated data of the corresponding convolution layer.

[0051] In one embodiment, as shown in FIG. 1, assuming that engine 0 is calculating input data batch(n), engine 1 is simultaneously calculating input data batch(n-1). Assuming that the length of the pipeline is m, when engine m-1 (i.e., the last engine) completes the calculation of the input data, the calculation result is stored in the double buffer connected to the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0052] Further, the single-layer multi-engine architecture acceleration component comprises a channel-by-channel convolution engine and a point-by-point convolution engine; the channel-by-channel convolution engine is used to complete a channel-by-channel convolution calculation task of the convolutional neural network; and the point-by-point convolution engine is used to complete a point-by-point convolution calculation task of the convolutional neural network.

[0053] The above architecture combines the advantages of the single-layer multi-engine and the single-layer single-engine, can improve the calculation flexibility, and overcome the limitations of the traditional accelerator architecture. However, the number of calculation units of the calculation platform is fixed, and the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component need to be allocated with specific calculation units based on the calculation load, so as to achieve optimal load balancing between the engines, achieve that the time for processing input data by the engines is basically the same, and thus improve the calculation efficiency of the pipeline.

[0054] For this purpose, the embodiment of the present application further provides a hybrid accelerator load balancing method for a convolutional neural network, as shown in FIG. 2, which specifically comprises steps S201-S204.

[0055] S201, initializing the hybrid accelerator and obtaining calculation unit data of the current calculation platform.

[0056] Specifically, the hybrid accelerator is initialized in the current calculation platform, and the calculation unit data of the current calculation platform is obtained.

[0057] The calculation unit data at least comprises: the total number of calculation units, a convolutional neural network model to be accelerated, a layer index of the segmented convolutional neural network model, an initial number of calculation units allocated to the single-layer single-engine architecture acceleration component, an initial distribution of the calculation units between the two acceleration components, and an overall throughput of the hybrid accelerator.

[0058] S202, traversing each convolution layer of the convolutional neural network, and sequentially determining a segmentation point.

[0059] Specifically, each layer of the convolutional neural network is traversed to determine each position in the convolutional neural network that can be segmented, and the position is determined as a segmentation point.

[0060] S203, for each segmentation point, determine the number of computing units required by the single-layer single-engine architecture acceleration component.

[0061] Specifically, based on the segmentation point, the convolutional neural network is divided into two parts, the former part is processed by the single-layer single-engine architecture acceleration component, and the latter part is processed by the single-layer multi-engine architecture acceleration component.

[0062] Further, the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each segmentation point is calculated in a loop, and the target segmentation point whose number of computing units is within the constraint range is screened out.

[0063] S204, according to the number of computing units corresponding to each segmentation point, determine the optimal computing unit distribution strategy.

[0064] Specifically, according to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target segmentation point, the accelerator throughput corresponding to each target segmentation point is calculated. The target segmentation point with the smallest accelerator throughput is determined as the optimal segmentation point.

[0065] Further, the convolutional neural network is segmented by the optimal segmentation point, and according to the final segmentation, the first number of computing units required by the single-layer single-engine architecture acceleration component and the second number of computing units required by the single-layer multi-engine architecture acceleration component are calculated respectively. According to the first number of computing units and the second number of computing units, the optimal computing unit distribution strategy is determined.

[0066] The above load balancing method is used to reasonably divide the limited computing unit resources to the single-layer single-engine architecture part and the single-layer multi-engine architecture part of the hybrid accelerator architecture for the convolutional neural network based on the convolutional neural network model, and to balance the execution time of each engine of the single-layer single-engine part.

[0067] The present application adjusts the allocation of computing units and the division of convolutional neural network layers through iterative optimization, ensures that the execution time of all single-layer single-engines is similar, and maximizes the resource utilization efficiency.

[0068] In addition, the method is also used to minimize the execution time of the bottleneck part. Since the overall throughput is limited by the part with the longest execution time, i.e. the bottleneck part. The system makes a trade-off between the single-layer single-engine part and the single-layer multi-engine part so that the execution time of the two parts is as close as possible and as short as possible under the given computing unit budget.

[0069] As a feasible implementation, the optimization concept of load balancing is realized by the following method:

[0070] 1. Initialization obtains the total number of computing units, the convolutional neural network model to be accelerated, the layer index of the partitioned convolutional neural network model, the number of computing units allocated to the single-layer single-engine component, the distribution of computing units between the two-part components, and the overall throughput of the accelerator.

[0071] 2. Traverse each layer of the convolutional neural network, and attempt to partition the convolutional neural network at each possible partition point.

[0072] 3. For each partition point, calculate the number of computing units required by the single-layer single-engine part, and ensure that the number of computing units is within the constraint range.

[0073] 4. Loop to try different single-layer single-engine computing unit numbers, calculate the throughput of the accelerator under each configuration, and update the optimal throughput corresponding to the computing unit distribution.

[0074] 5. Output the optimal accelerator throughput and computing unit distribution.

[0075] Further, in order to clearly and detailedly describe the above technical solution, the following takes FPGA as an example to realize a hybrid accelerator for convolutional neural network. It is noted that the hybrid accelerator is not limited to be realized by FPGA, but can also be realized by Chiplet IC and the like.

[0076] In this embodiment, the implementation mainly includes two parts: the first half is an acceleration component based on a single-layer single-engine architecture, and the second half is an acceleration component based on a single-layer multi-engine architecture; the two parts are connected through double buffering to ensure that the two parts can run simultaneously.

[0077] By writing computing unit code, computing units suitable for various computing tasks in the convolutional neural network and buffer units are generated. In addition, corresponding interface units, scheduling units, etc. should also be included.

[0078] Suppose that the acceleration component based on the single-layer multi-engine architecture is computing a batch of input data batch(n), then the acceleration component based on the single-layer single-engine architecture is computing the next batch of input data batch(n+1) at the same time.

[0079] The acceleration component based on the single-layer single-engine architecture adopts a pipeline design, and elastic buffers are inserted between the engines in the pipeline as needed. In addition, each single-layer single-engine contains an independent weight cache for storing the weight parameters of the convolutional neural network.

[0080] The input data is calculated in a pipelined manner in the single-layer single-engine architecture acceleration component by batch, that is, assuming that engine 0 calculates input data batch (n), engine 1 calculates input data batch (n-1) at the same time. Assuming that the length of the pipeline is m, when engine m-1 completes the calculation of the input data, the calculation result is stored in the double buffer connected to the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

[0081] Preferably, each engine is allocated a specific calculation unit by the construction system based on the calculation load, so as to achieve optimal load balancing between the engines, so that the time for processing the input data by each engine is basically the same, thereby improving the calculation efficiency of the pipeline.

[0082] The m layers of the convolutional neural network until the last layer are completed in the single-layer multi-engine architecture acceleration component. The single-layer multi-engine architecture acceleration component includes a depthwise convolution engine and a pointwise convolution engine. The depthwise convolution engine completes the depthwise convolution calculation task of the convolutional neural network, and the pointwise convolution engine completes the pointwise convolution calculation task of the convolutional neural network.

[0083] Further, a construction system for a hybrid accelerator for a convolutional neural network is constructed, which includes a load balancing method for reasonably dividing limited calculation unit resources to the single-layer single-engine architecture part and the single-layer multi-engine architecture part of the hybrid accelerator architecture for the convolutional neural network based on the convolutional neural network model.

[0084] The construction system is used to balance the execution time of each engine in the single-layer single-engine part. The construction system adjusts the allocation of calculation units and the division of convolutional neural network layers through iterative optimization to ensure that the execution time of all single-layer single-engines is similar, thereby maximizing resource utilization efficiency.

[0085] In addition, the construction system is used to minimize the execution time of the bottleneck part. Since the overall throughput is limited by the part with the longest execution time, that is, the bottleneck part. The construction system makes a trade-off between the single-layer single-engine part and the single-layer multi-engine part so that the execution time of the two parts is as close as possible and as short as possible under the given calculation unit budget.

[0086] The optimization concept of the above construction system is realized by the following methods:

[0087] 1. Initialize to obtain the total number of calculation units, the convolutional neural network model to be accelerated, the layer index of the segmented convolutional neural network model, the number of calculation units allocated to the single-layer single-engine component, the distribution of calculation units between the two component parts, and the overall throughput of the accelerator.

[0088] 2. Traverse each layer of the convolutional neural network, and attempt to split the convolutional neural network at each possible split point.

[0089] 3. For each split point, calculate the number of compute units required for the single-layer single-engine portion, and ensure that the number of compute units is within the constraint range.

[0090] 4. Loop to attempt different numbers of single-layer single-engine compute units, and calculate the throughput of the accelerator under each configuration, and update the optimal throughput corresponding to the compute unit distribution.

[0091] 5. Output the optimal accelerator throughput and compute unit distribution.

[0092] The above system is implemented by writing a corresponding software program, and each basic unit component obtained by writing a compute unit code is taken as a component library, an FPGA engineering code is constructed, and then is synthesized to a corresponding FPGA resource, to implement a hybrid accelerator for a convolutional neural network.

[0093] It should be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles, or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device that includes the element.

[0094] The specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform particular tasks or implement particular abstract data types. The specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media, including storage devices.

[0095] Each embodiment in the present application is described in a progressive manner, and the same and similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0096] The above-described embodiments of the application have special structure and can achieve particular advantages. It is to be understood that various equivalents, changes, and modifications can be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the described embodiments of the present application are to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.

[0097] The above-described embodiments of the application have special structure and can achieve particular advantages. It is to be understood that various equivalents, changes, and modifications can be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the described embodiments of the present application are to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.

Claims

1. A hybrid accelerator for a convolutional neural network, the hybrid accelerator comprising: The hybrid accelerator comprises a single-layer single-engine architecture acceleration component and a single-layer multi-engine architecture acceleration component; The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are connected through a double buffer; The single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component are used to process different batches of input data at the same time, wherein the input data is to-be-calculated data of each convolutional layer of a convolutional neural network. The double buffer is used to store output data of the single-layer single-engine architecture acceleration component and the single-layer multi-engine architecture acceleration component.

2. The hybrid accelerator for convolutional neural networks of claim 1, wherein, The single-layer single-engine architecture acceleration component is composed of a plurality of single-layer single-engines connected in sequence, and a flexible buffer is inserted between each single-layer single-engine according to actual requirements to form a pipeline structure.

3. The hybrid accelerator for convolutional neural networks of claim 2, wherein, Each single-layer single-engine comprises an independent weight buffer area for storing weight parameters of a convolutional neural network.

4. The hybrid accelerator for convolutional neural networks of claim 1, wherein, The input data passes through each single-layer single-engine in the single-layer single-engine architecture acceleration component in a pipeline manner. After the last single-layer single-engine in the single-layer single-engine architecture acceleration component completes the calculation of a batch of input data, the calculation result is stored in the double buffer. While the current single-layer single-engine is processing the current batch of input data, the previous single-layer single-engine processes the next batch of input data.

5. The hybrid accelerator for convolutional neural networks of claim 1, wherein, The single-layer multi-engine architecture acceleration component comprises a channel-wise convolution engine and a point-wise convolution engine. The channel-wise convolution engine is used to complete a channel-wise convolution calculation task of a convolutional neural network. The point-wise convolution engine is used to complete a point-wise convolution calculation task of a convolutional neural network. 6.A method for load balancing of a hybrid accelerator for convolutional neural network, applied to the hybrid accelerator for convolutional neural network according to claim 1, characterized in that, The method comprises: initializing a hybrid accelerator and obtaining computing unit data of a current computing platform; traversing each convolutional layer of a convolutional neural network to determine segmentation points in sequence; for each segmentation point, determining the number of computing units required by the single-layer single-engine architecture acceleration component; determining an optimal computing unit distribution strategy according to the number of computing units corresponding to each segmentation point.

7. The hybrid accelerator load balancing method for convolutional neural networks of claim 6, wherein, initializing a hybrid accelerator and obtaining computing unit data of a current computing platform, specifically comprising: initializing the hybrid accelerator to the current computing platform and obtaining the computing unit data of the current computing platform; wherein the computing unit data at least includes: the total number of computing units, a convolutional neural network model to be accelerated, a layer index of segmenting the convolutional neural network model, an initial number of computing units allocated to the single-layer single-engine architecture acceleration component, an initial distribution of computing units between the two acceleration components, and the overall throughput of the hybrid accelerator.

8. The hybrid accelerator load balancing method for convolutional neural networks of claim 6, wherein, traversing each convolutional layer of a convolutional neural network to determine segmentation points in sequence, specifically comprising: traversing each layer of the convolutional neural network to determine each segmentable position in the convolutional neural network, and determining the segmentation points.

9. The hybrid accelerator load balancing method for convolutional neural networks of claim 6, wherein, for each segmentation point, determining the number of computing units required by the single-layer single-engine architecture acceleration component, specifically comprising: based on the segmentation point, the convolutional neural network is divided into two parts, the former part is processed by the single-layer single-engine architecture acceleration component, and the latter part is processed by the single-layer multi-engine architecture acceleration component; The number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each split point is calculated in a loop, and a target split point whose number of computing units is within a constraint range is screened out.

10. The hybrid accelerator load balancing method for convolutional neural networks of claim 9, wherein, According to the number of computing units corresponding to each split point, an optimal computing unit distribution strategy is determined, specifically including: According to the number of computing units required by the single-layer single-engine architecture acceleration component corresponding to each target split point, the accelerator throughput corresponding to each target split point is calculated; The target split point with the minimum accelerator throughput is determined as the optimal split point; The convolutional neural network is split through the optimal split point, and according to the final split, the first number of computing units required by the single-layer single-engine architecture acceleration component and the second number of computing units required by the single-layer multi-engine architecture acceleration component are calculated respectively; According to the first number of computing units and the second number of computing units, the optimal computing unit distribution strategy is determined.

Images