A convolution computing device based on hybrid parallelism
By designing a hybrid parallel computing device in the convolutional computing accelerator, judging the parallelism degree based on the input data characteristics and optimizing parameters and data allocation, the problem of low parameter multiplexing efficiency is solved, and efficient calculation and energy efficiency improvement of the accelerator is achieved.
Patent Information
- Application Number
- CN202110461840.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-04-27
AI Technical Summary
The existing convolutional computing accelerator cannot be efficiently reused, resulting in low computational efficiency. It also causes performance losses due to bandwidth limitations when processing calculations such as deep separation convolution and fast connections, so it cannot achieve the best performance at hardware scale. .
Design a convolutional computing device based on hybrid parallelism, and judge the parallelism degree based on input data characteristics through input unit, simulation unit, on-chip processor, packet control unit and mapping unit, and group the processing units to optimize the allocation of parameters and data , realize efficient acceleration of convolutional calculation.
It improves the overall computing speed and energy efficiency of the accelerator, reduces the number of accesses to external storage, reduces the calculation delay and power consumption, and improves the utilization rate of hardware resources.
Smart Images

Figure CN112989270B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to a convolution computing device based on hybrid parallelism. Background Art
[0002] Nowadays, deep convolutional neural networks (DCNN) are widely used in the field of computer vision, such as image classification and target recognition. In order to achieve better performance, the convolutional computing model tends to become wider and deeper, which requires a large increase in the number of parameters introduced, and also brings about the problem of a sharp increase in computational complexity during the acceleration process. Therefore, it is necessary to perform appropriate block operations during the acceleration process.
[0003] Convolution calculations include calculations in multiple dimensions (output dimension, input dimension, height, width), and there are multiple data calculation reuse situations between these dimensions, such as 1) the reuse of input data for different output channels; 2) the accumulation of output data in the direction of the input channel; 3) the reuse of parameters in convolution calculations on a single calculation plane. How to divide the convolution calculation into blocks during the acceleration process and arrange the appropriate serial and parallel processing calculation dimensions has become a problem in the accelerator implementation process. These allocations will greatly affect the overall computing efficiency of the accelerator.
[0004] In order to solve the above problems, the CNN accelerator data flow widely used in the prior art is layer-by-layer calculation. This calculation method tends to traverse all the calculation blocks in a layer until all convolution layers are completed. After completion, the process is repeated from the beginning to calculate the next layer of convolution. In the case of a large feature map, the off-chip storage will be accessed multiple times for the same data, increasing the overall computing power consumption and computing delay. In addition, when performing some low-operational intensity calculations such as depth-wise convolution, shortcut connection, or even high-intensity sparse calculations, performance loss will occur due to bandwidth limitations, and the best performance under hardware scale cannot be achieved.
[0005] For this purpose, a fusion data flow was developed to complete the convolution calculations of several consecutive layers on the chip, which can avoid repeated access to off-chip storage in the middle. In this process, in order to ensure the correctness of the calculation, the feature maps of all channels need to be stored on the chip. Therefore, the feature map is divided into small blocks, and the overall calculation is completed in a cycle.
[0006] However, the fused data stream places the calculations of several consecutive layers on the chip containing the computing core, making it impossible to efficiently reuse the parameters in the convolution calculation, thereby increasing the access of parameters to external storage, increasing the overall computing delay and power consumption. Therefore, how to better utilize the fused data stream to improve the overall computing speed and energy efficiency of the accelerator has become a technical problem that needs to be solved urgently. Summary of the invention
[0007] The present application provides a convolution computing device based on hybrid parallelism to solve the problem of low acceleration efficiency caused by the inability to efficiently reuse parameters in the prior art.
[0008] The present application provides a convolution computing device based on hybrid parallelism, comprising:
[0009] An input unit is configured to obtain input convolution data and corresponding parameters, determine the convolution shape according to the input convolution data, and extract the feature map size, convolution kernel size and number of channels of the input convolution data;
[0010] A simulation unit, configured to obtain a parallelism corresponding to the input convolution data according to data features extracted by the input unit; the data features include convolution shape and parameters;
[0011] An on-chip processor including multiple parallel processing units;
[0012] a group control unit, connected to each processing unit, and configured to divide all processing units on the on-chip processor into G groups according to the parallelism, where G is equal to the parallelism and the number of processing units in each group is equal;
[0013] A mapping unit, connected to each processing unit, is configured to control the data and parameters input to each processing unit according to the parallelism, the input convolution data and the corresponding parameters; wherein the processing units in the same group input the same parameters and different data; and the processing units in different groups input different parameters;
[0014] The processing unit is used to complete the convolution acceleration behavior according to the input data and parameters and output the result.
[0015] In some embodiments, the processing unit is configured to adjust hardware parameter configuration accordingly according to the grouping command sent by the grouping control unit, so as to process parameters from different channels.
[0016] In some embodiments, the mapping unit is further configured to map the remaining channels onto the serial sequence.
[0017] In some embodiments, the simulation unit is further configured to:
[0018] Develop a performance simulator using a preset neural network for simulation;
[0019] Input the historical convolution shape and parameters, and generate an evaluation table based on the computational efficiency;
[0020] The evaluation table is used to evaluate the data features extracted by the current input unit to obtain the degree of parallelism corresponding to the data features.
[0021] In some embodiments, the different data refers to using Ph×Pw slicing on each feature map to obtain sub-feature maps of different sizes as data input to the processing unit.
[0022] The present application provides a convolution computing device based on hybrid parallelism, which supports hybrid parallelism that adapts to different convolutions during operation. Hybrid parallelism of different dimensions can be used to improve computing performance, especially in unstructured sparse accelerators; it can be coordinated with different high-level computing flows (layer by layer, partial fusion) to complete a more efficient computing process. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0024] Figure 1 Schematic diagram of the consumption of processing dense convolution and sparse convolution;
[0025] Figure 2 A structural diagram of a convolution computing device based on hybrid parallelism provided in this application;
[0026] Figure 3 This is an example diagram of the operation of the device provided in this application when G=4. DETAILED DESCRIPTION
[0027] Due to limited on-chip resources, feature maps are divided into many small blocks for processing during hardware accelerated CNN. In the previous fixed-size computing module cropping method, repeated access to external parameter storage is introduced. In addition, fixed-size computing modules limit the architecture's reuse of various CNN data types. The hybrid parallel convolution computing device of this scheme can achieve adjustable computing block size and adjustable channel parallelism during operation, which can be used to increase the hardware efficiency of dedicated accelerators.
[0028] Another beneficial effect of this scheme is that it allows dedicated hardware accelerators to better utilize unstructured sparsity. First, parallel processing elements (PEs) can be configured to process different convolution sizes, that is, different PEs share weights or output feature maps in the convolution. When the same weights are input, PEs are configured to process different parts of the same feature map. This configuration does not introduce load imbalance and memory access conflicts. However, highly parallel PEs cannot be fully utilized when they are configured to process single batch convolutions of small feature maps. When processing the same output feature map, different PEs are configured to process different input channels of the same convolution kernel. Each convolution kernel contains a large number of input channels, which ensures that highly parallel PEs are fully utilized. However, the high parallelism of the parameter kernel can lead to serious load imbalance, such as Figure 1 Therefore, choosing an appropriate PE parallelism can alleviate the above problems.
[0029] Depend on Figure 1 It can be seen that when processing dense convolution, assuming that it is processed by 4 parallel PEs, the processing consumption is 4PEs×9 processing clocks; when processing sparse convolution, if it is processed by 4 parallel PEs, the processing consumption is 4PEs×7 processing clocks; if it is processed by 2 parallel PEs, the processing consumption is 2PEs×12 processing clocks; here, the number of computing units×processing time is used to represent the overall performance, and the smaller the product value, the more efficient the calculation method. Therefore, in practical applications, the optimal number of parallel PEs should be selected according to the different characteristics of the input data. This application provides a device to achieve the above purpose.
[0030] See also Figure 2 , which is a structural diagram of a convolution computing device based on hybrid parallelism provided in this application.
[0031] Depend on Figure 2 It can be seen that the device provided in the embodiment of the present application includes:
[0032] The input unit 10 is configured to obtain input convolution data and corresponding parameters, determine the convolution shape according to the input convolution data, and extract the feature map size, convolution kernel size and number of channels of the input convolution data; wherein the input convolution data mainly refers to sparse convolution data, and the input data must first be analyzed to obtain the number of channels, corresponding feature map information, etc.
[0033] The simulation unit 20 is configured to obtain the parallelism corresponding to the input convolution data according to the data features extracted by the input unit 10; the data features include convolution shape and parameters;
[0034] In this embodiment, the role of the simulation unit 20 is to evaluate the characteristic information of the input data by pre-stored data or in real time to give the optimal parallelism value corresponding to the data, which can be achieved by developing a performance simulator. Specifically, when the hardware configuration of the simulation unit is a performance simulator, the simulation unit can be configured as follows:
[0035] Develop a performance simulator using a preset neural network for simulation;
[0036] Input the historical convolution shape and parameters, and generate an evaluation table based on the computational efficiency;
[0037] The evaluation table is used to evaluate the data features extracted by the current input unit 10 to obtain the degree of parallelism corresponding to the data features.
[0038] Simulation process:
[0039] For general convolutional neural networks, the computational efficiency of using different parallelism G for calculating various input data is shown in the following table (the convolution in the table is only an example, and more shapes can actually be involved in the calculation):
[0040]
[0041] The simulation results can be used to obtain the optimal parallelism (1×) under different input data. The simulation results can be used to determine the parallelism corresponding to the current input convolution data.
[0042] Similarly, the optimal degree of parallelism under different input data can be obtained from the simulation results, and the degree of parallelism corresponding to the current input convolution data can be determined using the simulation results.
[0043] According to the downsampling rules of multi-layer convolutional neural networks, narrower (fewer channels) convolution kernels tend to have larger feature maps. Therefore, when processing narrower convolutions, the parallelism in the feature map dimension will be expanded to ensure better processing results. When processing wider convolutions, small blocks are selected in the feature map dimension to increase the parallelism of channel processing. By selecting different parallel solutions for different convolutions, performance balance is achieved to achieve the purpose of overall acceleration.
[0044] The on-chip processor 30 includes a plurality of parallel processing units 31. A chip usually has a fixed number of processing units. The present application divides the fixed number of processing units into different groups so that each group processes different computing tasks.
[0045] The group control unit 40 is connected to each processing unit 31 respectively, and is configured to divide all processing units 31 on the on-chip processor 30 into G groups according to the parallelism, where G is equal to the parallelism, and the number of processing units 31 in each group is equal; the group control unit is essentially equivalent to sending instructions to each processing unit, indicating which group the processing unit is located in, and executing corresponding actions according to the data and parameters received by which group.
[0046] The mapping unit 50 is connected to each processing unit 31 respectively, and is configured to control the data and parameters input to each processing unit 31 according to the parallelism, the input convolution data and the corresponding parameters; wherein the processing units 31 in the same group input the same parameters and different data; and the processing units 31 in different groups input different parameters;
[0047] In this embodiment, the role of the mapping unit is to configure the variable parameters in the hardware and plan the data reading method (from off-chip storage to on-chip cache) according to the selected parallelism. Specifically, it reconfigures the variable parameters of the subsequent hardware processing according to the parallelism given by the simulation unit. This requires that the hardware device of the processing unit has a configurable feature, which can be configured to adjust the hardware parameter configuration accordingly according to the grouping command sent by the grouping control unit 40, so as to process the parameters from different channels to meet the requirements of different processing parallelisms and perform convolution operations with the data of the corresponding channels.
[0048] The processing unit 31 is used to complete the convolution acceleration behavior according to the input data and parameters and output the result.
[0049] The processing process of the above device in practical application can be Figure 3 To explain:
[0050] Figure 3 This is an operation example of the device provided in this application when G=4.
[0051] In the proposed hybrid parallel convolution computing device, multiple parallel PEs can be configured to process the same weights to calculate larger data blocks, and these PEs are represented as a PE group (PE_G). In a PE_G, the load imbalance is eliminated and the weights are reused. Assuming there are G_max PEs on the chip, they are divided into G groups, and different input channels from the same filter are processed according to the parallelism G.
[0052] Assume that each PE can derive Ph×Pw partial sums (psum). With this grouping, G_max / G×Ph×Pw partial sums (psum) are generated in each PE_G. To calculate the complete output, the partial sums from G input channels need to be accumulated.
[0053] Depend on Figure 3 It can be seen that the 8 processing units PE are divided into 4 groups, each of which has two processing units. Taking the first group as an example, the inputs of PE0 and PE1 both come from the feature map corresponding to the i-th channel, and the parameters used by the two are the same, but because the feature map is cut into Ph×Pw parts and blocks of different sizes, sub-feature maps of different sizes are obtained as data input to the processing units, so the data processed by the two processing units are different; PE1 and PE2 come from the feature maps corresponding to different channels, so the parameters used by the two are different and the data is also different.
[0054] As the weight parallelism increases, the load imbalance becomes more and more serious. This solution proposes that the load imbalance can be alleviated by reducing G. Therefore, in order to improve hardware efficiency, the choice of G can be optimized according to the network configuration. When processing convolutional layers with large feature maps, G tends to be configured to a smaller value to increase the reuse of weights. For convolutional layers with more channels, the parallelism of the input channels is increased, and more PE_G can be used for execution.
[0055] The hybrid parallel convolution computing device provided in the present application can be combined with a partially fused data stream to improve the overall computing efficiency of the accelerator. For partially fused data streams, in order to reduce repeated access to DRAM, several consecutive layers of convolution computing are completed on-chip. When segmenting a network with a stacked bottleneck (Bottleneck-liked Operation, BLO) structure, each BLO is divided into a unit of continuous computing in the fused data stream, and the size of the computing block can be kept consistent through the overlap buffer.
[0056] When using partially fused data streams, the hardware requires a lot of storage resources to store activation values and ensure that the computational blocks of all channels can be placed on-chip. Due to the limited on-chip resources and the huge weight requirements, it is impossible to allocate a weight buffer that can store all the weights of the fused layer. Therefore, when processing convolutional layers with many channels, repeated access to external storage to obtain weights is inevitable.
[0057] Therefore, this scheme divides the calculation by cutting the input feature map into blocks of indefinite sizes. The size of the blocks depends on the size of the input feature map, so that the computing resource utilization in the PE is maximized. Inside a BLO, a fused calculation flow is adopted, and the size of the input feature map is the block size each time it is calculated, and the block cutting is repeated many times until the overall calculation is completed. In this way, the calculation of a BLO is decomposed into multiple independent fused calculation processes. The hybrid parallel convolution computing device proposed in this scheme optimizes the tile-based data flow and increases the ability to reuse weights. Compared with the data flow with a fixed slicing scheme, this scheme can effectively reduce the number of repeated accesses to weights in external storage.
[0058] Through the simulation of access to different levels of storage, it can be seen that compared with a fixed channel parallelism in the calculation process, the proposed hybrid parallelism can reduce the parameter access to DRAM by 63.8%. Therefore, this scheme can significantly reduce the power consumption of memory access in the calculation process.
[0059] Furthermore, in a feasible embodiment, there may be some channels that are not allocated to the processing unit group. In this case, the mapping unit needs to map these channels to the serial sequence, and execute them in sequence according to the time sequence after the parallel processing is completed.
[0060] It can be seen from the above technical scheme that the present application provides a convolution computing device based on hybrid parallelism, including an input unit, a simulation unit, an on-chip processor including multiple parallel processing units, a grouping control unit and a mapping unit. The device provided by the present application can design a suitable channel parallelism according to the network shape, and can be successfully applied to hardware for acceleration. According to the convolution computing device based on hybrid parallelism proposed in this scheme, the hardware acceleration platform can better utilize the fused data stream and improve the overall computing speed and energy efficiency of the accelerator.
[0061] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0062] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A convolution computing device based on hybrid parallelism, characterized in that: The device comprises: An input unit (10) is configured to obtain input convolution data and corresponding parameters, determine the convolution shape according to the input convolution data, and extract the feature map size, convolution kernel size and number of channels of the input convolution data; A simulation unit (20) is configured to obtain the parallelism corresponding to the input convolution data according to the data features extracted by the input unit (10); the data features include convolution shape and parameters; An on-chip processor (30) comprising a plurality of parallel processing units (31); A grouping control unit (40) is connected to each processing unit (31) and is configured to divide all processing units (31) on the on-chip processor (30) into G groups according to the parallelism, where G is equal to the parallelism and the number of processing units (31) in each group is equal; A mapping unit (50) is connected to each processing unit (31) and is configured to control the data and parameters input into each processing unit (31) according to the parallelism, the input convolution data and the corresponding parameters; wherein the processing units (31) in the same group input the same parameters and different data; and the processing units (31) in different groups input different parameters; The processing unit (31) is used to complete the convolution acceleration behavior according to the input data and parameters and output the result.
2. A convolution computing device based on hybrid parallelism according to claim 1, characterized in that: The processing unit (31) is configured to adjust hardware parameter configuration accordingly according to the grouping command sent by the grouping control unit (40) so as to process parameters from different channels.
3. The hybrid parallel convolution computing device according to claim 1, characterized in that: The mapping unit (50) is further configured to map the remaining channels onto a serial sequence.
4. The hybrid parallel convolution computing device according to claim 1, characterized in that: The simulation unit (20) is further configured to: Develop a performance simulator using a preset neural network for simulation; Input the historical convolution shape and parameters, and generate an evaluation table based on the computational efficiency; The evaluation table is used to evaluate the data features extracted by the current input unit (10) to obtain the degree of parallelism corresponding to the data features.
5. The hybrid parallel convolution computing device according to claim 1, characterized in that: The different data refers to the use of Ph×Pw slicing for each feature map to obtain sub-feature maps of different sizes as data input to the processing unit (31).
