A fine-grained convolutional neural network acceleration architecture

Abstract

The application discloses a fine-grained convolutional neural network acceleration architecture, which comprises a row splitting module, a row array cache group, a convolution processing module group, and a pooling processing module; the row splitting module is connected with the row array cache group; the row array cache group is connected with the row splitting module and the convolution processing module group; the convolution processing module group is connected with the row array cache group and the pooling processing module; and the pooling processing module is connected with the convolution processing module group. The application adopts a parallel data processing mode, and multiple convolution function blocks can be simultaneously instantiated under the condition that resources are allowed, and the pooling calculation of a set size can be completed in a pipelined manner under the operation of one machine. The data processing efficiency is high, and the time consumption is short. The application can be used for assisting the deployment of a convolutional neural network, so that better real-time performance and operation efficiency are achieved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and communication engineering, and specifically to a fine-grained convolutional neural network acceleration architecture, suitable for assisting the deployment of convolutional neural networks (CNNs) to achieve better real-time performance and computational efficiency. Background Technology

[0002] Neural networks, represented by Convolutional Neural Networks (CNNs), are computational models that mimic the structure and function of biological neural networks. A classic neural network structure consists of three layers: an input layer, an output layer, and hidden layers. A CNN is composed of one or more convolutional layers, pooling layers, and fully connected layers.

[0003] The applications of convolutional neural networks (CNNs) are closely linked to real-time requirements, particularly in fields such as artificial intelligence-driven driving and military applications, where there is an urgent need for fast and accurate object recognition and motion direction prediction. However, current CNN architectures still suffer from high computational costs and slow processing speeds when handling large-scale image data, which limits their application in fields with extremely high real-time requirements. Summary of the Invention

[0004] The purpose of this invention is to provide a fine-grained convolutional neural network acceleration architecture, which aims to achieve parallel data processing effects. When resources permit, multiple convolutional function blocks can be instantiated simultaneously to achieve efficient data processing, thereby assisting in the deployment of convolutional neural networks and enabling them to achieve better real-time performance and computational efficiency.

[0005] The specific technical solution for achieving the objective of this invention is as follows:

[0006] A fine-grained convolutional neural network acceleration architecture includes a row splitting module, a row array cache group, a convolution processing module group, and a pooling processing module.

[0007] The row splitting module is connected to the row array cache group to split the input raw data into independent data rows and transmit the data rows to the row array cache group;

[0008] The row array cache group is connected to the row splitting module and the convolution processing module group. It is used to receive and store the split data rows, adjust them according to the convolution rules to make them match the data row content with the convolution kernel size, and transmit the integrated data row group to the convolution processing module group.

[0009] The convolution processing module group is connected to the row array cache group and the pooling processing module. It is used to perform convolution operations on the data row groups transmitted from the row array cache group and cache the convolution results. Subsequently, the direct convolution results and cached results are transmitted to the pooling processing module in sequence.

[0010] The pooling module is connected to the convolution module group and is used to perform pooling operations on the convolution results transmitted from the convolution module group, and to provide an output interface for the pooling results.

[0011] The row array cache group includes a level 1 row cache, a level 2 row cache, a level 3 row cache, and a level 4 row cache;

[0012] The first-level row cache is connected to the second-level row cache and the convolution processing module group to store the first n rows of data of the n*n convolution data row group and to transmit the stored data row group to the convolution processing module group.

[0013] The secondary row cache is connected to the primary row cache, the tertiary row cache and the convolution processing module group, and is used to store the first n rows of data starting from the second row of the n*n convolution data row group, and to transmit the stored data row group to the convolution processing module group.

[0014] The third-level row cache is connected to the second-level row cache, the fourth-level row cache and the convolution processing module group, and is used to store the first n rows of data starting from the third row of the n*n convolution data row group, and to transmit the stored data row group to the convolution processing module group.

[0015] The fourth-level row cache is connected to the row splitting module, the third-level row cache and the convolution processing module group. It is used to receive the split data rows and store the first n rows of data starting from the fourth row of the n*n convolution data row group, and transmit the stored data row group to the convolution processing module group.

[0016] The convolution processing module group includes four parallel convolution processing modules;

[0017] The convolution processing module includes a convolution operation unit, a convolution cache level, and a convolution configuration register;

[0018] The convolution operation unit is connected to the row array cache group, the convolution cache level, the convolution configuration register and the pooling processing module. It is used to receive n rows of data of n*n convolution and perform convolution operation on them. The n*n multiplication operations in the convolution kernel of each convolution are executed in parallel. After each convolution operation is completed, the convolution result is transmitted to the convolution cache level. Subsequently, the result of the last convolution operation is output to the pooling processing module.

[0019] The convolutional cache level contains multiple registers connected to the convolutional operation unit and the pooling processing module. It is used to store the convolutional result output by the convolutional operation unit each time, and then outputs the stored convolutional results to the pooling processing module in sequence.

[0020] The convolution configuration register is connected to the convolution operation unit and is used to control the running state and stepping of the convolution operation unit when performing convolution calculations. The pooling processing module includes a pooling configuration register, a multiplexer group, and a 2*2 basic pooling tuple;

[0021] The pooling configuration register is connected to the multiplexer group and is used to control the selection of the multiplexer group;

[0022] The multiplexer group is connected to the pooling configuration register and the 2*2 basic pooling tuple to select the convolution result entering the 2*2 basic pooling tuple;

[0023] The 2*2 basic pooling tuple is connected to the convolution operation unit and convolution buffer level of the multiplexer group and the convolution processing module group. It is used to receive the convolution results output by the convolution operation unit and the convolution buffer level. Under the control of the multiplexer group, it can perform pooling calculations based on 2*2 pooling tuples on the convolution results in a pipelined manner during a single machine operation, and provide a pooling result output interface.

[0024] The beneficial effects of this invention are:

[0025] This invention employs a parallel data processing approach, which, when resources permit, can simultaneously instantiate multiple convolutional function blocks and pipeline pooling calculations of a set size within a single machine operation. This results in high data processing efficiency and short processing time. It can be used to assist in the deployment of convolutional neural networks, enabling them to achieve better real-time performance and computational efficiency. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the architecture of the present invention; Figure 2 This is a schematic diagram of the architecture of Embodiment 1 of the present invention;

[0027] Figure 3 This is a schematic diagram of the architecture of Embodiment 2 of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0029] See Figure 1 The present invention includes a row splitting module 1, a row array cache group 2, a convolution processing module group 3, and a pooling processing module 4;

[0030] The row splitting module 1 is connected to the row array cache group 2 to split the input raw data into independent data rows and transmit the data rows to the row array cache group 2.

[0031] The row array cache group 2 is connected to the row splitting module 1 and the convolution processing module group 3. It is used to receive and store the split data rows, adjust them according to the convolution rules to make them data row content that matches the size of the convolution kernel, and transmit the integrated data row group to the convolution processing module group 3.

[0032] The convolution processing module group 3 is connected to the row array cache group 2 and the pooling processing module 4, and is used to perform convolution operations on the data row groups transmitted from the row array cache group 2 and cache the convolution results. Subsequently, the direct convolution results and cached results are transmitted to the pooling processing module 4 in sequence.

[0033] The pooling processing module 4 is connected to the convolution processing module group 3, and is used to perform pooling operations on the convolution results transmitted from the convolution processing module group 3, and to provide an output interface for the pooling results.

[0034] See Figure 1 The row array cache group 2 includes a first-level row cache 21, a second-level row cache 22, a third-level row cache 23, and a fourth-level row cache 24. The first-level row cache 21 is connected to the second-level row cache 22 and the convolution processing module group 3, and is used to store the first n rows of data in an n*n convolution data group, and to transmit the stored data group to the convolution processing module group 3. The second-level row cache 22 is connected to the first-level row cache 21, the third-level row cache 23, and the convolution processing module group 3, and is used to store the first n rows of data starting from the second row of an n*n convolution data group, and to transmit the stored data group to the convolution processing module group 3. The data is transmitted to the convolution processing module group 3; the third-level row buffer 23 is connected to the second-level row buffer 22, the fourth-level row buffer 24 and the convolution processing module group 3, and is used to store the first n rows of data from the third row of the n*n convolution data row group, and transmit the stored data row group to the convolution processing module group 3; the fourth-level row buffer 24 is connected to the row splitting module 1, the third-level row buffer 23 and the convolution processing module group 3, and is used to receive the split data rows and store the first n rows of data from the fourth row of the n*n convolution data row group, and transmit the stored data row group to the convolution processing module group 3.

[0035] See Figure 1 The convolution processing module group 3 includes four parallel convolution processing modules 31; each convolution processing module 31 includes a convolution operation unit 311, a convolution cache level 312, and a convolution configuration register 313.

[0036] The convolution operation unit 311 is connected to the row array cache group 2, the convolution cache level 312, the convolution configuration register 313, and the pooling processing module 4. It is used to receive n rows of data for n*n convolution and perform convolution operations on them. Each convolution kernel performs n*n multiplication operations in parallel. After each convolution operation is completed, the convolution result is transmitted to the convolution cache level 312. Subsequently, the result of the last convolution operation is output to the pooling processing module 4. The convolution cache level 312 contains multiple registers and is connected to the convolution operation unit 311 and the pooling processing module 4. It is used to store the convolution result output by the convolution operation unit 311 for each operation. Subsequently, the stored convolution results are output to the pooling processing module 4 in sequence. The convolution configuration register 313 is connected to the convolution operation unit 311 and is used to control the running state and step of the convolution operation unit 311 when performing convolution calculations.

[0037] See Figure 1 The pooling processing module 4 includes a pooling configuration register 41, a multiplexer group 42, and a 2*2 basic pooling tuple 43. The pooling configuration register 41 is connected to the multiplexer group 42 to control the selection of the multiplexer group 42. The multiplexer group 42 is connected to the pooling configuration register 41 and the 2*2 basic pooling tuple 43 to select the convolution result entering the 2*2 basic pooling tuple 43. The 2*2 basic pooling tuple 43 is connected to the multiplexer group 42, the convolution operation unit 311, and the convolution buffer level 312 of the convolution processing module group 3 to receive the convolution result output by the convolution operation unit 311 and the convolution buffer level 312. Under the control of the multiplexer group 42, it can perform pooling calculations based on 2*2 pooling elements in a pipelined manner during a single machine operation, and provide a pooling result output interface.

[0038] Example 1

[0039] See Figure 2 This embodiment takes the 4*4 convolution operation with a 3*3 pooling structure as an example.

[0040] The row splitting module 1 splits the raw data from the off-chip host into independent data rows, and then transmits the data rows to the row array cache group 2 one by one.

[0041] After receiving the data row groups transmitted from the row splitting module 1, the row array cache group 2 adjusts them according to the convolution rules to match the data row content with the convolution kernel size and stores them. In this embodiment, the first-level row cache 21 stores the first, second, third, and fourth rows of the data row group; the second-level row cache 22 stores the second, third, fourth, and fifth rows of the data row group; the third-level row cache 23 stores the third, fourth, fifth, and sixth rows of the data row group; and the fourth-level row cache 24 stores the fourth, fifth, sixth, and seventh rows of the data row group. After caching, each level of row cache transmits the data row groups in parallel to the corresponding convolution operation unit 311.

[0042] The convolution configuration register 313 of the fourth convolution processing module 31 from top to bottom in the convolution processing module group 3 disables its convolution operation unit 311 because the 3*3 pooling structure does not need the convolution operation result of the fourth convolution operation unit 311. The remaining three convolution operation units 311 perform convolution operations on the corresponding data row groups transmitted from the row array cache group 2 in parallel, and the step size of each convolution operation is controlled by the convolution configuration register 313. The result of each convolution operation of these three convolution operation units 311 is transmitted to the corresponding convolution cache level 312, and the result of the last operation is transmitted to the 2*2 basic pooling tuple 43. In this embodiment, the convolution cache level 312 uses two levels of registers, and the convolution operation results cached in each level of register are transmitted to the 2*2 basic pooling tuple 43.

[0043] In the pooling processing module 4, the multiplexer group 42, under the control of the pooling configuration register 41, selects the convolution result entering the 2*2 basic pooling tuple 43. The 2*2 basic pooling tuple 43 performs pooling calculations based on 2*2 pooling elements on the convolution result from the convolution operation unit 311 and the convolution buffer level 312, completing the configured pooling calculation in a pipelined manner within a single machine operation, and provides a pooling result output interface.

[0044] Example 2

[0045] See Figure 3 This embodiment takes 4*4 convolution operation with 4*4 pooling structure as an example.

[0046] The row splitting module 1 splits the raw data from the off-chip host into independent data rows, and then transmits the data rows to the row array cache group 2 one by one.

[0047] After receiving the data row groups transmitted from the row splitting module 1, the row array cache group 2 adjusts them according to the convolution rules to match the data row content with the convolution kernel size and stores them. In this embodiment, the first-level row cache 21 stores the first, second, third, and fourth rows of the data row group; the second-level row cache 22 stores the second, third, fourth, and fifth rows of the data row group; the third-level row cache 23 stores the third, fourth, fifth, and sixth rows of the data row group; and the fourth-level row cache 24 stores the fourth, fifth, sixth, and seventh rows of the data row group. After caching, each level of row cache transmits the data row groups in parallel to the corresponding convolution operation unit 311.

[0048] The four convolution operation units 311 in the convolution processing module group 3 perform convolution operations in parallel on the corresponding data row groups transmitted from the row array cache group 2. The step size of each convolution operation is controlled by the convolution configuration register 313. The result of each convolution operation by the four convolution operation units 311 is transmitted to the corresponding convolution cache level 312, and the result of the last operation is transmitted to the 2*2 basic pooling tuple 43. In this embodiment, the convolution cache level 312 uses three levels of registers, and the convolution operation results cached in each level of register are transmitted to the 2*2 basic pooling tuple 43.

[0049] In the pooling processing module 4, the multiplexer group 42, under the control of the pooling configuration register 41, selects the convolution result entering the 2*2 basic pooling tuple 43. The 2*2 basic pooling tuple 43 performs pooling calculations based on 2*2 pooling elements on the convolution result from the convolution operation unit 311 and the convolution buffer level 312, completing the configured pooling calculation in a pipelined manner within a single machine operation, and provides a pooling result output interface.

Claims

1. A fine-grained convolutional neural network acceleration architecture, characterized in that, The architecture includes a row splitting module (1), a row array cache group (2), a convolution processing module group (3), and a pooling processing module (4); The row splitting module (1) is connected to the row array cache group (2) to split the input raw data into independent data rows and transmit the data rows to the row array cache group (2); The row array cache group (2) is connected to the row splitting module (1) and the convolution processing module group (3) to receive and store the split data rows, adjust them according to the convolution rules to make them data row content that matches the size of the convolution kernel, and transmit the integrated data row group to the convolution processing module group (3). The convolution processing module group (3) is connected to the row array cache group (2) and the pooling processing module (4) to perform convolution operations on the data row group from the row array cache group (2) and cache the convolution results. Subsequently, the direct convolution results and cached results are transmitted to the pooling processing module (4) in sequence. The pooling module (4) is connected to the convolution processing module group (3) to perform pooling operations on the convolution results transmitted from the convolution processing module group (3) and to provide an output interface for the pooling results; wherein: The row array cache group (2) includes a level 1 row cache (21), a level 2 row cache (22), a level 3 row cache (23), and a level 4 row cache (24); The first-level row cache (21) is connected to the second-level row cache (22) and the convolution processing module group (3) to store the first n rows of data of the n*n convolution data row group and transmit the stored data row group to the convolution processing module group (3); The secondary row cache (22) is connected to the primary row cache (21), the tertiary row cache (23) and the convolution processing module group (3) to store the first n rows of data from the second row of the n*n convolution data row group, and to transmit the stored data row group to the convolution processing module group (3). The third-level row cache (23) is connected to the second-level row cache (22), the fourth-level row cache (24) and the convolution processing module group (3) to store the first n rows of data from the third row of the n*n convolution data row group, and to transmit the stored data row group to the convolution processing module group (3). The fourth-level row cache (24) is connected to the row splitting module (1), the third-level row cache (23) and the convolution processing module group (3) to receive the split data rows and store the first n rows of data starting from the fourth row of the n*n convolution data row group, and transmit the stored data row group to the convolution processing module group (3). The pooling processing module (4) includes a pooling configuration register (41), a multiplexer group (42), and a 2*2 basic pooling tuple (43); The pooling configuration register (41) is connected to the multiplexer group (42) and is used to control the selection of the multiplexer group (42); The multiplexer group (42) is connected to the pooling configuration register (41) and the 2*2 basic pooling tuple (43) to select the convolution result entering the 2*2 basic pooling tuple (43); The 2*2 basic pooling tuple (43) is connected to the convolution operation unit (311) and convolution buffer level (312) of the multiplexer group (42) and the convolution processing module group (3) to receive the convolution result output by the convolution operation unit (311) and the convolution buffer level (312). Under the control of the multiplexer group (42), the convolution result can be processed by pooling calculation of the configured size based on 2*2 pooling tuple in a single machine operation, and a pooling result output interface is provided.

2. The fine-grained convolutional neural network acceleration architecture according to claim 1, characterized in that, The convolution processing module group (3) includes four parallel convolution processing modules (31); The convolution processing module (31) includes a convolution operation unit (311), a convolution cache level (312), and a convolution configuration register (313); The convolution operation unit (311) is connected to the row array cache group (2), the convolution cache level (312), the convolution configuration register (313) and the pooling processing module (4) to receive n rows of data of n*n convolution and perform convolution operation on them. Each convolution kernel performs n*n multiplication operations in parallel. After each convolution operation is completed, the convolution result is transmitted to the convolution cache level (312). Subsequently, the result of the last convolution operation is output to the pooling processing module (4). The convolution cache level (312) includes multiple registers and is connected to the convolution operation unit (311) and the pooling processing module (4) to store each convolution result output by the convolution operation unit (311). Subsequently, the stored convolution results are output to the pooling processing module (4) in sequence. The convolution configuration register (313) is connected to the convolution operation unit (311) and is used to control the running state and stepping of the convolution operation unit (311) when performing convolution calculation.

Images