Reconfigurable processing element, array and method for convolutional neural networks
By designing reconfigurable processing elements and arrays, the problem of high storage access latency in convolutional neural networks is solved, enabling flexible data transfer and reuse between processing elements, thereby improving computational efficiency.
Patent Information
- Application Number
- CN202210951418.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-09
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2042-08-09
AI Technical Summary
Existing convolutional neural network computation suffers from low global reuse and high local reuse, leading to processors repeatedly accessing the same memory addresses, resulting in low real-time computation efficiency.
Design a reconfigurable processing element and array for convolutional neural networks, including a data selector, an ALU module, a PSUM part, an accumulation module, and a POOL pooling module. It supports data reuse and reduces storage access latency by flexibly transferring data for reuse among processing elements.
This effectively reduces repeated accesses to the same memory address by the processor, lowers the overall access latency, and improves the computational efficiency of convolutional neural networks.
Smart Images
Figure CN115329947B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of convolutional neural network technology, and in particular to a reconfigurable processing element, array, and method for convolutional neural networks. Background Technology
[0002] In recent years, convolutional neural networks, as an important direction in the research of multi-layer artificial neural networks, have always been a research hotspot.
[0003] In existing convolutional neural network computations, there is a phenomenon of low global reuse and high local reuse rate in convolutional computations. That is, the processor repeatedly accesses the same address in memory multiple times. To address this, the existing solution is to read the corresponding data directly from the cache or from the outside. However, due to hardware limitations, the data reading bandwidth is increasingly unable to meet the huge amount of computation, and the real-time computing efficiency is also unsatisfactory.
[0004] Therefore, there is an urgent need for a solution to address the total access latency of convolutional neural networks and improve their computational efficiency. Summary of the Invention
[0005] (a) Technical problems to be solved
[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a reconfigurable processing element, array and method for convolutional neural networks, which solves the problems of repeated access to the same address in storage and low real-time computing efficiency in existing convolutional calculations.
[0007] (II) Technical Solution
[0008] To achieve the above objectives, the main technical solutions adopted by the present invention include:
[0009] In a first aspect, embodiments of the present invention provide a reconfigurable processing element for convolutional neural networks, the processing element comprising:
[0010] A data selector that selects the direction of input and output data flow;
[0011] The ALU module performs a multiplication and accumulation operation on the feature image and the convolution kernel to obtain a partial sum;
[0012] A PSUM partial sum accumulation module is available to accumulate partial sums between different processing elements and to accumulate partial sums of different local processing elements; and,
[0013] A POOL pooling module that supports multiple pooling methods.
[0014] Optionally, the data selector includes:
[0015] A kernel source selector is used to select from the kernel data of several adjacent processing elements input;
[0016] A feature map source selector is used to select from feature map data inputs of several adjacent processing elements;
[0017] The output flow selector is used to select the output direction for the data input to adjacent processing units and the processed data of local processing units.
[0018] Optionally, the ALU module includes: a plurality of pipeline sub-units and an adder connected to the output of each of the pipeline sub-units;
[0019] When the ALU module performs a multiply-accumulate operation, it sequentially calls the corresponding pipeline subunits to perform several multiplication operations according to the pipeline order, and the adder sums and outputs the data obtained from each multiplication operation to obtain several partial sums.
[0020] Optionally, each pipeline subunit is provided with a feature map register, a convolution kernel register, and a multiplier connected to the feature map register and the convolution kernel register respectively;
[0021] The feature map register is used to receive input feature map data and update the feature map data when the data is enabled and when the clock rises.
[0022] The convolution kernel register is used to receive the input convolution kernel data and hold the convolution kernel data until the current convolution calculation in the ALU module is completed and then cleared to zero.
[0023] The multiplier is used to perform a multiplication operation based on the input feature map data and the input convolution kernel data.
[0024] Optionally, the PSUM section and the accumulation module include: an adjacency processing element selection unit, a local processing element selection unit, a first adder, a second adder, an accumulation register, and an accumulation result output unit;
[0025] The adjacency processing element selection unit includes: an input interface for accessing adjacency processing metadata and an accumulation permission port for determining whether the adjacency processing element participates in the accumulation.
[0026] The local processing element selection unit includes: an input interface for accessing local processing metadata and an accumulation permission port for selecting whether the local processing element participates in the accumulation.
[0027] The first adder is connected to the adjacent processing element selection unit and the local processing element selection unit, respectively.
[0028] When adjacent processing metadata is involved in the accumulation, the first adder is used to perform the accumulation operation of the adjacent processing metadata and the local processing metadata;
[0029] When only local processing metadata is involved in the accumulation, the first adder will hand over the received local processing metadata to the second adder to perform the accumulation operation of the local processing metadata;
[0030] The adjacent processing element is a unit that transmits data relative to the local processing element above, below, to the lower left, and to the upper right.
[0031] Optionally, the POOL pooling module includes:
[0032] A max-pooling unit, comprising: a comparator for comparing the size of data values and a data register for storing the maximum value;
[0033] The average pooling unit includes: a summation module for calculating the sum of the input data, a counting module for counting the number of input data, and a division module for calculating the average value based on the sum of the input data and the number of input data.
[0034] Optionally, the processing element further includes: an activation module, a normalization module, and a normalized data selector;
[0035] The activation module is connected to the PSUM section and the accumulation module through the low-level path of the normalized data selector;
[0036] One end of the normalization module is connected to the PSUM section and the accumulation module; the other end is connected to the activation module through the high-level path of the normalized data selector.
[0037] The activation module includes RELU, RELU6, and Sigmod.
[0038] Secondly, embodiments of the present invention provide a reconfigurable processing array for convolutional neural networks, comprising: a plurality of processing elements as described above;
[0039] Each of the aforementioned processing elements transmits the received raw data and / or processed data to the adjacent processing element to improve the computational efficiency of the convolutional neural network through data reuse.
[0040] Wherein, the original data is the image data or convolutional kernel data input to the reconfigurable processing array; the processed data is the pooled data output by the POOL pooling module, the partial sum data output by the PSUM part and the accumulation module, the normalized data output by the normalization module, or the activation data output by the activation module.
[0041] Optionally, the processing elements are arranged in an 8×8 structure, and each processing element has data transmission ports in six directions: up, down, left, right, lower left, and upper right.
[0042] Thirdly, an embodiment of the present invention provides a reconfigurable processing method for convolutional neural networks, applied to the reconfigurable processing element described above, comprising:
[0043] The processing element obtains the configuration information issued by the preset H-tree;
[0044] The processing element parses the configuration information to control the direction of input data flow, the direction of output data flow, and to switch internal paths within the processing element.
[0045] The input data consists of data and data enable signals from several adjacent processing units; the output data consists of data and data enable signals output from the local processing unit to several adjacent processing units; and the internal pathways of the processing unit include: pooling pathway, convolution pathway, convolution-activation pathway, and convolution-normalization-activation pathway.
[0046] (III) Beneficial Effects
[0047] The beneficial effects of this invention are as follows: In view of the characteristics of low global reuse and high local reuse rate in convolutional computation, this invention designs a reconfigurable processing element structure for convolutional neural networks that supports data reuse. The processing element completes the data processing in a data-driven manner. The original data and calculation results can be flexibly transferred and reused between different processing elements. This effectively reduces the processor's repeated access to the same address in memory, reduces the total access latency, and improves the computational efficiency of convolutional neural networks. Attached Figure Description
[0048] Figure 1 This is a schematic diagram of the structure of a reconfigurable processing element for a convolutional neural network provided by the present invention;
[0049] Figure 2 A diagram of an ALU unit structure for a reconfigurable processing element in a convolutional neural network, provided by the present invention;
[0050] Figure 3 This invention provides a convolution execution timing sequence for reconfigurable processing elements in convolutional neural networks;
[0051] Figure 4 The present invention provides a structural diagram of the PSUM part and accumulation module of a reconfigurable processing element for a convolutional neural network;
[0052] Figure 5The present invention provides a POOL pooling module structure diagram for a reconfigurable processing element in a convolutional neural network.
[0053] Figure 6 A schematic diagram of the interconnection structure of reconfigurable processing elements for convolutional neural networks provided by the present invention;
[0054] Figure 7 This invention provides H-tree configuration information for reconfigurable processing elements in convolutional neural networks.
[0055] Figure 8 A schematic diagram of a convolution-activation path for a reconfigurable processing element in a convolutional neural network, provided by the present invention;
[0056] Figure 9 This is a schematic diagram of a pooling path for a reconfigurable processing element in a convolutional neural network, as provided by the present invention. Detailed Implementation
[0057] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0058] like Figure 1 As shown in the embodiment of the present invention, a reconfigurable processing unit for convolutional neural networks includes: a data selector that selects the direction of input and output data flow; an ALU module that performs multiplication and accumulation operations on feature images and convolutional kernels to obtain several partial sums; a PSUM partial sum accumulation module that can accumulate partial sums between different processing units and accumulate different partial sums of local processing units; and a POOL pooling module that supports multiple pooling methods.
[0059] Given the characteristics of convolutional computation, which has low global reuse and high local reuse rate, this invention designs a reconfigurable processing element structure for convolutional neural networks that supports data reuse. The processing element completes data processing in a data-driven manner, and the original data and calculation results can be flexibly transferred and reused between different processing elements. This effectively reduces the processor's repeated access to the same address in memory, reduces the total access latency, and improves the computational efficiency of convolutional neural networks.
[0060] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.
[0061] like Figure 1As shown in Table 1, the processing units include: a data selector (S1, S2, right_up_out, right_out, left_down_out, up_out, left_out, and down_out) that selects the direction of input and output data flow; an ALU module that performs multiplication and accumulation operations on feature images and convolution kernels to obtain several partial sums; a PSUM partial sum accumulation module that can accumulate partial sums between different processing units and accumulate different partial sums of local processing units; and a POOL pooling module that supports multiple pooling methods.
[0062] It is worth mentioning that the processing unit also includes: an activation module, a normalization module (BN), and a normalization data selector; the activation module is connected to the PSUM section and the accumulation module through the low-level path of the normalization data selector; one end of the normalization module is connected to the PSUM section and the accumulation module; the other end is connected to the activation module through the high-level path of the normalization data selector; wherein, the activation module includes: RELU, RELU6, and Sigmod.
[0063] Table 1. Definitions and Functional Descriptions of Reconfigurable Processing Meta Interface Signals
[0064]
[0065]
[0066] Further, refer to Figure 1 It can be seen that the data selector includes: a convolution kernel source selector S1, used to select from the convolution kernel data input to several adjacent processing units; a feature map source selector S6, used to select from the feature map data input to several adjacent processing units; and output flow selectors (right_up_out, right_out, left_down_out, up_out, left_out, and down_out), used to select the output direction for the data input to adjacent processing units and the processing data of local processing units.
[0067] Furthermore, the ALU module includes: several pipeline sub-units and an adder connected to the output of each pipeline sub-unit; when the ALU module performs multiply-accumulate operations, it sequentially calls the corresponding pipeline sub-units according to the pipeline order to perform several multiplication operations, and the adder sums and outputs the data obtained from each multiplication operation to obtain several partial sums. Each pipeline sub-unit is equipped with a feature map register, a convolution kernel register, and a multiplier connected to the feature map register and the convolution kernel register respectively; the feature map register is used to receive the input feature map data and update the feature map data when the data is enabled and the clock rises; the convolution kernel register is used to receive the input convolution kernel data and keep the convolution kernel data until the convolution calculation in this ALU module is completed and cleared to zero; the multiplier is used to perform multiplication operations based on the input feature map data and the input convolution kernel data.
[0068] Therefore, the ALU module is the key unit for calculating the feature image and the convolution kernel, capable of performing multiplication and accumulation operations on multiple feature image pixels and convolution kernel pixels. For example... Figure 2 As shown in Table 2, the ALU module contains 8 feature map registers at the top, 8 convolution kernel registers at the bottom, 8 multipliers, and 1 adder. The convolution kernel registers retain data throughout the computation until the computation is complete, while the data in the feature map registers is updated on the data enable and rising clock edges. The configuration information provides the size of one row of convolution kernel data, the number of one row of feature map data, and the stride of the convolution kernel traversing the image. The number of multipliers and the number of convolution kernel registers required are determined based on the size of the convolution kernel. Each multiplier performs a multiplication operation between one input image pixel and one convolution kernel pixel per clock cycle.
[0069] Table 2. ALU Module Interface Signal Definitions and Function Descriptions
[0070]
[0071]
[0072] In a specific embodiment, the ALU module adopts a pipelined design. When the execution stride is 1, the input feature image block is 7×7, and the convolution kernel size is 3×3, the output feature map size is 5×5. Each row of the convolution kernel has 3 numbers, requiring 3 multipliers and a convolution kernel register for computation. Each row of the input image block has 7 numbers. (The last sentence appears to be incomplete and possibly refers to a different implementation.) Figure 2 and Figure 3 As shown, firstly, the data from the three convolution kernels flows into the weight port of the ALU. Once the data flows into the first three registers, the `en_weight_switch` signal goes high, controlling the convolution kernel data in the registers to remain unchanged. Then, the features... Figure 7 Data flows into the map port, then sequentially into all feature map registers. When feature map data is enabled, each feature map pixel is multiplied by its corresponding convolution kernel pixel to obtain three products. These three products are then summed by an adder to obtain the first partial sum output. This process is repeated four times to obtain a 1×3 convolution kernel on a 1×7 input image, ultimately resulting in a 5-pixel output.
[0073] Furthermore, the PSUM section and accumulation module include: an adjacency processing element selection unit, a local processing element selection unit, a first adder, a second adder, an accumulation register, and an accumulation result output unit; the adjacency processing element selection unit includes: an input interface for accessing adjacency processing metadata and an accumulation enable port for determining whether the adjacency processing element participates in the accumulation; the local processing element selection unit includes: an input interface for accessing local processing metadata and an accumulation enable port for selecting whether the local processing element participates in the accumulation; the first adder is connected to the adjacency processing element selection unit and the local processing element selection unit respectively; when adjacency processing metadata participates in the accumulation, the first adder is used to perform the accumulation operation of the adjacency processing metadata and the local processing element; when only local processing metadata participates in the accumulation, the first adder passes the accessed local processing metadata to the second adder to perform the accumulation operation of the local processing element; wherein, the adjacency processing element is a unit that transmits data relative to the top, bottom, lower left, and upper right of the local processing element.
[0074] The PSUM section and accumulation module can accumulate partial sums between different processing elements, and can also accumulate partial sums of different processing elements locally. (Reference) Figure 4 As per Table 3, when performing data accumulation between processing elements, `en_psum` must be high. When `add_present` is valid, the result from the ALU module of the local processing element is input to the adder in the PSUM module. Simultaneously, if `add_up` is valid, the output result of the processing element above or to the upper right is input to the PSUM module, achieving addition between the corresponding positions of the processing element above or to the upper right and the local ALU output data. Here, it is required that the corresponding data for the addition operation simultaneously enter the PSUM module to obtain the correct partial sum accumulation result `psum_in`. Finally, `psum_in` is output as the partial sum accumulation result of the local processing element and its neighboring processing elements. When performing accumulation of different partial sums of the local processing element, `en_psum` is low. Only when `add_present` is valid is the ALU calculation result in the local processing element input to PSUM and accumulated with the ALU calculation result of the next clock cycle.
[0075] Table 3. Definitions and Functional Descriptions of PSUM Section and Accumulation Module Interface Signals
[0076]
[0077]
[0078] Furthermore, the POOL pooling module includes: a max pooling unit, which includes a comparator for comparing the size of data values and a register for storing the maximum value; and an average pooling unit, which includes a summation module for calculating the sum of the input data, a counting module for counting the number of input data, and a division module for calculating the average value based on the sum of the input data and the number of input data.
[0079] refer to Figure 5 As shown in Table 4, the pooling module supports both max pooling and average pooling. For example, in 7×7 max pooling, 49 numbers are sequentially input into the Max pooling module. The comparison module compares the numbers after each input, storing the maximum value for the next input. This process is repeated multiple times until the maximum of the 49 numbers is obtained and output as the max pooling result. In average pooling, the data required for one pooling operation is summed, and the input data is counted. The sum is then input into the div division module to calculate the average, and finally, the average is output.
[0080] Table 4. Interface Signal Definitions and Function Descriptions for the POOL Pooling Module
[0081]
[0082]
[0083] Furthermore, embodiments of the present invention also provide a reconfigurable processing array for convolutional neural networks, comprising: a plurality of processing elements; each processing element transmits received raw data and / or processed data to adjacent processing elements to improve the computational efficiency of the convolutional neural network through data reuse. The raw data is image data or convolutional kernel data input to the reconfigurable processing array; the processed data is pooled data output by the POOL pooling module, the PSUM portion and the partial sum data output by the accumulation module, the normalized data output by the normalization module, or the activation data output by the activation module.
[0084] like Figure 6As shown, considering that the multiplication-accumulation operation in convolution calculation requires two operands—the feature image and the convolution kernel—and that convolution calculation has high locality of access and involves a large amount of data reuse, this embodiment of the invention proposes a reconfigurable array structure composed of 8×8 processing elements. That is, in addition to supporting top, bottom, left, and right adjacent interconnections at the input and output ports of each processing element, data transmission along the diagonal is added to better achieve data reuse. Therefore, each processing element can support data transmission in six directions: top, bottom, left, right, bottom left, and top right. Preferably, the reconfigurable array can simultaneously perform convolution on different pixels within a region of the image, or simultaneously perform convolution calculations on different regions of the image.
[0085] Furthermore, the present invention also provides a reconfigurable processing method for convolutional neural networks, applied to the reconfigurable processing unit as described above, comprising: the processing unit acquiring configuration information issued by a preset H-tree; the processing unit controlling the input data flow direction, the output data flow direction, and switching the internal pathways of the processing unit by parsing the configuration information; wherein, the input data is data and data enable from several adjacent processing units; the output data is data and data enable from the local processing unit to several adjacent processing units; and the internal pathways of the processing unit include: pooling pathway, convolution pathway, convolution-activation pathway, and convolution-normalization-activation pathway.
[0086] During reconfigurable processing, the first step is to send 72 bits of configuration information via the H-tree, where the high 7 bits are used to select the target processor for the configuration. The low 65 bits are used as follows: Figure 7 As shown, this is used to control the data flow of processor input and output, as well as to switch internal processor paths, thereby implementing different functions such as convolution, pooling, and activation. Table 5 explains the definition and function of the configuration information.
[0087] Table 5 Configuration Information Definitions and Function Descriptions
[0088]
[0089]
[0090] Figure 8The portion shown in the darker areas represents the convolution-activation pathway. First, s1 and s6 select the data sources for the convolution kernel and feature map, respectively. As shown, the convolution kernel data is input from the left port, and the feature map data comes from the upper right port. s4 and s7 control whether to perform convolution or pooling operations on the corresponding data. As shown, s4 and s7 are 0, indicating convolution. A row of convolution kernel data and a row of feature map data are convolved in the ALU to obtain the first partial sum of the current row. At this point, the configuration information `add_present` and `down` are active. The partial sum result is input to the PSUM module and added to the adjacent PE result below. The convolution kernel in the ALU then slides to the right to calculate the second convolution in the same row. Simultaneously, s17 selects whether to activate the convolution or normalization result. As shown, the convolution result is activated. s19 selects the output of the ReLU-activated result, and this process continues until a row of convolution kernels has traversed a row of image pixels.
[0091] like Figure 9 The part shown in the dark area is the pooling path. As shown in the figure, s1 selects the data source from the upper port, s4 selects to perform the pooling operation, the input data flows into the POOL pooling module from s10, and s16 selects to output the result of max pooling or average pooling as the final pooling result. Here, max pooling is selected.
[0092] In summary, to reduce data communication between the processor and main memory and further improve the speed of convolution operations, this invention provides a reconfigurable processing element, array, and method for convolutional neural networks, taking advantage of the characteristics of low global reuse and high local reuse in convolutional computation. In addition to supporting top, bottom, left, and right adjacency interconnections at the input and output ports of each processing element, data transmission along the diagonal is added. Inside each processing element, two data selectors select the data sources for the input feature map and the convolution kernel, respectively. The ALU module is responsible for calculating the convolution of one row of convolution kernels onto one row of input feature maps, using a pipelined approach and accumulating the partial sum of the current row in the ALU. The PSUM part and the accumulation module are responsible for accumulating the result of the current processing element's ALU calculation with the convolution result of one row output from the processing element above, to the upper right, below, or to the lower left. The accumulated data can be selected for BN normalization, ReLU, ReLU6, Sigmoid activation, etc., as needed. The POOL pooling module can perform pooling operations on all input data, including max pooling and average pooling. Ultimately, the input from all directions, as well as the data after convolution and pooling, can all be used by a data selector to choose the desired output direction.
[0093] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.
[0094] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0095] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.
[0096] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.
[0097] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0098] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0099] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.
Claims
1. A reconfigurable processing element for convolutional neural networks, characterized in that, The processing element includes: A data selector that selects the direction of input and output data flow; The ALU module performs a multiplication and accumulation operation on the feature image and the convolution kernel to obtain a partial sum; A PSUM partial sum accumulation module is provided, capable of accumulating partial sums between different processing elements and accumulating partial sums of different local processing elements. The PSUM partial sum accumulation module includes: an adjacent processing element selection unit, a local processing element selection unit, a first adder, a second adder, an accumulation register, and an accumulation result output unit. The adjacent processing element selection unit includes: an input interface for accessing adjacent processing metadata and an accumulation enable port for determining whether the adjacent processing element participates in the accumulation. The local processing element selection unit includes: an input interface for accessing local processing metadata and a port for selecting the local processing element. The accumulation allowed port for whether the processing element participates in the accumulation; the first adder is connected to the adjacent processing element selection unit and the local processing element selection unit respectively; when adjacent processing metadata participates in the accumulation, the first adder is used to perform the accumulation operation of the adjacent processing metadata and the local processing element; when only local processing metadata participates in the accumulation, the first adder passes the received local processing metadata to the second adder to perform the accumulation operation of the local processing element; wherein, the adjacent processing element is a unit that transmits data relative to the top, bottom, lower left, and upper right of the local processing element; and, A POOL pooling module that supports multiple pooling methods.
2. The reconfigurable processing element for convolutional neural networks as described in claim 1, characterized in that, The data selector includes: A kernel source selector is used to select from the kernel data of several adjacent processing elements input; A feature map source selector is used to select from feature map data inputs of several adjacent processing elements; The output flow selector is used to select the output direction for the data input to adjacent processing units and the processed data of local processing units.
3. The reconfigurable processing element for convolutional neural networks as described in claim 1, characterized in that, The ALU module includes: a plurality of pipeline sub-units and an adder connected to the output of each of the pipeline sub-units; When the ALU module performs a multiply-accumulate operation, it sequentially calls the corresponding pipeline subunits to perform several multiplication operations according to the pipeline order, and the adder sums and outputs the data obtained from each multiplication operation to obtain several partial sums.
4. The reconfigurable processing element for convolutional neural networks as described in claim 3, characterized in that, Each pipeline subunit at each stage is equipped with a feature map register, a convolution kernel register, and a multiplier connected to the feature map register and the convolution kernel register respectively. The feature map register is used to receive input feature map data and update the feature map data when the data is enabled and when the clock rises. The convolution kernel register is used to receive the input convolution kernel data and hold the convolution kernel data until the current convolution calculation in the ALU module is completed and then cleared to zero. The multiplier is used to perform a multiplication operation based on the input feature map data and the input convolution kernel data.
5. The reconfigurable processing element for convolutional neural networks as described in claim 1, characterized in that, The POOL pooling module includes: A max-pooling unit, comprising: a comparator for comparing the size of data values and a data register for storing the maximum value; The average pooling unit includes: a summation module for calculating the sum of the input data, a counting module for counting the number of input data, and a division module for calculating the average value based on the sum of the input data and the number of input data.
6. The reconfigurable processing element for convolutional neural networks as described in claim 1, characterized in that, The processing unit further includes: an activation module, a normalization module, and a normalized data selector; The activation module is connected to the PSUM section and the accumulation module through the low-level path of the normalized data selector; One end of the normalization module is connected to the PSUM section and the accumulation module; the other end is connected to the activation module through the high-level path of the normalized data selector. The activation module includes RELU, RELU6, and Sigmod.
7. A reconfigurable processing array for convolutional neural networks, characterized in that, include: Several processing elements as described in claim 6; Each of the aforementioned processing elements transmits the received raw data and / or processed data to the adjacent processing element to improve the computational efficiency of the convolutional neural network through data reuse. Wherein, the original data is the image data or convolutional kernel data input to the reconfigurable processing array; the processed data is the pooled data output by the POOL pooling module, the partial sum data output by the PSUM part and the accumulation module, the normalized data output by the normalization module, or the activation data output by the activation module.
8. The reconfigurable processing array for convolutional neural networks as described in claim 7, characterized in that, The processing elements are arranged in an 8×8 structure, and each processing element has data transmission ports in six directions: up, down, left, right, lower left, and upper right.
9. A reconfigurable processing method for convolutional neural networks, applied to the reconfigurable processing element as described in any one of claims 1-6, characterized in that, include: The processing element obtains the configuration information issued by the preset H-tree; The processing element parses the configuration information to control the direction of input data flow, the direction of output data flow, and to switch internal paths within the processing element. The input data consists of data and data enable signals from several adjacent processing units; the output data consists of data and data enable signals output from the local processing unit to several adjacent processing units; and the internal pathways of the processing unit include: pooling pathway, convolution pathway, convolution-activation pathway, and convolution-normalization-activation pathway.