Convolution circuit, convolution calculation method, chip and electronic device

Abstract

Provided in the embodiments of the present application are a convolution circuit, a convolution calculation method, a chip and an electronic device, which are applied to the technical field of chips to solve the problems of high power consumption and large layout area of convolution circuits in existing convolvers. The convolution circuit comprises a plurality of multipliers, a first adder coupled to the plurality of multipliers, and a second adder coupled to the first adder. Each multiplier comprises a plurality of precoders, a plurality of coder groups, and an adder tree circuit, wherein each precoder corresponds to one coder group on an one-to-one basis, and output ends of the plurality of coder groups and input lines of the adder tree circuit are the same in quantity and correspond to each other on an one-to-one basis. In addition, the adder tree circuits are coupled to the first adder. The second adder is further coupled to a memory. Partial products only related to weighting parameters and the constant 1 can be accumulated first in the multipliers, and then are added in the second adder to results output by the adder tree circuits, thereby reducing the area and power consumption of the adder tree circuits.

Description

Convolution circuit, convolution calculation method, chip and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on June 30, 2023, with application number 202310798299.9 and application name “A convolution circuit, convolution calculation method, chip and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of chip technology, and in particular to a convolution circuit, a convolution calculation method, a chip, and an electronic device. Background Art

[0003] With the development of artificial intelligence (AI) technology, more and more AI products are entering people's daily lives. In the implementation process of AI technology, its core mainly includes two aspects: 1. Advanced neural network algorithms; 2. Processors that provide massive hardware computing power. Among them, the calculation of neural network algorithms is mainly convolution calculation type. Traditionally, there are two hardware solutions to implement convolution calculation: (1) Convert convolution calculation into matrix operation, and integrate a large number of multiplier matrices by designing a general-purpose neural network processing unit (NPU) to implement matrix multiplication calculation. (2) Design a dedicated convolution device to directly implement convolution calculation. Among them, the first solution is the current mainstream solution. However, this solution requires a large number of repeated read and write operations on the data, which increases power consumption. The convolution device in the second solution is composed of multiple multipliers and adders. After the multiple multipliers independently multiply the input feature map, they are accumulated through the adder to obtain the output feature map. However, the structure of each multiplier is relatively complex, which increases the power consumption and wiring area of ​​the convolution device.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a convolution circuit, a convolution calculation method, a chip, and an electronic device to solve the problems of high power consumption and large wiring area in current convolver hardware circuits.

[0006] In order to solve the above problems, the embodiments of the present application provide the following technical solutions:

[0007] In a first aspect, a convolution circuit is provided, comprising: a plurality of multipliers, a first adder, and a second adder. The multipliers include a plurality of precoders, a plurality of encoder groups, and an adder tree circuit. The plurality of precoders are coupled to the plurality of encoder groups in a one-to-one correspondence. The adder tree circuit includes a plurality of input lines and a plurality of output lines. The plurality of encoder groups include a plurality of output terminals. The number of the plurality of output terminals is the same as the number of the plurality of input lines of the adder tree circuit, and the plurality of output terminals of the plurality of encoder groups are coupled to the plurality of input lines of the adder tree circuit in a one-to-one correspondence. The adder tree circuit is coupled to the first adder via the plurality of output lines. The second adder includes a plurality of input terminals and an output terminal. One of the plurality of input terminals is used to couple to a memory, and the other input terminals of the plurality of input terminals are used to couple to the first adder. When performing a convolution operation, the adder tree circuit only needs to operate on the data output by multiple encoder groups, and does not accumulate the partial products related to the odd bits of the weight parameters directly from the input in the multiplier and the constant 1, thereby reducing the number of full adders used in the adder tree circuit and further reducing the power consumption and wiring area of ​​the entire convolution circuit.

[0008] In one possible implementation, the multiple precoders include a first precoder. The multiple encoder groups include a first encoder group. The first precoder includes three input terminals, a first logic circuit, and two output terminals. The first encoder group includes a first input terminal and a second input terminal. The first input terminal of the first encoder group is coupled to the first input terminal of the first precoder via the first output terminal of the first precoder. The three input terminals are also coupled to the second of the two output terminals via the first logic circuit. The second input terminal of the first encoder group is coupled to the second of the two output terminals. During convolution, the first encoder group also inputs the corresponding odd-numbered bits in the weight parameters. The two input terminals of the first precoder input the two bits preceding the odd-numbered bits. Based on this, the first input terminal of the first encoder group can directly input the second bit preceding the odd-numbered bit, while the second input terminal of the first encoder group inputs the value calculated by the first logic circuit based on the two bits preceding the odd-numbered bit. Compared with the precoder in a traditional multiplier, this can reduce one logical operation, thereby reducing power consumption and improving operational efficiency.

[0009] In one possible implementation, the first logic circuit may perform an XOR operation on the signals input from the second and third inputs of the three inputs, then perform a NOR operation on the signals input from the first input of the three inputs, and output the result from the second output of the two outputs. As a result, compared to a traditional precoder, the first precoder can perform one fewer XOR operation, further reducing computational power consumption.

[0010] In one possible implementation, the first logic circuit includes an XOR gate and a NOR gate. The first input of the NOR gate is coupled to the output of the XOR gate. The second input of the NOR gate is coupled to the first input of the first precoder. The first input of the XOR gate is coupled to the second input of the first precoder. The second input of the XOR gate is coupled to the third input of the first precoder. The output of the NOR gate is coupled to the second output of the first precoder. Based on this, the first precoder can be equipped with one fewer XOR gate than a traditional precoder, further reducing the wiring area of ​​the convolution circuit.

[0011] In one possible implementation, the convolution circuit further includes a register circuit. The multiple precoders and multiple encoder groups are further coupled to the register circuit. The register circuit can cache parameters required for the precoder and multiple encoder groups during operation.

[0012] In one possible implementation, the memory is used to store computational constants; the computational constants are determined based on weight parameters. Based on this, the corresponding computational constants can be pre-calculated based on the weight parameters and stored in the memory. During convolution calculations, these constants can be directly transmitted to the second adder, thereby reducing the computational complexity of the convolution operation, improving operational efficiency, reducing circuit area, and lowering power consumption.

[0013] In a possible implementation, the above-mentioned calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 on the corresponding digits.

[0014] In the second aspect, a convolution calculation method applied to the convolution circuit in the first aspect is provided. The execution process of the convolution calculation method includes: first, receiving multiple input feature maps and weight parameters and calculation constants preset for each input feature map. The calculation constant is determined according to the weight parameter. Secondly, an input feature map and a weight parameter are transmitted to each multiplier, and the corresponding operation results are obtained by performing operations through multiple precoders, multiple encoder groups and addition tree circuits in the multiplier. Then, the operation results of each multiplier are accumulated through the first adder to obtain the first data. Finally, the calculation constant is transmitted to the second adder, and the first data and the calculation constant are added by the second adder to obtain the output feature map.

[0015] In a possible implementation, the calculation constant may be a result of accumulating the odd-numbered bits of the weight parameter of the input feature map and the constant 1 on the corresponding bits.

[0016] In one possible implementation, multiple precoders, multiple encoder groups, and an adder tree circuit in a multiplier perform operations to obtain corresponding operation results, including: first, each precoder outputs two precoded signals based on two corresponding bits in a weight parameter. Then, each encoder in the encoder group corresponding to each precoder outputs a corresponding partial product based on the two precoded signals and the corresponding odd-numbered bits in the weight parameter. Finally, the adder tree circuit performs a carry-addition operation on the partial products output by the multiple encoder groups to obtain the operation result.

[0017] In a third aspect, a computer-readable storage medium is provided, which stores computer program instructions. When the computer program instructions are executed by a processor, the convolution calculation method of the second aspect described above is implemented.

[0018] In a fourth aspect, a chip is provided, comprising a circuit board and a convolution circuit according to any possible implementation of the first aspect, arranged on the circuit board.

[0019] In a fifth aspect, an electronic device is provided, which includes a memory and the chip in the fourth aspect coupled to the memory.

[0020] In a sixth aspect, a computer program product is provided, which, when executed by a processor, implements the convolution calculation method of the second aspect.

[0021] The technical effects brought about by the second to sixth aspects and possible implementation methods can be found in the description of the technical effects brought about by the first aspect and possible implementation methods mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is a structural block diagram of an electronic device provided in an embodiment of the present application;

[0023] FIG2 is a schematic diagram of a convolution calculation based on matrix multiplication operation;

[0024] FIG3 is a schematic diagram of a convolution operation based on a convolver;

[0025] FIG4 is a schematic diagram of a convolution calculation process;

[0026] FIG5 is a hardware block diagram of a convolver;

[0027] FIG6 is a structural block diagram of a multiplication circuit;

[0028] FIG7 is a schematic diagram of the internal structure of a multiplier;

[0029] FIG8 is a schematic diagram of the structure of a precoder in a multiplier;

[0030] FIG9 is a schematic diagram of an adder tree circuit in a multiplier;

[0031] FIG10 is a schematic diagram of a convolution circuit provided in an embodiment of the present application;

[0032] FIG11 is a schematic diagram of an adder tree circuit provided in an embodiment of the present application;

[0033] FIG12 is a schematic diagram of the structure of a first precoder provided in an embodiment of the present application;

[0034] FIG13 is a schematic diagram of the structure of a second precoder provided in an embodiment of the present application;

[0035] FIG14 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0036] FIG15 is a schematic diagram of the structure of a multiplier provided in an embodiment of the present application;

[0037] FIG16 is a schematic diagram of a flow chart of a convolution method provided in an embodiment of the present application;

[0038] FIG17 is a schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0039] FIG18 is a schematic diagram of the structure of another chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0041] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit differences. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0042] When describing some embodiments, the terms "coupled" and "connected," and their derivatives, may be used. For example, when describing some embodiments, the term "connected" may be used to indicate that two or more components are in direct physical contact or point contact with each other. For another example, when describing some embodiments, the term "coupled" may be used to indicate that two or more components are in direct physical contact or electrical contact, or it may mean that two or more components are not in direct contact with each other but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited to the contents herein.

[0043] The present application is described in detail below with reference to the accompanying drawings and embodiments:

[0044] Currently, AI technology is used in many electronic devices 100. The electronic device 100 can be a terminal, server, or a chip, chipset, circuit board, or module within a terminal or server. As shown in Figure 1, the electronic device 100 may include a memory 110, a processor 120, a communication interface 130, and a bus 140. The memory 110, processor 120, and communication interface 130 are interconnected via bus 140. The memory 110 can be used to store data, software programs, and modules, and primarily includes a program storage area and a data storage area. The program storage area can store the operating system and applications required for at least one function, while the data storage area can store data generated during the use of the device. The processor 120 is used to control and manage the operation of the communication device, for example, by running or executing software programs and / or modules stored in the memory 110 and accessing data stored in the memory 110 to perform various functions of the device and process data. The communication interface 130 is used to support communication with the device.

[0045] The processor 120 includes, but is not limited to, a central processing unit (CPU), a network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a general-purpose processor. The processor 120 may include a convolution circuit 121, which includes one or more multipliers, such as a multiplier array, which is a device that implements multiplication operations in the processor. On the other hand, the convolution circuit 121 may also be provided on a circuit board as an independent chip and perform convolution operations based on the received data.

[0046] The bus 140 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Bus 140 may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, FIG1 shows bus 140 with a single bold line, but this does not imply that there is only one bus or only one type of bus.

[0047] The memory 110 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0048] In the electronic device 100 described above, the core of AI technology is mainly reflected in two aspects: advanced neural network algorithms and processors that can provide massive hardware computing power. Among them, the neural network algorithm requires a large number of convolution calculations. The formula for convolution operation is as follows:

[0049] Among them, F(Chi,m+i,n+j) represents the input feature map; W(Cho,Chi,i,j) represents the weight parameter; F'(Cho,m,n) represents the output feature map; Kw represents the length of the convolution kernel; Kh represents the width of the convolution kernel; C represents the number of input channels; Chi represents the input channel; Cho represents the output channel; m and n represent the coordinates corresponding to the output feature map.

[0050] In order to implement the above-mentioned convolution calculation, the following two hardware solutions are usually adopted: one is to convert the convolution calculation into a matrix operation, and by designing a general-purpose neural network processing unit (NPU), integrate a large number of multiplier matrices to implement matrix multiplication calculation. The other is to design a dedicated convolution device to directly implement the convolution calculation. Among them, when using a general-purpose neural network processor to implement matrix multiplication calculation, it is necessary to first copy the feature map of the C input channels input by Kw*Kh copies through the image to matrix (im2col) transformation, and then divide the input feature map of the C input channels into C*P small matrices, respectively, and perform matrix multiplication operations with the convolution kernel to generate an output feature map matrix. Finally, the Kw*Kh output feature map matrices are added to obtain the convolution operation result corresponding to the input feature map. Among them, P represents the number of matrices that each input feature map can be divided into. For example, taking a 3*3 convolution kernel for matrix multiplication as an example, Kw=3 and Kh=3. When Chi = 16, P = 16, C = 16, m = 0, n = 0 to 16, Cho = 16, the convolution calculation formula of each output feature map F'(Cho, 0, n) is as follows:

[0051] The process of matrix multiplication of the input feature map of each input channel through the convolution kernel is shown in Figure 2, and the corresponding principle is as follows:

[0052] In the above implementation process, because there are multiple im2col conversion processes, data requires more reading and writing time, which increases power consumption.

[0053] On the other hand, the implementation method of convolution calculation through a dedicated convolution unit is shown in Figure 3. Among them, the convolution unit can perform convolution calculation on the input multi-channel feature map F(Chi,m+i,n+j) according to the weight parameters W(Cho,Chi,i,j) stored in the register group, and output the corresponding feature map F'(Cho,m,n). For example, as shown in Figure 4, the convolution unit can use a 3*3 convolution kernel (i.e., Kw=3, Kh=3) to start convolution from the first row and the first column (i.e., m=1, n=1) in the multi-channel input feature map, and then slide one pixel and perform convolution again, ..., and continue convolution to the last row and the last column.

[0054] To perform convolution calculations, as shown in Figure 5 , the convolution circuit within the convolver typically includes multiple multipliers 510 and an adder 520 coupled to the multiple multipliers 510. Each multiplier 510 can independently perform a multiplication operation on a feature map and a weight parameter, and then input the result of the operation into the adder 520 for addition to obtain the final output result. As shown in Figure 6 , the multiplier 510 includes multiple precoders 511, multiple encoder groups 512, and an adder tree circuit 513. First, the multiple precoders 511 can precode the three bits set in the weight parameter corresponding to the input feature map currently undergoing convolution. Then, each encoder group 512 can encode the two encoded data output by the corresponding precoder 511 and the respective bits in the input feature map. After performing a carry addition operation through the adder tree circuit 513, the corresponding operation result is output. This operation result is accumulated with the operation results output by the other multipliers through the adder 520 to obtain the final output feature map. Where W[R-1:0] represents the weight parameter from bit 0 W[0] to bit R-1 W[R-1]. R is usually an even number, representing the total number of bits in the weight parameter. F[Z-1:0] represents the input feature map from bit 0 F[0] to bit Z-1 F[Z-1], where Z is the total number of bits in the input feature map. R and Z can be the same or different.

[0055] As shown in FIG7 , a plurality of feature input lines and a plurality of weight input lines are generally provided in the multiplier 510. The number of feature input lines and the number of encoders in each encoder group are the same as the number of bits in the feature map, and the number of pre-encoders and the number of weight input lines are half the number of feature input lines. For example, if the feature map of int8 includes 8 bits of data, the number of feature input lines is 8, the number of encoders is 8, and the number of pre-encoders and the number of weight input lines are both 4. Each feature input line is used to input one bit in the feature map. In each encoder group, except that the first encoder is only coupled with the feature input line corresponding to the 0th bit F[0], the remaining encoders are coupled with the feature input lines corresponding to two adjacent bits in sequence. For example, the second encoder in the encoder group can be coupled with the feature input line corresponding to the 0th bit F[0] and the feature input line corresponding to the 1st bit F[1] (denoted as F[0] & F[1]). The third encoder in the encoder group can be coupled with the feature input line corresponding to the 1st bit F[1] and the feature input line corresponding to the 2nd bit F[2]. The same applies to the other encoders, which will not be described in detail in this application. In addition, each encoder group is also coupled to a corresponding weight input line. Each precoder can provide two precoded signals (M1S and M2S) to the encoder according to the two bits set in the weight parameter. The two precoded signals output by each precoder in Figure 7 are distinguished by even numbers in the subscripts 0 to (R-2). Furthermore, each encoder in each encoder group operates on the two bit values ​​of the two feature input lines according to the corresponding two precoded signals and the values ​​of the corresponding weight input line input to obtain the corresponding partial product. The partial product is output after the carry addition operation is performed by the adder tree circuit. The input of each weight input line is the odd bit in the weight parameter corresponding to the input feature map. For example, in the weight parameter corresponding to the R-bit feature map, the value of each bit is W[0], W[1], ..., W[R-1] in sequence. Then the values ​​output by each weight input line in the corresponding multiplier are W[1], W[3], W[5], ..., W[R-1] respectively. The input of each feature input line is the value of a bit in the input feature map (one of F[0] to F[Z-1]).

[0056] As shown in FIG8 , among multiple precoders, some precoders include an XOR gate XOR1, an XNOR gate XNOR1, and a NOR gate NOR1. For example, if the value input to the weight input line corresponding to a certain encoder group is the value W[3] of the third bit in the weight parameter, then the value input to the first input of the XNOR gate XNOR1 in the corresponding precoder is W[3]. The value input to the second input of the XNOR gate XNOR1 is W[2]. At the same time, the value input to the first input of the corresponding XOR gate XOR1 is W[1]. The second input of the XOR gate XOR1 is coupled to the second input of the corresponding XNOR gate XNOR1. The first input of the NOR gate NOR1 and the first encoding input of the corresponding encoder group are both coupled to the output of the XOR gate XOR1. The second input of the NOR gate NOR1 is coupled to the output of the XNOR gate XNOR1. The output of the NOR gate NOR1 is coupled to the second encoding input of the corresponding encoder group. Furthermore, the structure of each subsequent precoder is the same as the structure of the precoder corresponding to the encoder group of input W[3], and the embodiments of the present application are not described in detail here.

[0057] In the above convolution circuit, the addition tree circuit can perform addition operations based on the coupling relationship between each encoder and the weight input line and the feature input line. For example, each encoder group can be regarded as a row; each encoder in each encoder group can be regarded as a column. The column where each encoder in each encoder group is located corresponds to a digit (20 to 2) in the partial product output by the addition tree circuit. n ). This digit can also be called a weight. When the addition tree circuit performs an operation, a carry addition operation can be performed on the outputs of the encoders of the same digit. Starting from the second row, the output of the encoder of the kth column (k≥1) in each row and the output of the encoder of the k+2th column of the previous row are binary bits on the same weight. In addition, when performing an addition operation on the outputs of each encoder corresponding to the first column of the first row to the first example of the last row, it is also necessary to add a partial product S related to the weight parameter at the end of each odd column. The partial product S can be the value of the odd bit of the weight parameter. When performing an addition operation on the outputs of each encoder corresponding to the last column of the first row to the last column of the last row, it is also necessary to add a constant 1 to each odd column first.

[0058] To implement the aforementioned addition operation, the adder tree circuit typically employs a multi-layer adder structure, as shown in Figure 9. The inputs to the adder tree circuit include the outputs of the encoder group, the partial product S, and a constant 1. Each layer of the adder tree circuit uses multiple full adders (the small rectangular boxes in Figure 9) to parallelize the addition of equally weighted binary bits (e.g., the columns in Figure 9). Each adder inputs a 3-bit binary number and outputs a sum bit (as shown in the dashed circle in Figure 9) and a carry bit (as shown in the dotted filled circle in Figure 9). The sum bits output by all adders and the remaining bits not added in the current layer are passed to the permutation columns of the same-weighted bits in the second layer of the adder tree, while the carry bits of the adders are passed to the permutation columns of the adjacent higher-weighted bits in the second layer, thus forming a second-layer permutation array. The adders in the second layer perform parallel addition on the binary bits of the same-weighted bits. The sum bits output by all adders and the remaining bits not added in the current layer are passed to the permutation columns of the same-weighted bits in the third layer of the adder tree, while the carry bits of the adders in the second layer are passed to the permutation columns of the adjacent higher-weighted bits in the third layer, and so on. Parallel addition continues until the last layer, and when no more than two bits of data remain in the permutation columns of each weighted bit, the parallel addition is complete.

[0059] As can be seen from the above, although the above multipliers can implement convolution calculations, the adder tree circuits within each multiplier not only need to perform carry addition operations on the outputs of each encoder, but also need to calculate the partial products S and the constant 1. Therefore, a large number of full adders are required, which increases the wiring area and power consumption.

[0060] Considering that the convolution kernel composed of weight parameters remains unchanged throughout the convolution process when performing convolution calculations on the input feature map, this feature can be utilized to simplify the hardware convolution circuit, thereby reducing the area and power consumption of the convolution circuit. Therefore, to address the above-mentioned issues, as shown in FIG10 , an embodiment of the present application provides a convolution circuit 1000, which includes multiple multipliers 1100, a first adder 1200, and a second adder 1300. The first adder 1200 is coupled to the multiple multipliers 1100. The second adder 1300 is coupled to the first adder 1200. Each multiplier 1100 includes multiple precoders, multiple encoder groups, and an adder tree circuit. The multiple precoders are coupled to the multiple encoder groups in a one-to-one correspondence. The adder tree circuit includes multiple input lines and multiple output lines. The multiple encoder groups include multiple output terminals. The number of output terminals of the multiple encoder groups is equal to the number of input lines, and the multiple output terminals are coupled to the multiple input lines in a one-to-one correspondence. The adder tree circuit is coupled to the first adder via a plurality of output lines. The second adder is further coupled to a memory. The memory is used to store a computation constant. The computation constant can be determined based on a weight parameter. For example, the computation constant can be the result of accumulating the odd-numbered bits of the weight parameter of the input feature map and a constant 1 at the corresponding bits.

[0061] When performing a convolution operation using the convolution circuit 1000, the adder tree circuit can perform an addition operation on the data output by multiple encoder groups. The first adder 1200 can add the outputs of each multiplier and transmit the result to the second adder 1300. Furthermore, the partial product S associated with the weight parameter of the input feature map and the constant 1 can be added together and transmitted to the second adder 1300 as a calculation constant Wconst. The second adder 1300 adds the calculation constant to the output of the first adder 1200 to obtain the corresponding output feature map. Based on this, the amount of computation of the adder tree circuit can be reduced, thereby reducing the number of full adders used by the adder tree circuit and reducing the power consumption and wiring area of ​​the convolution circuit 1000.

[0062] In the convolution circuit 1000, the number of multipliers can be adjusted according to the size of the convolution kernel. For example, if the size of the convolution kernel is 3*3, the number of multipliers is 9, and each multiplier can calculate one of the feature maps corresponding to the 9 pixels within the convolution range. If the size of the convolution kernel is 4*4, the number of multipliers is 16, and each multiplier can calculate one of the feature maps corresponding to the 16 pixels within the convolution range.

[0063] In one possible implementation, the structure of the adder tree circuit provided in an embodiment of the present application is shown in FIG11 . The adder tree circuit also uses multiple full adders (the small rectangular boxes in FIG11 ) to perform parallel addition of the binary bits of equal weight output by the encoder group (such as the columns in FIG11 ). However, the input of the adder tree circuit provided in an embodiment of the present application only includes the output of the encoder group. When performing the addition operation, each adder also inputs a 3-bit binary number and outputs one sum bit (such as the dotted circle in FIG11 ) and one carry bit (such as the dotted filled circle in FIG11 ). The summed bits output by all adders and the remaining bits not added in the current layer are passed to the columns of equally weighted bits in the second layer of the adder tree. The carry bits from the adders are passed to the columns of adjacent, higher-weighted bits in the second layer, thus forming a second-layer permutation array. The adders in the second layer perform parallel addition on the binary bits in equally weighted bits. The summed bits output by all adders and the remaining bits not added in the current layer are passed to the columns of equally weighted bits in the third layer of the adder tree. The carry bits from the adders in the second layer are passed to the columns of adjacent, higher-weighted bits in the third layer, and so on. Parallel addition continues until the last layer, when no more than two bits of data remain in each column of weighted bits. Because the input of the adder tree circuit only includes the output of the encoder group, the number of full adders used in each layer is fewer than in traditional adder tree circuits, thereby reducing the wiring area and power consumption of the adder tree circuit.

[0064] Furthermore, in one embodiment, the aforementioned multiple multipliers may share a common adder tree circuit. The number of input lines in the adder tree circuit is the sum of the number of encoders in all multipliers. Referring to FIG11 , in the adder tree circuit, the binary bits with the same weights output by the encoder groups in each multiplier may be added in parallel, and adders may be arranged layer by layer in the manner described in FIG11 .

[0065] In the above manner, the outputs of multiple multipliers can be directly added without independently setting an addition tree circuit in each multiplier and without setting a first adder, thereby further improving the efficiency of the operation.

[0066] In one possible implementation, the plurality of precoders mentioned above include a first precoder. The plurality of encoder groups mentioned above include a first encoder group. The first precoder can be any precoder other than precoder 0 in FIG. 7 . The first precoder includes three input terminals, a first logic circuit, and two output terminals. The first encoder group includes a first input terminal and a second input terminal. The first input terminal of the first encoder group is coupled to the first input terminal of the three input terminals via the first output terminal of the two output terminals. The three input terminals are also coupled to the second output terminal of the two output terminals via the first logic circuit. The second input terminal of the first encoder group is coupled to the second output terminal of the two output terminals. The first logic circuit is configured to perform an XOR operation on the signals input from the second input terminal and the third input terminal of the three input terminals, perform an NOR operation on the signals input from the first input terminal of the three input terminals, and output the result of the operation from the second output terminal of the two output terminals.

[0067] In the above manner, when the multiplier operates on the input feature map, the first precoder in each multiplier only needs to perform XNOR and NOR operations on the weight parameters input from the three input terminals, thereby reducing the power consumption of the operation.

[0068] Exemplarily, as shown in FIG12 , the first logic circuit A in the first precoder includes an XNOR gate XNOR1 and a NOR gate NOR1. A first input of the NOR gate NOR1 is coupled to the output of the XNOR gate XNOR1. A second input of the NOR gate NOR1 is coupled to the first input IN1 of the first precoder. A first input of the XNOR gate XNOR1 is coupled to the second input IN2 of the first precoder. A second input of the XNOR gate XNOR1 is coupled to the third input IN3 of the first precoder. The first input IN1 of the first precoder is coupled to the first output OUT1 of the first precoder. The output of the NOR gate NOR1 is coupled to the second output OUT2 of the first precoder.

[0069] With the above structure, the first precoder in the embodiment of the present application can reduce one XOR gate compared to the conventional precoder in Figure 8. Convolution circuit 1000 typically includes multiple multipliers, so using the above first precoder can reduce both the amount of logical operations and the wiring area.

[0070] Furthermore, the above-mentioned multiple precoders also include a second precoder. The above-mentioned multiple encoder groups also include a second encoder group. The second precoder can be used as precoder 0 in Figure 7. The second precoder includes two input terminals, a second logic circuit and two output terminals; the second encoder group includes a first input terminal and a second input terminal. The first input terminal of the two input terminals of the second precoder is coupled to the first input terminal of the second encoder group through the first output terminal of the two output terminals. The two input terminals are also coupled to the second output terminal of the two output terminals through the second logic circuit. The second logic circuit can invert the signal input from the first input terminal of the two input terminals, perform an AND logic operation with the signal input from the second input terminal, and output it from the second output terminal of the two output terminals.

[0071] In one possible embodiment, as shown in FIG13 , the second precoder includes a first AND gate AND1 and a NOT gate NOT1. The input of the NOT gate NOT1 is coupled to the first input In1 of the second precoder. The output of the NOT gate NOT1 is coupled to the first input of the first AND gate AND1. The second input of the first AND gate AND1 is coupled to the second input In2 of the second encoder. The first output out1 of the second precoder is coupled to the first input in1 of the second precoder. The output of the first AND gate AND1 is coupled to the second output out2 of the second precoder. The value of the 0th bit in the weight parameter corresponding to the feature map can be input to the input of the NOT gate NOT1. The value of the 1st bit in the weight parameter corresponding to the feature map can be input to the second input of the first AND gate AND1.

[0072] In another embodiment, the NOT gate NOT1 in the second precoder can be optionally arranged between the first input and the first output of the second precoder, and the first input of the first AND gate AND1 can be directly coupled to the first input of the second precoder. In this case, the input of the NOT gate NOT1 can input the inverted value of bit 0 in the weight parameter corresponding to the feature map.

[0073] In one embodiment, the first and second encoder groups each include multiple encoders. As shown in FIG14 , each encoder includes a second AND gate AND2, a third AND gate AND3, a first OR gate OR1, and a first XOR gate XOR2. The first input of the second AND gate AND2 is used to input the first encoded data M1S output by the corresponding precoder. The second input of the second AND gate AND2 is used to input the value F[a] of the a-th bit in the feature map. The first input of the third AND gate AND3 is used to input the second encoded data M2S output by the corresponding precoder. The second input of the third AND gate AND3 is input with the value F[a-1] of the a-1-th bit in the input feature map. Here, a is an integer not less than 1. Furthermore, if a=0, the second inputs of the second AND gate AND2 and the third AND gate AND3 are both input with the value F[0] of the 0th bit in the feature map. The first input of the first OR gate OR1 is coupled to the output of the second AND gate AND2; the second input of the first OR gate OR1 is coupled to the output of the third AND gate AND3. The first input of the first XOR gate XOR2 is coupled to the output of the first OR gate OR1. The second input terminal of the first XOR gate XOR2 is used to input the value WC[i] of the odd bit in the weight parameter, where i is an odd number between 0 and R.

[0074] In the above implementation process, the data and weight parameters of each bit in the feature map can be transmitted to the corresponding device through the register circuit. The register circuit can be coupled to the memory, and the register circuit includes a plurality of registers. Each weight parameter and the value of each bit in the input feature map can be cached through an independent register. In addition, when the above-mentioned first precoder and the second precoder need to input the same weight parameter, they can be coupled to the same register. When the encoders in the above-mentioned encoder groups need to input the value of the same bit in the feature map, they can also be coupled to the same register. Based on this, while ensuring that the multipliers operate in an orderly manner, the number of devices can be reduced and the wiring area can be reduced.

[0075] Furthermore, when the above-mentioned first precoder and a traditional precoder are used in the multiplier, the number of the first precoders can be set according to actual needs.

[0076] For example, in one embodiment, as shown in FIG15 , a multiplier may include K-1 first precoders and 1 second precoder, where K represents the total number of precoders in the multiplier. Each first precoder and second precoder is coupled to a corresponding encoder group. The first input of the second precoder is used to input the value WC[0] of the 0th bit in the weight parameter. The second input of the second precoder is used to input the value WC[1] of the 1st bit in the weight parameter. The third input of each first precoder can input the value of the corresponding odd bit after WC[1] in the weight parameter. The value input by each weight input line is the same as the value input by the third input of the corresponding first precoder. The second input of the first precoder inputs the value of the first bit before the corresponding odd bit. The second input of the first precoder inputs the value of the second bit before the corresponding odd bit. For example, if the third input terminal of a first precoder inputs the value WC[3] of the third bit in the weight parameter, then the second input terminal of the first precoder inputs the value WC[2] of the second bit in the weight parameter, and the first input terminal of the first precoder inputs the value WC[1] of the first bit in the weight parameter. The conversion relationship between the values ​​of each bit in the weight parameter of the multiplier provided in the embodiment of the present application and the values ​​of each bit in the weight parameter of the traditional multiplier is as follows:

[0077] Among them, XNOR represents the exclusive-or operation; the transformation relationship between the values ​​of each bit in the weight parameter is shown in Table 1:

[0078] Table 1

[0079] In Table 1, W[i] represents the value of each bit in the weight parameter of the traditional multiplier. WC[i] represents the value of each bit in the weight parameter of the multiplier provided in the embodiment of the present application. Among them, WC[i] can be pre-calculated according to the transformation relationship in Table 1 and stored in the memory. When a convolution operation is required, WC[i] can be cached by a register coupled to the memory and then transmitted to the traditional precoder and each first precoder.

[0080] In the above implementation process, the number of first precoders used in the multiplier can be adjusted according to actual needs and is not limited to the number in the above implementation scheme.

[0081] In one embodiment, as shown in FIG16 , the present application also provides a convolution calculation method applied to the above-mentioned convolution circuit. The execution process of the convolution calculation method is as follows:

[0082] S161. Receive multiple input feature maps and preset weight parameters and calculation constants for each input feature map.

[0083] Among them, the above-mentioned calculation constant is the constant obtained by adding the odd bits and the constant 1 in the weight parameters of the input feature map. The weight parameters corresponding to each input feature map include the values ​​of each bit in each feature map. Because when performing convolution calculations, the odd bits of the weight parameters and the constant 1 are pre-configured known parameters, and the data finally output by the convolver also needs to be added with these two fixed parameters, the two can be added as pre-configured calculation constants, and there is no need to calculate through the addition tree circuit. Each input feature map, weight parameter and calculation constant can be stored in a memory. The preset weight parameters and calculation constants of each input feature map can be pre-stored in a memory. When convolution calculations are required, the values ​​of each bit of the weight parameter, the values ​​of each bit in the input feature map and the calculation constant can be read into the corresponding registers in the register circuit.

[0084] S162. Transmit an input feature map and a weight parameter to each multiplier, and perform operations through multiple precoders, multiple encoder groups, and adder tree circuits in the multiplier to obtain corresponding operation results.

[0085] Among them, the value of the pixel point at a certain position in the input feature map of each multiplier in each convolution process. When executing the above steps, the number of bits of the weight parameter can be the same as the number of bits of the input feature map. For example, if the input feature map is int8 data. Then, when performing the convolution operation, the weight parameter input by each multiplier can include an 8-bit value. Each precoder can first output two precoded signals according to the values ​​of the corresponding two bits in the weight parameter. Then, the encoder group corresponding to each precoder outputs multiple partial products based on the two precoded signals and the values ​​of the corresponding odd bits in the weight parameter. Finally, the adder tree circuit performs a carry addition operation on the partial products output by multiple encoder groups to obtain the above operation results.

[0086] In the above implementation process, the number of bits of the weight parameter and the number of bits of the input feature map may also be different. The specific situation can be set according to the actual convolution operation requirements.

[0087] S163. Accumulate the operation results of each multiplier through the first adder to obtain first data.

[0088] S164. Transmit the above-mentioned calculation constant to the second adder, and add the above-mentioned first data and the calculation constant through the second adder to obtain an output feature map.

[0089] During the convolution calculation, the constant is the sum of the odd-numbered bits of the input feature map weight parameter and the constant 1. This constant can be cached in a register to ensure more orderly execution of the corresponding calculation process in a high-speed computing environment.

[0090] In one possible implementation, an embodiment of the present application further provides a computer-readable storage medium, which can store computer program instructions. When the computer program instructions are executed by a processor, the above-mentioned convolution calculation method can be implemented. Among them, the above-mentioned processor can be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor (MCU), a microcontroller, a programmable logic device (PLD) or any combination thereof. The processor can also be other devices with processing functions, such as circuits, devices or software modules, which are not limited by this application. The computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, a USB flash drive and an optical data storage device.

[0091] In a possible implementation, an embodiment of the present application further provides a chip, which includes a circuit board and the above-mentioned convolution circuit 1000 arranged on the circuit board. The above-mentioned circuit board may be a printed circuit board (PCB) or a substrate (including but not limited to a silicon substrate). In one example, as shown in FIG17 , the chip can be sold or used as an independent convolution chip 1700. An interface 1711 coupled to the convolution circuit 1000 can be provided on the circuit board 1710. Through the interface 1711, the convolution circuit 1000 can receive an input feature map sent by an external device (such as a processor) and output a processed output feature map to the external device. Of course, the convolution chip can also be set on a circuit board together with one or more processors for sale or use as a chip system. The embodiment of the present application does not impose specific restrictions on this.

[0092] In another example, as shown in FIG18 , the chip may also be a processor 1800. The processor 1800 includes a control unit 1811 and a convolution circuit 1000 provided on a circuit board 1810. The convolution circuit 1000 may be coupled to the control unit 1811, and the control unit 1811 may receive an input feature map via a bus 1812. The input feature map is then sent to the convolution circuit 1000 for convolution operation to obtain a corresponding output feature map. Finally, the output feature map is then sent to the next-level processing unit in the processor 1800 via the control unit 1811 or to other devices via the bus 1812 for processing. Of course, the convolution circuit 1000 may also be coupled to the control unit 1811 via the bus 1812. This embodiment of the present application does not impose any specific restrictions on this.

[0093] In the above implementation process, the processor can be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a system on chip (SoC), or any combination thereof. The embodiments of the present application do not impose specific restrictions on this.

[0094] Furthermore, in the above implementation process, the chips in the above two examples may also include other types of devices, and the embodiments of the present application do not impose specific limitations on this.

[0095] Those skilled in the art will appreciate that the functions of the circuits described in the various examples of the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed circuits and electronic devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0097] In addition, the chips in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0098] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A convolution circuit, characterized in that: comprising: a plurality of multipliers; a first adder coupled to the plurality of multipliers; and a second adder coupled to the first adder; The multiplier includes a plurality of precoders, a plurality of encoder groups, and an adder tree circuit; the plurality of precoders are coupled to the plurality of encoder groups in a one-to-one correspondence; The adder tree circuit includes a plurality of input lines and a plurality of output lines; the plurality of encoder groups include a plurality of output terminals; the number of the plurality of output terminals is the same as the number of the plurality of input lines, and the plurality of output terminals are coupled to the plurality of input lines in a one-to-one correspondence; the adder tree circuit is coupled to the first adder through the plurality of output lines; The second adder includes a plurality of input terminals and an output terminal, one of the plurality of input terminals is used to couple with a memory, and the other input terminals of the plurality of input terminals are used to couple with the first adder.

2. The convolution circuit according to claim 1, characterized in that The plurality of precoders include a first precoder; the plurality of encoder groups include a first encoder group; The first precoder includes three input terminals, a first logic circuit and two output terminals; the first encoder group includes a first input terminal and a second input terminal; the first input terminal of the first encoder group is coupled to the first input terminal of the three input terminals through the first output terminal of the two output terminals; the three input terminals are also coupled to the second output terminal of the two output terminals through the first logic circuit; the second input terminal of the first encoder group is coupled to the second output terminal of the two output terminals.

3. The convolution circuit according to claim 2, characterized in that: The first logic circuit is used to perform an XOR operation on the signals input from the second input terminal and the third input terminal among the three input terminals, and then perform an NOR operation on the signal input from the first input terminal among the three input terminals, and output the operation result from the second output terminal among the two output terminals.

4. The convolution circuit according to claim 2 or 3, characterized in that: The first logic circuit includes an XNOR gate and a NOR gate; The first input terminal of the NOR gate is coupled to the output terminal of the XNOR gate; the second input terminal of the NOR gate is coupled to the first input terminal of the first precoder; The first input terminal of the XOR gate is coupled to the second input terminal of the first precoder; the second input terminal of the XOR gate is coupled to the third input terminal of the first precoder; An output terminal of the NOR gate is coupled to a second output terminal of the first precoder.

5. The convolution circuit according to any one of claims 1 to 3, characterized in that: A register circuit is also included; the plurality of precoders and the plurality of encoder groups are also coupled to the register circuit.

6. The convolution circuit according to any one of claims 1 to 5, characterized in that: The memory is used to store calculation constants; the calculation constants are determined according to weight parameters.

7. The convolution circuit according to claim 6, characterized in that: The calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 at the corresponding digits.

8. A convolution calculation method, applied to a convolution circuit; the convolution circuit includes a plurality of multipliers; a first adder coupled to the plurality of multipliers; and a second adder coupled to the first adder; the multipliers include a plurality of precoders, a plurality of encoder groups and an adder tree circuit; the plurality of precoders are coupled to the plurality of encoder groups in a one-to-one correspondence; the adder tree circuit includes a plurality of input lines and a plurality of output lines; the plurality of encoder groups include a plurality of output terminals; the number of the plurality of output terminals is the same as the number of the plurality of input lines, and the plurality of output terminals are coupled to the plurality of input lines in a one-to-one correspondence; the adder tree circuit is coupled to the first adder through the plurality of output lines; the second adder includes a plurality of input terminals and an output terminal, and one of the plurality of input terminals is used to couple with a memory, characterized in that The method comprises: Receiving a plurality of input feature maps and weight parameters and calculation constants preset for each of the input feature maps; the calculation constants are determined according to the weight parameters; Transmitting one of the input feature maps and one of the weight parameters to each of the multipliers, and performing operations through the multiple precoders, the multiple encoder groups, and the adder tree circuit in the multiplier to obtain corresponding operation results; Accumulating the operation results of each of the multipliers through the first adder to obtain first data; The calculation constant is transmitted to the second adder, and the first data and the calculation constant are added by the second adder to obtain an output feature map.

9. The method according to claim 8, characterized in that The calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 at the corresponding digits.

10. The method according to claim 8, characterized in that The method further comprises: performing operations on the plurality of precoders, the plurality of encoder groups and the addition tree circuit in the multiplier to obtain corresponding operation results, including: Each of the precoders outputs two precoded signals according to two corresponding bits in the weight parameter; The encoder groups corresponding to the respective precoders output a plurality of partial products according to the two precoded signals and the corresponding odd-numbered bits in the weight parameters; The addition tree circuit performs a carry addition operation on the plurality of partial products to obtain the operation result.

11. A computer-readable storage medium, characterized in that: Computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described in any one of claims 8 to 10 is implemented.

12. A chip, characterized in that: It comprises a circuit board and a convolution circuit as claimed in any one of claims 1 to 7 arranged on the circuit board.

13. An electronic device, characterized in that: The invention comprises a memory, and the chip as claimed in claim 12 coupled to the memory.

Images