Convolution circuit, convolution calculation method, chip and electronic equipment

Abstract

The embodiment of the invention provides a convolution circuit, a convolution calculation method, a chip and electronic equipment, is applied to the technical field of chips, and aims to solve the problems of high power consumption and large wiring area of a convolution circuit in an existing convolver. The convolution circuit comprises a plurality of multipliers; a first adder coupled to the plurality of multipliers; and a second adder coupled with the first adder. Each multiplier comprises a plurality of precoders, a plurality of encoder groups and an adder tree circuit. Wherein each precoder is in one-to-one correspondence with one coder group. The number of the output ends of the encoder groups is the same as the number of the input lines of the adder tree circuit, and the output ends are in one-to-one correspondence with the input lines. In addition, an adder tree circuit is coupled with the first adder. The second adder is also coupled to the memory. In the multiplier, only a partial product related to a weight parameter and a constant 1 can be accumulated firstly and then added with results output by each adder tree circuit in the second adder, so that the area and the power consumption of the adder tree circuit are reduced.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to a convolution circuit, a convolution calculation method, a chip and an electronic device. Background Art

[0002] With the development of artificial intelligence (AI) technology, more and more AI products are entering people's daily lives. In the implementation process of AI technology, its core mainly includes two aspects: 1. Advanced neural network algorithms; 2. Processors that provide massive hardware computing power. Among them, the calculation of neural network algorithms is mainly convolution calculation type. Traditionally, there are two hardware solutions to implement convolution calculation: (1) Convert convolution calculation into matrix operation, and integrate a large number of multiplier matrices by designing a general neural network processing unit (NPU) to implement matrix multiplication calculation. (2) Design a dedicated convolution device to directly implement convolution calculation. Among them, the first solution is the current mainstream solution. However, this solution requires a large number of repeated read and write operations on the data, which increases power consumption. The convolution device in the second solution is composed of multiple multipliers and adders. After multiple multipliers independently multiply the input feature map, they are accumulated through the adder to obtain the output feature map. However, the structure of each multiplier is relatively complex, which increases the power consumption and wiring area of ​​the convolution device. Summary of the invention

[0003] The embodiments of the present application provide a convolution circuit, a convolution calculation method, a chip and an electronic device to solve the problems of high power consumption and large wiring area in the current convolution hardware circuit.

[0004] In order to solve the above problems, the embodiments of the present application provide the following technical solutions:

[0005] In a first aspect, a convolution circuit is provided, which includes: a plurality of multipliers, a first adder and a second adder. The multipliers include a plurality of precoders, a plurality of encoder groups and an adder tree circuit. The plurality of precoders are coupled to the plurality of encoder groups in a one-to-one correspondence. The adder tree circuit includes a plurality of input lines and a plurality of output lines. The plurality of encoder groups include a plurality of output terminals. The number of the plurality of output terminals is the same as the number of the plurality of input lines of the adder tree circuit, and the plurality of output terminals of the plurality of encoder groups are coupled to the plurality of input lines of the adder tree circuit in a one-to-one correspondence; the adder tree circuit is coupled to the first adder through the plurality of output lines. The second adder includes a plurality of input terminals and an output terminal. One of the plurality of input terminals is used to couple with a memory, and the other input terminals of the plurality of input terminals are used to couple with the first adder. When performing convolution operations, the adder tree circuit only needs to operate on the data output by multiple encoder groups, and does not accumulate the partial products and constant 1 directly from the input related to the odd bits of the weight parameters in the multiplier, thereby reducing the number of full adders used in the adder tree circuit, thereby reducing the power consumption and wiring area of ​​the entire convolution circuit.

[0006] In a possible implementation, the above-mentioned multiple precoders include a first precoder. The above-mentioned multiple encoder groups include a first encoder group. Among them, the first precoder includes three input terminals, a first logic circuit and two output terminals. The first encoder group includes a first input terminal and a second input terminal. The first input terminal of the first encoder group is coupled to the first input terminal of the first precoder through the first output terminal of the first precoder. The three input terminals are also coupled to the second output terminal of the two output terminals through the first logic circuit. The second input terminal of the first encoder group is coupled to the second output terminal of the two output terminals. When performing convolution, the first encoder group will also input the corresponding odd bits in the weight parameter. The two input terminals of the first precoder input the first two bits of the odd bit. Based on this, the first input terminal of the first encoder group can directly input the second bit before the odd bit, and the second input terminal of the first encoder group inputs the value calculated by the first logic circuit according to the first two bits of the odd bit. Compared with the precoder in the traditional multiplier, one logical operation can be reduced, thereby reducing power consumption and improving operation efficiency.

[0007] In a possible implementation, the first logic circuit can perform an XOR operation on the signals inputted from the second input terminal and the third input terminal among the three input terminals, and then perform an XOR operation on the signals inputted from the first input terminal among the three input terminals, and output the operation result from the second output terminal among the two output terminals. Based on this, compared with the traditional precoder, the first precoder can perform one less XOR operation, further reducing the power consumption of the operation.

[0008] In a possible implementation, the first logic circuit includes an XOR gate and a NOR gate. The first input of the NOR gate is coupled to the output of the XOR gate. The second input of the NOR gate is coupled to the first input of the first precoder. The first input of the XOR gate is coupled to the second input of the first precoder. The second input of the XOR gate is coupled to the third input of the first precoder. The output of the NOR gate is coupled to the second output of the first precoder. Based on this, compared with the traditional precoder, the first precoder can be provided with one less XOR gate, further reducing the wiring area of ​​the convolution circuit.

[0009] In a possible implementation, the convolution circuit further includes a register circuit. The multiple precoders and multiple encoder groups are also coupled to the register circuit. The register circuit can cache parameters required during the operation of the precoders and multiple encoder groups.

[0010] In a possible implementation, the memory is used to store calculation constants; the calculation constants are determined according to weight parameters. Based on this, the corresponding calculation constants can be calculated in advance according to the weight parameters and stored in the memory. When performing convolution calculations, they can be directly transmitted to the second adder, thereby reducing the amount of calculation of the convolution operation, improving the operation efficiency, reducing the circuit area, and reducing power consumption.

[0011] In a possible implementation, the above calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 at the corresponding bits.

[0012] In the second aspect, a convolution calculation method applied to the convolution circuit in the first aspect is provided. The execution process of the convolution calculation method includes: first, receiving multiple input feature maps and weight parameters and calculation constants preset for each input feature map. Among them, the calculation constant is determined according to the weight parameter. Secondly, an input feature map and a weight parameter are transmitted to each multiplier, and the corresponding operation results are obtained by performing operations through multiple precoders, multiple encoder groups and addition tree circuits in the multiplier. Then, the operation results of each multiplier are accumulated through the first adder to obtain the first data. Finally, the calculation constant is transmitted to the second adder, and the first data and the calculation constant are added by the second adder to obtain the output feature map.

[0013] In a possible implementation, the above calculation constant may be the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 at the corresponding bits.

[0014] In a possible implementation, a plurality of precoders, a plurality of encoder groups, and an addition tree circuit in a multiplier perform operations to obtain corresponding operation results, including: first, each precoder outputs two precoded signals according to two corresponding bits in a weight parameter. Then, each encoder in an encoder group corresponding to each precoder outputs a corresponding partial product according to the two precoded signals and the corresponding odd bits in the weight parameter. Finally, the addition tree circuit performs a carry addition operation on the partial products output by the plurality of encoder groups to obtain an operation result.

[0015] In a third aspect, a computer-readable storage medium is provided, which stores computer program instructions. When the computer program instructions are executed by a processor, the convolution calculation method of the second aspect is implemented.

[0016] In a fourth aspect, a chip is provided, comprising a circuit board and a convolution circuit in any possible implementation of the first aspect arranged on the circuit board.

[0017] In a fifth aspect, an electronic device is provided, which includes a memory, and the chip in the fourth aspect coupled to the memory.

[0018] In a sixth aspect, a computer program product is provided, which, when executed by a processor, implements the convolution calculation method of the second aspect.

[0019] The technical effects brought about by the second to sixth aspects and possible implementation methods can be found in the description of the technical effects brought about by the first aspect and possible implementation methods mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present application;

[0021] Figure 2 It is a schematic diagram of a convolution calculation based on matrix multiplication operation;

[0022] Figure 3 It is a schematic diagram of a convolution operation based on a convolution device;

[0023] Figure 4 A schematic diagram of a convolution calculation process;

[0024] Figure 5 It is a hardware structure block diagram of a convolution device;

[0025] Figure 6 It is a structural block diagram of a multiplication circuit;

[0026] Figure 7is a schematic diagram of the internal structure of a multiplier;

[0027] Figure 8 A schematic diagram of the structure of a precoder in a multiplier;

[0028] Fig. 9 It is a schematic diagram of an addition tree circuit in a multiplier;

[0029] Fig.10 A schematic diagram of a convolution circuit provided in an embodiment of the present application;

[0030] Fig.11 A schematic diagram of an addition tree circuit provided in an embodiment of the present application;

[0031] Fig.12 A schematic diagram of the structure of a first precoder provided in an embodiment of the present application;

[0032] Fig.13 A schematic diagram of the structure of a second precoder provided in an embodiment of the present application;

[0033] Fig.14 A schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0034] Fig.15 A schematic diagram of the structure of a multiplier provided in an embodiment of the present application;

[0035] Fig.16 A schematic diagram of a convolution method provided in an embodiment of the present application;

[0036] Fig.17 A schematic diagram of the structure of a chip provided in an embodiment of the present application;

[0037] Fig.18 A schematic diagram of the structure of another chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0039] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the differences. At the same time, in the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0040] When describing some embodiments, the expressions "coupled" and "connected" and their derivatives may be used. For example, when describing some embodiments, the term "connected" may be used to indicate that two or more components are in direct physical contact or point contact with each other. For another example, when describing some embodiments, the term "coupled" may be used to indicate that two or more components are in direct physical contact or electrical contact with each other, or it may mean that two or more components are not in direct contact with each other, but still cooperate or interact with each other. The embodiments disclosed herein are not necessarily limited to the contents of this document.

[0041] The present application is described in detail below with reference to the accompanying drawings and embodiments:

[0042] At present, AI technology is used in many electronic devices 100. The electronic device 100 may be a terminal or a server, or may be a chip, a chipset, a circuit board or a module in a terminal or a server. Figure 1 As shown, the electronic device 100 may include a memory 110, a processor 120, a communication interface 130 and a bus 140. The memory 110, the processor 120 and the communication interface 130 are interconnected via the bus 140. The memory 110 may be used to store data, software programs and modules, and mainly includes a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function, etc., and the data storage area may store data created when the device is used, etc. The processor 120 is used to control and manage the actions of the communication device, such as executing various functions of the device and processing data by running or executing software programs and / or modules stored in the memory 110, and calling data stored in the memory 110. The communication interface 130 is used to support the device to communicate.

[0043] The processor 120 includes, but is not limited to, a central processing unit (CPU), a network processing unit (NPU), a graphics processing unit (GPU), a digital signal processor (DSP), or a general-purpose processor. The processor 120 may include a convolution circuit, which includes one or more multipliers, such as a multiplier array, which is a device for implementing multiplication operations in the processor. On the other hand, the convolution circuit may also be arranged on a circuit board as an independent chip, and perform convolution operations according to received data.

[0044] The bus 140 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus 140 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 1 Although only one thick line is used to represent the bus 140, this does not mean that there is only one bus or only one type of bus.

[0045] The memory 110 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0046] In the above electronic device 100, the core of AI technology is mainly reflected in two aspects: advanced neural network algorithm and processor that can provide massive hardware computing power. Among them, the neural network algorithm needs to perform a large number of convolution calculations, and the formula of convolution operation is as follows:

[0047]

[0048] Among them, F(Chi,m+i,n+j) represents the input feature map; W(Cho,Chi,i,j) represents the weight parameter; F'(Cho,m,n) represents the output feature map; Kw represents the length of the convolution kernel; Kh represents the width of the convolution kernel; C represents the number of input channels; Chi represents the input channel; Cho represents the output channel; m and n represent the coordinates corresponding to the output feature map.

[0049] In order to realize the above convolution calculation, the following two hardware solutions are usually adopted: one is to convert the convolution calculation into matrix operation, and integrate a large number of multiplier matrices by designing a general neural network processor (neural-network processing unit, NPU) to realize matrix multiplication calculation. The other is to design a dedicated convolution device to directly realize convolution calculation. Among them, when using a general neural network processor to realize matrix multiplication calculation, it is necessary to first copy Kw*Kh copies of the feature map (featuremap) input by C input channels through the image to matrix (imagetocolumn, referred to as im2col), and then divide the input feature map of C input channels into C*P small matrices, respectively, and perform matrix multiplication operations with the convolution kernel (kernel) to generate an output feature map matrix, and finally add the Kw*Kh output feature map matrices to obtain the convolution operation result corresponding to the input feature map. Among them, P represents the number of matrices that can be divided for each input feature map. Exemplarily, taking the matrix multiplication operation of a 3*3 convolution kernel as an example, Kw=3 and Kh=3 at this time. When Chi = 16, P = 16, C = 16, m = 0, n = 0 to 16, Cho = 16, the convolution calculation formula of each output feature map F'(Cho, 0, n) is as follows:

[0050]

[0051] The process of matrix multiplication of the input feature map of each input channel through the convolution kernel is as follows: Figure 2 The corresponding principles are as follows:

[0052]

[0053] In the above implementation process, because there are multiple im2col conversion processes, the data needs to consume more reading and writing time, which increases power consumption.

[0054] On the other hand, the convolution operation is realized by a dedicated convolution unit. Figure 3 As shown. The convolver can perform convolution calculation on the input multi-channel feature map F(Chi,m+i,n+j) according to the weight parameter W(Cho,Chi,i,j) stored in the register group, and output the corresponding feature map F'(Cho,m,n). For example, Figure 4 As shown, the convolution unit can use a 3*3 convolution kernel (i.e., Kw=3, Kh=3) to start convolution from the first row and the first column (i.e., m=1, n=1) of the multi-channel input feature map, and then slide one pixel and perform convolution again, ..., and continue convolution to the last row and the last column.

[0055] In order to realize the convolution calculation, such as Figure 5As shown, the convolution circuit inside the convolver usually includes multiple multipliers 510 and adders 520 coupled to the multiple multipliers 510. Each multiplier 510 can independently perform a multiplication operation of a feature map and a weight parameter, and then input its operation results into the adder 520 for addition to obtain the final output result. Figure 6 As shown, the multiplier 510 includes a plurality of precoders 511, a plurality of encoder groups 512 and an addition tree circuit 513. First, the three bits set in the weight parameter corresponding to the input feature map currently undergoing convolution can be precoded by the plurality of precoders 511. Then, each encoder group 512 can encode according to the two coded data output by the corresponding precoder 511 and each bit in the input feature map, and output the corresponding operation result after performing a carry addition operation by the addition tree circuit 513. The operation result can be accumulated by the adder 520 with the operation results output by other multipliers to obtain the final output feature map. Among them, W[R-1:0] represents the 0th bit W[0] in the weight parameter to the R-1th bit W[R-1] in the weight parameter. R is usually an even number, which represents the total number of bits of the weight parameter. F[Z-1:0] represents the 0th bit F[0] to the Z-1th bit F[Z-1] in the input feature map, and Z is the total number of bits of the input feature map. R and Z may be the same or different.

[0056] like Figure 7 As shown, a plurality of feature input lines and a plurality of weight input lines are usually provided in the multiplier 510. The number of feature input lines and the number of encoders in each encoder group are the same as the number of bits of the feature map, and the number of pre-encoders and the number of weight input lines are half the number of feature input lines. For example, if the feature map of int8 includes 8 bits of data, the number of feature input lines is 8, the number of encoders is 8, and the number of pre-encoders and the number of weight input lines are both 4. Each feature input line is used to input a bit in the feature map. In each encoder group, except that the first encoder is only coupled with the feature input line corresponding to the 0th bit F[0], the remaining encoders are coupled with the feature input lines corresponding to two adjacent bits in sequence. For example, the second encoder in the encoder group can be coupled with the feature input line corresponding to the 0th bit F[0] and the feature input line corresponding to the 1st bit F[1] (represented as F[0]&F[1]). The third encoder in the encoder group can be coupled with the feature input line corresponding to the 1st bit F[1] and the feature input line corresponding to the 2nd bit F[2]. The same is true for other encoders, which are not described in detail in this application. In addition, each encoder group is also coupled to a corresponding weight input line. Each pre-encoder can provide two pre-encoded signals (M1S and M2S) to the encoder according to the two bits set in the weight parameter. Figure 7 The two precoded signals output by each precoder in are distinguished by an even number in the subscript 0 to (R-2). Further, each encoder in each encoder group operates on the values ​​of the two bits of the two feature input lines according to the corresponding two precoded signals and the values ​​of the corresponding weight input line input to obtain the corresponding partial product. The partial product is output after the carry addition operation is performed by the adder tree circuit. Among them, the input of each weight input line is the odd bit in the weight parameter corresponding to the input feature map. For example, in the weight parameter corresponding to the R-bit feature map, the value of each bit is W[0], W[1], ..., W[R-1] in sequence. Then the values ​​output by each weight input line in the corresponding multiplier are W[1], W[3], W[5], ..., W[R-1] respectively. The input of each feature input line is the value of a bit in the input feature map (one of F[0] to F[Z-1]).

[0057] like Figure 8 As shown, among the multiple precoders, some precoders include an XOR gate XOR1, an XNOR gate XNOR1 and a NOR gate NOR1. Exemplarily, if the value input to the weight input line corresponding to a certain encoder group is the value W[3] of the 3rd bit in the weight parameter. Then the value input to the first input end of the XNOR gate XNOR1 in the corresponding precoder is W[3]. The value input to the second input end of the XNOR gate XNOR1 is W[2]. At the same time, the value input to the first input end of the corresponding XOR gate XOR1 is W[1]. The second input end of the XOR gate XOR1 is coupled to the second input end of the corresponding XNOR gate XNOR1. The first input end of the NOR gate NOR1 and the first encoding input end of the corresponding encoder group are both coupled to the output end of the XOR gate XOR1. The second input end of the NOR gate NOR1 is coupled to the output end of the XNOR gate XNOR1. The output end of the NOR gate NOR1 is coupled to the second encoding input end of the corresponding encoder group. Furthermore, the structure of each subsequent precoder is the same as the structure of the precoder corresponding to the encoder group of the input W[3], and the embodiments of the present application are not described in detail here.

[0058] In the above convolution circuit, the addition tree circuit can perform addition operations according to the coupling relationship between each encoder and the weight input line and the feature input line. For example, each encoder group can be regarded as a row; each encoder in each encoder group can be regarded as a column. The column where each encoder in each encoder group is located corresponds to a digit (2 0 ~2 n). This digit can also be called a weight. When the addition tree circuit performs an operation, a carry addition operation can be performed on the outputs of the encoders with the same digit. Starting from the second row, the output of the encoder of the kth column (k≥1) in each row and the output of the encoder of the k+2th column of the previous row are binary bits on the same weight. In addition, when performing an addition operation on the outputs of each encoder corresponding to the first column of the first row to the first example of the last row, it is also necessary to add a partial product S related to the weight parameter at the end of each odd column. The partial product S can be the value of the odd bit of the weight parameter. When performing an addition operation on the outputs of each encoder corresponding to the last column of the first row to the last column of the last row, it is also necessary to add a constant 1 to each odd column first.

[0059] In order to implement the above addition operation, Fig. 9 As shown, the above-mentioned addition tree circuit usually adopts a multi-layer adder structure. The input of the addition tree circuit includes not only the output of the encoder group, but also the partial product S and the constant 1. Among them, each layer of the addition tree circuit is connected through multiple full adders ( Fig. 9 The rectangular box in the figure) is used to compare the binary bits with the same weight (such as Fig. 9 Each adder inputs a 3-bit binary number and outputs a sum bit (such as Fig. 9 The dashed circle in Fig. 9 The dotted filled circles in the figure), the summed bits output by all adders and the remaining bits not added in the current layer are transmitted together to the arrangement columns of the same-weighted bits in the second layer of the addition tree, and the carry bits of the adders are transmitted to the arrangement columns of the adjacent high-weighted bits in the second layer, thereby forming a second-layer arrangement array. The adders in the second layer perform parallel addition on the binary bits on the same-weighted bits, and the summed bits output by all adders and the remaining bits not added in the current layer are transmitted together to the arrangement columns of the same-weighted bits in the third layer of the addition tree, and the carry bits of the adders in the second layer are transmitted to the arrangement columns of the adjacent high-weighted bits in the third layer, ..., and the parallel addition is performed until the last layer. When there are no more than 2 bits of data left in the arrangement columns of each weighted bit, the parallel addition is completed.

[0060] From the above content, it can be seen that although the above multipliers can be used to implement convolution calculation, the adder tree circuit inside each multiplier needs to perform carry addition operations on the outputs of each encoder as well as the partial product S and the constant 1. Therefore, a large number of full adders need to be set, which increases the wiring area and also increases the power consumption.

[0061] Considering that the convolution kernel composed of weight parameters remains unchanged during the entire convolution process when calculating the convolution of the input feature map, this feature can be used to simplify the hardware convolution circuit, thereby reducing the convolution circuit area and power consumption. Therefore, in order to solve the above problems, Fig.10As shown, an embodiment of the present application provides a convolution circuit 1000, which includes multiple multipliers 1100, a first adder 1200 and a second adder 1300. The first adder 1200 is coupled to the multiple multipliers 1100. The second adder 1300 is coupled to the first adder 1200. Each multiplier 1100 includes multiple precoders, multiple encoder groups and an addition tree circuit. The multiple precoders are coupled to the multiple encoder groups in a one-to-one correspondence. The addition tree circuit includes multiple input lines and multiple output lines. The multiple encoder groups include multiple output terminals. The number of output terminals of the multiple encoder groups is the same as the number of the multiple input lines, and the multiple output terminals are coupled to the multiple input lines in a one-to-one correspondence. The addition tree circuit is coupled to the first adder through the multiple output lines. The second adder is also coupled to the memory. The memory is used to store calculation constants. The calculation constant can be determined according to the weight parameter. For example, the calculation constant may be the result of accumulating the odd-numbered bits of the weight parameter of the input feature map and the constant 1 at the corresponding bits.

[0062] Through the above-mentioned convolution circuit 1000, when performing a convolution operation, the adder tree circuit can perform an addition operation on the data output by multiple encoder groups. The first adder 1200 can add the outputs of each multiplier and transmit it to the second adder 1300. Furthermore, the partial product S related to the weight parameter of the input feature map and the constant 1 can be added and transmitted to the second adder 1300 as a calculation constant Wconst. The second adder 1300 adds the calculation constant to the output of the first adder 1200 to obtain the corresponding output feature map. Based on this, the amount of calculation of the adder tree circuit can be reduced, thereby reducing the number of full adders used by the adder tree circuit, and reducing the power consumption and wiring area of ​​the convolution circuit 1000.

[0063] In the above convolution circuit 1000, the number of multipliers can be adjusted according to the size of the convolution kernel. For example, if the size of the convolution kernel is 3*3, the number of multipliers is 9, and each multiplier can calculate one of the feature maps corresponding to the 9 pixels within the convolution range. If the size of the convolution kernel is 4*4, the number of multipliers is 16, and each multiplier can calculate one of the feature maps corresponding to the 16 pixels within the convolution range.

[0064] In a possible implementation, the structure of the addition tree circuit provided in the embodiment of the present application is as follows: Fig.11 As shown, the addition tree circuit also uses multiple full adders ( Fig.11 The rectangular box in the figure) is used to calculate the binary bits with the same weight on the output of the encoder group (such as Fig.11However, the input of the adder tree circuit provided in the embodiment of the present application only includes the output of the encoder group. When performing addition operation, each adder also inputs a 3-bit binary number and outputs 1 sum bit (such as Fig.11 The dashed circle in Fig.11 The dotted filled circles in the figure). The summed bits output by all adders and the remaining bits not added in the current layer are transmitted to the arrangement columns of the same weighted bits in the second layer of the addition tree, and the carry bits of the adders are transmitted to the arrangement columns of the adjacent high weighted bits in the second layer, thereby forming a second layer arrangement array. The adders in the second layer perform parallel addition on the binary bits on the same weighted bits, and the summed bits output by all adders and the remaining bits not added in the current layer are transmitted to the arrangement columns of the same weighted bits in the third layer of the addition tree, and the carry bits of the adders in the second layer are transmitted to the arrangement columns of the adjacent high weighted bits in the third layer, ..., and the parallel addition is performed until the last layer. When there are no more than 2 bits of data left on the arrangement columns of each weighted bit, the parallel addition is completed. Because the input of the addition tree circuit only includes the output of the encoder group, the number of full adders used in each layer of the addition tree circuit is less than that of the traditional addition tree circuit, thereby reducing the wiring area and power consumption of the addition tree circuit.

[0065] Furthermore, in one embodiment, the above-mentioned multiple multipliers may also share an adder tree circuit. The number of input lines in the adder tree circuit is the sum of the number of encoders in all multipliers. Fig.11 In the addition tree circuit, the binary bits of the same weight output by the encoder groups in each multiplier can be added in parallel, and refer to Fig.11 The adders are set layer by layer in a .

[0066] In the above manner, the outputs of multiple multipliers can be directly added without independently setting an addition tree circuit in each multiplier and without setting a first adder, which further improves the efficiency of the operation.

[0067] In a possible implementation, the plurality of precoders include a first precoder. The plurality of encoder groups include a first encoder group. The first precoder may be used as Figure 7Any precoder except precoder 0. The first precoder includes three input terminals, a first logic circuit and two output terminals. The first encoder group includes a first input terminal and a second input terminal. The first input terminal of the first encoder group is coupled to the first input terminal of the three input terminals through the first output terminal of the two output terminals. The three input terminals are also coupled to the second output terminal of the two output terminals through the first logic circuit. The second input terminal of the first encoder group is coupled to the second output terminal of the two output terminals. The first logic circuit is used to perform an XOR operation on the signals input from the second input terminal and the third input terminal of the three input terminals, and then perform an XOR operation on the signals input from the first input terminal of the three input terminals, and output the operation result from the second output terminal of the two output terminals.

[0068] In the above manner, when the multiplier operates on the input feature map, the first precoder in each multiplier only needs to perform XEN_OR operation and NEN_OR operation on the weight parameters input from the three input terminals, thereby reducing the power consumption of the operation.

[0069] For example, Fig.12 As shown, the first logic circuit A in the above-mentioned first precoder includes an XNOR gate XNOR1 and a NOR gate NOR1. The first input end of the NOR gate NOR1 is coupled to the output end of the XNOR gate XNOR1. The second input end of the NOR gate NOR1 is coupled to the first input end IN1 of the first precoder. The first input end of the XNOR gate XNOR1 is coupled to the second input end IN2 of the first precoder. The second input end of the XNOR gate XNOR1 is coupled to the third input end IN3 of the first precoder. The first input end IN1 of the first precoder is coupled to the first output end OUT1 of the first precoder. The output end of the NOR gate NOR1 is coupled to the second output end OUT2 of the first precoder.

[0070] Through the above structure, the first precoder in the embodiment of the present application is Figure 8 Compared with the conventional precoder, one XOR gate can be reduced. The convolution circuit 1000 usually includes a plurality of multipliers, so the use of the first precoder can reduce both the amount of logic operations and the wiring area.

[0071] Furthermore, the plurality of precoders mentioned above further include a second precoder. The plurality of encoder groups mentioned above further include a second encoder group. The second precoder can be used as Figure 7Precoder 0 in the embodiment. The second precoder includes two input terminals, a second logic circuit and two output terminals; the second encoder group includes a first input terminal and a second input terminal. The first input terminal of the two input terminals of the second precoder is coupled to the first input terminal of the second encoder group through the first output terminal of the two output terminals. The two input terminals are also coupled to the second output terminal of the two output terminals through the second logic circuit. The second logic circuit can invert the signal input to the first input terminal of the two input terminals, perform an AND logic operation with the signal input to the second input terminal, and output the signal from the second output terminal of the two output terminals.

[0072] In one possible embodiment, Fig.13 As shown, the second precoder includes a first AND gate AND1 and a NOT gate NOT1. The input end of the NOT gate NOT1 is coupled to the first input end In1 of the second precoder. The output end of the NOT gate NOT1 is coupled to the first input end of the first AND gate AND1. The second input end of the first AND gate AND1 is coupled to the second input end In2 of the second encoder. The first output end out1 of the second precoder is coupled to the first input end in1 of the second precoder. The output end of the first AND gate AND1 is coupled to the second output end out2 of the second precoder. The input end of the NOT gate NOT1 can input the value of the 0th bit in the weight parameter corresponding to the feature map. The second input end of the first AND gate AND1 can input the value of the 1st bit in the weight parameter corresponding to the feature map.

[0073] In another embodiment, the NOT gate NOT1 in the second precoder can also be selectively arranged between the first input terminal and the first output terminal of the second precoder, and the first input terminal of the first AND gate AND1 is directly coupled to the first input terminal of the second precoder. In this case, the input terminal of the NOT gate NOT1 can input the inverted value of the 0th bit in the weight parameter corresponding to the feature map.

[0074] In one embodiment, the first encoder group and the second encoder group each include a plurality of encoders. Fig.14As shown, each encoder includes a second AND gate AND2, a third AND gate AND3, a first OR gate OR1 and a first XOR gate XOR2. The first input end of the second AND gate AND2 is used to input the first coded data M1S output by the corresponding precoder. The second input end of the second AND gate AND2 is used to input the value F[a] of the ath bit in the feature map. The first input end of the third AND gate AND3 is used to input the second coded data M2S output by the corresponding precoder. The input of the second input end of the third AND gate AND3 is the value F[a-1] of the a-1th bit in the input feature map. Wherein, a is an integer not less than 1. In addition, if a=0, the second input end of the second AND gate AND2 and the second input end of the third AND gate AND3 are both input with the value F[0] of the 0th bit in the feature map. The first input end of the first OR gate OR1 is coupled to the output end of the second AND gate AND2; the second input end of the first OR gate OR1 is coupled to the output end of the third AND gate AND3. The first input end of the first XOR gate XOR2 is coupled to the output end of the first OR gate OR1. The second input terminal of the first XOR gate XOR2 is used to input the value WC[i] of the odd bit in the weight parameter, where i is an odd number between 0 and R.

[0075] In the above implementation process, the data and weight parameters of each bit in the feature map can be transmitted to the corresponding device through a register circuit. Among them, the register circuit can be coupled to a memory, and the register circuit includes a plurality of registers. Each weight parameter and the value of each bit in the input feature map can be cached through an independent register. In addition, when the first precoder and the second precoder need to input the same weight parameter, they can be coupled to the same register. When the encoders in the above-mentioned encoder groups need to input the value of the same bit in the feature map, they can also be coupled to the same register. Based on this, while ensuring that the multipliers operate in an orderly manner, the number of devices can be reduced and the wiring area can be reduced.

[0076] Furthermore, when the above-mentioned first precoder and the traditional precoder are used in the multiplier, the number of the first precoders can be set according to actual requirements.

[0077] For example, in one embodiment, Fig.15As shown, the multiplier may include K-1 first precoders and 1 second precoder, where K represents the total number of precoders in the multiplier. Each first precoder and second precoder are coupled to a corresponding encoder group. The first input end of the second precoder is used to input the value WC[0] of the 0th bit in the weight parameter. The second input end of the second precoder is used to input the value WC[1] of the 1st bit in the weight parameter. The third input end of each first precoder can input the value of the corresponding odd bit after WC[1] in the weight parameter. The value input by each weight input line is the same as the value input by the third input end of the corresponding first precoder. The second input end of the first precoder inputs the value of the first bit before the corresponding odd bit. The second input end of the first precoder inputs the value of the second bit before the corresponding odd bit. For example, if the third input terminal of a first precoder inputs the value WC[3] of the third bit in the weight parameter, the second input terminal of the first precoder inputs the value WC[2] of the second bit in the weight parameter, and the first input terminal of the first precoder inputs the value WC[1] of the first bit in the weight parameter. The conversion relationship between the value of each bit of the weight parameter in the multiplier provided in the embodiment of the present application and the value of each bit of the weight parameter in the traditional multiplier is as follows:

[0078]

[0079] Among them, XNOR represents the exclusive OR operation; the transformation relationship between the values ​​of each bit in the weight parameter is shown in Table 1:

[0080] Table 1

[0081] Weight parameters for this application Traditional weight parameters WC[0] W[0] WC[1] W[1] WC[2] W[1]XNRW[2] WC[3] W[3] WC[4] W[3]XNRW[4] WC[5] W[5] WC[6] W[5]XNRW[6] WC[7] W[7]

[0082] In Table 1, W[i] represents the value of each bit in the weight parameter of the traditional multiplier. WC[i] represents the value of each bit in the weight parameter of the multiplier provided in the embodiment of the present application. Among them, WC[i] can be pre-calculated according to the transformation relationship in Table 1 and stored in the memory. When a convolution operation is required, WC[i] can be cached by a register coupled to the memory and then transmitted to the traditional precoder and each first precoder.

[0083] In the above implementation process, the number of first precoders used in the multiplier can be adjusted according to actual needs and is not limited to the number in the above implementation scheme.

[0084] In one embodiment, Fig.16 As shown, the embodiment of the present application also provides a convolution calculation method applied to the above convolution circuit, and the execution process of the convolution calculation method is as follows:

[0085] S161. Receive multiple input feature maps and preset weight parameters and calculation constants for each input feature map.

[0086] Among them, the above-mentioned calculation constant is a constant obtained by accumulating the odd bits and constant 1 in the weight parameters of the input feature map. The weight parameters corresponding to each input feature map include the values ​​of each bit in each feature map. Because when performing convolution calculation, the odd bits and constant 1 of the weight parameter are pre-configured known parameters, and the data finally output by the convolver also needs to add these two fixed parameters, the two can be accumulated as pre-configured calculation constants without the need to calculate through the addition tree circuit. Each input feature map, weight parameter and calculation constant can be stored in a memory. The preset weight parameters and calculation constants of each input feature map can be pre-stored in the memory. When convolution calculation is required, the values ​​of each bit of the weight parameter, the values ​​of each bit of the input feature map and the calculation constant can be read into the corresponding register in the register circuit.

[0087] S162. An input feature map and a weight parameter are transmitted to each multiplier, and corresponding operation results are obtained by performing operations through multiple precoders, multiple encoder groups and addition tree circuits in the multiplier.

[0088] Among them, the value of the pixel point at a certain position in the input feature map of each multiplier in each convolution process. When executing the above steps, the number of bits of the weight parameter can be the same as the number of bits of the input feature map. For example, if the input feature map is int8 data. When performing the convolution operation, the weight parameter input by each multiplier can include an 8-bit value. Each precoder can first output two precoded signals according to the values ​​of the corresponding two bits in the weight parameter. Then, the encoder group corresponding to each precoder outputs multiple partial products according to the two precoded signals and the values ​​of the corresponding odd bits in the weight parameter. Finally, the addition tree circuit performs a carry addition operation on the partial products output by multiple encoder groups to obtain the above operation results.

[0089] In the above implementation process, the number of bits of the weight parameter may be different from the number of bits of the input feature map, which may be set according to the actual convolution operation requirements.

[0090] S163. Accumulate the operation results of each multiplier through the first adder to obtain first data.

[0091] S164. Transmit the above-mentioned calculation constant to the second adder, and add the above-mentioned first data and the calculation constant through the second adder to obtain an output feature map.

[0092] In the convolution calculation process, the above calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 on the corresponding digits. The calculation constant can be cached by a register to ensure that the corresponding calculation process can be executed more orderly in a high-speed computing environment.

[0093] In a possible implementation, the embodiment of the present application also provides a computer-readable storage medium, in which computer program instructions can be stored. When the computer program instructions are executed by the processor, the above-mentioned convolution calculation method can be implemented. Among them, the above-mentioned processor can be a central processing unit (CPU), a general processor, a network processor (NP), a digital signal processor (DSP), a microprocessor (MCU), a microcontroller unit, a programmable logic device (PLD) or any combination thereof. The processor can also be other devices with processing functions, such as circuits, devices or software modules, which are not limited by this application. The computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, a USB flash drive and an optical data storage device, etc.

[0094] In a possible implementation, the present application embodiment further provides a chip, which includes a circuit board and the above-mentioned convolution circuit 1000 arranged on the circuit board. The above-mentioned circuit board may be a printed circuit board (PCB) or a substrate (including but not limited to a silicon substrate). In one example, Fig.17 As shown, the chip can be sold or used as an independent convolution chip 1700. Among them, an interface 1711 coupled to the convolution circuit 1000 can be set on the circuit board 1710. Through the interface 1711, the convolution circuit 1000 can receive an input feature map sent by an external device (such as a processor) and output a processed output feature map to the external device. Of course, the convolution chip can also be set on a circuit board together with one or more processors for sale or use as a chip system. The embodiment of the present application does not impose specific restrictions on this.

[0095] In another example, Fig.18As shown, the chip can also be a processor 1800. The processor 1800 includes a control unit 1811 and a convolution circuit 1000 arranged on a circuit board 1810. Among them, the convolution circuit 1000 can be coupled with the control unit 1811, and the control unit 1811 can receive the input feature map through the bus 1812. Then, the input feature map is sent to the convolution circuit 1000 for convolution operation to obtain the corresponding output feature map. Finally, the output feature map is sent to the next level processing unit in the processor 1800 through the control unit 1811 or sent to other devices through the bus 1812 for processing. Of course, the convolution circuit 1000 can also choose to be coupled with the control unit 1811 through the bus 1812. The embodiment of the present application does not impose specific restrictions on this.

[0096] In the above implementation process, the processor can be a central processing unit (CPU), a general processor, a network processor (NP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or a system on chip (SoC), or any combination thereof. The embodiments of the present application do not impose specific restrictions on this.

[0097] Furthermore, in the above implementation process, the chips in the above two examples may also include other types of devices, and the embodiments of the present application do not impose specific limitations on this.

[0098] Those of ordinary skill in the art will appreciate that the functions of the circuits of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0099] In the several embodiments provided in the present application, it should be understood that the disclosed circuits and electronic devices can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0100] In addition, the chip in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0101] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A convolution circuit, characterized in that: comprising: a plurality of multipliers; a first adder coupled to the plurality of multipliers; and a second adder coupled to the first adder; The multiplier includes a plurality of precoders, a plurality of encoder groups, and an adder tree circuit; the plurality of precoders are coupled to the plurality of encoder groups in a one-to-one correspondence; The adder tree circuit includes a plurality of input lines and a plurality of output lines; the plurality of encoder groups include a plurality of output terminals; the number of the plurality of output terminals is the same as the number of the plurality of input lines, and the plurality of output terminals are coupled to the plurality of input lines in a one-to-one correspondence; the adder tree circuit is coupled to the first adder through the plurality of output lines; The second adder includes a plurality of input terminals and an output terminal, one of the plurality of input terminals is used to couple with a memory, and the other input terminals of the plurality of input terminals are used to couple with the first adder.

2. The convolution circuit according to claim 1, characterized in that The plurality of precoders include a first precoder; the plurality of encoder groups include a first encoder group; The first precoder includes three input terminals, a first logic circuit and two output terminals; the first encoder group includes a first input terminal and a second input terminal; the first input terminal of the first encoder group is coupled to the first input terminal of the three input terminals through the first output terminal of the two output terminals; the three input terminals are also coupled to the second output terminal of the two output terminals through the first logic circuit; the second input terminal of the first encoder group is coupled to the second output terminal of the two output terminals.

3. The convolution circuit according to claim 2, characterized in that: The first logic circuit is used to perform an XOR operation on the signals input from the second input terminal and the third input terminal among the three input terminals, and then perform an NOR operation on the signal input from the first input terminal among the three input terminals, and output the operation result from the second output terminal among the two output terminals.

4. The convolution circuit according to claim 2 or 3, characterized in that: The first logic circuit includes an XNOR gate and a NOR gate; The first input terminal of the NOR gate is coupled to the output terminal of the XNOR gate; the second input terminal of the NOR gate is coupled to the first input terminal of the first precoder; The first input terminal of the XOR gate is coupled to the second input terminal of the first precoder; the second input terminal of the XOR gate is coupled to the third input terminal of the first precoder; An output terminal of the NOR gate is coupled to a second output terminal of the first precoder.

5. The convolution circuit according to any one of claims 1 to 3, characterized in that: A register circuit is also included; the plurality of precoders and the plurality of encoder groups are also coupled to the register circuit.

6. The convolution circuit according to any one of claims 1 to 5, characterized in that: The memory is used to store calculation constants; the calculation constants are determined according to weight parameters.

7. The convolution circuit according to claim 6, characterized in that: The calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 at the corresponding digits.

8. A convolution calculation method, applied to a convolution circuit; the convolution circuit includes a plurality of multipliers; a first adder coupled to the plurality of multipliers; and a second adder coupled to the first adder; the multipliers include a plurality of precoders, a plurality of encoder groups and an adder tree circuit; the plurality of precoders are coupled to the plurality of encoder groups in a one-to-one correspondence; the adder tree circuit includes a plurality of input lines and a plurality of output lines; the plurality of encoder groups include a plurality of output terminals; the number of the plurality of output terminals is the same as the number of the plurality of input lines, and the plurality of output terminals are coupled to the plurality of input lines in a one-to-one correspondence; the adder tree circuit is coupled to the first adder through the plurality of output lines; the second adder includes a plurality of input terminals and an output terminal, and one of the plurality of input terminals is used to couple with a memory, characterized in that The method comprises: Receiving a plurality of input feature maps and weight parameters and calculation constants preset for each of the input feature maps; the calculation constants are determined according to the weight parameters; Transmitting one of the input feature maps and one of the weight parameters to each of the multipliers, and performing operations through the multiple precoders, the multiple encoder groups, and the adder tree circuit in the multiplier to obtain corresponding operation results; Accumulating the operation results of each of the multipliers through the first adder to obtain first data; The calculation constant is transmitted to the second adder, and the first data and the calculation constant are added by the second adder to obtain an output feature map.

9. The method according to claim 8, characterized in that The calculation constant is the result of accumulating the odd bits of the weight parameter of the input feature map and the constant 1 at the corresponding digits.

10. The method according to claim 8, characterized in that The method further comprises: performing operations on the plurality of precoders, the plurality of encoder groups and the addition tree circuit in the multiplier to obtain corresponding operation results, including: Each of the precoders outputs two precoded signals according to two corresponding bits in the weight parameter; The encoder groups corresponding to the respective precoders output a plurality of partial products according to the two precoded signals and the corresponding odd-numbered bits in the weight parameters; The addition tree circuit performs a carry addition operation on the plurality of partial products to obtain the operation result.

11. A computer-readable storage medium, characterized in that: Computer program instructions are stored, and when the computer program instructions are executed by a processor, the method described in any one of claims 8 to 10 is implemented.

12. A chip, characterized in that: It comprises a circuit board and a convolution circuit as claimed in any one of claims 1 to 7 arranged on the circuit board.

13. An electronic device, characterized in that: The invention comprises a memory, and the chip as claimed in claim 12 coupled to the memory.

Images