High-speed matrix computation circuit

Abstract

A high-speed matrix computation circuit, relating to electronic technology. The present invention comprises M+N-1 addition modules and a predetermined number of macro computation modules. Each macro computation module comprises M row lines and N column lines which are equivalently and orthogonally arranged. Multipliers are arranged at intersections of the row lines and the column lines to form a multiplier array composed of M x N multipliers. A first input terminal of each multiplier is connected to the row line in which the multiplier is located, and a second input terminal thereof is connected to the column line in which the multiplier is located. M and N are predetermined natural numbers. U[M-x][N-y] is used to represent a multiplier at each position, and x and y are both integers and satisfy the following conditions: 0≤x≤M-1, and 0≤y≤N-1. Among the macro computation modules, an output terminal of a multiplier satisfying x+y=S is connected to an input terminal of an addition module having a serial number S. On the basis of serial numbers, output terminals of the addition modules are connected to different bit weight input terminals of a bit weight addition circuit in a one-to-one correspondence. The present invention greatly reduces the amount of digital computation.

Description

High-speed matrix calculation circuit TECHNICAL FIELD

[0001] The present application relates to the electronic technology, and particularly to the matrix calculation technology applied to neural network, deep learning, artificial intelligence. BACKGROUND

[0002] In the field of artificial intelligence, especially deep learning, matrix multiplication is the most core computing task. According to the official CUDA programming guide and architecture document of NVIDIA, a large number of operations performed by GPU in deep learning tasks are matrix multiplication and convolution operation (https: / / docs.nvidia.com / cuda / cuda-c-programming-guide / index.html); the white paper of Google's TPU describes in detail how the TPU architecture focuses on matrix multiplication. The paper points out that TPU is designed specifically for the key computing operations of neural networks, especially matrix multiplication and convolution operation.

[0003] The paper "In-Datacenter Performance Analysis of a Tensor Processing Unit" (published by Google research team) explains how the matrix multiplication unit of TPU accelerates the neural network operation of TensorFlow (https: / / arxiv.org / abs / 1704.04760). Therefore, matrix multiplication is widely used in the forward propagation, back propagation and weight update process of neural network, especially in convolutional neural network (CNN) and fully connected layer, and in the Transformer architecture for natural language processing. The computing efficiency of matrix multiplication directly determines the training and inference performance of the entire model. How to effectively improve the computing speed and efficiency of matrix multiplication has always been the focus of artificial intelligence hardware acceleration technology.

[0004] At present, the mainstream matrix multiplication calculation technology is mainly based on pure digital circuit. For example, Nvidia introduces the design and implementation of tensor core in its official white paper and GPU architecture document (including the white paper of Volta, Ampere, Hopper and other architectures), which shows that they are based on all-digital circuit to realize matrix operation.

[0005] (https: / / images.nvidia.com / content / volta-architecture / pdf / volta-architecture- whitepaper.pdf), and on its official website developer page also clearly indicates that the matrix calculation unit of its most advanced H100 tensor core is also based on pure digital circuit (https: / / developer.nvidia.com / blog / nvidia-hopper-architecture-in-depth / , https: / / resources.nvidia.com / en-us-tensor-core / gtc22-whitepaper-hopper). Similarly, in the classic paper “In-Datacenter Performance Analysis of a Tensor Processing Unit” published by Google, the hardware design of TPU is described in detail, including the use of full digital circuit to realize the matrix multiplication of systolic array structure (Jouppi et al., “In-Datacenter Performance Analysis of a Tensor Processing Unit,” ISCA 2017).

[0006] Digital circuit architecture needs to frequently transfer, store and operate numerical values when performing large-scale matrix multiplication. Whether it is GPU, TPU or ASIC, it relies on a large number of multiplication and addition operations, which need to go through precise bit-level operations in digital circuits, resulting in a serious impact on speed and extremely high energy consumption when performing massive calculations. With the increase of model size, especially in super large scale deep neural networks and super high parameter artificial intelligence large models, the problems of computing speed and energy consumption become particularly prominent, limiting the application of such computing units.

[0007] According to the discussion in the classic chip design textbook Digital Integrated Circuits: A Design Perspective, the core reason affecting the speed and power consumption of multiplication operation is the carry operation of digital adder. Since the addition calculation needs to be carried out in series, the high bit must wait for the carry of the low bit to complete the calculation, which will cause a large delay. Although the delay can be reduced by techniques such as lookahead carry, ripple carry, tree structure, etc., the effect is still limited. In high-dimensional matrix multiplication, in addition to the addition of the results of multiplication of corresponding bits to obtain the final product, multiple product results also need to be added. For example, in the autoregressive transformer architecture of Chatgpt 3.5, the vector dimension of token is 512. Even if only one double-precision matrix multiplication calculation is performed, 512 64-bit binary floating-point multiplication calculations need to be completed first, and then 512 binary multiplication results need to be added. The accumulated addition carry delay will bring serious delay to the above calculation. Artificial intelligence, whether training or reasoning, will perform a large number of matrix multiplication operations, so it will consume a lot of time and power consumption.

[0008] Compared with digital adders, analog adders have obvious advantages in speed and power consumption. According to Kirchhoff's law, the addition result can be quickly obtained by physically superimposing two or more currents. Typical technologies such as In-Memory Computing (IMC) architecture (In-memory hyperdimensional computing, nature electronics), programmable analog current mirror matrix vector multiplier (Analog Vector-Matrix Multiplier Based on Programmable Current Mirrors for Neural Network Integrated Circircuits, IEEE access), SRAM-based in-memory computing matrix multiplier (A 42pJ / decision 3.12TOPS / W robust in-memory machine learning classifierwith on-chip training, IEEE International Solid-State Circuits Conference Digest of Technical Papers), and the like, realize matrix multiplication operation through analog devices. The advantages of this scheme are that the operation speed and power consumption are much lower than those of pure digital matrix multipliers. However, due to the non-ideal characteristics of analog quantities and devices, combined with the errors of the final analog-to-digital converter, the accuracy of the calculation result will be much lower than that of the digital multiplier. If a memristor, magnetic memory, ferroelectric memory or other device is used to store the weight, due to the non-linear characteristics of the device, the result accuracy will be further reduced. It can only be applied to edge computing devices and cannot be applied to high-parameter strong artificial intelligence.

[0009] In summary, although the existing matrix multiplication acceleration technology has made breakthroughs in performance, there are still significant deficiencies in energy efficiency, delay, calculation accuracy, and analog-to-digital conversion delay. Therefore, there is an urgent need for a new computing scheme that can further improve the calculation speed while ensuring high accuracy and reducing energy consumption. SUMMARY

[0010] The present application provides a matrix multiplication calculation circuit, which realizes the cross product and calculation of multiplication by current addition, thereby completing the matrix multiplication operation.

[0011] The algorithm technical scheme adopted by the present application is:

[0012] The high-speed matrix calculation circuit comprises M+N-1 addition modules and a predetermined number of macro calculation modules, each of which comprises M row lines and N column lines arranged in equivalent quadrature; a multiplier is arranged at the intersection of the row line and the column line to form a multiplier array comprising M*N multipliers; each multiplier has a first input end, a second input end and an output end, wherein the first input end is connected with the row line at the position of the multiplier, and the second input end is connected with the column line at the position of the multiplier; M and N are predetermined natural numbers, U[M-x][N-y] represents the multiplier at each position, x and y are integers and satisfy 0≤x≤M-1 and 0≤y≤N-1;

[0013] In each macro calculation module, the output ends of the multipliers satisfying x+y=S are connected to the input ends of the addition module with the serial number S, S is each integer in the closed interval [0, M+N-2], and the serial number of the addition module is unique;

[0014] According to the serial number, the output ends of the addition modules are connected to different bit weight input ends of the bit weight addition circuit in a one-to-one corresponding manner, and the bit weight addition circuit has an output end;

[0015] The addition module is used to realize the addition calculation of the input quantity, the multiplier is used to realize the multiplication calculation of the input quantity, and the bit weight addition circuit is used to perform the addition calculation on the input quantity of each bit weight input end according to the bit weight.

[0016] The bit weight values of the bit weight input ends of the bit weight addition circuit are sorted from small to large, and the serial numbers thereof are incremented from 0 and unique; the addition module and the bit weight input end are connected in the same serial number.

[0017] The output of the multiplier is an analog quantity, and the input and output of the addition module are analog quantities.

[0018] The analog quantity is a current value.

[0019] The multiplier has an analog quantity input end and a digital quantity input end.

[0020] More specifically:

[0021] Each digit of each multiplicand and multiplier in matrix multiplication is separated by a traditional digital decoder, multiplied respectively and subjected to analog addition calculation according to the weight of each digit, subjected to analog-to-digital conversion, and subjected to digital addition to obtain the final result.

[0022] More specifically, for example, the following matrix multiplication operation is performed:

[0023] According to the matrix operation rule, the following calculation needs to be performed: A*D+B*E+C*F

[0024] Assuming that all data of A~F are at most three digits, the digit of A hundred place is A1, the digit of ten place is A2, and the digit of unit place is A3, i.e. A = 100*A1 + 10*A2 + A3, the digit of B hundred place is B1, the digit of ten place is B2, and the digit of unit place is B3, i.e. B = 100*B1 + 10*B2 + B3, and the same is true for C~F.

[0025] As shown in Fig. 1, the first step is consistent with the calculation step of traditional multiplication, and three multiplication operations of A*D, B*E, and C*F are respectively performed. Each digit of A~F will first be multiplied according to the traditional multiplication algorithm, and the intermediate multiplication results of each digit analog signal A1*D1, A3*D3, etc. are obtained. However, the final results of the three multiplications are not calculated, but the analog signal results of all the same digits in the three multiplications are added, i.e. the unit place calculation results of the three multiplications are added (A3D3+B3E3+C3F3), the ten place calculation results are added (A2D3+A3D2+B2E3+B3E2+C2F3+C3F2), the hundred place calculation results are added (A1D+A2D2+A3D1+B1E3+B2E3+B3E1+C1F3+C2F2+C3F1), and so on, so as to obtain the corresponding analog signal results, thereby avoiding the digital addition requiring carry. Then, the analog signals of each digit are converted into binary digital signals through an analog-to-digital converter (ADC), and finally the binary digital signals of each digit are weighted and added, so that the final calculation result is obtained. The foregoing A3D3 is a short writing of A3*D3, and the same is true for others.

[0026] As shown in FIG. 4, the macro calculation module (24) receives analog signals or digital signals as inputs of the multipliers A, B, C and the multiplicands D, E, F of the matrix, and the digit separator (5) separates the digits of the multipliers and the multiplicands respectively, and sends the digits (for example, the units, the tens, the hundreds, etc. in the decimal system) to the multipliers (1) in analog or digital form through different channels; the analog multiplication of the corresponding digits of the multipliers and the multiplicands is completed through the analog multipliers (1), and the product is output in the form of analog quantities such as voltage, current, charge, etc. For the same-digit product of the digits that do not need to be added in the vertical multiplication, the same-digit analog product (20) is directly output from the macro calculation module (24); for the cross-digit product of the digits that need to be added in the vertical multiplication, the macro calculation module (24) internally completes the addition calculation through the addition of analog quantities such as current, voltage, charge, etc. to obtain the cross-digit analog sum product (13), which is then output from the macro calculation module (24), and finally the multi-digit product without carry is obtained, which is represented by the size of analog quantities such as current, voltage, charge, etc. The macro calculation module (24) can be single or multiple. In the case of a single macro calculation module (24), the cross-digit analog sum product (13) and the same-digit analog product (20) of the multiplication result of each digit are directly sent to the analog-to-digital converter (2). In the case of multiple macro calculation modules (24), the output cross-digit analog sum product (13) and the same-digit analog product (20) of each macro calculation module (24) are added on each digit, and then the analog quantity on each digit is sent to the analog-to-digital converter (2). The analog-to-digital converter (2) converts the analog quantity into a digital signal and inputs it into the weighted calculator (3) to complete the digital carry addition and generate the final digital calculation result (4).

[0027] The input multiplier is converted into data of any base and any number of digits by the digit separator (5) according to the actual situation, and the digits (for example, the units, the tens, the hundreds, etc. in the decimal system) are separated, and the size of the value on each digit is represented by one analog signal or multiple digital signals, which is called the multiplier digit (14). The digit separator (5) outputs the signals of each digit, which are multiplied by each other through the analog multipliers (1).

[0028] About the analog multiplier (1), the input multiplicand is the analog data on a certain digit converted by the digit separator (5), that is, the multiplicand digit (15), and the input multiplier is the digital data on a certain digit converted by the digit separator (5), that is, the multiplier digit (14). The digital signal of the multiplier digit (14) controls the on-off of the switching devices such as MOSFET, EEPROM, EPROM, PROM, and ferroelectric devices, realizes the linear addition of the analog signal on the multiplicand digit (15), and obtains the product result current (21).

[0029] About the analog multiplier (1), the input multiplicand is the multi-bit bus digital data on a certain digit converted by the digit separator (5), that is, the multiplicand digit (15), and the input multiplier is the digital data on a certain digit converted by the digit separator (5), that is, the multiplier digit (14). The bus digital signal on the multiplicand digit (15) controls the on-off of the current path of the switching devices such as MOSFET, EEPROM, EPROM, PROM, and ferroelectric devices, generates the corresponding size analog current sum; the digital signal of the multiplier digit (14) also controls the on-off of the switching devices, realizes the linear addition of the current value converted by the multiplicand digit (15), and obtains the product result current (21).

[0030] About the analog multiplier (1), the input multiplicand digit (15) and the multiplier digit (14) are both analog signals, and the multiplier analog signal drives the memristor device, multiplies the analog signal of the multiplicand digit (15) by changing the conductance value, and obtains the product result current (21).

[0031] The analog-digital converter (2) converts the analog signals of the cross analog sum product (13) or the same position analog product (20) obtained by each digit into binary digital signals in parallel.

[0032] About the weighted calculator (3), through the internal conventional digital multiplier circuit, the result output by the analog-digital converter (2) on each digit is multiplied by the weight corresponding to the current base of each digit, and the conventional digital adder is used to complete the binary carry addition, to generate the final matrix multiplication calculation result.

[0033] The beneficial effects of the present application are: without affecting the calculation accuracy, the matrix multiplication operation speed and performance are greatly improved, thereby significantly improving the speed and efficiency of deep learning, neural network and artificial intelligence training and reasoning. The present application separates and adds the cross product of high and low bits and the current, and then independently performs analog-to-digital conversion, ensuring that the high-bit data does not affect the accuracy of the low-bit data, so the accuracy is significantly higher than the traditional analog current addition scheme, and the accuracy requirement of the analog-to-digital converter is lower, allowing the analog-to-digital converter to be faster with lower accuracy, thereby further improving the calculation speed. The present application greatly reduces the digital calculation amount, so that the calculation speed is much higher than the matrix operation scheme based on pure digital adder and multiplier. BRIEF DESCRIPTION OF DRAWINGS

[0034] Fig. 1 is a schematic diagram of the algorithm of the present application.

[0035] Fig. 2 is a schematic diagram of the overall structure of the present application.

[0036] Fig. 3 is an example of circuit calculation of the present application.

[0037] Fig. 4 is a schematic diagram of a three-bit three-matrix multiplication structure of the present application.

[0038] Fig. 5 is a possible internal structure of an analog multiplier.

[0039] Fig. 6 is a schematic diagram of a multiplication switch device.

[0040] Fig. 7 is a schematic diagram of the matrix multiplication principle structure of embodiment 1.

[0041] Fig. 8 is a schematic diagram of the analog multiplier principle structure used in embodiment 1.

[0042] Fig. 9 is a schematic diagram of the analog multiplier principle structure used in embodiment 3.

[0043] Fig. 10 is a schematic diagram of the analog multiplier principle structure used in embodiment 4.

[0044] Fig. 11 is a schematic diagram of the overall structure of embodiment 5.

[0045] Fig. 12 is a schematic diagram of the overall structure of embodiment 6.

[0046] Fig. 13 is a schematic diagram of the principle structure of embodiment 7.

[0047] Fig. 14 is a schematic diagram of the two-bit binary matrix calculation topology structure of embodiment 7.

[0048] Fig. 15 is a schematic diagram of the four-bit binary matrix calculation topology structure of embodiment 7.

[0049] Fig. 16 is a schematic diagram of the addition calculation topology structure of embodiment 7.

[0050] Fig. 17 is a schematic diagram of the matrix structure of embodiment 8.

[0051] Figure 18 is a schematic diagram of matrix calculation of Example 8.

[0052] Figure 19 is a coordinate schematic diagram of a 4*8 matrix.

[0053] Figure 20 is an operation schematic diagram of a 4*8 matrix. DETAILED DESCRIPTION

[0054] The present application provides a high-speed matrix calculation circuit, comprising M+N-1 addition modules and a predetermined number of macro calculation modules, each macro calculation module comprising M row lines and N column lines arranged in equivalent quadrature; a multiplier is arranged at the intersection of a row line and a column line, forming a multiplier array comprising M*N multipliers; each multiplier has a first input end, a second input end and an output end, wherein the first input end is connected to the row line at the position of the multiplier, and the second input end is connected to the column line at the position of the multiplier; M and N are predetermined natural numbers, U[M-x][N-y] represents the multiplier at each position, x and y are integers and satisfy 0≤x≤M-1 and 0≤y≤N-1.

[0055] The definitions of row lines and column lines are described in Example 8.

[0056] In each macro calculation module, the output ends of the multipliers satisfying x+y=S are connected to the input end of the addition module with serial number S, S is each integer in the closed interval [0, M+N-2], and the serial number of the addition module is unique;

[0057] According to the serial number, the output ends of each addition module are connected to different bit weight input ends of a bit weight addition circuit in a one-to-one correspondence, and the bit weight addition circuit has an output end.

[0058] The addition module is used to realize the addition calculation of the input quantity, the multiplier is used to realize the multiplication calculation of the input quantity, and the bit weight addition circuit is used to perform addition calculation on the input quantity of each bit weight input end according to the bit weight.

[0059] Further, the bit weight values of each bit weight input end of the bit weight addition circuit are sorted from small to large, and the serial numbers are incremented from 0 and have uniqueness; the addition module and the bit weight input end are connected in the same serial number.

[0060] Further, the output of the multiplier is an analog quantity, and the input and output of the addition module are both analog quantities.

[0061] Further, the analog quantity is a current value.

[0062] Further, the multiplier has an analog quantity input end and a digital quantity input end.

[0063] More specifically:

[0064] Referring to Fig. 2, the present application provides a high-speed high-precision matrix multiplication calculation circuit, the core module is macro calculation module (24), all the multiplicand M1~Mk and all the multiplier N1~Nk are converted into bit data (51) on each digit according to actual needs through digit separator (5), for example, the multiplicand M1 is decimal number 321, the bit data on the unit place is 1, the bit data on the ten place is 2, and the bit data on the hundred place is 3. The multiplicand M1~Mk and the multiplier N1~Nk can be analog quantities of any size and any range; they can also be digital quantities of any size and any range, which can take any form of coding according to actual conditions. The digit separator (5) converts each multiplicand M1~Mk and the multiplier N1~Nk into multi-bit data (51) of any base and any number of bits according to actual application requirements, and the bit data (51) on each digit can be a single-bit analog or digital quantity, or a multi-bit analog or digital quantity.

[0065] Each bit data (51) obtained by the multiplicand and the multiplier through the digit separator (5) will be multiplied by each other through each multiplier (1), and the product will be output by the multiplier (1) in the form of an analog quantity. In the case of a single macro calculation module (24), the cross analog sum and product (13) and the same bit analog product (20) of the multiplication calculation result of each digit will be directly sent to the analog-to-digital converter (2). In the case of multiple macro calculation modules (24), the output cross analog sum and product (13) and the same bit analog product (20) of each macro calculation module (24) will be added on each digit corresponding to the analog quantity, and then the analog quantity on each digit will be sent to the analog-to-digital converter (2), which will convert the analog quantity into a digital signal and send it to the weighted calculator (3) to complete the digital carry addition and generate the final digital calculation result (4).

[0066] Taking the multiplication calculation process of the decimal number 258 and the decimal number 357 as an example, as shown in Fig. 3, the multiplicand 258 is processed by the digit separator (5) to represent the numbers on the hundred, ten and unit places by three current intensities, respectively, and the decimal three-digit number 258 can be represented by three current intensities of 2uA, 5uA and 8uA (which can also be represented by other analog or digital means according to actual conditions and algorithms), and the multiplier 357 is processed by the digit separator (5) to represent the numbers on the hundred, ten and single places by the on-off of the switching devices on the three 9-bit buses. The decimal number 357 has three switches on the 9-bit bus on the hundred place in the on state, and the other six switches are in the off state; similarly, the ten and unit places have five and seven switches in the on state, respectively, and the other switches are in the off state (which can also be represented by other analog or digital means according to actual conditions and algorithms).

[0067] The multiplier digits (15) will be multiplied by the multiplier digits (14) through the analog multiplier (1), and Fig. 5 is a possible internal structure of the analog multiplier (1), and each intersection is provided with a multiplication switch device (30), as shown in Fig. 6, the multiplier digits (14) will drive the multiplication switch device (30) to open the corresponding switch, as described above, for the decimal number 357, 7 multiplication switch devices (30) will be opened at the unit place, so as to provide 7 mirror current paths for the multiplier digits (15), to realize the analog multiplication operation; at the same time, the analog multiplier (1) at the ten place and the hundred place will open 3 and 5 switches respectively, to provide three and five mirror current paths respectively, and the total current forms the product result current (21).

[0068] As shown in Fig. 2, after the analog multiplication calculation of the analog multiplier (1) at each digit is completed, the digit current output by each macro calculation module (24) will be superimposed at each digit to complete the analog addition, and then the cross analog product (13) and the same place analog product (20) are sent to the analog-digital converter (2). Each analog-digital converter (2) converts the final current at the digit into a digital signal, which is then input to the weighted calculator (3). In this embodiment, it is a matrix operation of three pairs of decimal three-digit numbers, so in the weighted calculator (3), the digital-to-analog conversion result data at the hundred place needs to be multiplied by 100, the digital-to-analog conversion result data at the ten place needs to be multiplied by 10, and then the unit place data is added to generate the final digital calculation result (4).

[0069] Embodiment 1

[0070] Referring to Fig. 7, this embodiment is a matrix multiplication calculation of one row and two columns multiplied by two rows and one column, (7), (8), (9), and (10) are two-digit decimal data of different matrix inputs, and the digit separator (5) converts the matrix input data (7) and (9) into two 9-bit bus digital signals respectively, wherein the number of 1s on the bus corresponds to the data of the unit place and the ten place of the two-digit decimal of the respective input data (7) and (9), and the rest are 0. Similarly, the digit separator (5) converts the matrix input data (8) and (10) into two two-digit digital signals respectively, wherein the number of 1s on the bus corresponds to the data of the unit place and the ten place of the two-digit decimal of the respective input data (8) and (10), and the rest are 0.

[0071] After being separated by the digit separator (5), the single digit of the multiplier and the multiplier data is sent to the analog multiplier (1) for calculation, the internal structure is shown in Figure 5, (the single digit of the multiplier is multiplied by the single digit of the multiplicand, the single digit of the multiplier is multiplied by the ten digit of the multiplicand, etc.), the signal of 1 on the bus of the digit separator (5) will open the corresponding switch in the multiplicand digit (15), so that the corresponding number of longitudinal path voltage is Vdd; At the same time, the signal of 1 on the bus of the digit separator (5) will open the corresponding multiplication switch (11) in the multiplier digit (14) to generate the corresponding number of mirror current. Therefore, the number of longitudinal path with Vdd voltage and the number of transverse current mirror will produce the product effect to obtain the product result current (21).

[0072] As shown in Figure 4, the product result current (21) will be output as a digital current from the analog multiplier (1), the output of the same bit analog product (20) of the result of the multiplication of the single digit of the two pairs of multiplicand and multiplier in the two macro calculation modules (24) will be added by current and then converted to binary by the analog-digital converter (2); The current addition, that is, the current convergence, realizes the addition operation, and the current addition is a mature technology, which will not be described here.

[0073] Similarly, the same bit analog product (20) of the result of the multiplication of the hundred digit of the multiplicand and the multiplier will also be directly added and converted to binary by the analog-digital converter (2). In the macro calculation module (24), the results of the multiplication of the single digit and the ten digit of the multiplicand and the multiplier will generate two cross product currents, the cross product currents will be added by current inside the macro calculation module (24) to form the cross analog and product (13) and then output from the macro calculation module (24), and added with the cross analog and product (13) output from the other macro calculation module (24), and finally sent to the analog-digital converter (2) to convert to binary.

[0074] After the final current on each digit is converted to a digital signal by each analog-digital converter (2), it is input to the weighted calculator (3), and in this embodiment, the matrix operation of two pairs of decimal two-digit numbers is used, so in the weighted calculator (3), the traditional digital multiplier and adder structure can be used, the ten digit analog-digital conversion result data is multiplied by 10, and then added with the single digit data to generate the final digital calculation result (4).

[0075] The embodiment is the most intuitive and the most accurate implementation case of the application. The nine longitudinal and transverse paths of the analog multiplier (1) in Figure 5 can realize multiplication within 9*9, meeting the requirements of one-digit decimal multiplier and multiplicand. If the number of longitudinal and transverse paths is changed, multiplication of different sizes can be realized to meet the calculation requirements of different bases. If the number of longitudinal and transverse paths is both 1, binary multiplication of one digit can be realized. If the number of longitudinal and transverse paths is both 7, octal calculation of one digit can be realized. The number of longitudinal and transverse paths can also be different, meeting more flexible algorithm requirements.

[0076] The two-digit multiplication of the macro calculation module (24) in Figure 7 in the embodiment can also be expanded to multiplication of any number of digits according to actual requirements. Only the analog multiplier (1) needs to be expanded into an array according to data requirements. One analog multiplier (1) processes multiplication of one digit of data. The number of the largest multiplicand and multiplier corresponds to the number of analog multipliers (1). The corresponding current addition of the cross analog of each digit in Figure 4 and the product (13) and the same analog product (20) can be realized.

[0077] Embodiment 2

[0078] Referring to Figure 8, the same overall calculation architecture and macro calculation module (24) as in Embodiment 1 are adopted in the embodiment, but the digit separator (5) converts the input multiplicand signal into analog signals represented by the strength of multiple paths. For example, the two-digit decimal multiplicand 58 can be represented by 0.5V and 0.8V voltages on two paths. Each analog multiplier (1) will accept the voltage on each path output by the multiplicand digit (15). The digit separator (5) separates the input multiplier signal into digits represented by digital signals. The signal of 1 on the bus of the digit separator (5) will open the multiplication switch (11) in the corresponding multiplier digit (14) to generate the corresponding number of mirror current paths. The current intensity on each path is proportional to the voltage size on the path. The product result current (21) is obtained. Then, the currents of various outputs are added according to the same principle as in Embodiment 1 to realize matrix multiplication calculation of any number of digits and any base. Nine transverse paths are adopted in the embodiment. The maximum analog current multiplication is nine. The number of transverse paths can be adjusted according to actual conditions, combined with the quantization number of analog current, to realize multiplication of any size of multiplicand and multiplier, meeting the requirements of different algorithms.

[0079] In the embodiment, the digit separator (5) converts the input multiplicand signal into multiple analog currents. The analog multiplier (1) performs direct multiplication calculation on the analog currents, which is simpler and more efficient in circuit structure.

[0080] Example 3

[0081] Referring to Fig. 9, in this embodiment, the multiplier digit (14) is modified to output a three-bit binary code. In the analog multiplier (1), when the multiplication switch amplifier (22) of the highest bit is turned on, the current generated by the multiplier digit (15) is amplified by a factor of 4, when the multiplication switch amplifier (22) of the middle bit is turned on, the current generated by the multiplier digit (15) is amplified by a factor of 2, and the low bit uses the multiplication switch (12) without amplification in Example 2. Three horizontal paths generate currents with different binary weights, and the final product current (21) is obtained. Similarly, the three-bit binary code can be arbitrarily extended to multiple bits, combined with the quantized number of analog current, to realize the multiplication of any size of the multiplier and the multiplier, and meet the needs of different algorithms. The rest of this embodiment is consistent with Example 2.

[0082] The advantage of this embodiment is that the internal circuit structure of the analog multiplier (1) can be further simplified, but the calculation accuracy will be slightly reduced.

[0083] Example 4

[0084] Referring to Fig. 10, in this embodiment, in the analog multiplier (1), the multiplier digit (14) is represented by an analog signal, and the multiplier digit (14) is also represented by an analog signal. The analog multiplication switch (25) is a memory switch device such as a memistor and a ferroelectric memory. The multiplier digit (14) will turn on the analog multiplication switch (25) to realize the multiplication operation of two analog signals and obtain the product current (21). The rest of the external part is completely consistent with the above embodiment.

[0085] The advantage of this embodiment is that the internal circuit of the analog multiplier (1) is the most simplified, but since pure analog devices are used, the calculation accuracy is lower.

[0086] Example 5

[0087] Referring to Fig. 11, in this embodiment, there are a large number of data in the matrix that need to be multiplied. At this time, the current generated by multiple digit products will have a large range of variation, which requires a high precision of the analog-to-digital converter (9) and reduces the calculation accuracy. In this embodiment, multiple sets of macro calculation modules (24) are used for parallel calculation. In this embodiment, nine macro calculation modules (24) are used for parallel calculation in groups of three bits. The calculation of the sum of the currents of each digit is converted into a binary code by the respective analog-to-digital converter (2) and sent to the respective weighted calculator (3) for calculation to obtain the matrix multiplication result of each group. The calculation results of each group output by the weighted calculator (3) are sent to the second-order adder (31) for further addition to obtain the final calculation result (4). Meanwhile, the number of stages of the adder can also be increased according to the actual size in a tree structure.

[0088] This embodiment is suitable for multiplication of large matrix. By parallel grouping, the problem of precision reduction caused by current and large variation range of too many digit products can be avoided.

[0089] Embodiment 6

[0090] Referring to Fig. 12, this embodiment only uses one macro calculation module (24), but adds clock (50) control. When the clock arrives, the macro calculation module (24) will complete multiplication calculation on the current input multiplicand and multiplier. After processing by the analog-digital converter (2) and the weighting calculator (3), the result is sent to the accumulation register (32) which is also controlled by the clock. The accumulation register (32) will save the current calculation result. When the next clock arrives, the macro calculation module (24) will complete multiplication calculation on the next group of multiplicand and multiplier. After processing by the analog-digital converter (2) and the weighting calculator (3), the result is sent to the accumulation register (32) to add to the saved data of the last time. In this way, the third multiplication operation will be completed at the next clock, and the result will be added to the accumulated data before. In this way, multiplication and addition operation of any size can be realized.

[0091] This embodiment changes the parallel calculation in embodiment 5 to clock-controlled pipeline serial calculation. Therefore, this embodiment can realize large-scale matrix calculation in a smaller circuit size, but the calculation speed will be affected. This embodiment and embodiment 5 can be combined to realize reasonable combination of serial and parallel and pipeline structure according to actual application, to meet the needs of different application occasions.

[0092] Embodiment 7

[0093] Referring to FIG. 13, this embodiment shows how to use multiple two-bit binary matrix calculation units (55) to complete the multiplication and addition calculation of matrices of any bit number and any dimension. The analog multiplier (1) of this embodiment only completes one-bit binary multiplication, that is, the AND logic operation of two binary inputs is performed to determine whether to output an analog current. All multipliers M1-Mk and all multipliers N1-Nk are two-bit binary numbers, and the position separator (5) exchanges the positions of each two-bit multiplier without operating on the two-bit multipliers. Therefore, the bit data (51) on each bit of the two-bit multipliers and the two-bit multipliers are multiplied by each other through the analog multiplier (1), and then the matrix analog addition is completed and sent to the analog-to-digital converter (2) of each bit to be converted into binary. Since this embodiment uses binary, the weighting calculator (3) can complete the weight multiplication operation through binary shifting when performing the weighting operation, and then the traditional digital adder can complete the final addition operation to obtain the final binary result. Therefore, the two-bit binary matrix calculation unit (55) can output the addition operation of multiple two-bit binary products. On this basis, the two-bit binary matrix calculation unit (55) can be topologically connected in the manner shown in FIG. 14 to realize the addition operation of multiple four-bit binary products and obtain a four-bit binary matrix calculation unit (56); as shown in FIG. 15, the four-bit binary matrix calculation unit (56) can be topologically connected to realize the addition operation of multiple eight-bit binary products and obtain an eight-bit binary matrix calculation unit (57). In this way, the topology is expanded according to this manner, and the 64-bit binary multiplication and addition for ChatGPT3.5 can be realized and can continue to be expanded. ChatGPT3.5 uses a 512-dimensional token vector, and if the precision of the analog-to-digital converter (2) is not sufficient to meet the analog-to-digital conversion requirement after 512 analog quantities are added, that is, k in Mk and Nk is 512, then the mode conversion can be performed after 256 analog quantities are added to obtain a binary digital intermediate result (43), and then a standard digital adder (32) is used to perform pure digital addition calculation to obtain the final result, as shown in FIG. 16.

[0094] Through topology expansion, this embodiment can realize binary multiplication of bit number and vector addition of any dimension, and can meet the matrix multiplication calculation requirement of any scale.

[0095] Embodiment 8

[0096] Referring to FIGS. 17, 18, 19, and 20.

[0097] The high-speed matrix calculation circuit of the embodiment comprises M+N-1 addition modules and a predetermined number of macro calculation modules, each of which comprises M row lines and N column lines arranged in an equivalent orthogonal manner; a multiplier is arranged at the intersection of the row line and the column line to form a multiplier array comprising M*N multipliers; each multiplier has a first input end, a second input end and an output end, wherein the first input end is connected with the row line at the position of the multiplier, and the second input end is connected with the column line at the position of the multiplier; M and N are predetermined natural numbers, U[M-x][N-y] represents the multiplier at each position, x and y are integers and satisfy 0≤x≤M-1 and 0≤y≤N-1;

[0098] In each macro calculation module, the output ends of the multipliers satisfying x+y=S are connected to the input end of the addition module with serial number S, S is each integer in the closed interval [0, M+N-2], and the serial number of the addition module is unique;

[0099] According to the serial number, the output ends of the addition modules are connected to different bit weight input ends of the bit weight addition circuit in a one-to-one corresponding manner, and the bit weight addition circuit has an output end;

[0100] The addition module is used to implement the addition calculation of the input quantity, the multiplier is used to implement the multiplication calculation of the input quantity, and the bit weight addition circuit is used to implement the addition calculation of the input quantity at each bit weight input end according to the bit weight.

[0101] The bit weight values of each bit weight input end of the bit weight addition circuit are sorted from small to large, and the serial numbers thereof are incremented from 0 and unique; the addition module and the bit weight input end are connected in the same serial number.

[0102] The output of the multiplier is an analog quantity, and the input and output of the addition module are both analog quantities. The analog quantity is a current value. The multiplier has an analog quantity input end and a digital quantity input end.

[0103] As a more specific example, the macro calculation module 24 in FIG. 4 has 3*3=9 multipliers, and the arrangement of the 9 multipliers can be equivalent to a 3*3 array, which is suitable for multiplication calculation of two three-digit numbers in decimal. For example, the two inputs A and D of the macro calculation module shown by the dashed box need to satisfy 0≤A≤999, 0≤D≤999 and A and D are integers. Obviously, it can be extended to the case of N*N. In FIG. 4, the vertical downward thick arrow represents the addition module of the embodiment.

[0104] In fact, the present application can be applied regardless of whether the number of the multiplier and the multiplicand is the same or not. Referring to FIG. 17 and FIG. 18, the row line is taken as the input of A and the column line is taken as the input of D. FIG. 17 shows the distribution of the multipliers when A is a 4-digit number and D is a 3-digit number. As in FIG. 4, the circles in FIG. 17 represent the multipliers. The representation of FIG. 17(a) is the same as that of FIG. 14, and FIG. 17(b) is obtained by rotating the representation of FIG. 17(a) clockwise by 45°. At this time, the 3*5 array can be used to represent the multipliers (M=3, N=5). Referring to FIG. 17(c), a two-dimensional array U[M-x][N-y] is used to represent the multipliers at different positions. The multiplier at the lower right corner is U[M-0][N-0], i.e., U[3, 5], and the multiplier at the upper left corner is U[M-2][N-4], i.e., U[1, 1]. The other multipliers can be represented in the same way. FIG. 17(c) and FIG. 18(c) show the coordinates of the multipliers in a simplified form. For example, the multiplier at the lower right corner is represented by the divided row M-0 and the divided column N-0, i.e., U[M-0][N-0].

[0105] The outputs of the multipliers satisfying x+y=S are connected to the inputs of the addition module with the serial number S, and S is each integer in the closed interval [0, M+N-2]. The serial numbers of the addition modules are unique. The outputs of the multipliers connected by the same dashed line in FIG. 18 are added together, and the result is taken as one output of the macro calculation module. The output will be calculated by the addition module and then taken as the input of the subsequent bit weight addition circuit. For example, U[M-0][N-0] is taken as the input of the bit weight 10 0 . The outputs of U[M-3][N-0], U[M-2][N-1], U[M-1][N-2], and U[M-0][N-3] will be added together with the outputs of the same serial number of the other macro calculation modules by the addition module, and the result is taken as the input of the bit weight 10 3 . Referring to the form of U[M-x][N-y], for each integer S in the closed interval [0, M+N-2], the outputs of the multipliers satisfying the coordinate relationship x+y=S will be connected to the same addition module, i.e., the addition module with the serial number S.

[0106] FIG. 19 and FIG. 20 show the case of 4 rows and 8 columns (M=4, N=8). In the same way, the present application can be applied to more rows and more columns.

Claims

1. A high-speed matrix computing circuit, characterized in that, It includes an M+N-1 number of addition modules and a predetermined number of macro calculation modules. Each macro calculation module includes M rows and N columns arranged orthogonally. Multipliers are set at the intersections of the rows and columns, forming a multiplier array consisting of M×N multipliers. Each multiplier has a first input terminal, a second input terminal, and an output terminal. The first input terminal is connected to the row line where the multiplier is located, and the second input terminal is connected to the column line where the multiplier is located. M and N are predetermined natural numbers, and U[Mx][Ny] represents the multiplier at each position. x and y are both integers and satisfy 0≤x≤M-1 and 0≤y≤N-1. In each macro calculation module, the output of the multiplier that satisfies x+y=S is connected to the input of the addition module with the sequence number S, where S is each integer in the closed interval [0, M+N-2], and the sequence number of the addition module is unique. According to the sequence number, the output terminals of each addition module are connected to the different bit weight input terminals of the bit weight addition circuit in a one-to-one correspondence manner. The bit weight addition circuit has an output terminal. The addition module is used to perform addition calculations on the input quantities, the multiplier is used to perform multiplication calculations on the input quantities, and the bit weight addition circuit is used to perform addition calculations on the input quantities at each bit weight input terminal according to the bit weight.

2. The high-speed matrix computing circuit as described in claim 1, characterized in that, The bit weights of each bit weight input terminal of the bit weight adder circuit are sorted from smallest to largest, and their serial numbers are unique; the adder module is connected to the bit weight input terminals in the same serial number manner.

3. The high-speed matrix computing circuit as described in claim 1, characterized in that, The input and output of the addition module are both analog quantities.

4. The high-speed matrix computing circuit as described in claim 2, characterized in that, The analog quantity is a current value.

5. The high-speed matrix computing circuit as described in claim 1, characterized in that, The multiplier has an analog input terminal and a digital input terminal. The analog input terminal is used to input the multiplicand, and the digital input terminal is used to input the multiplier. The output of the multiplier is an analog signal.

Images