Circuit structure and calculation method for data sharing between computing units

Abstract

This invention discloses a circuit structure that enables data sharing between computing units, a method for optimizing computing performance and energy efficiency across multiple computing units using this structure, and a scalable computing system based on the combination of these multiple computing units. This invention enables data reuse and sharing during computing, reducing repeated data read and write operations, thereby improving computing system energy efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of integrated circuits, and more specifically, relates to the system architecture and working mode of integrated circuit computing chips. Background Art

[0002] With the development of the Internet of Things (IoT) and cloud computing, the complexity of computing tasks is increasing, placing greater pressure on computing chips. Key performance indicators for computing chips can be categorized as power consumption and computing power. Power consumption is crucial in both energy-constrained scenarios like the IoT and energy-intensive scenarios like cloud computing. Therefore, reducing chip power consumption while maintaining a given computing power is a key research topic for computing chips.

[0003] Computing chip power consumption primarily comes from two operations: data reading and writing, and data computation. Specific computations, such as the inference and training involved in deep neural networks, require a large number of data read and write operations. The target memory for data read and write can be divided into on-chip and off-chip storage. Since on-chip storage widely uses static random access memory (SRAM), its density is relatively low, and therefore its storage capacity is often relatively small. When encountering large numbers of data read and write operations, off-chip storage must be accessed. Examining data read and write operations, the charging and discharging of parasitic capacitance around interconnects is the primary source of energy consumption. Accessing data from off-chip storage through larger interconnects and pins results in significantly higher energy consumption than on-chip storage. To address this issue, maximizing data storage on on-chip storage and further reducing access to on-chip storage are key strategies for optimizing computing chip energy efficiency.

[0004] The integrated storage and computing architecture is one specific implementation. By constructing high-density memristor-based computing units, this architecture can partially solidify computational data within the units, reducing data access operations. Furthermore, because it uses analog computing, it offers robustness in data storage, enabling larger-scale on-chip storage. A typical computational approach involves inputting a voltage vector V via a digital-to-analog converter (DAC) to one side of a memristor array, whose conductance matrix G is represented by a matrix. By measuring the output current vector I on the other orthogonal side, this allows for matrix-vector multiplication: I = GV. In neural network inference scenarios, G corresponds to network weights and is updated less frequently, while V corresponds to data feature maps and is updated more frequently. Because the DACs themselves require data writing, frequent DAC updates contribute significantly to energy consumption. Furthermore, if serial control is used, sequentially updating hundreds of DACs in the array consumes significant computational time. Therefore, reducing DAC data updates will significantly improve computing system energy efficiency. Summary of the Invention

[0005] The purpose of the present invention is to address the problem of difficulty in data reuse in current computing systems, and to propose a circuit structure that can realize data sharing between computing units, a working method that uses this structure to optimize the computing performance and energy efficiency between multiple computing units, and a scalable computing system based on the combination of the above multiple computing units.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] In a first aspect, a circuit structure for sharing data between computing units is provided, wherein the circuit structure has a data port;

[0008] The data port can provide data to multiple computing units simultaneously, and multiple computing units can access the data and perform calculations simultaneously;

[0009] The data ports can be independently configured, that is, each data port can be connected to the computing unit in a different manner;

[0010] The connection relationship corresponds to a computing operation in which input data is reused;

[0011] The circuit can cover part or all of the calculations in multiple calculation operations within one clock cycle.

[0012] Furthermore, the data port provides data to the computing unit in a wired manner, and the computing unit receives the data in a wired manner without buffering or storing the data.

[0013] Furthermore, the data port is a digital-to-analog converter, the data is an analog signal, and the analog signal output form of the digital-to-analog converter is time, charge, current or voltage.

[0014] Furthermore, the calculation unit outputs a calculation result signal in an analog form, and the calculation result signals output by multiple calculation units are further output after being superimposed.

[0015] Furthermore, the computing unit implements the computing process by analog computing, and the output computing result signal is given by an analog circuit; the output form of the data port is voltage, the analog computing is implemented by a memristor, and the output computing result signal is in the form of current.

[0016] In a second aspect, a method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure described in the first aspect is provided, the method comprising:

[0017] For a fixed calculation process, it is expressed as follows:

[0018] Y=g(f1(A1,B1),f2(A2,B2),…,f i (A i ,B i ),…,f N (A N ,B N ))

[0019] Where N is the number of computing units involved in the calculation, {A1, A2, ..., A N} is the input data set, {B1,B2,…,B N} is the weight data set, A i is the input data of computing unit i, B i is the weight data stored in computing unit i; f i is the calculation process implemented by calculation unit i, and g is the calculation process between the output results of multiple calculation units;

[0020] Define using N input data A i Among all the unions that can be constructed, the largest set that satisfies the condition of no identical elements is the non-overlapping union set C, and the length of the non-overlapping union set C is recorded as N. C ;

[0021] The above calculation process is mapped to the circuit structure in the following manner:

[0022] Map C to N C On a data port;

[0023] Provide the data port corresponding to A1 to the first computing unit, which is the computing unit containing the weight data B1;

[0024] The data port corresponding to A2 is provided to the second computing unit, which is the computing unit containing the weight data B2;

[0025] And so on, A1, A2, ..., A N Provided to N computing units;

[0026] The output results of N computing units are used to calculate g, and the result Y is output.

[0027] Furthermore, the calculation process f i is a convolution operation, the calculation process g is point-by-point addition, C is the input feature map, and B i is the convolution kernel, the Y is the input feature map C and N convolution kernels B i The convolution calculation results between .

[0028] Furthermore, the circuit structure performs three consecutive convolution operations at a time, and each data port is connected to three computing units and data is shared among the three computing units.

[0029] Furthermore, there are more than two configurations of the connection relationship of the circuit structure; in the working mode, the above connection configuration is periodically used during calculation to connect some data ports to different computing units that need the data without refreshing the data.

[0030] In a third aspect, a scalable computing system based on the circuit structure of the first aspect is provided, wherein the system adopts multiple circuit structures;

[0031] Data exchange can be performed between the plurality of circuit structures;

[0032] The data ports of the plurality of circuit structures can be configured independently.

[0033] The beneficial effects of the present invention are: the present invention can realize the reuse and sharing of data in the calculation process, reduce the repeated and repetitive reading and writing operations on data, and thus improve the energy efficiency of the computing system.

[0034] Those skilled in the art will recognize additional features and advantages upon reading the following detailed description, and upon viewing the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0036] Figure 1 A circuit is shown that can realize data sharing of computing units between two 3*2 arrays;

[0037] Figure 2 A circuit is shown that can realize data sharing of computing units between two 3*2 arrays;

[0038] Figure 3 A circuit is shown that can realize data sharing of computing units between two 3*2 arrays;

[0039] Figure 4 A circuit is shown that can realize data sharing of computing units between two 3*2 arrays;

[0040] Figure 5 A computing mode based on a memristor array is shown;

[0041] Figure 6 A 2*2 computing array and its computing circuit are shown;

[0042] Figure 7 The input feature map and convolution kernel of the convolutional neural network are shown;

[0043] Figure 8 The input feature map of the convolutional neural network shows the calculation process between three rows and a convolution kernel;

[0044] Figure 9 The figure shows a working method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure proposed in this application, specifically, the calculation process between three rows and a convolution kernel in the input feature map of a convolutional neural network;

[0045] Figure 10 The figure shows a working method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure proposed in this application, specifically, the calculation process between three rows and a convolution kernel in the input feature map of a convolutional neural network;

[0046] Figure 11 The figure shows a working method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure proposed in this application, specifically, the calculation process between three rows and a convolution kernel in the input feature map of a convolutional neural network;

[0047] Figure 12 The figure shows a working method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure proposed in this application, specifically, the calculation process between three rows and a convolution kernel in the input feature map of a convolutional neural network;

[0048] Figure 13 The figure shows a working method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure proposed in this application, specifically, the calculation process between three rows and a convolution kernel in the input feature map of a convolutional neural network;

[0049] Figure 14 The input feature map of the same depth of the convolutional neural network shows the calculation process between three rows and three convolution kernels;

[0050] Figure 15 The present invention illustrates a method for optimizing computing performance and energy efficiency among multiple computing units using the circuit structure proposed in this application. Specifically, the computation process between three rows and three convolution kernels of an input feature map of the same depth in a convolutional neural network is shown. Figure 16 The calculation process between three rows and nine convolution kernels in the input feature map of different depths of the convolutional neural network is shown;

[0051] Figure 17 The figure shows a method for using the circuit structure proposed in this application to optimize the computing performance and energy efficiency among multiple computing units, specifically, the calculation process between three rows and nine convolution kernels of the input feature map of different depths of the convolutional neural network;

[0052] Figure 18 Shows a circuit structure and working mode of different column summation;

[0053] Figure 19 A 3*3 scalable computing system based on the combination of the above-mentioned multiple computing units is shown;

[0054] Figure 20 A 3*3 scalable computing system based on the combination of the above-mentioned multiple computing units is shown. DETAILED DESCRIPTION

[0055] In the following specific embodiments, reference is made to the accompanying drawings which form a part of the present invention, wherein the specific embodiments in which the present invention can be implemented are shown by way of diagrams. It should be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. For example, features shown or described for one embodiment may be used or combined with other embodiments to produce another embodiment. It is intended that the present invention include these modifications and variations. Specific language (which should not be interpreted as limiting the scope of the appended claims) is used to describe the embodiments. The accompanying drawings are not drawn to scale and are for illustrative purposes only.

[0056] The technical problem that this invention aims to solve is how to reduce the number of data read and write operations in a given computational process. To address this problem, the present invention decomposes the computational process and analyzes its common data, creatively proposing a computational method that transfers the same data port to multiple computational units.

[0057] To achieve this, the present invention needs to further address the problem of how to enable multiple computing units to share the same data port through circuit structure. To address this problem, the present invention creatively constructs a signal connection structure that matches the computing mode to achieve data transmission in a specified mode.

[0058] refer to Figure 1 As shown, the first embodiment of the present application proposes a circuit capable of realizing data sharing of computing units between two 3*2 arrays, including:

[0059] Digital signal input ports, 101, 102, 103;

[0060] Computing units arranged in an array, such as 111, 112, 113, etc.;

[0061] routing module, 104, 105;

[0062] First type signal lines 11A, 11B, and 11C connecting the digital signal input port to the computing unit under the allocation of the routing module;

[0063] A second type of signal line connecting different computing units, such as 119;

[0064] A digital signal output port, such as 117, 118, etc., for receiving output data of the computing unit via a connection of a second type of signal line;

[0065] Specifically, data is shared between array 110 and array 120 .

[0066] refer to Figure 2 As shown, the second embodiment of the present application proposes a circuit capable of realizing data sharing of computing units between two 3*2 arrays, including:

[0067] Digital-to-analog converter input ports, 201, 202, 203;

[0068] Computing units arranged in an array, such as 211, 212, 213, etc.;

[0069] routing module, 204, 205;

[0070] First type signal lines 21A, 21B, and 21C connecting the digital-to-analog converter input port and the computing unit under the allocation of the routing module;

[0071] A second type of signal line connecting different computing units, such as 219;

[0072] An analog-to-digital converter output port, such as 217 and 218, that receives output data of the computing unit via a connection of a second type of signal line;

[0073] Specifically, data is shared between array 210 and array 220 .

[0074] refer to Figure 3 As shown, the third embodiment of the present application proposes a circuit capable of realizing data sharing of computing units between two 3*2 arrays, including:

[0075] Digital-to-analog converter input ports, 301, 302, 303;

[0076] Computing units arranged in an array, such as 311, 312, 313, etc.;

[0077] First type signal lines 31A, 31B, and 31C connecting the digital-to-analog converter input ports to the computing units in the array 310 according to a predetermined circuit design;

[0078] First type signal lines 32A, 32B, and 32C connecting the digital-to-analog converter input ports to the computing units in the array 320 according to a predetermined circuit design;

[0079] A second type of signal line connecting different computing units, such as 319;

[0080] An analog-to-digital converter output port, such as 317, 318, etc., that receives output data of the computing unit via a connection of a second type of signal line;

[0081] Specifically, data is shared between array 310 and array 320 .

[0082] refer to Figure 4 As shown, the fourth embodiment of the present application proposes a circuit capable of realizing data sharing of computing units between two 3*2 arrays, including:

[0083] Digital-to-analog converter input ports, 401, 402, 403;

[0084] Computing units arranged in an array, such as 411, 412, 413, etc.;

[0085] A gating unit, such as 404, 405, and 407, that controls whether the digital-to-analog converter input port is connected to the corresponding computing unit in a certain array;

[0086] The first type of signal line for controlling the gating unit, such as 41D;

[0087] Second type signal lines, such as 41A, 41B, 41C, etc., connecting the digital-to-analog converter input ports to the computing units in the array 410 according to the control of the gating unit;

[0088] A third type of signal line connecting different computing units, such as 419;

[0089] An analog-to-digital converter output port, such as 417, 418, etc., that receives output data of the computing unit via a connection of a third type of signal line;

[0090] Specifically, data is shared between array 410 and array 420 .

[0091] In the above embodiment, the calculation unit may be as follows: Figure 5 、 6 The memristor shown in Figure 1 is a Figure 5As shown in , the memristor array implements multiplication through resistance characteristics (Ohm's law) and addition through current continuity. Figure 6 As shown, for a 2*2 memristor array, during the calculation process, the voltages of the signal lines 61A and 61B are fixed at 0V by the operational amplifiers 617 and 618, and the voltage is sampled through the feedback loop of the resistor.

[0092] refer to Figure 7-18 As shown, the fifth embodiment of the present application proposes a working method using the circuit structure proposed in the present application to optimize the computing performance and energy efficiency among multiple computing units. The working method includes:

[0093] like Figure 7 As shown, in the convolutional neural network, the input feature map 710 has a width of W, a height of H, and a depth of D, and the convolution kernel 720 has a width of K, a height of K, and a depth of M. The process of convolution kernel (variable filter) convolving the input feature map (variable input) and obtaining the output feature map (variable output) can be represented as multiple loops as shown below:

[0094]

[0095]

[0096] for Figure 7 The three rows 711 of the input feature map data and one layer of data 721 in the convolution kernel are used as the corresponding operations. Figure 8 As shown. The numbers represent the input feature map data, and the letters represent the convolution kernel data. Furthermore, as in the standard convolution definition, overlapping data indicates corresponding multiplication, and after the product is completed, all data needs to be added. In the first clock cycle, the corresponding relationship is shown in 810; in the second clock cycle, the corresponding relationship is shown in 820; and so on. 810 to 890 show all the calculation processes.

[0097] like Figure 9 As shown, consider the input feature map data 910 and the convolution kernel data 920. The circuit structure 930 shows a data sharing method between computing units. Among them, the input data port 932 is shared by three computing arrays 931, 933, and 934. The specific sharing method is represented by arrows, and its physical correspondence is the interconnection line in the integrated circuit. For example, input 7 is shared by the computing unit storing A in 931, the computing unit storing G in 933, and the computing unit storing D in 934. Its basic principle is that, as Figure 8As shown, calculation cycles 810, 820, and 830 all use data 7. In calculation cycle 810, data 7 is multiplied by data G, in calculation cycle 820, data 7 is multiplied by data D, and in calculation cycle 830, data 7 is multiplied by data A. Since data 7 is used three times, efficiency can be improved by sharing the data with the three calculation units simultaneously instead of reading data 7 three times separately.

[0098] In some embodiments, as Figure 10 As shown, the routing of interconnects can be simplified by calculating the changes in array position.

[0099] Furthermore, if Figure 11 As shown, during a complete convolution process involving input data 1100 and convolution kernel 1110, the computation process is implemented by updating the input data port state for a given interconnection relationship and data within the computational unit. In computation cycle 1120, under input data port state 1121, computation operations for computation cycles 810, 820, and 830 are implemented; in computation cycle 1130, under input data port state 1131, computation operations for computation cycles 840, 850, and 860 are implemented; and in computation cycle 1140, under input data port state 1141, computation operations for computation cycles 870, 880, and 890 are implemented. As can be seen, the input data port undergoes a total of 15 times 3 refreshes, or 45 times. Traditional computation methods, on the other hand, require a data refresh for each computation, which translates to 9 times 9 refreshes, or 81 times.

[0100] In some embodiments, as Figure 12 As shown, consistent data storage can be achieved in the computing unit by changing the routing of the interconnection lines.

[0101] Furthermore, if Figure 13 As shown, by defining two data-sharing interconnect routing structures, the number of data load and refresh times can be further reduced. Specifically, in computation cycle 1320, data imports were performed 15 times. In computation cycle 1330, by switching the data-sharing interconnect routing structure, data 10 to 15 were reused, resulting in only 9 data port refreshes. In computation cycle 1340, by switching the data-sharing interconnect routing structure, data 19 to 24 were reused, resulting in only 9 data port refreshes. In summary, a total of 33 data port refreshes were performed, a further reduction from the 45 previously mentioned. Furthermore, computation cycles 1320 and 1340 used the same data-sharing interconnect routing structure. This demonstrates that this approach corresponds to periodic interconnect routing structure switching, which in turn corresponds to a larger input feature map dimension. As computation proceeds, only the two interconnect routing structures described above are required to achieve the aforementioned advantages.

[0102] like Figure 14 As shown, for the input feature map data 1412 of the same depth (1412, 1422, 1432 are exactly the same), it needs to perform convolution calculations with the corresponding multiple convolution kernels 1411, 1421, 1431 respectively.

[0103] like Figure 15 As shown, this process can be achieved by constructing the convolution kernel data in the same computational array. In the same computational array 1540, computational units in the same row naturally share data through the source lines of that row. For example, A1 connected to bit line 1541, A2 connected to bit line 1542, and A3 connected to bit line 1543 can share data 1 in data port 1530 through source line 1531 in the same row.

[0104] like Figure 16 As shown, for input feature map data 1612 of different depths (1612, 1622, 1632 are different), the input feature map data of each depth needs to be convolved with the corresponding multiple convolution kernels. As shown in the above pseudo code, the output data of different depths needs to be further summed. Specifically, for a neural network convolution layer with an input dimension of 3 and an output dimension of 3, 3 input feature maps and 9 convolution kernels are used for calculation, and 3 feature maps are output to the next layer of the network.

[0105] like Figure 17 As shown, this process can be implemented by constructing the convolution kernel data in the same computing array and further stacking the input data ports 1731, 1732, and 1733.

[0106] In some embodiments, when the input feature maps of the same depth are convolved with different convolution kernels and the results need to be added, the convolution operation can be performed as follows: Figure 18 Specifically, in the structure of 1842, the calculation can be performed column by column and then added, or in the structure of 1852, analog addition can be performed directly by constructing connected bit lines and then sampling and outputting.

[0107] refer to Figure 19 As shown, the sixth embodiment of the present application proposes a 3*3 scalable computing system based on the combination of the above-mentioned multiple computing units, including:

[0108] Control and input data buffer 1941;

[0109] Control and output data buffer 1942;

[0110] a communication signal line 1943 between the control and input data buffer 1941 and the control and output data buffer 1942;

[0111] Multiple computing units 1911 to 1919 sharing a data port;

[0112] Buses 1921 , 1922 , 1931 and 1932 between multiple computing units 1911 to 1919 sharing data ports;

[0113] The system can be expanded by further increasing the number of computing arrays in two or three dimensions.

[0114] In some instances, such as Figure 20 As shown, a richer control logic and data interaction can be achieved by using routing to index signals between buses 2021, 2022, 2031, and 2032. In addition, 2041 can achieve both input and output.

[0115] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0116] The above description is only a preferred embodiment of the present invention. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present invention without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment with equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A circuit structure for data sharing between computing units, characterized in that: The circuit structure has a data port; The data port can provide data to multiple computing units simultaneously, and multiple computing units can access the data and perform calculations simultaneously; The data ports can be independently configured, that is, each data port can be connected to the computing unit in a different manner; The connection relationship corresponds to a computing operation in which input data is reused; The circuit can cover part or all of the calculations in multiple calculation operations within one clock cycle.

2. The circuit structure according to claim 1, wherein: The data port provides data to the computing unit in a wired manner, and the computing unit receives the data in a wired manner without buffering or storing the data.

3. The circuit structure according to claim 1, wherein: The data port is a digital-to-analog converter, the data is an analog signal, and the analog signal output form of the digital-to-analog converter is time, charge, current or voltage.

4. The circuit structure according to claim 1, wherein: The calculation unit outputs a calculation result signal in an analog form, and the calculation result signals output by multiple calculation units are superimposed and then output.

5. The circuit structure according to claim 4, characterized in that: The calculation unit implements the calculation process by analog calculation, and the output calculation result signal is given by an analog circuit; the output form of the data port is voltage, the analog calculation is implemented by a memristor, and the output calculation result signal is in the form of current.

6. A method of using the circuit structure according to any one of claims 1 to 5 to optimize computing performance and energy efficiency among multiple computing units, characterized in that: For a fixed calculation process, it is expressed as follows: Y=g(f1(A1,B1),f2(A2,B2),…,f i (A i ,B i ),…,f N (A N ,B N )) Where N is the number of computing units involved in the calculation, {A1, A2, ..., A N } is the input data set, {B1,B2,…,B N } is the weight data set, A i is the input data of computing unit i, B i is the weight data stored in computing unit i; f i is the calculation process implemented by calculation unit i, and g is the calculation process between the output results of multiple calculation units; Define using N input data A i Among all the unions that can be constructed, the largest set that satisfies the condition of no identical elements is the non-overlapping union set C, and the length of the non-overlapping union set C is recorded as N. C ; The above calculation process is mapped to the circuit structure in the following manner: Map C to N C On a data port; {A1,A2,…,A N } Provided to the corresponding computing unit according to their corresponding data ports; The output results of N computing units are used to calculate g, and the result Y is output.

7. The working method according to claim 6, characterized in that: The calculation process f i is a convolution operation, the calculation process g is point-by-point addition, C is the input feature map, and B i is the convolution kernel, the Y is the input feature map C and N convolution kernels B i The convolution calculation results between .

8. The working method according to claim 7, characterized in that: The circuit structure performs three consecutive convolution operations at a time, and each data port is connected to three computing units and data is shared among the three computing units.

9. The working method according to claim 6, characterized in that: There are more than two configurations of the connection relationship of the circuit structure; in the working mode, the above connection configuration is periodically used during calculation to connect some data ports to different computing units that need the data without refreshing the data.

10. A scalable computing system based on the circuit structure according to any one of claims 1 to 5, characterized in that: The scalable computing system: adopting a plurality of said circuit structures; Data exchange can be performed between the plurality of circuit structures; The data ports of the plurality of circuit structures can be configured independently.

Images