A memory-computing chip and a memory-computing method supporting convolution operation and sine-cosine function operation

Abstract

The application provides a memory-computing chip and a memory-computing method supporting convolution operation and sine and cosine function operation. The memory-computing chip comprises a preprocessing module, a memory-computing array module and a shift-add module. The preprocessing module is configured to control input data to be input to the memory-computing array module in the form of a single data bit, wherein the input data comprises convolution operation data and sine and cosine function data. The memory-computing array module is configured to perform multiplication and accumulation operation according to preloaded data and the input data to obtain multiplication and accumulation results of different data bits. The shift-add module is configured to perform shift and accumulation on the multiplication and accumulation results to obtain array calculation results. The convolution operation and the sine and cosine function operation are multiplexed in the memory-computing array module and the shift-add module in the hardware implementation process, thereby increasing the operation function of the traditional memory-computing chip and expanding the application scenarios that can be adapted by the memory-computing chip under the premise of sacrificing a small amount of area.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of chip design, and in particular to a memory-compute chip supporting convolution operation and sine-cosine function operation, and a memory-compute method. BACKGROUND

[0002] In the face of AI scenarios that need to process large quantities of frequent data movement, a memory-compute integrated architecture can eliminate unnecessary data movement delay and power consumption, reduce the bandwidth pressure of the chip, and use the storage unit to improve the computing power, greatly improve the computing efficiency, and reduce the cost. Therefore, memory-compute integrated computing has developed rapidly, and high-energy efficiency, low power consumption, and widely applicable memory-compute chip design are currently the focus of research in academia and even the industry.

[0003] The hierarchical structure of the array in the memory-compute chip is Tile->PE (Processing Element)->Array, and the array is composed of calculation units arranged in rows and columns. Such a large-scale and regularly arranged array structure is suitable for operations such as convolution operators that require a large amount of data movement, because the calculation unit itself can store a certain amount of data. Therefore, the memory-compute chip is mostly used for the acceleration operation of neural networks. However, the pure support of convolution operators makes the chip function relatively single, and the application scenarios are greatly limited. For example, in some application scenarios that require general computing (such as sin, cos, and other functions), the existing memory-compute chip cannot be adapted thereto.

[0004] Therefore, there is an urgent need for a memory-compute chip supporting multiple operator calculations to expand the application scenarios that the memory-compute chip can adapt to. SUMMARY

[0005] In view of the above problems, the embodiments of the present application provide a memory-compute chip supporting convolution operation and sine-cosine function operation, and a memory-compute method, so as to overcome the above problems or at least partially solve the above problems.

[0006] The first aspect of the embodiments of the present application discloses a memory-compute chip supporting convolution operation and sine-cosine function operation, comprising a preprocessing module, a memory-compute array module, and a shift-add module.

[0007] The preprocessing module is configured to control input data to be input to the memory-compute array module in the form of a single data bit, and the input data comprises convolution operation data and sine-cosine function data.

[0008] The memory-compute array module is configured to perform multiply-accumulate operation according to preloaded data and the input data to obtain multiply-accumulate results of different data bits, and the preloaded data comprises: convolution weight data and fixed-point results of sine-cosine function fixed-point data at a target index bit width.

[0009] The shift and add module is configured to perform shift and accumulation on the multiplication and accumulation result to obtain an array calculation result.

[0010] Optionally, the preprocessing module comprises a convolution preprocessing module.

[0011] The convolution preprocessing module is configured to input the convolution operation data to the compute-in-memory array module in the form of single data bit.

[0012] Optionally, the preprocessing module comprises a sine and cosine function preprocessing module and a first selection module.

[0013] The sine and cosine function preprocessing module is configured to sequentially perform an exponential interval adaptation implementation algorithm and floating point data fixed-point processing on the sine and cosine function data to obtain sine and cosine function fixed-point input data.

[0014] The first selection module is configured to determine a target array in the compute-in-memory array module in which the fixed-point result of preloading is located according to the target index bit width, and input fixed-point data other than the target index bit width of the sine and cosine function fixed-point input data to a specified position in the target array in the form of single data bit.

[0015] Optionally, the compute-in-memory chip further comprises an accumulation module and a floating point output module, and the array calculation result comprises convolution calculation results of multiple different arrays and fixed-point number results of a sine and cosine function.

[0016] The accumulation module is configured to accumulate the convolution calculation results of the multiple different arrays to obtain a convolution operation result.

[0017] The floating point output module is configured to convert the fixed-point number results of the sine and cosine function into floating point type data to obtain a sine and cosine function operation result.

[0018] Optionally, the compute-in-memory chip further comprises a second selection module, a convolution state module and a sine and cosine function state module.

[0019] The second selection module is configured to determine a target result in multiplication and accumulation results of different data bits of sine and cosine function data.

[0020] The convolution operation state module is configured to control state scheduling of a convolution operation process.

[0021] The sine and cosine function state module is configured to control state scheduling of a sine and cosine function calculation process.

[0022] Optionally, the storage-computing array module includes a first sub storage-computing array module and a second sub storage-computing array module, the first sub storage-computing array module and the second sub storage-computing array module each include a plurality of storage-computing array groups composed of computing units, all word lines of static random access memories (SRAMs) of the computing units in a same row in the storage-computing array group are connected together, and all bit lines of the SRAMs of the computing units in a same column in the storage-computing array group are connected together.

[0023] In a second aspect, a convolution storage-computing method is disclosed, applied to the storage-computing chip in the first aspect, and the method includes:

[0024] Preloading convolution weight data to the storage-computing array module;

[0025] Inputting convolution operation data in a single data bit form to the storage-computing array module and performing multiplication and accumulation operation with the preloaded convolution weight data to obtain multiplication and accumulation results of different data bits of the convolution operation data;

[0026] Performing shift and accumulation on the multiplication and accumulation results of the different data bits of the convolution operation data to obtain convolution calculation results of a plurality of different arrays;

[0027] Performing accumulation on the convolution calculation results of the plurality of different arrays to obtain a convolution operation result.

[0028] In a third aspect, a sine-cosine function storage-computing method is disclosed, applied to the storage-computing chip in the first aspect, and the method includes:

[0029] Preloading a fixed-point result of sine-cosine function fixed-point data at a target index bit width to a storage-computing array module;

[0030] Sequentially performing index interval fitting implementation algorithm and floating-point data fixed-point processing on sine-cosine function data to obtain sine-cosine function fixed-point input data;

[0031] Determining a target array of the preloaded fixed-point result in the storage-computing array module according to the target index bit width;

[0032] Inputting sine-cosine function fixed-point input data excluding the target index bit width in a single data bit form to a specified position in the target array and performing multiplication and accumulation operation with the preloaded fixed-point result to obtain multiplication and accumulation results of different data bits of the sine-cosine function data;

[0033] Determining a target result from the multiplication and accumulation results and performing shift and accumulation on the target result to obtain a fixed-point result of the sine-cosine function;

[0034] Convert the fixed-point result of the sine function into floating-point type data to obtain a sine function operation result.

[0035] Optionally, the fixed-point result includes a sine function fixed-point result and a cosine function fixed-point result; and the fixed-point data of the sine-cosine function is preloaded into the storage-computing array module at the target index bit width, including:

[0036] The sine function fixed-point result is preloaded into a first storage-computing array submodule of the storage-computing array module.

[0037] The cosine function fixed-point result is preloaded into a second storage-computing array submodule of the storage-computing array module.

[0038] Optionally, the specified position in the target array refers to a specified position in the second storage-computing array submodule; and the fixed-point data of the sine-cosine function input data, except for the target index bit width, is input into the specified position in the target array in the form of a single data bit, and is multiplied and accumulated with the preloaded fixed-point result to obtain multiplication and accumulation results of different data bits of the sine-cosine function data, including:

[0039] The sine function fixed-point result is indexed from the first storage-computing array submodule as a first term result.

[0040] The fixed-point data of the sine-cosine function input data, except for the target index bit width, is input into the specified position of the second storage-computing array submodule, and is shifted and accumulated in each cycle to obtain a second term result.

[0041] The first term result and the second term result are accumulated to obtain multiplication and accumulation results of different data bits of the sine-cosine function data.

[0042] Embodiments of the present application include the following advantages:

[0043] The embodiment of the present application provides a storage-computing chip supporting convolution operation and sine-cosine function operation, which comprises a preprocessing module, a storage-computing array module and a shift-add module; the preprocessing module is used for controlling input data to be input into the storage-computing array module in the form of a single data bit, wherein the input data includes convolution operation data and sine-cosine function data; the storage-computing array module is used for multiplying and accumulating according to preloaded data and input data to obtain multiplication and accumulation results of different data bits; and the shift-add module is used for shifting and accumulating the multiplication and accumulation results to obtain array calculation results. In the storage-computing chip, the convolution operation and the sine-cosine function operation are multiplexed in the hardware implementation process to occupy the largest resources of the storage-computing array (i.e. the storage-computing array module) and the shift-add module, so that the operation function of the traditional storage-computing chip is increased and the application scenarios that can be adapted by the storage-computing chip are expanded on the premise of sacrificing a small amount of area. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0045] Figure 1 is a structure schematic diagram of a memory-computing chip supporting convolution operation and sine and cosine function operation provided by an embodiment of the present application;

[0046] Figure 2 is a structure schematic diagram of a memory-computing array module provided by an embodiment of the present application;

[0047] Figure 3 is a structure schematic diagram of an array group provided by an embodiment of the present application;

[0048] Figure 4 is a hardware schematic diagram of a sine and cosine function hardware implementation algorithm provided by an embodiment of the present application;

[0049] Figure 5 is a structure schematic diagram of another memory-computing chip supporting convolution operation and sine and cosine function operation provided by an embodiment of the present application;

[0050] Figure 6 is a step flowchart of a convolution memory-computing method provided by an embodiment of the present application;

[0051] Figure 7 is a step flowchart of a sine and cosine function memory-computing method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0053] Referring to Figure 1 , as shown in Figure 1 is a structure schematic diagram of a memory-computing chip supporting convolution operation and sine and cosine function operation provided by an embodiment of the present application. As shown in Figure 1 , the memory-computing chip includes a preprocessing module, a memory-computing array module and a shift-add module;

[0054] The preprocessing module is configured to control input of input data in the form of single data bits to the compute-in-memory array module, the input data including convolution operation data and sine and cosine function data.

[0055] The compute-in-memory array module is configured to perform multiply-accumulate operation according to preloaded data and the input data to obtain multiply-accumulate results of different data bits, the preloaded data including fixed-point results of convolution weight data and sine and cosine function fixed-point data at a target index bit width.

[0056] The shift-add module is configured to perform shift-accumulate operation on the multiply-accumulate results to obtain array calculation results.

[0057] In the embodiments of the present application, the compute-in-memory array module refers to an array structure in a compute-in-memory chip. Specifically, the compute-in-memory array module includes a first sub-compute-in-memory array module and a second sub-compute-in-memory array module, and each of the first sub-compute-in-memory array module and the second sub-compute-in-memory array module includes a plurality of compute-in-memory array groups composed of compute units. The word lines of all static random access memories (SRAMs) of the compute units in the same row in the compute-in-memory array group are connected together, and the bit lines of all SRAMs of the compute units in the same column in the compute-in-memory array group are connected together.

[0058] In an example, Figure 2 The embodiments of the present application provide a structural diagram of a compute-in-memory array module. The scale of the compute-in-memory array module is 32x128x128. Based on the compute-in-memory array module and certain peripheral circuits, the compute-in-memory array module supports INT type convolution operation and FP32 type sine and cosine function operation. Specifically, the compute-in-memory array module includes a first sub-compute-in-memory array module (Tile1) and a second sub-compute-in-memory array module (Tile2), each of the sub-compute-in-memory array modules includes 4 PEs, and each of the PEs includes 4 arrays with a size of 128x128. By grouping 8 columns of compute-in-memory arrays as a compute-in-memory array group, that is, an array with a size of 128x128 includes 16 compute-in-memory array groups. Figure 3The structure diagram of the array group is shown. By connecting the word lines (WL) of the SRAM of the calculation units in the same row of the array group together and connecting the bit lines (BL) of all the SRAM of the calculation units in the same column together, the writing in the row-by-row manner can be realized when data is written. Specifically, when the data of the first row is written, WL[0] is set high, the BL end corresponding to each SRAM of the row flows into the data to be written, that is, the data writing of the SRAM of the row is completed in one period, and WL[0] is set to zero and WL[1] is set high in the next period to write the data of the second row. In this way, the writing of all the data of 128 rows in the array can be completed in 128 periods. In this way, each array is written in the above-mentioned manner. In the specific implementation, the convolution weight data (that is, the convolution kernel in the convolution operation) and the fixed-point data of the sine and cosine functions are preloaded in the target index bit width fixed-point result in the compute-in-memory array module according to the above-mentioned writing manner.

[0059] When the operation is performed, the pre-processing module is used to control the input data to be input into a row of an array group of the compute-in-memory array module in the form of single data bit (that is, 1 bit of data is input per calculation period). Since one SRAM unit stores 1 bit of data, each row of the array group maps at most 8 bits of weight data. By adding an AND gate beside the SRAM unit, the multiplication operation of two single-bit data is realized, and each array group is equipped with a 7-stage addition tree to realize the accumulation of the result. Further, the array group can realize the multiplication and accumulation operation of 1 bit of input data and 1-8 bits of weight data per period to obtain the multiplication and accumulation result of different data bits. Since the input data is input in the form of single data bit, the result of the multiplication and accumulation of each bit of data needs to be shifted to the right by one bit to be accumulated, and then the shift and addition module is used to complete the shift and accumulation of the multiplication and accumulation results of different data bits to obtain the array calculation result.

[0060] In the specific implementation, 1) for the convolution operation: the convolution operation data is INT8 type (that is, 8 bits) data, and the convolution weight data supports INT1-INT8 type (that is, 1-8 bits) data. Under the clock frequency of 100 MHz, the theoretical peak computing power of INT8xINT8 is 0.82 TOPS. The pre-processing module is used to input the convolution operation data in the form of single data bit into the compute-in-memory convolution module to perform the multiplication and accumulation result with the preloaded 1-8 bit convolution weight data in the compute-in-memory convolution module to obtain the multiplication and accumulation result of different data bits of the convolution operation data, and then the shift and addition module is used to shift and accumulate the multiplication and accumulation result of different data bits of the convolution operation data to obtain the convolution calculation result of multiple different arrays.

[0061] By preloading the convolution weight data to the compute-in-memory array module, only the convolution operation data needs to be input during the convolution operation, thereby reducing the data flow in the convolution operation process, effectively reducing the energy consumption, and improving the parallelism, thereby realizing the convolution operation acceleration. Meanwhile, after the weight data is preloaded to the compute-in-memory array module, the weight data can be reused. During each convolution operation, the convolution operation data is directly input to the compute-in-memory array module, and the compute-in-memory array module automatically performs the convolution operation based on the preloaded weight data. The repeated use of the weight data further reduces the data flow in the convolution operation process.

[0062] 2) For the sine and cosine function calculation: the sine and cosine function data is FP32 type data (including 1 bit sign, 8 bit exponent, and 23 bit mantissa), the pre-processing module is used to process the sine and cosine function data into sine and cosine function fixed-point input data, and based on the Taylor first-order expansion form, the calculation of the sine and cosine function data is converted into the form of two groups of multiplication and addition, and the target array of the preloaded sine and cosine function fixed-point data in the target index bit width is determined according to the target index bit width. The fixed-point result of the sine and cosine function fixed-point input data other than the target index bit width is input to the specified position in the target array to perform multiplication and accumulation operation with the preloaded fixed-point result, so as to obtain the multiplication and accumulation result of the sine and cosine function data in different data bits. Similarly, after the sine and cosine function fixed-point data in the target index bit width is preloaded to the compute-in-memory array module, the sine and cosine function fixed-point data in the target index bit width can be reused. During each sine and cosine function operation, the sine and cosine function fixed-point input data other than the target index bit width is input to the compute-in-memory array module to realize the operation. Thus, the data flow in the operation process of the sine and cosine function is reduced, and the expansion of the sine and cosine function operation on the compute-in-memory chip is realized.

[0063] In an optional embodiment, the data types of the convolution operation data and the sine and cosine function data are different, so the pre-processing processes of the convolution operation data and the sine and cosine function data are also different. Therefore, in order to realize the convolution operation, the pre-processing module includes a convolution pre-processing module, and the convolution pre-processing module is configured to input the convolution operation data to the compute-in-memory array module in the form of a single data bit.

[0064] In the embodiment of the present application, considering that the data type of the convolution operation data is INT8 type, a convolution preprocessing module is configured in the preprocessing module to control each convolution operation data to be input to the compute-in-memory array module in the form of single data bit by using the convolution preprocessing module. Specifically, the convolution operation data is input to a row in one array group in the compute-in-memory array module in the form of single data bit (1 bit), for example, the data of the first bit is controlled to be input to the first row in the array group in the first period, and the data of the second bit is controlled to be input to the first row in the array group in the second period. Then, the convolution operation data is multiplied and accumulated with the convolution weight data preloaded in the compute-in-memory array module to obtain convolution calculation results of multiple different arrays, and the shift accumulation module is used to shift and accumulate the convolution calculation results of the multiple different arrays to obtain the convolution operation result.

[0065] Further, the compute-in-memory chip further comprises an accumulation module, the array calculation result comprises convolution calculation results of multiple different arrays; and the accumulation module is configured to accumulate the convolution calculation results of the multiple different arrays to obtain the convolution operation result.

[0066] In the embodiment of the present application, the compute-in-memory array module comprises multiple 128*128 size arrays, and the convolution operation result obtained by the shift accumulation of the shift accumulation module is the multiplication and accumulation result of one array, therefore, the accumulation module is used to accumulate the convolution calculation results of the multiple different arrays to obtain the convolution operation result, that is, the final calculation result output by the compute-in-memory array module.

[0067] Further, in order to ensure the smooth progress of the convolution operation, the compute-in-memory chip further comprises a convolution state module, and the convolution operation state module is configured to control the state scheduling of the convolution operation process.

[0068] In an optional embodiment, in order to implement the sine and cosine function operation, the preprocessing module comprises a sine and cosine function preprocessing module and a first selection module.

[0069] The sine and cosine function preprocessing module is configured to sequentially perform the exponential interval adaptation implementation algorithm and the floating point data fixed-point processing on the sine and cosine function data to obtain sine and cosine function fixed-point input data.

[0070] The first selection module is configured to determine the target array in the compute-in-memory array module in which the preloaded fixed-point result is located according to the target index bit width, and input the fixed-point data of the sine and cosine function fixed-point input data except the target index bit width to a specified position in the target array in the form of single data bit.

[0071] In the embodiments of the present application, the processing of the sine and cosine functions is mainly based on the form of Taylor first-order expansion, and the calculation of the sine and cosine function data is converted into the form of two sets of multiplication and addition. Specifically, since the sine and cosine function data is FP32 type data, in order to facilitate calculation, the FP32 type floating point data needs to be converted into fixed point data. At the same time, considering that the sine and cosine function data is periodic data, different adaptive implementation algorithms can be specified in different exponential intervals, for example, the values corresponding to the sine and cosine function data in the interval of 2πn (n is an integer) can be 0, 1, -1, and in this interval (such as 0, 2πn, etc.), no adaptive implementation algorithm needs to be set, and the corresponding values can be mapped, while in other intervals, the corresponding adaptive implementation algorithm needs to be specified.

[0072] In the sine and cosine function operation, first, the sine and cosine function preprocessing module is used to sequentially perform the exponential interval adaptive implementation algorithm and floating point data fixed point processing on the sine and cosine function data to obtain sine and cosine function fixed point data, which is a 39-bit fixed point number F, and the fixed point number F includes 1-bit integer and 38-bit decimal. Secondly, the decimal value f is further reduced to 36-bit decimal f by using the induction formula, and the decimal value f is divided into the main part F[35:36-m] (f_main) and the secondary part F[35-m:0] (f_minor). Further, the Taylor first-order expansion formula can be obtained as follows:

[0073] sin(2π2 -38 f)

[0074] =sin(2π2 -(m+2) f_main)

[0075] +2πcos(2π2 -(m+2) f_main)2 -38 f_minor (1)

[0077] Wherein, m is the target index bit width. Specifically, by using the induction formula and algorithm modeling analysis (for example, using C language modeling to determine the target index bit width), the target index bit width is determined to be 13 under the premise of meeting the accuracy requirement, and the output fixed point bit width is 32 and there are 8-bit exponents, which meets the demand of high-precision sine and cosine function calculation. The simulation value and the calculation accuracy of the single-precision sine and cosine function of the C language math library are at most only 1 error unit, and through modeling statistics and analysis, the 8-bit exponents respectively exist 14 and 13 cases, so it can be directly judged.

[0078] It can be understood that taking 13-bit data F[35:23] (f_main) as an index, the fixed point result of the sine and cosine function fixed point data in the target index bit width is:-15 f_main) and 2πcos(2π2 -15 f_main) are 32-bit mantissa fixed-point results. Therefore, the sin(2π2 -15 f_main) and 2πcos(2π2 -15 f_main) are 32-bit mantissa fixed-point results preloaded into the compute-in-memory array module.

[0079] And, the target index bit width is 13 bits, the output is 32 bits, and the data amount of each item is 2 13 ×32=262144bit, i.e., 256Kb, which is exactly the amount of data that can be stored in a single compute-in-memory array sub-module in the compute-in-memory array module, so the first compute-in-memory array sub-module Tile1 corresponds to sin(2π2 -15 f_main), and the second compute-in-memory array sub-module Tile2 corresponds to 2πcos(2π2 -15 f_main), at which time the utilization rate of the compute-in-memory array is exactly 100%.

[0080] Further, the compute-in-memory chip further includes a second selection module, which is configured to determine a target result in multiplication and accumulation results of different data bits of the sine / cosine function data.

[0081] In the embodiments of the present application, considering that the specified array outputs four different multiplication results, in order to obtain a target result (i.e., a required multiplication result), the second selection module is used to output the target result. For example, for a result 1, a result 2, a result 3, and a result 4, if the target result is the result 1, the second selection module is used to output the result 1, and not to output the result 2, the result 3, and the result 4.

[0082] Further, the compute-in-memory chip further includes a floating-point output module, the array calculation result includes a fixed-point number result of the sine / cosine function; and the floating-point output module is configured to convert the fixed-point number result of the sine / cosine function into a floating-point type of data to obtain a sine / cosine function operation result.

[0083] In the embodiments of the present application, since the sine / cosine function calculation is a calculation on a fixed-point number, in order to ensure that the sine / cosine function calculation result is accurately output, the fixed-point number result needs to be converted into a floating-point type of data after the calculation is completed.

[0084] Further, in order to ensure that the sine / cosine function operation is successfully performed, the compute-in-memory chip further includes a sine / cosine function state module; and the sine / cosine function state module is configured to control state scheduling of the sine / cosine function calculation process.

[0085] For example, Figure 4A hardware schematic diagram of a hardware implementation algorithm of a sine and cosine function provided by an embodiment of the present application is shown. On the basis of an existing memory-computing chip, a small number of digital unit circuits are added, so that the memory-computing chip additionally supports FP32 type sine and cosine function operations, and the array utilization rate reaches 100%. The supported operators are increased, and the application scenarios of the memory-computing chip are expanded.

[0086] For example, Figure 5 A structure schematic diagram of another memory-computing chip supporting convolution operation and sine and cosine function operation provided by an embodiment of the present application is shown, which includes a preprocessing module, a memory-computing array module, a shift-add module, an accumulation module, a floating-point output module, a second selection module, a convolution state module and a sine and cosine function state module. The preprocessing module includes a convolution preprocessing module, a sine and cosine function preprocessing module and a first selection module. In specific implementation, for different application scenarios, convolution operation or sine and cosine function operation can be selectively used to realize the required specified module. Among them, the corresponding functional modules are selected to realize data operation in different scenarios, and for the memory-computing array (i.e. the memory-computing array module) and the shift-add module which occupy the most resources, multiplexing can be realized in the convolution operation and the sine and cosine function operation process, thereby increasing the operation function of the traditional memory-computing chip under the premise of sacrificing a small amount of area, and expanding the application scenarios that the memory-computing chip can adapt to.

[0087] An embodiment of the present application provides a convolution memory-computing method, which is applied to the memory-computing chip provided by an embodiment of the present application, and the convolution memory-computing method is shown in Figure 6 Figure 6 is a step flowchart of a convolution memory-computing method provided by an embodiment of the present application. As shown in Figure 6 The convolution memory-computing method provided by an embodiment of the present application includes steps S610 to S640:

[0088] Step S610: Preload convolution weight data to the memory-computing array module.

[0089] Step S620: Input the convolution operation data in the form of a single data bit to the memory-computing array module, and perform multiplication and accumulation operation with the preloaded convolution weight data to obtain multiplication and accumulation results of different data bits of the convolution operation data.

[0090] Step S630: Shift and accumulate the multiplication and accumulation results of different data bits of the convolution operation data to obtain convolution calculation results of multiple different arrays.

[0091] Step S640: Accumulate the convolution calculation results of the multiple different arrays to obtain a convolution operation result.

[0092] ​In the embodiments of the present application, it is considered that the process of convolution operation is a process of operation using convolution operation data and convolution weight data. First, the convolution weight data is preloaded to the compute-in-memory array module (i.e., Tile1 and Tile2), and then only the convolution operation data needs to be input during the convolution operation, thereby reducing the data flow in the convolution operation process, effectively reducing the energy consumption, and improving the parallelism.

[0093] In specific implementation, the convolution weight data supports 1-8bit, and only the corresponding column needs to be mapped in the calculation unit of each array group (i.e., the array group shown in the figure), and the remaining columns are set to zero. For example, when the convolution weight data is 1bit, since each convolution weight data only needs to occupy one SRAM unit, only the first column of the array group is mapped into the convolution weight data, and the remaining columns are set to zero. Similarly, different convolution weight data bit widths correspond to the preloaded specified column number, so that the compute-in-memory chip supports 1-8bit convolution weight data. Figure 3

[0094] After the convolution weight data preloading is completed, the convolution operation data is serially streamed into the array by bit through the convolution preprocessing module (i.e., input to the compute-in-memory array module in the form of single data bit), and the multiplication operation is performed with the data preloaded in each SRAM unit. The multiplication results of 128 rows in the array group are accumulated using the addition tree, and then the accumulation addition operation in the convolution operation is completed, wherein the bit-serial streaming mode is scheduled by the convolution state module. Then, the shift-add single module is used to shift and accumulate the multiplication and accumulation results of different data bits of the convolution operation data, obtain the convolution calculation results of multiple different arrays, and accumulate the convolution calculation results of different arrays Array, to obtain the convolution operation results, i.e., according to the specific configuration requirements of different neural networks, the multiplication and accumulation results in Tile1 and Tile2 are output, and then the complete process of accelerating the convolution operation in the compute-in-memory array is realized.

[0095] ​In the embodiment of the present application, for the convolution operation process, the convolution preprocessing module, the storage and calculation array module, the convolution state module, the shift and add module and the accumulation module in the storage and calculation chip are selectively used to realize the convolution operation. Since the convolution operation data supports INT8 type and the convolution weight data supports INT1-INT8 type, in the case of INT8xINT8, the embodiment of the present application can provide a theoretical peak computing power of 0.82TOPS. By preloading the convolution weight data to the storage and calculation array module, only the convolution operation data needs to be input during the convolution operation, thereby reducing the data flow in the convolution operation process and effectively reducing the energy consumption, and the convolution operation is accelerated. At the same time, the weight data can be reused after being preloaded to the storage and calculation array module. That is, after step S610 is completed, the subsequent convolution operation only needs to perform steps S620 to S640. Each time the convolution operation is performed, the convolution operation data is directly input to the storage and calculation array module, and the storage and calculation array module automatically performs the convolution operation based on the preloaded weight data, thereby reducing the data flow in the convolution operation process by repeatedly using the preloaded convolution weight data.

[0096] The embodiment of the present application provides a sine and cosine function storage and calculation method, which is applied to the storage and calculation chip provided by the embodiment of the present application, and the sine and cosine function storage and calculation method is used for Figure 7 , as shown in Figure 7 is a step flow chart of the sine and cosine function storage and calculation method provided by the embodiment of the present application. As shown in Figure 7 , the sine and cosine function storage and calculation method provided by the embodiment of the present application includes steps S710 to S760:

[0097] Step S710: Preload the fixed-point result of the sine and cosine function fixed-point data at the target index bit width to the storage and calculation array module.

[0098] Step S720: Perform the index interval fitting implementation algorithm and the floating-point data fixed-point processing on the sine and cosine function data in sequence to obtain the sine and cosine function fixed-point input data.

[0099] Step S730: Determine the target array of the preloaded fixed-point result in the storage and calculation array module according to the target index bit width.

[0100] Step S740: Input the fixed-point data of the sine and cosine function fixed-point input data except the target index bit width to the specified position in the target array in the form of single data bit, and perform multiplication and accumulation operation with the preloaded fixed-point result to obtain the multiplication and accumulation result of different data bits of the sine and cosine function data.

[0101] Step S750: Determine the target result from the multiplication and accumulation result, and perform shift and accumulation on the target result to obtain the fixed-point number result of the sine and cosine function.

[0102] Step S760: converting the fixed-point result of the cosine function into data of a floating-point type to obtain a cosine function operation result.

[0103] In the embodiment, the cosine function data is FP32 type data, in step S710, the 1-bit sign bit of the output result is determined by using the cosine function preprocessing module, and the cosine function data is converted into a 39-bit (1-bit integer bit and 38-bit decimal bit) cosine function fixed-point data F by using the 8-bit exponent of the input FP32 data, the 23-bit mantissa, and the periodicity of the sin and cos functions. Then, by induction formula and algorithm modeling analysis, the target index bit width m is determined as 13 bits, and then the data F[35:23] (f_main) is used as the index to determine the target array of the preloaded fixed-point result (i.e., the 32-bit mantissa fixed-point result of sin(2π2 -15 f_main) and 2πcos(2π2 -15 f_main) in the compute-in-memory array module; and then the fixed-point data of the cosine function input data except the target index bit width (i.e., f_minor in formula 1) is input to the specified position in the target array to perform multiply-accumulate operation with the preloaded fixed-point result, to obtain the multiply-accumulate result of different data bits of the cosine function data.

[0104] Moreover, after the fixed-point result of the cosine function data at the target index bit width is preloaded to the compute-in-memory array module, it can be reused. That is, after step S710 is completed, the subsequent cosine function operation only needs to perform steps S720 to S750, and each time the cosine function operation is performed, the fixed-point result is directly indexed from the compute-in-memory array module, and the fixed-point data of the cosine function input data except the target index bit width is input to the compute-in-memory array module to achieve the operation, thereby reducing the data flow in the operation process of the cosine function.

[0105] In a specific embodiment, the fixed-point result includes a sine function fixed-point result and a cosine function fixed-point result; the fixed-point result of the cosine function data at the target index bit width is preloaded to the compute-in-memory array module; the sine function fixed-point result is preloaded to a first compute-in-memory array submodule of the compute-in-memory array module; and the cosine function fixed-point result is preloaded to a second compute-in-memory array submodule of the compute-in-memory array module.

[0106] For example, the 32-bit mantissa fixed-point result of sin(2π2 -15 f_main) and 2πcos(2π2 -15 f_main) preloaded to the compute-in-memory array module can be converted into formula:

[0107] sin(2π2 -38 f)=Tile1(f_main)+Tile2(f_main)2 -38 f_minor (2)

[0108] Wherein, the index f_main determines the position of the array that needs to be flowed in for f_minor, and the fixed-point data is input to the specified position of the compute-in-memory array module in the form of a single data bit according to the second fixed-point result of the target index bit width (i.e., f_minor). That is, after the fixed-point data f_minor is serially flowed in, a single data is flowed in per cycle to realize the multiplication operation.

[0109] Specifically, when the 32-bit mantissa fixed-point results of sin(2π2 -15 f_main) and 2πcos(2π2 -15 f_main) are preloaded to the compute-in-memory array module, it is necessary to divide the 4 consecutive array groups in the compute-in-memory array module into a set, and one row of each set can exactly preload 32-bit data results. Then, according to this form, sin(2π2 -15) f_main) (i.e., the fixed-point result of the sine function) is preloaded into the first compute-in-memory array submodule Tile1, and 2πcos(2π2 -15 f_main) (i.e., the fixed-point result of the cosine function) is preloaded into the second compute-in-memory array submodule Tile2, and then the preloading of the fixed-point data of the sine and cosine functions at the first fixed-point result of the target index bit width is realized.

[0110] In a specific embodiment, the specified position in the target array refers to the specified position in the second compute-in-memory array submodule; the fixed-point input data of the sine and cosine functions other than the target index bit width is input to the specified position in the target array in the form of a single data bit, and multiplication and accumulation operation is performed with the preloaded fixed-point result to obtain the multiplication and accumulation result of different data bits of the sine and cosine function data, including:

[0111] Indexing the fixed-point result of the sine function from the first compute-in-memory array submodule as a first term result;

[0112] Inputting the fixed-point data of the sine and cosine function input data other than the target index bit width to the specified position of the second compute-in-memory array submodule, and performing shift and accumulation per cycle to obtain a second term result;

[0113] Accumulating the first term result and the second term result to obtain the multiplication and accumulation result of different data bits of the sine and cosine function data.

[0114] In the embodiments of the present application, it can be seen from the above formula (2) that the cosine function operation is equivalent to the multiplication-addition logic, which is perfectly matched with the operation supported by the designed storage-computing array. Therefore, after f_main determines the positions in the first storage-computing array submodule Tile1 and the second storage-computing array submodule Tile2, for the first term in formula (2), the corresponding first term result is obtained by indexing in the first storage-computing array submodule Tile1, that is, Tile1(f_main); for the second term in formula (2), after obtaining its position in the second storage-computing array submodule Tile2, the F[22:0] data (that is, f_minor) is serially flowed into the specified position of the array, and is shifted and accumulated in each cycle, thereby realizing the mantissa calculation of Tile2(f_main)f_minor, and obtaining the second term result. By accumulating the first term result and the second term result, the multiplication-accumulation result of the cosine function data at different data bits is obtained.

[0115] In the specific implementation, the first selection module is used to determine the target array of the preloaded fixed-point result in the storage-computing array module, and the second selection module is used to output the target result (that is, the required result). Specifically, 2-bit data of f_[12:11] is flowed into the first selection module to determine the PE to which it belongs, 2-bit data of f_main[10:9] is flowed into the first selection module to determine the array Array to which the specified PE belongs, and f_main[6:0] determines the array Array row position of f_minor flowed into the array Array, so that when the data is flowed into the specified array Array in each cycle, the 23-bit f_minor is placed in the specified row position, and the remaining positions are all set to zero. At this time, the specified array outputs four different multiplication results, and f_main[8:7] determines the required multiplication result in the second selection module. The 23-bit f_minor data is serially flowed into the array, and is shifted and accumulated in the shift-add module, and finally the 80-bit fixed-point result is obtained.

[0116] In the embodiment of the present application, the fixed-point data of the sine and cosine function is preloaded to the storage and computing array module in the target index bit width fixed-point result, and then when the sine and cosine function calculation is performed, the target array of the preloaded fixed-point result in the storage and computing array module is determined, and then the fixed-point data of the sine and cosine function input data other than the target index bit width is input to the specified position of the target array to perform multiplication and accumulation operation with the preloaded fixed-point result, to obtain the multiplication and accumulation result of the sine and cosine function data in different data bits, and finally the multiplication and accumulation result of the sine and cosine function data in different data bits is converted into floating-point type data output to obtain the sine and cosine function operation result. Since the storage and computing array module and the shift and add module used for the sine and cosine function operation are multiplexed with the sine and cosine operation, the expansion of the sine and cosine function operation is realized by adding a small amount of digital circuits on the basis of the traditional storage and computing chip, thereby increasing the operation function of the traditional storage and computing chip and expanding the application scenarios that the storage and computing chip can adapt to.

[0117] Each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between the embodiments can be referred to each other.

[0118] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the system according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal equipment realize the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more blocks.

[0119] These computer program instructions can also be stored in a computer readable memory that can guide the computer or other programmable data processing terminal equipment to work in a specific way, so that the instructions stored in the computer readable memory generate a product including instruction apparatus, which realizes the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more blocks.

[0120] These computer program instructions can also be loaded into a computer or other programmable data processing terminal equipment, so that a series of operation steps are performed on the computer or other programmable terminal equipment to generate a computer-implemented process, so that the instructions executed on the computer or other programmable terminal equipment provide a process for realizing the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the steps of a function specified in one or more blocks.

[0121] While preferred embodiments of the application have been described, those skilled in the art will appreciate that other modifications than those specifically described can be made within the scope of the application. Accordingly, the appended claims are intended to embrace all such alternatives as well.

[0122] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are not intended to denote a physical or logical relationship between such elements. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0123] The above provides a kind of support convolution operation and sine and cosine function operation of memory chip, memory calculation method, detailed introduction is carried out in the application, the principle and implementation mode of the present application are described in this paper with specific examples, the above example is only for helping to understand the method of the present application and its core idea;For those skilled in the art, according to the idea of the present application, there will be changes in specific implementation mode and application range, as described above, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A memory chip supporting convolution and sine / cosine function operations, characterized in that, It includes a preprocessing module, a memory array module, and a shift-add module; The preprocessing module is used to control the input data to be input into the storage array module in the form of single data bits. The input data includes convolution operation data and sine and cosine function data. The storage array module is used to perform multiplication and accumulation operations based on the preloaded data and the input data to obtain multiplication and accumulation results of different data bits. The preloaded data includes: the fixed-point results of convolution weight data and sine and cosine function fixed-point data at the target index bit width. The shift-add module is used to shift and accumulate the multiplication-accumulation result to obtain the array calculation result; The fixed-point results include sine function fixed-point results and cosine function fixed-point results. The in-memory array module includes a first sub-in-memory array module and a second sub-in-memory array module. The preprocessing module is further configured to: preload the fixed-point results of the sine and cosine function fixed-point data at the target index bit width into the in-memory array module, including: The fixed-point result of the sine function is preloaded into the first in-memory array submodule of the in-memory array module; the fixed-point result of the cosine function is preloaded into the second in-memory array submodule of the in-memory array module; The result of the multiplication and accumulation is obtained according to the following steps: The first result is obtained by indexing the fixed-point result of the sine function from the first storage array submodule; the fixed-point data of the sine and cosine function, excluding the target index bit width, is input to the specified position of the second storage array submodule and shifted and accumulated in each cycle to obtain the second result; the first result and the second result are accumulated to obtain the multiplication and accumulation result of different data bits of the sine and cosine function data.

2. The memory chip according to claim 1, characterized in that, The preprocessing module includes a convolution preprocessing module; The convolution preprocessing module is used to input the convolution operation data into the storage array module in the form of single data bits.

3. The memory chip according to claim 1, characterized in that, The preprocessing module includes a sine and cosine function preprocessing module and a first selection module; The sine and cosine function preprocessing module is used to sequentially perform exponential interval adaptation algorithm and floating-point data fixed-point processing on the sine and cosine function data to obtain sine and cosine function fixed-point input data. The first selection module is used to determine the target array of the preloaded fixed-point result in the storage array module according to the target index bit width, and to input the fixed-point data of the sine and cosine function fixed-point input data other than the target index bit width into the specified position in the target array in the form of single data bits.

4. The memory chip according to claim 1, characterized in that, The in-memory computing chip also includes an accumulation module and a floating-point output module. The array calculation results include convolution calculation results of multiple different arrays and fixed-point results of sine and cosine functions. The accumulation module is used to accumulate the convolution calculation results of the multiple different arrays to obtain the convolution operation result; The floating-point output module is used to convert the fixed-point result of the sine and cosine functions into floating-point data to obtain the sine and cosine function calculation result.

5. The memory chip according to claim 1, characterized in that, The in-memory computing chip also includes a second selection module, a convolution state module, and a sine and cosine function state module; The second selection module is used to determine the target result in the multiplication and accumulation results of different data bits of the sine and cosine function data; The convolution operation state module is used to control the state scheduling of the convolution operation process; The sine and cosine function state module is used to control the state scheduling of the sine and cosine function calculation process.

6. The memory chip according to claim 1, characterized in that, Both the first sub-in-memory array module and the second sub-in-memory array module include multiple in-memory array groups composed of computing units. The word lines of all static random access memory (SRAM) of computing units in the same row of the in-memory array group are connected together, and the bit lines of all SRAM of computing units in the same column of the in-memory array group are connected together.

7. A convolution storage computation method, characterized in that, Applied to the in-memory computing chip according to any one of claims 1 to 6, the method includes: Preload the convolution weight data into the in-memory array module; The convolution operation data is input into the storage array module in the form of single data bits, and multiplied and accumulated with the pre-loaded convolution weight data to obtain the multiplication and accumulation result of different data bits of the convolution operation data. The multiplication and accumulation results of different data bits of the convolution operation data are shifted and accumulated to obtain convolution calculation results of multiple different arrays; The convolution calculation results of the multiple different arrays are summed to obtain the convolution operation result.

8. A method for storing and calculating sine and cosine functions, characterized in that, Applied to the in-memory computing chip according to any one of claims 1 to 6, the method includes: The fixed-point results of the sine and cosine function fixed-point data at the target index bit width are preloaded into the in-memory array module; The sine and cosine function data are sequentially processed by the exponential interval adaptation algorithm and the floating-point data fixed-point conversion to obtain the fixed-point input data of the sine and cosine functions. The target array of the preloaded fixed-point result in the in-memory array module is determined based on the target index bit width. The fixed-point input data of the sine and cosine functions, excluding the target index bit width, is input into the specified position in the target array in the form of single data bits, and multiplied and accumulated with the pre-loaded fixed-point results to obtain the multiplied and accumulated results of different data bits of the sine and cosine function data. The target result is determined from the multiplication and accumulation results, and the target result is shifted and accumulated to obtain the fixed-point result of the sine and cosine functions; The fixed-point result of the sine and cosine functions is converted into floating-point data to obtain the result of the sine and cosine function operation.

9. The method according to claim 8, characterized in that, The fixed-point results include fixed-point results for the sine function and fixed-point results for the cosine function; The fixed-point results of the sine and cosine functions at the target index bit width are preloaded into the in-memory array module, including: The fixed-point result of the sine function is preloaded into the first in-memory array submodule of the in-memory array module; The fixed-point result of the cosine function is preloaded into the second in-memory array submodule of the in-memory array module.

10. The method according to claim 9, characterized in that, The specified position in the target array refers to the specified position in the second storage array submodule; the fixed-point data of the sine and cosine function fixed-point input data, excluding the target index bit width, is input into the specified position of the target array in single data bit form, and multiplied and accumulated with the pre-loaded fixed-point result to obtain the multiplied and accumulated result of different data bits of the sine and cosine function data, including: The fixed-point result of the sine function is indexed from the first storage array submodule as the first result; The fixed-point input data of the sine and cosine functions, excluding the target index bit width, is input to the designated position of the second storage array submodule, and shifted and accumulated in each cycle to obtain the second result; The first and second results are summed to obtain the sum of the multiplication of different data bits of the sine and cosine function data.

Images