In-memory calculation module and related method, chip and equipment
By removing the data reading unit in the in-memory computing module, only the data input unit, the data writing unit and the computing array are used to realize in-memory computing module, the problem of large area and power consumption in the prior art is solved. More efficient computing and lower power consumption are achieved.
Patent Information
- Application Number
- CN202310959998.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-31
- Publication Date
- 2025-02-14
AI Technical Summary
The existing in-memory computing module has a large area and power consumption, mainly due to the large amount of hardware and the existence of data reading units.
By removing the data reading unit in the in-memory computing module, in-memory computing is implemented using only the data input unit, the data writing unit, and the computing array, thereby reducing area and power consumption.
The area and power consumption of the in-memory computing module are reduced, the processing efficiency is improved, and the power consumption of the circuit is reduced.
Smart Images
Figure CN119440462A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of in-memory computing technology, and in particular to an in-memory computing module and related methods, chips and devices. Background Art
[0002] In-memory computing is a new computing architecture that achieves high-efficiency computing. Different from the storage-computing separation structure of the traditional von Neumann architecture, the in-memory computing architecture can implement computing functions inside the memory. For example, the in-memory computing module in the in-memory computing architecture can efficiently perform operations such as matrix multiplication, so it can be widely used in scenarios such as artificial intelligence that require a large amount of storage and computing.
[0003] However, current in-memory computing modules are usually provided with storage units for storing input data. The in-memory computing modules have a large area and consume a large amount of power during application due to the large amount of hardware. Summary of the invention
[0004] The embodiment of the present application provides an in-memory computing module, which requires a small area and consumes less power. The embodiment of the present application also provides corresponding methods, chips, devices, etc.
[0005] The first aspect of the present application provides an in-memory computing module, which includes a computing array, a data writing unit and a data input unit; the data writing unit is used to write first data into the computing array so that the computing array stores the first data; the data input unit is used to input second data into the computing array; the computing array is used to store the first data, and after the data input unit inputs the second data, performs a matrix multiplication operation according to the first data and the second data to obtain a calculation result.
[0006] At present, in conventional in-memory computing modules, a data reading unit is usually reserved in addition to the computing array to read and store the data to be calculated for subsequent matrix multiplication operations. However, the data reading unit usually brings certain area and power consumption overhead.
[0007] In the first aspect, there is no need to additionally set up a data reading unit in the in-memory computing module. Instead, in-memory computing can be achieved through a data input unit, a data writing unit and a computing array, thereby reducing the area and power consumption overhead of the in-memory computing module.
[0008] In a possible implementation manner of the first aspect, the in-memory computing module further includes a data output unit; the data output unit is used to output the computing result.
[0009] In a possible implementation of the first aspect, when it is necessary to read the i-th row in the first data, the i-th element in the second data is 1, so that the calculation result output by the data output unit includes the i-th row read from the first data stored in the calculation array; when it is not necessary to read the j-th row in the first data, the j-th element in the second data is 0, so that the calculation result output by the data output unit does not include the j-th row in the first data.
[0010] In this possible implementation, there is no need to additionally set up a data reading unit in the in-memory computing module. Instead, in-memory computing can be achieved through a data input unit, a data writing unit and a computing array, thereby reducing the area and power consumption of the in-memory computing module.
[0011] In a possible implementation of the first aspect, the computing array includes a combinational logic circuit and at least one register, the combinational logic circuit includes multiple levels of sub-circuits, and sub-circuits at different levels are used to implement combinational logic at different levels; a register is located between sub-circuits at adjacent levels and is respectively connected to adjacent sub-circuits, and the register is used to store output data of the adjacent upper-level sub-circuit.
[0012] In this possible implementation, by inserting registers in the combinational logic circuit, a multi-stage pipeline operation method for matrix multiplication operations can be realized, without having to wait for the entire combinational logic circuit to complete a complete operation before inputting new data into the combinational logic circuit, thereby greatly improving the processing efficiency of the in-memory computing module.
[0013] In a possible implementation of the first aspect, the computing array includes multiple computing units, each computing unit includes a multiplier and a shutdown circuit; the shutdown circuit is used to generate a control signal based on an element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, and the control signal is used to control whether the multiplier of the corresponding computing unit is working.
[0014] In this possible implementation, a shutdown circuit may be provided in the calculation unit, so that the shutdown circuit may be used to control whether the multiplier works, that is, whether the corresponding multiplication operation is performed is controlled according to whether the element in the second data input in the current operation cycle is 0. In this way, when the element in the second data input in the current operation cycle is 0, the multiplier may be turned off, thereby reducing the power consumption of the circuit.
[0015] In a possible implementation of the first aspect, each computing unit also includes a data selection circuit and multiple storage units; any storage unit is used to store at least one element of the first data; the data selection circuit is used to select at least one storage unit from the multiple storage units of the corresponding computing unit as a data input terminal in the current operation cycle of the matrix multiplication operation, so as to input the element of the first data stored in the at least one storage unit into the multiplier of the corresponding computing unit; the shutdown circuit is used to generate a control signal according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, and the control signal is used to control whether the data selection circuit of the corresponding computing unit and / or the multiplier of the corresponding computing unit are working.
[0016] In this possible implementation, the data selection circuit and / or the multiplier can be controlled to work by shutting down the circuit, that is, whether the corresponding multiplication operation is performed is controlled according to whether the element in the second data input in the current operation cycle is 0. In this way, when the element in the second data input in the current operation cycle is 0, the data selection circuit and / or the multiplier can be shut down, thereby reducing the power consumption of the circuit.
[0017] A second aspect of the present application provides a data processing method, which is applied to an in-memory computing module, the in-memory computing module including a computing array, a data writing unit and a data input unit, the method comprising: writing first data into the computing array through the data writing unit so that the computing array stores the first data; inputting second data into the computing array through the data input unit; storing the first data through the computing array, and after the second data is input into the data input unit, performing a matrix multiplication operation based on the first data and the second data to obtain a calculation result.
[0018] In a possible implementation manner of the second aspect, the in-memory computing module further includes a data output unit, and the method further includes: outputting the calculation result through the data output unit.
[0019] In a possible implementation of the second aspect, when it is necessary to read the i-th row in the first data, the i-th element in the second data is 1, so that the calculation result output by the data output unit includes the i-th row read from the first data stored in the calculation array; when it is not necessary to read the j-th row in the first data, the j-th element in the second data is 0, so that the calculation result output by the data output unit does not include the j-th row in the first data.
[0020] In a possible implementation of the second aspect, the computing array includes multiple computing units, each computing unit includes a multiplier and a shutdown circuit, and the method also includes: generating a control signal through the shutdown circuit according to the element of the second data input into the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, and the control signal is used to control whether the multiplier of the corresponding computing unit is working.
[0021] In a possible implementation of the second aspect, each computing unit also includes a data selection circuit and multiple storage units; the method also includes: storing at least one element of the first data through any storage unit; selecting at least one storage unit from the multiple storage units of the corresponding computing unit as a data input terminal through the data selection circuit in the current operation cycle of the matrix multiplication operation, so as to input the element of the first data stored in the at least one storage unit into the multiplier of the corresponding computing unit; generating a control signal through the shutdown circuit according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, including: generating a control signal through the shutdown circuit according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, the control signal is used to control whether the data selection circuit of the corresponding computing unit and / or the multiplier of the corresponding computing unit is working.
[0022] A third aspect of the present application provides a chip, which includes an in-memory computing module according to the first aspect or any possible implementation of the first aspect.
[0023] A fourth aspect of the present application provides an electronic device, which includes an in-memory computing module according to the first aspect or any possible implementation of the first aspect.
[0024] Among them, the technical effects brought about by the second to fourth aspects or any possible implementation methods thereof can refer to the technical effects brought about by the first aspect or the relevant possible implementation methods of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 is an exemplary schematic diagram of a chip provided in an embodiment of the present application;
[0026] Figure 2 is an exemplary schematic diagram of an in-memory computing module provided in an embodiment of the present application;
[0027] Figure 3a is an exemplary schematic diagram of a computing unit provided in an embodiment of the present application;
[0028] Figure 3b It is an exemplary circuit diagram of a shutdown circuit provided in an embodiment of the present application to control whether a corresponding data selection circuit and a corresponding multiplier work;
[0029] Figure 4a is an exemplary schematic diagram of a traditional data output unit provided in an embodiment of the present application;
[0030] Figure 4b is an exemplary schematic diagram of a computing unit in a computing array of an in-memory computing module provided in an embodiment of the present application;
[0031] Figure 5a It is an exemplary schematic diagram of the combinational logic in the traditional in-memory computing module provided in the embodiment of the present application;
[0032] Figure 5b is an exemplary schematic diagram of inserting a register in a multi-level sub-circuit provided in an embodiment of the present application;
[0033] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0034] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.
[0035] Those skilled in the art may appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.
[0036] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of singular or plural items. The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable where appropriate, and this is merely a description of the way in which objects with the same attributes are described in the embodiments of the present application. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent to such process, method, product, or apparatus.
[0037] In-memory computing is a new computing architecture that achieves high-efficiency computing. Different from the storage-computing separation structure of the traditional von Neumann architecture, the in-memory computing architecture can implement computing functions inside the memory. For example, the in-memory computing module in the in-memory computing architecture can efficiently perform operations such as matrix multiplication, so it can be widely used in scenarios such as artificial intelligence that require a large amount of storage and computing.
[0038] Current in-memory computing modules are usually provided with storage units etc. for storing input data, which makes the in-memory computing module larger in area. In addition, due to the large amount of hardware, the in-memory computing module consumes a large amount of power during application.
[0039] Based on this, an embodiment of the present application provides an in-memory computing module with smaller required area and power consumption.
[0040] In the embodiment of the present application, the in-memory computing module is a hardware module that performs in-memory computing.
[0041] There are many specific deployment methods for the in-memory computing module.
[0042] In one example, the in-memory computing module may be one or more computing cores in a chip, which are deployed as intellectual property (IP) cores in various integrated circuit chips or electronic devices.
[0043] Take chips as an example. Figure 1 In the example shown, the chip may include an in-memory computing module, and may also include one or more of a storage module, a control module, and other logic modules.
[0044] Of course, in other examples, the in-memory computing module may also be in other hardware forms, which is not limited here.
[0045] The in-memory computing module may include a digital circuit or an analog circuit, which is not limited in the embodiments of the present application.
[0046] The in-memory computing module can be used to perform matrix multiplication operations.
[0047] Specifically, the in-memory computing module is used to perform a matrix multiplication operation between the first data and the second data.
[0048] The dimensions and sizes of the first data and the second data may be in various situations, which are not limited here. The first data and the second data are fixed-point integers of arbitrary bit width.
[0049] Exemplarily, the first data may be a matrix, and the second data may be a vector. In this case, the matrix multiplication operation may be a matrix-vector multiplication operation.
[0050] The sizes of the various dimensions of the first data are not limited here.
[0051] The second data may be a set of vectors in a matrix, or may also be a matrix. When the second data is a vector, the in-memory computing module is used to perform a matrix-vector multiplication operation.
[0052] like Figure 2 The figure shows an exemplary structural diagram of the in-memory computing module.
[0053] exist Figure 2 In the example shown, the in-memory computing module includes a computing array, a data writing unit, a data input unit, and a data output unit. In addition, in some examples, the in-memory computing module may also include other units such as a timing control unit.
[0054] The data writing unit is used to write the first data into the computing array so that the computing array stores the first data.
[0055] The data writing unit writing the first data into the computing array may refer to the data writing unit causing the states of the storage units in the computing array to be flipped accordingly according to the written first data, so that the storage units in the computing array store the first data.
[0056] The data input unit is used to input the second data into the calculation array.
[0057] The data input unit can transmit the second data to a data interface in the calculation array, for example, to an interface of a multiplier in the calculation array, to be used as data in an input of the multiplier.
[0058] The computing array generally includes a plurality of storage units to store the first data. The structure and form of the storage units in the computing array are not limited here.
[0059] Exemplarily, the plurality of storage units may be included in a storage array, and different storage units may be of the same type or may be different in type.
[0060] There may be many types of storage cells. For example, any storage cell may be a static random-access memory (SRAM), a dynamic random access memory (DRAM), a resistive random access memory (ReRAM), a ferroelectric random access memory (FeRAM), or other storage devices.
[0061] In addition, the computing array may also include a data selection circuit, a multiplier, an adder, etc. Among them, the data selection circuit can select one or more storage units from a plurality of storage units as data input terminals, so that the elements of the first data stored in the selected one or more storage units are used as input data of the corresponding multiplier. The multiplier, adder, etc. are used to implement matrix multiplication operations.
[0062] In some embodiments, the computation array includes a plurality of computation units, each computation unit including a multiplier and a shutdown circuit.
[0063] The shutdown circuit is used to generate a control signal according to the element of the second data input to the multiplier of the corresponding calculation unit in the current operation cycle of the matrix multiplication operation, and the control signal is used to control whether the multiplier of the corresponding calculation unit works.
[0064] In the current traditional in-memory computing module, multiplication and accumulation operations are usually performed directly on the input data to achieve matrix multiplication operations.
[0065] In the embodiment of the present application, it is considered that the matrix multiplication operation is usually divided into multiple operation cycles, and since there may be elements of 0 in the input data for the matrix multiplication operation, there may be one or more operation cycles in which the operation is multiplied by 0, so that the output of the operation is 0. Therefore, in the embodiment of the present application, it is considered to reduce or avoid the multiplication operation related to the 0 value in the input data of the matrix multiplication operation, so as to reduce the power consumption of the in-memory operation module.
[0066] Specifically, the computing array may include a plurality of computing units, each of which includes a multiplier and a shutdown circuit.
[0067] The multiplier can perform operations in each operation cycle of the matrix multiplication operation, and the number and structure of the multipliers in each computing unit are not limited here.
[0068] In addition, in the embodiment of the present application, a shutdown circuit can be set in the calculation unit, so that the shutdown circuit can be used to control whether the multiplier works, that is, whether the corresponding multiplication operation is performed is controlled according to whether the element in the second data input in the current operation cycle is 0. In this way, when the element in the second data input in the current operation cycle is 0, the multiplier can be turned off, thereby reducing the power consumption of the circuit.
[0069] The specific circuit structure of the data selection circuit is not limited here. Exemplarily, the data selection circuit includes multiple data input channels and a multi-way switch, etc. In addition, in some examples, the data selection circuit may also include circuit structures such as an amplifier circuit.
[0070] In some embodiments, each computing unit further includes a data selection circuit and a plurality of storage units.
[0071] Any storage unit is used to store at least one element of the first data.
[0072] The data selection circuit is used to select at least one storage unit from multiple storage units of the corresponding computing unit as a data input terminal in the current operation cycle of the matrix multiplication operation, so as to input the elements of the first data stored in the at least one storage unit into the multiplier of the corresponding computing unit.
[0073] The shutdown circuit is used to generate a control signal according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, and the control signal is used to control whether the data selection circuit and / or the multiplier of the corresponding computing unit works.
[0074] In the embodiment of the present application, the computing array can implement the storage of the first data through the storage units in each computing unit, that is, each element in the first data is stored in the storage unit of each computing unit respectively, and any storage unit can store at least one element in the first data.
[0075] The data amount of the elements of the first data stored in the storage unit in each computing unit is not limited here.
[0076] The structures of the circuits in the various computing units in the computing array may be the same or different. For example, the amount of data that can be stored in the storage units of the various computing units may be the same or different, and the structures of the data selection circuits of the various computing units may be the same or different. For example, the data selection circuit may include multiple data input channels and a multi-way switch. In addition, in some examples, the data selection circuit may also include circuit structures such as an amplifier circuit, which is not limited in the embodiments of the present application.
[0077] For example, Figure 3a In the example shown, a computing unit of the computing array includes storage unit 1, storage unit 2, ... storage unit P, where P is an integer greater than 1. Storage units 1-P are respectively connected to different channels in the data selection circuit in the computing unit, so that in the current operation cycle of the matrix multiplication operation, the data selection circuit can select any storage unit in the storage units 1-P as the data input terminal according to the corresponding signal, and input the data stored in the selected storage unit into the multiplier as an input data of the multiplier.
[0078] Another input data of the multiplier may be an element of the second data x[x1, x2, ... xn] in the current operation cycle, for example, Figure 3a The element xi is shown.
[0079] In the embodiment of the present application, a shutdown circuit can be set in the calculation unit, so that the shutdown circuit can be used to control whether the data selection circuit and / or the multiplier work, that is, whether the corresponding multiplication operation is performed is controlled according to whether the element in the second data input in the current operation cycle is 0. In this way, when the element in the second data input in the current operation cycle is 0, the data selection circuit and / or the multiplier can be turned off, thereby reducing the power consumption of the circuit.
[0080] Exemplarily, when generating a control signal based on an element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, the signal of the element can be used as the control signal, or a new signal can be generated based on the signal of the element and input into the corresponding data selection circuit and / or multiplier.
[0081] For example, Figure 3b As shown, it is an exemplary circuit diagram of a shutdown circuit in a computing unit controlling whether a corresponding data selection circuit and a corresponding multiplier work.
[0082] Among them, according to the data of the element of the second data input to the multiplier of the corresponding calculation unit in the current operation cycle is ACT, then ACT can be used as a control signal of the data selection circuit and the multiplier to control whether the data selection circuit and the multiplier work.
[0083] If the ACT is 0, the data selection circuit and the multiplier do not work in this operation cycle, and the output result of the output terminal of the multiplier can be 0. If the ACT is 1, the data selection circuit and the multiplier are in a working state in this operation cycle, and the output result of the output terminal of the multiplier is the multiplication result after the multiplier performs the multiplication operation according to the corresponding input.
[0084] The data output unit is used to output the calculation results.
[0085] The data output unit can output the final calculation result of the matrix multiplication operation.
[0086] Among them, outputting the calculation results refers to transmitting the calculation results to the outside of the in-memory computing module. For example, it can be transmitted to other modules other than the in-memory computing module in an electronic device deployed with the in-memory computing module, or it can also be transmitted to other devices. The embodiments of the present application do not limit this.
[0087] At present, an exemplary structural diagram of a conventional data output unit is as follows: Figure 4a shown.
[0088] It can be seen that the traditional data output unit reads data from the storage cells of the storage array of the in-memory computing module (for example, storage cells 0, 1-0, n, storage cells 1, 1-1, n and storage cells m, 1-m, n in the figure), and each time data is read, a selected row of data in the storage array is output through the data read unit.
[0089] In one example, an exemplary structural diagram of the data output unit in the embodiment of the present application is as follows: Figure 4b shown.
[0090] exist Figure 4b In the example shown, the computing array may include multiple computing units, each of which may include a storage unit, a data selection circuit, a multiplier, and the like.
[0091] The number of storage units is not limited here.
[0092] like Figure 4b The figure shows an exemplary structural diagram of a computing unit in a computing array of an in-memory computing module.
[0093] In this example, the calculation array can store M rows and N columns of data. Take the first column in the calculation array as an example. Figure 4b As shown, the first column in the computing array may include Q computing units, where Q is a positive integer. Each computing unit may include P storage units. For example, computing unit 11 may include storage units 11,1 to 11,P, which can store part of the first data X, and computing unit 1Q may include storage units 1Q,1 to 1Q,P, which can store another part of the first data X.
[0094] When performing a matrix multiplication operation, the first data may be an m*n matrix, where m is not greater than M, and n is not greater than N. In this way, the elements of the first data may be stored in storage units of the calculation array, respectively, and in each operation, a data input terminal is selected from the storage unit through a data selection circuit to input the data stored in the corresponding storage unit into the multiplier of the calculation unit.
[0095] In this example, the element x1 in the second data x[x1, ...xQ, ...xn] can be used as the input of the calculation unit 11, and the element xQ can be used as the input of the calculation unit 1Q. In this way, based on the matrix multiplication operation in this example, the output of the calculation unit associated in this matrix multiplication operation in the calculation array can be obtained and output through the data output unit to obtain the final calculation result. The data output unit may include hardware such as an adder and / or a register to implement operations such as data integration and storage.
[0096] It can be seen that in this example, in the actual calculation process, each computing unit can output the corresponding multiplication result, and perform a complete matrix multiplication operation through the adder or even the register in the data output unit to obtain the final calculation result of the matrix multiplication operation, and then output the final calculation result of the matrix multiplication operation.
[0097] It can be seen that the data output unit in this example is different from the traditional data output unit in at least the following two aspects:
[0098] 1. The data read out is different.
[0099] A traditional data output unit reads data from a storage array of an in-memory computing module, whereas the data output unit in this example outputs a calculation result after a matrix multiplication operation between first data stored in the in-memory computing module and second data input.
[0100] 2. The data paths are different.
[0101] Each time a traditional data output unit reads data, it outputs a selected row of data in the storage array. In this example, in the computing array, the first data can be stored by multiple columns and rows of storage units, and the data output unit in the in-memory computing module can obtain the calculation result of the matrix multiplication operation of the first data stored by the multiple columns and rows of storage units and the second data through an adder or even a register, and then output it to the outside of the in-memory computing module.
[0102] At present, in conventional in-memory computing modules, a data reading unit is usually reserved in addition to the computing array to read and store the data to be calculated for subsequent matrix multiplication operations. However, the data reading unit usually brings certain area and power consumption overhead.
[0103] In the embodiment of the present application, there is no need to additionally set up a data reading unit in the in-memory computing module. Instead, in-memory computing can be achieved through a data input unit, a data writing unit and a computing array, thereby reducing the area and power consumption of the in-memory computing module.
[0104] In some scenarios, the in-memory computing module can still meet the data reading needs through in-memory computing operations.
[0105] Specifically, in some embodiments, when the i-th row in the first data needs to be read, the i-th element in the second data is 1, so that the calculation result output by the data output unit includes the i-th row read from the first data stored in the calculation array;
[0106] When it is not necessary to read the j-th row in the first data, the j-th element in the second data is 0, so that the calculation result output by the data output unit does not include the j-th row in the first data.
[0107] Among them, the number of rows that need to be read from the first data can be at least one row. At this time, the calculation result output by the data output unit includes the data of the at least one row. If there is data that does not need to be read in the first data, then at this time, in the calculation result output by the data output unit, the value of the element with the same position as the data that does not need to be read in the first data is 0.
[0108] For example, if the read instruction indicates that only the i-th row of the first data W needs to be taken out, then the i-th data in the second data X[x1, x2...xm] only needs to be set to 1, and the other data in the second data except the i-th data is set to 0, so as to achieve the data reading requirement. For example, if i=1, the second data X is [1,0,0,0,...], which is used to read the first row of the first data; if i=2, the second data X is [0,1,0,0,...], which is used to read the second row of the first data.
[0109] In the embodiment of the present application, the control unit in the in-memory computing module may receive a first control instruction, the first control instruction instructing to read at least one row of the first data, the at least one row including the i-th row. Then, the control unit may generate the second data, or the first control instruction may carry the second data, so that the in-memory computing module obtains an output result through a matrix multiplication operation between the first data and the generated second data, thereby outputting at least one row of the first data to be read through the output result.
[0110] Furthermore, in some embodiments, registers may be inserted into the in-memory computing module to improve the computational efficiency of the in-memory computing module.
[0111] Specifically, in some embodiments, the matrix multiplication operation is implemented by multiple levels of combinatorial logic.
[0112] The computing array includes a combinational logic circuit and at least one register. The combinational logic circuit includes multiple levels of subcircuits. Subcircuits at different levels are used to implement combinational logics at different levels.
[0113] A register is located between sub-circuits of adjacent levels and is connected to the adjacent sub-circuits respectively. The register is used to store output data of the adjacent upper-level sub-circuit.
[0114] Currently, traditional in-memory computing modules only include combinational logic for matrix multiplication operations, but do not include sequential logic.
[0115] like Figure 5a An exemplary schematic diagram of the combinational logic in a traditional in-memory computing module is shown. After the first data and the second data are input into the computing array, the combinational logic of the matrix multiplication operation must be executed before the calculation result can be obtained and output. Then, the next set of data can be input into the computing array and the next matrix multiplication operation can be performed.
[0116] However, when the parallelism of the matrix multiplication operation to be executed is large (such as the length of the second data as a vector is long), or the bit width of each row of data in the matrix multiplication operation is large, the delay of the combinational logic will increase significantly, resulting in a decrease in the computing efficiency of the in-memory computing module.
[0117] In the embodiment of the present application, a register is inserted in the middle of the combinatorial logic of the matrix multiplication operation, thereby reducing the delay of the combinatorial logic within a single clock cycle through one or more levels of register buffering, thereby improving the computing efficiency of the in-memory computing module.
[0118] The computing array may include a combinational logic circuit to implement multi-level combinational logic through the combinational logic circuit.
[0119] The specific circuit structure of the combinational logic circuit is not limited here. Exemplarily, the combinational logic circuit may include a multiplier and an adder implemented by combinational logic.
[0120] The division method of the multi-level combinational logic can also be determined based on the actual application scenario.
[0121] When the combinational logic is long, in order to reduce the delay of the combinational logic, in an embodiment of the present application, the combinational logic circuit can be divided into multiple levels of sub-circuits, and sub-circuits of different levels are used to implement combinational logic of different levels.
[0122] Then, the register can be set between two adjacent sub-circuits of different levels and connected to the adjacent sub-circuits respectively, so that the output data of the upper-level sub-circuit will be input into the register for storage, and then the lower-level sub-circuit can obtain the output data of the upper-level sub-circuit from the register.
[0123] After the output data of the upper-level subcircuit is input into the register, the upper-level subcircuit can obtain new input data and process it. It can be seen that in the embodiment of the present application, when a register is inserted between two adjacent subcircuits, the register can be used to implement pipeline processing of one level. If in a combinational logic circuit, two adjacent subcircuits are considered to be an adjacent node, and registers are provided at the positions of at least two adjacent nodes in the combinational logic circuit, it can be considered that multiple levels of registers are inserted into the combinational logic circuit, thereby implementing multi-level pipeline processing of matrix multiplication operations.
[0124] For example, for the addition of the second data, which is a vector with a length of 128, the combinational logic is a pairwise addition operation logic, that is, in this example, the combinational logic includes the following calculation logic: 128*1b->64*2b->32*3b->16*4b->8*5b->4*6b->2*7b->1*8b. At this time, the combinational logic of the matrix multiplication operation includes 7 levels of operations, and the corresponding combinational logic circuit of the calculation array can include 7 levels of sub-circuits to perform the above 7 levels of operations in sequence. In this example, a register can be inserted after any level of combinational logic except the last level of combinational logic. For example, in one example, registers can be inserted into 4 adjacent nodes in the above 7-level sub-circuit.
[0125] like Figure 5b FIG. 1 is an exemplary schematic diagram of inserting registers in a multi-stage sub-circuit.
[0126] Figure 5b In the embodiment, registers can be inserted into the first-level subcircuit (i.e., subcircuit 1 implementing combinational logic 1) and the second-level subcircuit (i.e., subcircuit 2 implementing combinational logic 2). In this way, after subcircuit 1 completes the operation of combinational logic 1 and stores the corresponding result in the register. Then, subcircuit 1 can obtain new input data and perform the operation of combinational logic 1 again, while subcircuit 2 can read data from the register and perform the operation of combinational logic 2. The setting method of registers between subcircuits of other levels can refer to the setting method of registers between the first-level subcircuit and the second-level subcircuit.
[0127] It can be seen that in the embodiment of the present application, by inserting registers in the combinational logic circuit, a multi-stage pipeline operation method for matrix multiplication operations can be implemented, without having to wait for the entire combinational logic circuit to complete a complete operation before inputting new data into the combinational logic circuit, thereby greatly improving the processing efficiency of the in-memory computing module.
[0128] During the application process, the data processing method for performing matrix multiplication operations can be implemented through the in-memory computing module in any of the above embodiments.
[0129] Specifically, the data processing method includes the following steps:
[0130] Writing the first data into the computing array through the data writing unit so that the computing array stores the first data;
[0131] Inputting the second data into the calculation array through the data input unit;
[0132] The first data is stored by a calculation array, and after the second data is input into a data input unit, a matrix multiplication operation is performed according to the first data and the second data to obtain a calculation result.
[0133] In some embodiments, the in-memory computing module further includes a data output unit, and the method further includes:
[0134] The calculation results are output through the data output unit.
[0135] In some embodiments, when the i-th row in the first data needs to be read, the i-th element in the second data is 1, so that the calculation result output by the data output unit includes the i-th row read from the first data stored in the calculation array;
[0136] When it is not necessary to read the j-th row in the first data, the j-th element in the second data is 0, so that the calculation result output by the data output unit does not include the j-th row in the first data.
[0137] In some embodiments, the computing array includes a combinational logic circuit and at least one register, the combinational logic circuit includes multiple levels of subcircuits, and the subcircuits at different levels are used to implement different levels of combinational logic;
[0138] A register is located between sub-circuits of adjacent levels and is connected to the adjacent sub-circuits respectively. The register is used to store output data of the adjacent upper-level sub-circuit.
[0139] In some embodiments, the computing array includes a plurality of computing units, each computing unit includes a multiplier and a shutdown circuit, and the method further includes:
[0140] By shutting down the circuit, a control signal is generated according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, and the control signal is used to control whether the multiplier of the corresponding computing unit works.
[0141] In some embodiments, each computing unit further includes a data selection circuit and a plurality of storage units;
[0142] The method also includes:
[0143] Storing at least one element of the first data by any storage unit;
[0144] By means of the data selection circuit, in a current operation cycle of the matrix multiplication operation, at least one storage unit is selected from a plurality of storage units of the corresponding computing unit as a data input terminal, so as to input an element of the first data stored in the at least one storage unit into a multiplier of the corresponding computing unit;
[0145] Generate a control signal by shutting down the circuit according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, including:
[0146] By shutting down the circuit, a control signal is generated according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, and the control signal is used to control whether the data selection circuit and / or the multiplier of the corresponding computing unit work.
[0147] The in-memory computing module in the embodiment of the present application can be set in an electronic device.
[0148] Figure 6 The figure is a possible logical structure diagram of an electronic device provided in an embodiment of the present application.
[0149] The specific type of the electronic device is not limited here.
[0150] Exemplarily, the electronic device may be a terminal device, a single server, or a server cluster.
[0151] There is no restriction on the type of terminal device.
[0152] Exemplarily, the terminal device may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a terminal in industrial control, a terminal in self driving, a terminal in remote medical, a terminal in a smart grid, a terminal in transportation safety, a terminal in a smart city, a terminal in a smart home, a terminal in the Internet of Things (IoT), a wearable device, a robot, or a combination of one or more thereof.
[0153] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0154] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0155] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0156] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0157] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk, etc. Various media that can store program codes.
Claims
1. An in-memory computing module, characterized in that: The in-memory computing module includes a computing array, a data writing unit and a data input unit; The data writing unit is used to write the first data into the computing array so that the computing array stores the first data; The data input unit is used to input second data into the calculation array; The calculation array is used to store the first data, and after the data input unit inputs the second data, perform a matrix multiplication operation according to the first data and the second data to obtain a calculation result.
2. The in-memory computing module according to claim 1, characterized in that: The in-memory computing module also includes a data output unit; The data output unit is used to output the calculation result.
3. The in-memory computing module according to claim 1 or 2, characterized in that: When it is necessary to read the i-th row in the first data, the i-th element in the second data is 1, so that the calculation result output by the data output unit includes the i-th row read from the first data stored in the calculation array; When there is no need to read the j-th row in the first data, the j-th element in the second data is 0, so that the calculation result output by the data output unit does not include the j-th row in the first data.
4. The in-memory computing module according to any one of claims 1 to 3, characterized in that: The computing array includes a combinational logic circuit and at least one register, wherein the combinational logic circuit includes multiple levels of subcircuits, and subcircuits of different levels are used to implement combinational logic of different levels; One of the registers is located between the sub-circuits of adjacent levels and is connected to the adjacent sub-circuits respectively. The register is used to store the output data of the adjacent upper-level sub-circuit.
5. The in-memory computing module according to any one of claims 1 to 4, characterized in that: The computing array includes a plurality of computing units, each computing unit includes a multiplier and a shutdown circuit; The shutdown circuit is used to generate a control signal according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, and the control signal is used to control whether the multiplier of the corresponding computing unit works.
6. The in-memory computing module according to claim 5, characterized in that: Each computing unit also includes a data selection circuit and a plurality of storage units; Any of the storage units is used to store at least one element of the first data; The data selection circuit is used to select at least one storage unit from a plurality of storage units of the corresponding computing unit as a data input terminal in a current operation cycle of the matrix multiplication operation, so as to input an element of the first data stored in the at least one storage unit into a multiplier of the corresponding computing unit; The shutdown circuit is used to generate a control signal according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, and the control signal is used to control whether the data selection circuit and / or the multiplier of the corresponding computing unit are working.
7. A data processing method, characterized in that: Applied to an in-memory computing module, the in-memory computing module includes a computing array, a data writing unit and a data input unit, the method includes: Writing the first data into the computing array through the data writing unit, so that the computing array stores the first data; Inputting second data into the computing array through the data input unit; The first data is stored through the calculation array, and after the second data is input by the data input unit, a matrix multiplication operation is performed according to the first data and the second data to obtain a calculation result.
8. The method according to claim 7, characterized in that The in-memory computing module further includes a data output unit, and the method further includes: The calculation result is outputted through the data output unit.
9. The method according to claim 7 or 8, characterized in that: When it is necessary to read the i-th row in the first data, the i-th element in the second data is 1, so that the calculation result output by the data output unit includes the i-th row read from the first data stored in the calculation array; When there is no need to read the j-th row in the first data, the j-th element in the second data is 0, so that the calculation result output by the data output unit does not include the j-th row in the first data.
10. The method according to any one of claims 7 to 9, characterized in that: The computing array includes a plurality of computing units, each computing unit includes a multiplier and a shutdown circuit, and the method further includes: Through the shutdown circuit, a control signal is generated according to the elements of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation, and the control signal is used to control whether the multiplier of the corresponding computing unit works.
11. The method according to claim 10, characterized in that Each computing unit also includes a data selection circuit and a plurality of storage units; The method further comprises: Storing at least one element of the first data by any of the storage units; By means of the data selection circuit, in a current operation cycle of the matrix multiplication operation, at least one storage unit is selected from a plurality of storage units of the corresponding computing unit as a data input terminal, so as to input an element of the first data stored in the at least one storage unit into a multiplier of the corresponding computing unit; The generating of the control signal by the shutdown circuit according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle of the matrix multiplication operation comprises: Through the shutdown circuit, a control signal is generated according to the element of the second data input to the multiplier of the corresponding computing unit in the current operation cycle, and the control signal is used to control whether the data selection circuit of the corresponding computing unit and / or the multiplier of the corresponding computing unit works.
12. A chip, characterized in that: The chip includes the in-memory computing module described in any one of claims 1-6.
13. An electronic device, characterized in that: The electronic device comprises the in-memory computing module as described in any one of claims 1-6.