A memory-computing integrated device and a computing method

Abstract

Embodiments of the present application provide a memory-computing integrated device and a computing method, and relate to the chip technical field, which are used to reduce the computing overhead when performing neural network computing and improve the computing efficiency. The method comprises: calculating a plurality of input data by a bit width calculation module to obtain a plurality of effective data; inputting the plurality of effective data to a calculation module; obtaining the calculation result of each column in the calculation array according to the plurality of effective data and the bit of the weight data stored in each memory calculation unit by the calculation module; inputting the calculation result of each column to a result processing module; and finally performing weighted calculation on the calculation result of each column by the result processing module to obtain the final result. The embodiments of the present application are used in the process of the memory-computing integrated device for computing.

Description

Technical Field

[0001] This application relates to the field of chip technology, and in particular to an in-memory computing device and computing method. Background Technology

[0002] In recent years, neural networks (NNs) have developed rapidly and are widely used in fields such as robotics, speech recognition, image recognition, natural language processing, and expert systems. The core computation of neural networks is matrix-vector multiplication, which is characterized by computational and memory-intensive operations. When using general-purpose chips for neural network computations, these chips have significant limitations in terms of power consumption, performance, and size. Therefore, to improve the computational efficiency of neural networks, it is necessary to customize dedicated chips (neural network accelerators) for neural network computations.

[0003] In-memory computing devices retain the storage and read / write functions of the original storage circuitry while also supporting parallel multiplication and addition operations. This reduces data movement and improves energy efficiency, providing a highly efficient solution for neural network accelerator design. However, in-memory computing devices typically require expanding multi-bit data into single-bit / low-bit (e.g., 2-bit or 4-bit) data based on the data width before performing calculations and then merging the results. This process involves numerous expansion operations, resulting in significant overhead. Summary of the Invention

[0004] This application provides an in-memory computing device and a computing method, which can reduce overhead and improve computing efficiency when performing neural network calculations.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide an in-memory computing device, which includes a bit-width calculation module, a calculation module, and a result processing module. The calculation module includes a calculation array, which includes multiple storage calculation units for storing weight data. The bit-width calculation module calculates multiple input data to obtain multiple valid data, inputting the multiple valid data into the calculation module. Each input data corresponds one-to-one with a valid data, and a first input data corresponds to a first valid data, with the bit width of the first input data being greater than the bit width of the first valid data. The calculation module calculates the result of each column in the calculation array based on the bits of the multiple valid data and the weight data, inputting the result of each column into the result processing module. A column's calculation result is the sum of the products of the same bit of the multiple valid data and the calculations of a column's storage calculation units. The result processing module performs a weighted calculation on the calculation results of each column to obtain the final result.

[0007] Therefore, compared with the prior art, which expands multi-bit input data into multiple single-bit / low-bit input data according to the data bit width for input and calculation, resulting in too many expansion calculations and large overhead, the calculation method provided in this application can dynamically calculate the effective data of the input data, thereby only calculating the effective bits of the input data, effectively reducing the number of calculations performed by the calculation module, reducing the calculation overhead, and improving the calculation efficiency of the in-memory computing device.

[0008] In one possible design, the bit-width calculation module is specifically used to perform mask calculations on multiple input data to obtain mask values. Based on the valid bits of the mask values, multiple valid data are determined, and these valid data are input bit by bit into the calculation module, allowing the calculation module to perform calculations on each bit of the valid data. Therefore, the calculation method provided in this application enables the bit-width calculation module to obtain the valid data of the input data through mask calculations and input the valid data bit by bit into the calculation module, thereby significantly reducing the number of calculations performed by the calculation array.

[0009] In one possible design, when the computing array receives the Nth bit corresponding to multiple valid data points, where N is an integer greater than or equal to 0, the computing array calculates the product of the Nth bit corresponding to each of the multiple valid data points and the bit of the weight data. The computing module also includes an accumulation circuit, which sums the products calculated by the same column of storage computing units in the computing array to obtain the sum of the products calculated by each column of storage computing units in the computing array. Therefore, in the computing method provided by this application, the computing module calculates the Nth bit corresponding to each of the multiple valid data points each time. The number of calculations performed by the computing module corresponds to the bit width of the valid data. Since the bit width of the valid data is smaller than the bit width of the input data, the number of calculations performed by the computing array can be effectively reduced.

[0010] In one possible design, the weight data includes multiple weight data types, and the in-memory computing device further includes a weight bit-width configuration module. This module stores bit-width information for the multiple weight data types, including the bit-width of each weight data type and the identifier of the starting column of each weight data type in the computing array. At least two of the weight data types have different bit widths. Therefore, the calculation method provided in this application, compared to existing technologies where the weight data has a fixed bit-width, cannot achieve mixed-precision weight data calculations, resulting in low computational efficiency. This application utilizes the bit-width information of multiple weight data types stored in the weight bit-width configuration module to deploy and calculate weight data with multiple bit widths within a single computing array, thereby supporting mixed-precision weight data calculations and effectively improving the computational efficiency of the in-memory computing device.

[0011] In one possible design, the in-memory computing device also includes a control module, which is used to write multiple weight data into multiple storage computing units according to the bit width information. Thus, the computing method provided in this application allows the control module to deploy weight data to each storage computing unit in the computing array according to the bit width information, thereby including the bit width of multiple weight data in a single computing array, realizing mixed-precision calculation of weight data, and improving the computing efficiency of the in-memory computing device.

[0012] In one possible design, the control module is further configured to determine the valid bits of the mask value bit by bit. When any bit of the mask value is determined to be valid, a first control signal and a second control signal are generated. The first control signal instructs the calculation module to calculate the sum of the products of each column of storage computing units in the calculation array. The second control signal instructs the result processing module to perform a weighted calculation on the sum of the products of multiple columns of storage computing units corresponding to each type of weight data in the calculation array according to the bit width information, obtaining multiple weighted results for the Nth bit corresponding to multiple valid data, where each weighted result corresponds to a type of weight data. Thus, in the calculation method provided by this application, the control module can generate control signals based on the valid bits of the mask value to control the calculation module and the result processing module. Since the number of valid bits of the mask value is the same as the bit width of the valid data, and is usually smaller than the bit width of the input data, generating control signals based on the valid bits of the mask value can reduce the number of calculations performed by the calculation module and reduce computational overhead.

[0013] In one possible design, the control module is further configured to generate a third control signal when the bit width of the mask value is equal to the bit width of the input data. This third control signal instructs the result processing module to perform a weighted calculation based on the bit weights corresponding to the valid bits of the mask value and the weighted results of each bit of the multiple valid data, yielding a final result that includes the weighted result for each type of weighted data. Therefore, the calculation method provided in this application, after the calculation module finishes its calculation, allows the result processing module to perform a weighted calculation based on the bit width information and the bit weights of the valid bits of the mask value. This accurately transforms the calculation results of multiple single-bit valid data and multi-bit weighted data into the calculation results of multi-bit input data and multi-bit weighted data. While maintaining the same calculation accuracy, this effectively reduces the number of calculations and lowers overhead.

[0014] Secondly, embodiments of this application provide a calculation method applied to a memory computing device. The memory computing device includes a computing array, which includes multiple storage computing units for storing weight data. The method includes: calculating multiple input data to obtain multiple valid data, where each input data corresponds one-to-one with a valid data, and a first input data corresponds to a first valid data, where the bit width of the first input data is greater than the bit width of the first valid data; obtaining the calculation result for each column in the computing array based on the bits of the multiple valid data and the weight data, wherein a column's calculation result is the sum of the products of the same bit of the multiple valid data and the calculations of a column's storage computing units; and performing a weighted calculation on the calculation results of each column to obtain the final result. The beneficial effects achieved in this second aspect can be found in the beneficial effects described in the first aspect.

[0015] In one possible design, calculating multiple input data to obtain multiple valid data includes: performing mask calculation on multiple input data to obtain mask values, determining multiple valid data based on the valid bits of the mask values, and obtaining the calculation result of each column in the calculation array based on the multiple valid data and the bits of the weight data.

[0016] In one possible design, the calculation result of each column in the computing array is obtained based on the bits of multiple valid data and weight data, including: when the computing array receives the Nth bit corresponding to multiple valid data respectively, where N is an integer greater than or equal to 0, the product of the Nth bit corresponding to the multiple valid data and the bit of the weight data is calculated, and the products calculated by the storage computing units in the same column of the computing array are added together to obtain the sum of the products calculated by the storage computing units in each column of the computing array.

[0017] In one possible design, the method further includes: storing bit width information of multiple weight data, the bit width information including the bit width of each weight data and the identifier of the starting column of each weight data in the calculation array, wherein at least two of the multiple weight data have different bit widths.

[0018] In one possible design, the weight data includes multiple weight data, and the method further includes writing multiple weight data into multiple storage computing units according to bit width information.

[0019] In one possible design, the method further includes: determining the valid bits of the mask value bit by bit; and generating a first control signal and a second control signal when any bit of the mask value is determined to be valid. The first control signal is used to calculate the sum of the products of each column of storage computing units in the computing array, and the second control signal is used to perform weighted calculation on the sum of the products of multiple columns of storage computing units corresponding to each type of weight data in the computing array according to the bit width information, to obtain multiple weighted results for the Nth bit corresponding to multiple valid data, wherein each weighted result corresponds to a type of weight data.

[0020] In one possible design, the method further includes: when the bit width of the mask value is equal to the bit width of the input data, generating a third control signal, the third control signal being used to perform weighted calculations based on the bit weights corresponding to the valid bits of the mask value and the weighted results of each bit of the multiple valid data, to obtain a final result, the final result including the weighted result of each type of weighted data.

[0021] Thirdly, a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform the methods described in the second aspect and any possible design of the second aspect.

[0022] Fourthly, a computer program product, when run on a computer, causes an electronic device to perform the method described in the second aspect and any possible design of the second aspect.

[0023] For the other beneficial effects mentioned above, please refer to the description of the beneficial effects of the first aspect, which will not be repeated here. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of an analog computing array;

[0025] Figure 2 This is a schematic diagram of a digital computing array;

[0026] Figure 3 A schematic diagram of the structure of an in-memory computing device provided in an embodiment of this application;

[0027] Figure 4 A schematic diagram of a computing array provided for an embodiment of this application;

[0028] Figure 5 A flowchart illustrating a calculation method provided in an embodiment of this application;

[0029] Figure 6 This is a schematic diagram illustrating the calculation of effective data provided in an embodiment of this application;

[0030] Figure 7 A schematic diagram of a computing module provided in an embodiment of this application;

[0031] Figure 8 A schematic diagram of a control module provided in an embodiment of this application;

[0032] Figure 9 A flowchart illustrating a calculation method provided in an embodiment of this application;

[0033] Figure 10 This is a schematic diagram of the structure of an in-memory computing device provided in an embodiment of this application. Detailed Implementation

[0034] For ease of understanding, some concepts related to the embodiments of this application are illustrated below for reference.

[0035] Artificial neural networks (ANNs), also known as neural networks or neural-like networks, are mathematical or computational models that mimic the structure and function of biological neural networks (central nervous systems, such as the brain). They are used to estimate or approximate functions. A neural network consists of a large number of interconnected nodes (neurons). Each node represents a specific output function, called the activation function. The connection between any two nodes represents a weighted value for the signal passing through that connection, called the weight data.

[0036] Neural network accelerator: An application-specific integrated circuit (ASIC) chip suitable for inference or training of artificial neural networks, used to perform neural network calculations and improve the computational efficiency of neural networks.

[0037] In-memory computing: Algorithms are embedded in memory, transferring computer operations from the central processing unit (CPU) to memory, enabling computation within the memory computing unit (Cell), which can significantly reduce data exchange time and energy consumption for data access during computation.

[0038] There are two ways to implement in-memory computing devices: one is to use analog devices (such as resistive random-access memory, ReRAM) to build the computing array, and the other is to use digital devices (such as static random-access memory, SRAM) to build the computing array.

[0039] Figure 1 The diagram shows a schematic of an analog computing array constructed using analog devices. During neural network computation, the analog devices can be understood as storage and computing units, arranged in an array. Analog devices in the same row share a word line, and analog devices in the same column share a bit line. The conductance in the analog devices can be understood as weight data, and the voltage as input data. The input voltage is the same for the same word line. The current value output by each bit line represents the sum of the products of the conductance and voltage of the analog devices (located in the same column) sharing that bit line, which in turn represents the sum of the products of the weight data and the input data for that column. For example, in a 4×4 analog computing array, the conductivities of the first column are G1, G2, G3, and G4, which means the weight data of the first column are G1, G2, G3, and G4. The input voltages of each row are V1, V2, V3, and V4, which means the input data are V1, V2, V3, and V4. The input data are input in parallel. Then the current I1 output by the first column is I1 = G1×V1 + G2×V2 + G3×V3 + G4×V4, which represents the sum of the products of the weight data of this column and the multiple input data.

[0040] Figure 2 The diagram shows a digital computing array constructed using digital devices. During neural network computation, each storage computing unit stores one weight data. The input unit inputs input data to each storage computing unit in the digital computing array. Storage computing units in the same row have the same input data. The weight data and input data are multiplied in the storage computing unit. The multiplication results in the same column are accumulated by an external accumulation circuit to obtain the sum of the products of the weight data and multiple input data in each column.

[0041] Both implementations allow multiple input data to be input in parallel on the row level and multiple multiplication and addition calculations to be performed in parallel on the column level.

[0042] Data bit width: Abbreviated as bit, it represents the number of binary digits transmitted on a bus at one time. A bit is the smallest unit of data storage inside a computer. For example, 11010100 is an 8-bit binary number, meaning its bit width is 8 bits, and it can be called 8-bit data.

[0043] Crossbar (XB): In this application, it refers to a crossbar constructed from storage computing units, each crossbar containing several rows and several columns.

[0044] Place value: The unit value corresponding to each fixed position in a number is called the place value. For multi-digit numbers, the magnitude of the value represented by "1" at a certain position is called the place value of that position. For example, in a decimal number, the place value of the second digit from the right is 10, and the place value of the third digit is 100; while in a binary number, the place value of the second digit from the right is 2, and the place value of the third digit is 4. For an N-ary number, the place value of the i-th digit from the right in the integer part is N. i-1 The place value of the j-th digit from left to right in the fractional part is N. -j .

[0045] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments, unless otherwise stated, "multiple" means two or more.

[0046] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "coupling" means that two or more circuit elements are directly or indirectly connected; for example, coupling A to B can mean that A is directly connected to B, or A is connected to B through C.

[0047] Currently, when neural network accelerators use in-memory computing devices for computation, if the computing array is an analog computing array built with analog devices, it tends to perform low-bit computations due to the limitations of the analog devices' precision and the overhead of analog-to-digital converters (ADCs) and digital-to-analog converters (DACs). For example, both input data and weight data use 16 bits, and the storage computing unit uses 2 bits. That is, each storage computing unit stores 2 bits of data, and 16 bits of weight data requires 8 storage computing units to store. This can be understood as 8 columns of storage computing units representing one column of weight data. During neural network computation, the 16-bit input data is represented as a 0 / 1 voltage sequence of length 16. Each clock cycle, starting from the least significant bit, one bit of input data is input in parallel for computation. That is, the storage computing unit performs one computation per clock cycle, calculating the product of 1 bit of input data and 2 bits of weight data each time. It takes 16 clock cycles to complete the computation of 16 bits of input data and 16 bits of weight data. After each clock cycle, the storage computation unit completes one calculation, and each column storage computation unit obtains a sum of products (the sum of multiple products obtained from parallel calculations of the same single bit of multiple input data). After 16 clock cycles, each column storage computation unit outputs the sum of 16 products obtained from 16 calculations. The eight sums output by eight consecutive columns of storage computation units are then combined using a shift-add operation, resulting in the sum of the products of each column's weight data and multiple input data. This can be understood as... Figure 1 I1 in the middle.

[0048] When the computing array is a digital computing array constructed using digital devices, since digital computing arrays typically tend to perform single / low-bit calculations, multi-bit calculations need to be implemented through multiple single / low-bit calculations. For example, if both input data and weight data use 4 bits, and the storage computing unit is a single-bit multiplier (meaning each storage computing unit stores 1 bit of data), then 4 bits of weight data need to be stored in 4 storage computing units. This can be understood as 4 columns of storage computing units representing one column of weight data. During neural network calculations, the input data is input bit by bit into the storage computing units located in the same row. Each single bit of the input data is multiplied by all bits of the weight data. That is, each single bit of the input data is multiplied by 4 storage computing units (each of which stores one weight data). Each storage computing unit calculates the product of 1 bit of input data and 1 bit of weight data. The product result is a 4-bit data (the product of a single bit of the input data and the 4 storage computing units), and this product result is output to the external accumulation circuit. After each calculation, the external accumulator circuit adds up the multiple product results obtained from the parallel calculation of the same single bit of multiple input data in the same column of weight data, resulting in four accumulated product results corresponding to the four bits of multiple input data. Finally, the external accumulator circuit performs corresponding shifting and summation on these four accumulated product results to obtain the sum of the product of the column of weight data and multiple input data.

[0049] As can be seen, when using in-memory computing devices for computation, it is typically necessary to expand multi-bit input data into multiple single-bit / low-bit input data for input and computation, based on the data bit width. Since the bit width of the multi-bit input data is fixed, the number of expansion calculations is also fixed, regardless of the size of the input data. For example, if the 8-bit input data is 00001010, expanding it into multiple single-bit input data for multiplication requires 8 expansion calculations, multiplying the single-bit input data 0, 0, 0, 0, 1, 0, 1, and 0 respectively. It can be understood that since the result of multiplying the 8-bit input data at the bit position containing the single-bit input data 0 is 0, it can be interpreted that the calculation of the single-bit input data 0 is invalid in these 8 calculations. During computation, most multi-bit data values ​​are small and do not require extensive expansion calculations. Therefore, expanding multi-bit input data into multiple single-bit / low-bit input data for input and computation based on the data bit width results in redundant calculations and significant overhead. Furthermore, when using in-memory computing devices for computation, the weight data bit width is fixed. This means that regardless of the size of the weight data value, the number of storage computing units required to deploy on the computing array is the same, leading to low computational efficiency.

[0050] Therefore, this application proposes an in-memory computing device, which can be understood as a chip, such as a neural network accelerator. Considering that in existing technologies using in-memory computing devices for neural network computation, multi-bit input data is expanded into multiple single-bit / low-bit input data for input and computation based on the data bit width, and the input data bit width and weight data bit width are fixed, resulting in high computational overhead and low computational efficiency, this application, when using an in-memory computing device for neural network computation, calculates multiple input data through a bit width calculation module to obtain multiple valid data corresponding one-to-one with the multiple input data. These valid data are then input into a calculation module, which, based on the multiple valid data and the bit width of the weight data, obtains the calculation result for each column in the computation array. The calculation result for each column is then input into a result processing module, which finally performs a weighted calculation on the calculation results of each column to obtain the final result. This effectively reduces the number of computations performed by expanding the computation array, lowers computational overhead, and improves computational efficiency.

[0051] The in-memory computing device proposed in this application can be applied to computing scenarios, such as neural network computing scenarios. When performing neural network computing, the in-memory computing device calculates the weight data of multiple neural networks and multiple input data.

[0052] like Figure 3 The diagram shows a schematic representation of a memory computing device, which can be a chip. Figure 3 The example chip 300 includes a data processing element (PE) 301, a data exchange module (switch) 302, and an input / output module (TxRx) 303, etc.

[0053] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on chip 300. In other embodiments of this application, chip 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0054] The data processing unit 301 may include one or more data processing units, and each data processing unit includes multiple computing engines. One part of the computing engines is used to perform multiplication-addition calculations in the neural network. In this embodiment, the computing engines used to perform multiplication-addition calculations in the neural network include a bit-width calculation module 3011, a calculation module 3012, a weight bit-width configuration module 3013, a control module 3014, and a result processing module 3015. Another part of the computing engines is used to perform calculations in the neural network such as activation, dot multiplication, dot addition, and division.

[0055] The bit width calculation module 3011 can be used to calculate the valid data of the input data. For example, it can perform a logical OR operation on multiple input data to obtain a mask value, determine multiple valid data of the multiple input data based on the mask value, and input the calculated multiple valid data into the calculation module.

[0056] The calculation module 3012 includes a calculation array and an accumulation circuit. The calculation array includes multiple storage calculation units arranged in an array. Each storage calculation unit can be used to store bits of weight data, such as 1 bit, 2 bits, or 4 bits of multi-bit weight data. See also... Figure 4 The 8×8 computing array shown comprises 8 columns of storage computing units, each column containing 8 storage computing units. Taking the storage computing unit storing 1 bit of data and the weight data using 4 bits as an example, one 4-bit weight data requires 4 storage computing units. This can be understood as 4 columns of storage computing units representing one column of weight data, with one column containing 8 4-bit weight data. The computing array can be used to perform calculations on multiple valid data and multiple weight data. For example, it can perform multiplication calculations on the same bits (single bit / low bit) of multiple valid data and the bits of the weight data stored in each storage computing unit, obtaining multiple product results (the number of product results obtained in one calculation is equal to the number of storage computing units in the computing array), and input these multiple product results into the accumulation circuit.

[0057] The accumulation circuit can be used to accumulate multiple product results output by the computing array. For example, it can accumulate multiple product results obtained from the same column of storage computing units to obtain the sum of the products of each column of storage computing units, and input the sum of the multiple products obtained into the result processing module 3015.

[0058] The weight bit-width configuration module 3013 can be used to store bit-width information for multiple weight data. Each column of weight data represents one type of weight data; therefore, it can be understood that the weight bit-width configuration module 3013 stores bit-width information for multiple columns of weight data. The bit-width of weight data within the same column is the same, while the bit-widths of weight data in different columns may be the same or different. The bit-width information includes the bit-width of each type of weight data and the identifier of the starting column corresponding to each type of weight data in the computation array. This can be understood as including the bit-width of each column of weight data and the identifier of the starting column corresponding to each column of weight data in the computation array. Figure 4Taking the 8×8 computing array shown as an example, the computing array from left to right represents the 0th column of the storage computing unit, the 1st column of the storage computing unit, ..., the 7th column of the storage computing unit. If the bit width of the weight data in the 0th column of the weight bit width configuration module 3013 is 4 bits, and the identifier of the starting column in the computing array is the 0th column storage computing unit, then the weight data in the 0th column is as follows: Figure 4 As shown, it includes the storage and calculation units in columns 0 through 3.

[0059] The control module 3014 can be used to write multiple weight data stored in the memory 302 into multiple storage computing units according to the bit width information in the weight bit width configuration module 3013. The control module 3014 can also be used to generate control signals to control the calculation module 3012 and the result processing module 3015. For example, when the control module 3014 determines that any bit of the mask value obtained by the bit width calculation module 3011 is valid, it generates a first control signal and a second control signal. The first control signal instructs the calculation module 3012 to perform multiplication calculations on the same bit (single bit / low bit) of multiple valid data and the bit of the weight data stored in each storage computing unit, and inputs the sum of the multiple products to the result processing module 3015. The second control signal instructs the result processing module 3015 to perform weighted calculations on the sum of the products of multiple columns of storage computing units corresponding to each weight data in the calculation array according to the bit width information, obtaining multiple weighted results for the Nth bit corresponding to the multiple valid data, where the least significant bit is the 0th bit, and N is an integer greater than or equal to 0. When the control module 3014 determines that the bit width of the mask value is equal to the bit width of the input data, it can also generate a third control signal. The third control signal is used to instruct the result processing module 3015 to perform weighted calculations based on the bit weights corresponding to the valid bits of the mask value and the multiple weighted results of each bit of multiple valid data, so as to obtain the weighted result of each type of weighted data.

[0060] The result processing module 3015 can be used to perform corresponding actions according to the control signals sent by the control module 3014 after receiving them. For example, when the second control signal is received, the module performs a weighted calculation on the sum of the products of the multiple column storage calculation units corresponding to each type of weight data in the calculation array according to the bit width information, to obtain multiple weighted results for the Nth bit corresponding to each of the multiple valid data. When the third control signal is received, the module performs a weighted calculation on the bit weights corresponding to the valid bits of the mask value and the multiple weighted results for each bit of the multiple valid data, to obtain a weighted result for each type of weight data.

[0061] The data exchange module 302 can be used to realize data exchange between various units inside the chip, such as data exchange between the input / output module 303 and multiple data processing units 301.

[0062] The input / output module 303 can be used to receive input data and weight data, and also to output the final result obtained in the data processing unit 301. For example, the input / output module 303 can interact with an external memory (which stores input data and weight data), receive input data and weight data, and input the input data and weight data into the data processing unit 301 through the data exchange module 302. It can also output the final result obtained in the data processing unit 301 to an external memory or an internal cache. Figure 3 (not shown in the image) This application is not limited.

[0063] Using the in-memory computing device provided in this application, the following describes the calculation method proposed in this application for the in-memory computing device in conjunction with the accompanying drawings. Taking the in-memory computing device as a chip as an example, the process of calculating multiple valid data and multiple weight data by calculating the effective data of each input data in multiple input data during the neural network calculation process of the chip is introduced.

[0064] like Figure 5 As shown in the figure, this application provides a calculation method applied to a memory computing device. Taking a chip 300 as an example, the memory computing device includes a bit-width calculation module 3011, a calculation module 3012, and a result processing module 3015. The calculation module includes a calculation array, which includes multiple storage calculation units. Each storage calculation unit stores bits of weight data. (See also...) Figure 4 The description of the computational array shown. The method includes:

[0065] Step 501: Calculate multiple input data to obtain multiple valid data.

[0066] In this process, when expanding the input data for multiplication, the result of multiplication on bits that are 0 is 0, which can be considered invalid. Multiplication on bits that are 1 is considered valid. Therefore, valid input data can be understood as data composed of the valid bits (bits that are 1). Multiple input data correspond one-to-one with multiple valid data, and the first input data corresponds to the first valid data, with the bit width of the first input data being greater than the bit width of the first valid data. The first input data can be any one of the multiple input data. Since the result obtained by performing neural network calculations on the input data and on the valid data is the same, the accuracy of the calculation result can be guaranteed. Furthermore, because the bit width of the first input data is greater than the bit width of the first valid data, the number of multiplications performed on the valid data is less than the number of multiplications performed on the expanded input data, effectively reducing the number of calculations by the computation module and lowering overhead.

[0067] Specifically, step 501 involves the bit width calculation module 3011 calculating multiple input data to obtain multiple valid data, and then inputting the multiple valid data into the calculation module 3012.

[0068] For example, the bit width calculation module 3011 can obtain multiple input data from the input / output module 303, calculate the valid data of each input data in the multiple input data, and input the calculated multiple valid data into the calculation module 3012 for calculation.

[0069] In some optional embodiments, step 501 includes: performing mask calculation on multiple input data to obtain a mask value, and determining multiple valid data based on the valid bits of the mask value. Specifically, the bit width calculation module 3011 performs mask calculation on multiple input data to obtain a mask value, and determines multiple valid data based on the valid bits of the mask value.

[0070] The valid data for multiple input data sets needs to be determined based on these multiple input data sets. Methods for calculating the valid data for multiple input data sets include masking the input data. Taking a logical OR operation as an example, the valid data for each input data set is determined bit by bit. That is, the same bits of the multiple input data sets are logically ORed from the highest bit to the lowest bit, resulting in a mask value. Based on the valid bits (bits that are 1) of the mask value, the valid data for each input data set can be determined.

[0071] For example, such as Figure 6As shown, taking four 8-bit input data sets as an example, the four 8-bit input data sets are 00001101, 00010100, 00001001, and 00000001. Following the order from the most significant bit to the least significant bit, a logical OR operation is performed on the same bits of these four 8-bit input data sets. For example, if the most significant bit (the 7th bit) of all four 8-bit input data sets is 0, the logical OR result is 0. If the least significant bit (the 0th bit) of the four 8-bit input data sets is 1, 0, 1, and 1 respectively, the logical OR result is 1. After performing a logical OR operation on each bit of these four 8-bit input data sets, the mask value is 00011101. The valid bits of this mask value are the 4th, 3rd, 2nd, and 0th bits. The numbers corresponding to the 4th, 3rd, 2nd, and 0th bits of the multiple input data sets are extracted, which are the valid data for each input data set. Therefore, the valid data for the four 8-bit input data are 0111, 1010, 0101, and 0001, respectively.

[0072] After obtaining the valid data for each of the multiple input data, the bit-width calculation module 3011 inputs the multiple valid data bit by bit to the calculation module 3012. This allows the calculation module 3012 to perform calculations bit by bit on the multiple valid data and the bits of the weight data stored in each storage calculation unit, obtaining the calculation result for each column in the calculation array. The result of the calculation by the calculation module 3012 on the multiple valid data is consistent with the result of the calculation on the multiple input data. For example, using... Figure 6 Taking the multiple valid data 0111, 1010, 0101, and 0001 as an example, the multiple valid data are input into the calculation module 3012 bit by bit in parallel, from the most significant bit to the least significant bit. For example, the most significant bits 0, 1, 0, and 1 of the multiple valid data are first input into the calculation module 3012 in parallel, and then the remaining bits are input into the calculation module 3012 in parallel, so that the calculation module 3012 performs calculations on the multiple valid data bit by bit.

[0073] In some embodiments, the bit-width calculation module 3011 can also determine the valid bits of multiple input data bit by bit (i.e., calculate the mask value of the four input data bit by bit). When any bit is determined to be valid, the valid bit of the multiple input data is input to the calculation module 3012 for calculation. For example, taking the four input data as 00001101, 00010100, 00001001, and 00000001, the bit-width calculation module 3011 determines the valid bits of the four input data bit by bit. When the fourth bit is determined, it is determined to be valid, and the fourth bit of the four input data is input to the calculation module 3012 for calculation. This process continues, and invalid bits are not input to the calculation module 3012.

[0074] Understandably, the bit width calculation module 3011 acquires multiple input data from the input / output module 303. Each time multiple input data is acquired, the valid data of the acquired input data is calculated, and the calculated valid data is input to the calculation module 3012. The bit width of the valid data is related to the multiple input data acquired each time; the bit width of the calculated valid data may be the same or different each time. Therefore, the bit width calculation module 3011 can dynamically calculate the valid data of multiple input data.

[0075] In some optional embodiments, mask calculation can also be performed in other ways, such as determining whether the high-order bits of the mask are zero by determining the maximum value of multiple input data. This application does not limit this method. Furthermore, when there is only one input data, the mask value is that input data, and the bit width calculation module 3011 can directly determine the valid data of the input data based on whether each bit of the input data is 1.

[0076] In some optional embodiments, the bit width calculation module 3011 may also expand the calculated multiple valid data into the remaining low bits and input them into the calculation module 3012, depending on the different devices and circuit implementations. For example, the multiple valid data may be expanded into 2 bits and input into the calculation module 3012. This application does not limit this.

[0077] Step 502: Based on the bits of multiple valid data and weight data, obtain the calculation result of each column in the calculation array.

[0078] In each column, the calculation result is the sum of the products of the same bit from multiple valid data and the calculations of a column of storage computing units. Specifically, step 502 involves the calculation module 3012 calculating the result of each column in the calculation array based on the multiple valid data and the bit values ​​of the weight data stored in each storage computing unit, and then inputting the result of each column into the result processing module 3015. The calculation module 3012 includes a calculation array, which includes multiple storage computing units. A weight data is expanded into multiple single-bit / low-bit weight data stored in multiple storage computing units. The bit values ​​of the weight data stored in each storage computing unit can be understood as a portion of the bit values ​​of a weight data stored in each storage computing unit; this portion of the bit values ​​can be a single bit or multiple bits. The calculation module 3012 performs multiplication calculations on the multiple valid data input from the bit-width calculation module 3011 and the bits of the weight data stored in each storage calculation unit. Specifically, each valid data is input into a different row in the calculation array, that is, each valid data corresponds to a row of storage calculation units, and each valid data is multiplied by the bits of the weight data stored in the corresponding storage calculation unit. After the calculation is completed, each column in the calculation array corresponds to a calculation result. The calculation result of each column is the sum of the products of multiple valid data and that column. The calculation module 3012 inputs the calculation result of each column into the result processing module 3015.

[0079] In some optional embodiments, step 502 includes: when the computing array receives the Nth bit corresponding to the multiple valid data respectively, the computing array calculates the product of the Nth bit corresponding to the multiple valid data and the bit of the weight data.

[0080] Where N is an integer greater than or equal to 0. The statement "each valid data point is multiplied by a bit of the weight data stored in each corresponding storage computing unit" specifically refers to multiple calculations. In each calculation, a single bit of each valid data point is multiplied by a bit of the weight data stored in each corresponding storage computing unit. The number of calculations is determined by the bit width of the valid data. For example, 4 bits of valid data will be calculated 4 times, with each calculation performed on a single bit of that valid data point. Furthermore, in each calculation, the same single bits of multiple valid data points are calculated in parallel; that is, the Nth bits corresponding to multiple valid data points are calculated in parallel. This can be understood as the computing array performing one calculation when it receives the Nth bits corresponding to multiple valid data points.

[0081] For example, Figure 7An example of a computing module 700 is provided, which includes a 4×8 computing array 701. The effective data and weight data of the input data are both 4 bits, and the storage computing unit is 1 bit. That is, the storage computing unit stores 1 bit of weight data and performs multiplication calculation with 1 bit of input data. The effective data of multiple input data are a1b1c1d1, a2b2c2d2, a3b3c3d3 and a4b4c4d4, and the weight data in a column of computing array 701 are A1B1C1D1, A2B2C2D2, A3B3C3D3 and A4B4C4D4, respectively. The third bit (highest bit) corresponding to these multiple valid data points are a1, a2, a3, and a4, respectively. When the computing array 701 receives a1, a2, a3, and a4, it inputs a1, a2, a3, and a4 into different rows of the computing array 701. Specifically, it inputs a1, a2, a3, and a4 into each storage computing unit in the corresponding row. Taking a1 as an example, a1 is multiplied by the bits of the weight data stored in each storage computing unit in the corresponding row to obtain multiple product results, such as a1×A1, a1×B1, a1×C1, and a1×D1. Similarly, a2, a3, and a4 are also multiplied to obtain multiple product results. After the calculations for a1, a2, a3, and a4 are completed, it signifies the completion of one calculation by the computing array 701. It is understandable that the above calculation process needs to be performed 4 times for 4 bits of valid data to complete the calculation of the entire valid data. After the calculation of a1, a2, a3 and a4 is completed, b1, b2, b3 and b4, c1, c2, c3 and c4, and d1, d2, d3 and d4 will be calculated 3 times each.

[0082] In some optional embodiments, the computing module further includes an accumulation circuit that adds the products calculated by the same column of storage computing units in the computing array to obtain the sum of the products calculated by each column of storage computing units in the computing array.

[0083] After each calculation by the computing array, the accumulation circuit accumulates the multiple results obtained by the computing array. Specifically, it accumulates the multiple product results obtained by the same column of the computing array storage computing unit to obtain the calculation result of each column in the computing array, that is, the sum of the products of each column of the computing array storage computing unit, and inputs the sum of the products of each column of the computing array storage computing unit into the result processing module 3015.

[0084] For example, such as Figure 7As shown, after the computation array 701 completes the calculations for a1, a2, a3, and a4, the accumulation circuit 702 accumulates the multiple product results calculated by the same column of the computation array 701. The accumulation circuit 702 accumulates the multiple product results calculated by the 0th column of the computation array 701 to obtain the sum of the products of the 0th column of the computation array 701, S3 = a1×A1 + a2×A2 + a3×A3 + a4×A4. It also accumulates the multiple product results calculated by the 1st column of the computation array 701 to obtain the sum of the products of the 1st column of the computation array 701, S2 = a1×B1 + a2×B2 + a3×B3 + a4×B4. Finally, it accumulates the products of the 2nd column of the computation array 701... The multiple product results obtained from the initial calculation are accumulated to obtain the sum of the products of the second column of storage calculation units, S1 = a1×C1 + a2×C2 + a3×C3 + a4×C4. The multiple product results obtained from the third column of storage calculation units in the calculation array 701 are accumulated to obtain the sum of the products of the third column of storage calculation units, S0 = a1×D1 + a2×D2 + a3×D3 + a4×D4, and so on. The sums of the multiple products S3, S2, S1, and S0 are then input into the result processing module 3015. It can be understood that each time the calculation array 701 receives multiple valid data corresponding to the Nth bit and completes one calculation, the accumulation circuit 702 calculates the sum of the products obtained from each column of storage calculation units and inputs the calculated sum of the products of each column of storage calculation units into the result processing module 3015. Four bits of valid data require the accumulation circuit 702 to input four calculation results into the result processing module 3015.

[0085] In some optional embodiments, the in-memory computing device further includes a weight bit width configuration module, which stores bit width information of various weight data.

[0086] The weight data includes various weight data types, and the weight bit width configuration module can be used for... Figure 3 The weight bit width configuration module 3013 in the array. Bit width information includes the bit width of each weight data type and the identifier of the starting column of each weight data type in the calculation array. A weight data type can be understood as a column of weight data, for example... Figure 7 The 4×8 computing array 701 in the diagram represents a column of weight data (one type of weight data) stored in columns 0 through 3. The computing array 701 can include multiple columns of weight data (multiple types of weight data), with at least two types of weight data having different bit widths. In the computing array 701, columns 0 through 3 represent the weight data of column 0, which has a bit width of 4 bits. The starting column of the weight data in the computing array 701 is identified by the column 0 storage unit.

[0087] For example, the bit width information is shown in Table 1 below, corresponding to... Figure 7 The computation array 701 shown in Table 1 includes three types of weight data: column 0, column 1, and column 2. The first type of weight data (column 0) has a 4-bit width, with the starting column identified as column 0 (the storage computation unit). That is, storage computation units 0 through 3 represent the first type of weight data. The second type of weight data (column 1) has a 2-bit width, with the starting column identified as column 4 (the storage computation unit). That is, storage computation units 4 and 5 represent the second type of weight data. The third type of weight data (column 2) has a 2-bit width, with the starting column identified as column 6 (the storage computation unit). That is, storage computation units 6 and 7 represent the third type of weight data.

[0088] Table 1

[0089] Weighted data identifier Bit width Start column identifier Weight data in column 0 4 bits Column 0 stores the computation unit. Column 1 Weight Data 2 bits Column 4 storage computing unit Column 2 Weight Data 2 bits Column 6 storage computing unit

[0090] As can be seen, the weight bit width configuration module 3013 of this application can store bit width information of multiple weight data, and at least two of the weight data have different bit widths. That is, a single computing array of this application can include weight data with multiple bit widths, and supports the calculation of weight data with mixed precision, thus effectively improving the computing efficiency of the in-memory computing device.

[0091] In some optional embodiments, the in-memory computing device further includes a control module that writes various weight data into multiple storage computing units based on bit width information.

[0092] The control module can be Figure 3 The control module 3014 is used in the memory. The control module 3014 can write various weight data stored in the memory 302 into multiple storage calculation units according to the bit width information in the weight bit width configuration module 3013.

[0093] For example, the bit width information shown in Table 1 above and Figure 7Taking the computing array 701 as an example, the control module 3014 writes each bit of the weight data (A1B1C1D1, A2B2C2D2, A3B3C3D3 and A4B4C4D4) stored in the memory 302 into each of the storage computing units in the 0th column storage computing unit to the 3rd column storage computing unit according to the bit width and starting column identifier of the weight data in the 0th column shown in Table 1. This process continues until all the weight data in the memory 302 is written into each storage computing unit of the computing array 701 according to the bit width information shown in Table 1.

[0094] In some optional embodiments, the control module determines the valid bits of the mask value bit by bit, and when any bit of the mask value is determined to be valid, it generates a first control signal and a second control signal.

[0095] The control module 3014 generates control signals to control the calculation module 3012 and the result processing module 3015 based on the mask value calculated by the bit-width calculation module 3011. Specifically, the bit-width calculation module 3011 inputs the mask value bit by bit into the control module 3014. The control module 3014 determines whether each bit of the mask value is valid (i.e., whether it is 1). When any bit of the mask value is determined to be valid, the control module 3014 generates a first control signal and a second control signal. It can be understood that the control module 3014 generates the first and second control signals as many times as there are valid bits in the mask value.

[0096] The first control signal is used to instruct the calculation module 3012 to calculate the sum of the products of each column of storage calculation units in the calculation array, which can be understood as instructing the calculation module 3012 to perform one operation. Figure 7 The calculation is shown for the Nth bit corresponding to multiple valid data, and the sum of the products of each column of storage computing units in the computing array is obtained.

[0097] The second control signal instructs the result processing module 3015 to perform a weighted calculation on the sum of the products of multiple storage computing units corresponding to each weight data in the computing array, based on the bit width information, to obtain multiple weighted results for the Nth bit corresponding to multiple valid data. Since the bit width information reveals which columns of storage computing units in the computing array correspond to a particular weight data (a column of weight data), the result processing module 3015 can determine the multiple storage computing units corresponding to each weight data based on the bit width information. The result processing module 3015 performs a weighted calculation on the sum of the products of the multiple storage computing units corresponding to each weight data, specifically by performing a weighted calculation based on the bit weight of the weight data bits. For example, for a column of storage computing units corresponding to the lowest bit (0th bit) of the weight data, during the weighted calculation, the sum of the products of this storage computing unit is multiplied by 2.0 After multiplying and summing, the column of stored computational units corresponding to the most significant 2 bits of the weighted data is used for weighted calculation. The sum of the products in this column is then summed to 2. 2 Multiplication and summation are performed, and multiplication with powers of 2 can be achieved through bit shifting in the circuit. It can be understood that if the calculation array includes several weighted data types (several columns of weighted data), the result processing module 3015 will obtain several weighted results after performing a single weighted calculation. When the calculation array includes multiple weighted data types, the result processing module 3015, upon receiving the second control signal, will obtain multiple weighted results for the Nth bit corresponding to multiple valid data types, where each weighted result corresponds to one type of weighted data.

[0098] For example, such as Figure 8 The control module 800 shown includes a first comparator, which compares whether the bit input to the control module 800 is the same as 1. If they are the same, a first control signal and a second control signal are generated; if they are different, no first control signal and a second control signal are generated.

[0099] by Figure 6 Taking a mask value of 00011101 as an example, the bit width calculation module 3011 inputs the mask value into the control module 800 bit by bit, from the most significant bit to the least significant bit. First, the bit width calculation module 3011 inputs the most significant bit (the 7th bit) of the mask value (0) into the control module 800. The first comparator in the control module 800 compares 0 and 1; they are different, thus determining that the bit is invalid and no first or second control signal is generated. Similarly, when the bit width calculation module 3011 inputs the 4th bit (1) of the mask value into the control module 800, the first comparator in the control module 800 compares 1 and 1; they are the same, determining that the bit is valid and generating the first and second control signals.

[0100] The first control signal generated by the control module 800 is input to the calculation module 3012, instructing the calculation module 3012 to perform a calculation on the Nth bit corresponding to each of the multiple valid data. Correspondingly, the first control signal generated based on the 4th bit of the mask value instructs the calculation module 3012 to perform a calculation on the highest bit (3rd bit) of the multiple valid data (0, 1, 0, and 0), obtaining the sum of the products of each column of storage calculation units in the calculation array, i.e. Figure 7 The numbers S3, S2, S1, and S0 are shown.

[0101] The second control signal generated by the control module 800 is input to the result processing module 3015, instructing the result processing module 3015 to perform a weighted calculation on the sum of multiple products generated by the calculation module 3012 in a single calculation. The bit width information is shown in Table 1. Figure 7 Taking the calculation module 700 as an example, the result processing module 3015 determines, based on the 4-bit width of the weight data in column 0 of Table 1 and the starting column identifier being the storage calculation unit in column 0, that the storage calculation unit in column 0-column 3 of the calculation array 701 represents the first type of weight data (weight data in column 0). The calculation module 700 then performs a weighted calculation on the sum of the products S3, S2, S1, and S0 corresponding to the storage calculation units in column 0-column 3, respectively, with the bit weights of the weight data bits, resulting in a weighted result sum0. This sum0 corresponds to one type of weight data (weight data in column 0), and sum0 = S3 × 2. 3 +S2×2 2 +S1×2 1 +S0×2 0 And so on. It is understandable that the calculation module 700 can obtain 3 weighted results, corresponding to the weighted data in column 0, column 1, and column 2, respectively.

[0102] In some optional embodiments, the control module generates a third control signal when it determines that the bit width of the mask value is equal to the bit width of the input data.

[0103] Among them, by the pair Figure 6 As shown in the description, the bit width of the input data is the same as the bit width of the mask value. Since the mask value is input bit by bit into the control module 3014, when the control module 3014 determines that the bit width of the mask value is the same as the bit width of the input data, it can determine that the mask value input is complete, thus generating the third control signal. This can be understood as the control module 3014 outputting the third control signal after outputting the first and second control signals.

[0104] The third control signal is used to instruct the result processing module 3015 to perform weighted calculations based on the bit weights corresponding to the valid bits of the mask value and the weighted results of each bit of multiple valid data to obtain the final result, which includes the weighted result of each type of weighted data.

[0105] For example, such as Figure 8As shown, the control module 800 also includes a counter and a second comparator. For each bit of the input mask value, the counter increments by 1, recording the bit width of the mask value. The second comparator compares the bit width of the mask value recorded in the counter with the bit width of the input data. If they are the same, a second control signal is generated; otherwise, no second control signal is generated.

[0106] by Figure 6 The input data shown is 8 bits wide, with a mask value of 00011101 as an example. The valid bits of the mask value are the 4th, 3rd, 2nd, and 0th bits, with bit weights of 2 for each valid bit. 4 2 3 2 2 and 2 0 The bit width calculation module 3011 inputs the highest bit (7th bit) of the mask value (0) into the control module 800. The first comparator in the control module 800 compares 0 and 1 and finds they are different, thus determining that the bit is invalid and not generating the first and second control signals. At the same time, the counter records the bit width of the mask as 1. The second comparator compares the bit width (1) of the mask recorded by the counter and finds it is different from the bit width (8) of the input data, thus not generating the third control signal. Similarly, when the bit width calculation module 3011 inputs the lowest bit (0th bit) of the mask value (1) into the control module 800, the first comparator in the control module 800 compares 1 and 1 and finds them to be the same, determining that the bit is valid and generating the first and second control signals. At the same time, the counter records the bit width of the mask as 8. The second comparator compares the bit width (8) of the mask recorded by the counter and finds it is the same as the bit width (8) of the input data, thus generating the third control signal.

[0107] The third control signal generated by the control module 800 is input to the result processing module 3015. At this time, the result processing module 3015 has received the second control signal four times, meaning it has performed four weighted calculations on the sum of multiple products from the calculation module 3012. Each weighted calculation yields multiple weighted results (e.g., the first weighted calculation yields sum0, etc.). The third control signal instructs the result processing module 3015 to perform another weighted calculation based on the bit weights corresponding to the valid bits of the mask value and the multiple weighted results obtained from the previous weighted calculations, to obtain the final result. Taking the result processing module 3015's first weighted calculation of the weighted data in column 0 yielding sum0, the second weighted calculation yielding sum1, the third weighted calculation yielding sum2, and the fourth weighted calculation yielding sum3 as an example, the multiple weighted results sum0, sum1, sum2, and sum3 are respectively related to the bit weights of the valid bits of the mask value. 4 2 3 2 2and 2 0 Correspondingly, this can be understood as the bit weight of the valid bits of the mask value corresponding to the weighted result being the same as the bit weight of the valid bits of the mask value corresponding to the second control signal that generated the weighted result. The result processing module 3015 performs weighted calculation again to obtain the final result out0 = sum0 × 2 of the weighted data in column 0. 4 +sum1×2 3 +sum2×2 2 +sum3×2 0 And so on, the final result includes the weighted result of each type of weighted data. It can be understood that the calculation module 700 can obtain three final results, corresponding to the weighted data in column 0, column 1, and column 2, respectively.

[0108] Step 503: Perform a weighted calculation on the results of each column to obtain the final result.

[0109] Specifically, step 503 involves the result processing module 3015 performing a weighted calculation on the results of each column to obtain the final result. The calculation result of each column is the sum of the products of the storage calculation units in each column, which can be understood as the calculation results S3, S2, S1, and S0 in step 502 above. The weighted calculation of the results of each column by the result processing module 3015 specifically involves performing a weighted calculation based on the bit weight of the weighted data bits to obtain multiple sum values, and then performing a weighted calculation based on the bit weight corresponding to the valid bits of the mask value to obtain multiple out values, thus obtaining the final result. Please refer to the description of the control module 3014 (control module 800) above, which will not be elaborated on here.

[0110] In some optional embodiments, the input data and weight data include unsigned numbers and signed numbers. The calculation method for unsigned numbers can be found in the examples in the embodiments of this application. Signed numbers can be calculated using methods such as two's complement calculation and difference calculation. This application does not limit the calculation method.

[0111] Therefore, the calculation method provided in this application can be applied to in-memory computing devices, such as chips. During neural network calculations, a bit-width calculation module calculates multiple input data, and inputs multiple valid data points of the calculated input data into the calculation module. The calculation module then obtains the calculation result for each column in the calculation array based on the multiple valid data points and the bit bits of the weight data stored in each storage calculation unit. The calculation result for each column is then input into the result processing module. Finally, the result processing module performs a weighted calculation on the calculation results of each column to obtain the final result. Compared to the prior art, which expands multi-bit input data into multiple single-bit / low-bit input data points based on the data bit width for input and calculation, resulting in too many expansion calculations and significant overhead, the bit-width calculation module of this application can dynamically calculate the valid data of the input data, thereby calculating only the valid bits of the input data, effectively reducing the number of calculations by the calculation module and lowering overhead. Furthermore, existing technologies cannot achieve mixed-precision calculation of weighted data, resulting in low computational efficiency. This application can utilize the bit width information of various weighted data stored in the weight bit width configuration module to realize the deployment and calculation of weighted data with multiple bit widths in a single computing array, thereby supporting mixed-precision calculation of weighted data and effectively improving the computational efficiency of the in-memory computing device.

[0112] With the above Figure 5 The provided calculation method corresponds to, in Figure 3 Based on the structure of the in-memory computing device shown, such as Figure 9 As shown in the diagram, this application provides a flowchart of a calculation method. The bit-width calculation module is represented by module 3011, the calculation module by module 3012, the weight bit-width configuration module by module 3013, the control module by module 3014, and the structure processing module by module 3015. Multiple input data are 00011, 00101, and 00010. The calculation array is a 3×3 array, and the storage unit uses 1 bit. Taking the weight bit-width configuration module 3013 as an example, it stores only one type of weight data. The calculation process includes:

[0113] Step 1: The bit width calculation module 3011 calculates multiple input data to obtain the mask value of multiple input data and the valid data corresponding to each input data, and inputs the multiple valid data into the calculation module 3012.

[0114] The input data includes 00011, 00101, and 00010. A mask calculation is performed on these input data (taking a logical OR operation as an example). The calculated mask value is 00111, thus determining the valid data as 011, 101, and 010. These 011, 101, and 010 are then input bit by bit into the calculation module. See the description of step 501 above for details; further elaboration is omitted here.

[0115] Step 2: The control module 3014 writes multiple weight data into multiple storage and computing units according to the weight bit width configuration module 3013.

[0116] The weight bit-width configuration module 3013 stores only one type of weight data, with a bit width of 3 bits, and the starting column identifier is the 0th column storage calculation unit. Taking multiple weight data sets of 101, 011, and 111 as an example, the control module 3014 writes the multiple weight data sets into multiple storage calculation units according to the bit-width information in the weight bit-width configuration module 3013. (See [link / reference]). Figure 9 The computing array is shown. See the description of the control module above for details; further elaboration will not be repeated here.

[0117] Step 3: The control module 3014 generates a first control signal and a second control signal based on the valid bits of the mask value calculated by the bit width calculation module 3011.

[0118] In this process, the most significant bit (the fourth bit) of the mask value, 0, is first input to the control module 3014. The control module 3014 determines that this most significant bit is invalid and does not generate the first and second control signals. Next, the third bit of the mask value, 0, is input to the control module 3014. The control module 3014 determines that this third bit is invalid and does not generate the first and second control signals. Then, the second bit of the mask value, 1, is input to the control module 3014. The control module 3014 determines that this second bit of the mask value is valid and generates the first and second control signals. The first control signal generated based on the second bit of the mask value is used to control the calculation module 3012 to calculate the highest (second bit) bits (0, 1, and 0) of multiple valid data, resulting in S0 = 0×1 + 1×0 + 0×1 = 0, S1 = 0×0 + 1×1 + 0×1 = 1, and S2 = 0×1 + 1×1 + 0×1 = 1, corresponding to the 0th column storage calculation unit, the 1st column storage calculation unit, and the 2nd column storage calculation unit, respectively. These are then input to the result processing module 3015. The second control signal generated based on the second bit of the mask value is used to control the result processing module 3015 to perform a weighted calculation on S0, S1, and S2 according to the bit weights of the weighted data bits, resulting in sum = 0×2. 2 +1×2 1 +1×2 0=3. Next, the first bit (1) of the mask value is input to the control module 3014. The control module 3014 determines that the first bit of the mask value is a valid bit and generates a first control signal and a second control signal. The first control signal generated based on the first bit of the mask value is used to control the calculation module 3012 to calculate the first bits (1, 0, and 1) of multiple valid data, obtaining S0' = 1×1 + 0×0 + 1×1 = 2, S1' = 1×0 + 0×1 + 1×1 = 1, and S2' = 1×1 + 0×1 + 1×1 = 2, corresponding to the 0th column storage calculation unit, the 1st column storage calculation unit, and the 2nd column storage calculation unit, respectively, and inputting them to the result processing module 3015. The second control signal generated based on the first bit of the mask value is used to control the result processing module 3015 to perform a weighted calculation on S0', S1', and S2' according to the bit weights of the weighted data bits, obtaining sum' = 2×2 2 +1×2 1 +2×2 0 =12. Finally, the 0th bit of the mask value (1) is input to the control module 3014. The control module 3014 determines that the 0th bit of the mask value is a valid bit and generates a first control signal and a second control signal. The first control signal generated based on the 0th bit of the mask value is used to control the calculation module 3012 to calculate the 0th bits (1, 1, and 0) of multiple valid data, resulting in S0'' = 1×1 + 1×0 + 0×1 = 1, S1'' = 1×0 + 1×1 + 0×1 = 1, and S2'' = 1×1 + 1×1 + 0×1 = 2, which correspond to the 0th column storage calculation unit, the 1st column storage calculation unit, and the 2nd column storage calculation unit, respectively, and are then input to the result processing module 3015. The second control signal generated based on the 0th bit of the mask value is used to control the result processing module 3015 to perform a weighted calculation on SO'', S1'', and S2'' according to the bit weights of the weighted data bits, resulting in sum'' = 1 × 2. 2 +1×2 1 +2×2 0 =8. For details, please refer to the description of the control module 3014 mentioned above, which will not be elaborated on here.

[0119] Step 4: When the control module 3014 determines that the bit width of the mask value is equal to the bit width of the input data, it generates a third control signal.

[0120] Specifically, when control module 3014 determines that the width of the mask value is 5 bits, it generates a third control signal. This third control signal controls result processing module 3015 to perform a weighted calculation on sum, sum', and sum'' obtained in step 3 according to the bit weights of the valid bits of the mask value, resulting in out = sum × 2. 2 +sum'×2 1 +sum''×20 =44, which is the final result. See the description of control module 3014 above for details; further explanation is omitted here.

[0121] At this point, the in-memory computing device has completed the calculation of multiple input data and multiple weight data. It is understood that steps 1-4 above only use the example of a computing array containing one type of weight data (a column of weight data); in reality, there may be multiple types of weight data. The calculation method provided in this application dynamically calculates the effective data of multiple input data, performing calculations only on the effective bits of the input data. This effectively reduces the number of calculations by the computing module, lowers overhead, and also supports mixed-precision calculations of weight data, improving the computing efficiency of the in-memory computing device.

[0122] The calculation results using the YOLOv3-tiny target detection model according to the calculation method provided in this application are shown in Table 2 (dataset is COCO2017val dataset). Taking the prior art using an 8-bit model (8-bit weight data) with 100% bit operands and array computations as an example, when using the 8-bit model and the in-memory computing device of this application, the number of bit operands can be reduced to 81.38% of the prior art while maintaining computational accuracy, and the number of array computations can be reduced to 78.31% of the prior art. When using a 4 / 8-bit hybrid model (weight data including 4 bits and 8 bits) and the in-memory computing device of this application, the number of bit operands can be reduced to 69.14% of the prior art while maintaining computational accuracy, and the number of array computations can be reduced to 72.23% of the prior art. It can be seen that the method provided in this application can effectively reduce the number of computations, and when the weight data is calculated with mixed precision, the number of computations can be significantly reduced, thus effectively reducing computational overhead and improving computational efficiency.

[0123] Table 2

[0124] Model Bit operands Array computation times 8-bit model 100％ 100％ 8-bit model + in-memory computing device of this application 81.38％ 78.31％ 4 / 8-bit hybrid model + in-memory computing device of this application 69.14％ 72.23％

[0125] It is understood that, in order to achieve the aforementioned functions, the in-memory computing device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0126] This application embodiment can divide the above-described in-memory computing device into functional modules based on the method example described above. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated modules can be implemented in hardware or as software functional modules. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0127] When using integrated units, such as Figure 10 As shown in the illustration, this application discloses a memory computing device 1000, which can be the chip 300 in the above embodiments. The memory computing device 1000 may include a processing module, a storage module, and a communication module. The processing module can be used to control and manage the operation of the memory computing device 1000, for example, it can support the memory computing device 1000 in executing the steps performed by the bit width calculation module 3011, the calculation module 3012, the weight bit width configuration module 3013, the control module 3014, and the result processing module 3015. The storage module can be used to support the memory computing device 1000 in storing program code and data, for example, it can be used to store input data and weight data. The communication module can be used to support communication between the memory computing device 1000 and other devices, for example, it can be used to input multiple input data and weight data from external devices, and it can also be used to output the final result obtained by the result processing module 3015 to an external device.

[0128] Of course, the unit modules in the aforementioned in-memory computing device 1000 include, but are not limited to, the aforementioned processing module, storage module, and communication module.

[0129] The processing module can be a processor or a controller. It can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor can also be a combination of functions that implement computing capabilities, such as a combination of one or more microprocessors, a neural network processing unit (NPU), a digital signal processing unit (DSP), and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other external devices.

[0130] For example, the processing module can be a processor 1001, the storage module can be a memory 1002, and the communication module can be a communication interface 1003. The in-memory computing device 1000 provided in this embodiment can be... Figure 3The chip 300 shown. The processor 1001, memory 1002, communication interface 1003, etc. can be connected together, for example, through a bus.

[0131] This application also provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the aforementioned method steps to implement the computation method in the above embodiments.

[0132] This application also provides an electronic device, which includes one or more communication interfaces and one or more processors, wherein the communication interfaces and processors are interconnected by a line, and the processor receives and executes computer instructions from the memory of the electronic device through the communication interface, so that the electronic device performs the above-mentioned related method steps to realize the calculation method in the above embodiments.

[0133] This application also provides a computer-readable storage medium storing computer program code, which, when computer instructions are executed on a computer or processor, causes the computer or processor to perform the calculation method described in the above embodiments.

[0134] Embodiments of this application also provide a computer program product, which includes computer instructions. When the computer instructions are run on a computer or processor, they cause the computer or processor to perform the aforementioned related steps to implement the calculation method executed by the electronic device in the above embodiments.

[0135] In this embodiment, the in-memory computing device, electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0136] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0139] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0140] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A storage-and-computing integrated device, characterized in that, The in-memory computing device includes a bit-width computing module, a computing module, and a result processing module; the computing module includes a computing array, and the computing array includes multiple storage computing units, which are used to store weight data. The bit width calculation module is used to perform mask calculation on multiple input data to obtain a mask value, and determine multiple valid data based on the valid bits of the mask value. The bit width of the valid data is smaller than the bit width of the input data. The multiple valid data are input bit by bit into the calculation; The calculation module is used to multiply the multiple valid data bits by the weight data bits to obtain the calculation result of each column in the calculation array. The number of calculations is determined by the bit width of the valid data. The result processing module is used to perform weighted calculations on the results of the calculation module to obtain the final result.

2. The in-memory computing device according to claim 1, characterized in that, When the computing array receives the Nth bit corresponding to each of the plurality of valid data, where N is an integer greater than or equal to 0, The computing array is used to calculate the product of the Nth bit corresponding to the plurality of valid data and the bit of the weight data; The calculation module also includes an accumulation circuit; The accumulation circuit is used to add the products calculated by the same column of storage computing units in the computing array to obtain the sum of the products calculated by each column of storage computing units in the computing array.

3. The in-memory computing device according to claim 1 or 2, characterized in that, The weight data includes multiple weight data, and the in-memory computing device also includes a weight bit width configuration module; The weight bit width configuration module is used to store bit width information of the multiple weight data. The bit width information includes the bit width of each weight data and the identifier of the starting column of each weight data in the computing array. Among the multiple weight data, at least two weight data have different bit widths.

4. The in-memory computing device according to claim 3, characterized in that, The in-memory computing device also includes a control module; The control module is used to write the various weight data into the multiple storage and computing units according to the bit width information.

5. The in-memory computing device according to claim 4, characterized in that, The control module is also used to determine the valid bits of the mask value bit by bit, and when any bit of the mask value is determined to be valid, generate a first control signal and a second control signal. The first control signal is used to instruct the computing module to calculate the sum of the products calculated by each column of storage computing units in the computing array; The second control signal is used to instruct the result processing module to perform a weighted calculation on the sum of the products of the multiple column storage calculation units corresponding to each weight data in the calculation array according to the bit width information, so as to obtain multiple weighted results of the Nth bit corresponding to the multiple valid data, and each weighted result in the multiple weighted results corresponds to a weight data.

6. The in-memory computing device according to claim 5, characterized in that, The control module is further configured to generate a third control signal when the bit width of the mask value is equal to the bit width of the input data; the third control signal is configured to instruct the result processing module to perform weighted calculations based on the bit weights corresponding to the valid bits of the mask value and the weighted results of each bit of the multiple valid data to obtain the final result, the final result including the weighted result of each type of weighted data.

7. A calculation method, characterized in that, The method is applied to a storage-computing device, the storage-computing device including a computing array, the computing array including multiple storage computing units, the multiple storage computing units being used to store weight data; the method includes: A mask calculation is performed on multiple input data to obtain a mask value. Multiple valid data are determined based on the valid bits of the mask value. The bit width of the valid data is smaller than the bit width of the input data. The calculation results for each column in the calculation array are obtained by multiplying the multiple valid data bits by the weight data bits one by one, and the number of calculations is determined by the bit width of the valid data. The calculation results are weighted to obtain the final result.

8. The method according to claim 7, characterized in that, The step of obtaining the calculation result of each column in the computing array based on the plurality of valid data and the weight data bits includes: when the computing array receives the Nth bit corresponding to the plurality of valid data respectively, where N is an integer greater than or equal to 0, Calculate the product of the Nth bit corresponding to each of the multiple valid data and the bit of the weight data; The products calculated by the storage computing units in the same column of the computing array are added together to obtain the sum of the products calculated by the storage computing units in each column of the computing array.

9. The method according to claim 7 or 8, characterized in that, The weight data includes multiple weight data types, and the method further includes: The bit width information of the various weight data is stored. The bit width information includes the bit width of each weight data and the identifier of the starting column of each weight data in the computing array. Among the various weight data, at least two weight data have different bit widths.

10. The method according to claim 9, characterized in that, The method further includes: The various weight data are written into the multiple storage and computing units according to the bit width information.

11. The method according to claim 10, characterized in that, The method further includes: The effective bits of the mask value are determined bit by bit. When any bit of the mask value is determined to be effective, a first control signal and a second control signal are generated. The first control signal is used to calculate the sum of the products calculated by each column of storage computing units in the computing array; the second control signal is used to perform weighted calculation on the sum of the products of multiple columns of storage computing units corresponding to each weight data in the computing array according to the bit width information, so as to obtain multiple weighted results of the Nth bit corresponding to the multiple valid data, and each weighted result in the multiple weighted results corresponds to a weight data.

12. The method according to claim 11, characterized in that, The method further includes: When the bit width of the mask value is determined to be equal to the bit width of the input data, a third control signal is generated; the third control signal is used to perform weighted calculation based on the bit weight corresponding to the effective bits of the mask value and the weighted results of each bit of the multiple effective data to obtain the final result, the final result including the weighted result of each type of weighted data.

13. A computer-readable storage medium, characterized in that, The device stores computer instructions that, when executed on the electronic device, cause the electronic device to perform the method described in any one of claims 7-12.

Images