In-memory computing system, method and device with dynamically adjustable precision

Abstract

The present invention provides an in-memory computing system, method, and device with dynamically adjustable precision. The system includes a module for dynamically dividing eigenvalue bits, a configurable in-memory computing module for dynamically dividing weight bits, and a multiplication-accumulation result shift-addition module. The module for dynamically dividing eigenvalue bits is used to dynamically and adjustably split eigenvalues ​​into different precisions and send them to the configurable in-memory computing module; the configurable in-memory computing module contains eight identical in-memory computing core units, each of which stores different bits of weights and can dynamically map 1 bit of the weight; the multiplication-accumulation result shift-addition module is used to shift-add the multiplication-accumulation results of the in-memory computing core units with different weights to obtain a complete output eigenvalue. The present invention supports acceleration of neural network model inference with dynamically adjustable precision, thereby improving the efficiency of neural network inference.

Description

Technical Field

[0001] The present invention relates to the field of in-memory computing technology, and in particular to an in-memory computing system, method, and device with dynamically adjustable precision. Background Art

[0002] As satellite missions become increasingly complex, the demands for data processing capabilities and computing efficiency continue to rise. However, traditional computing systems typically rely on data transfer between the central processing unit (CPU) and memory. The main bottleneck of this architecture is the "memory wall" problem between the processor and memory: data transmission speeds are far lower than processing speeds, limiting overall system performance. To address this issue, in-memory computing technology has emerged. By embedding processing units within memory, in-memory computing reduces the need for data transmission and significantly improves data processing speed and energy efficiency. This architecture is suitable for applications requiring large amounts of data processing and low latency, such as real-time satellite image processing, data compression, and complex signal processing.

[0003] However, in satellite applications, different tasks require varying computational accuracy, necessitating the system's ability to dynamically adjust computational accuracy. For example, image processing tasks require high-precision calculations to ensure accurate results, while low-power tasks require lower precision to conserve computing resources. In this context, a dynamically adjustable in-memory computing system and method can flexibly adjust computational accuracy based on specific mission requirements, optimizing computational efficiency and extending the satellite's mission lifespan.

[0004] Therefore, designing an in-memory computing system and method that can dynamically adjust the calculation accuracy can not only improve the satellite's data processing capabilities, but also flexibly respond to different accuracy requirements in various complex tasks, thus having important technical significance and broad application prospects in satellite applications. Summary of the Invention

[0005] In view of the defects in the prior art, the purpose of the present invention is to provide an in-memory computing system, method and device with dynamically adjustable precision.

[0006] According to the present invention, a dynamically adjustable precision in-memory computing system includes: a dynamic feature value bit division module, a configurable in-memory computing module for dynamic weight bit division, and a multiplication and accumulation result shift addition module;

[0007] The eigenvalue bit dynamic division module is used to dynamically and adjustably split the eigenvalue into different precisions and send it to the configurable in-memory calculation module, the configurable in-memory calculation module includes multiple in-memory calculation core units, each in-memory calculation core unit stores different bits of weights, and the multiplication and accumulation result shift addition module is used to shift and add the multiplication and accumulation results of the in-memory calculation core units with different weights to obtain a complete output eigenvalue;

[0008] The bit copying and bit splitting functions are set in the characteristic value bit dynamic division module. The bit copying is: copying 1 bit value twice and sending it to two forks; the bit splitting is: splitting 2 bit value into two 1 bit values ​​and sending them to two forks.

[0009] Preferably, the bit copying and bit splitting functions include:

[0010] If the input characteristic value is 8 bits abcdefgh, the allocation relationship is a:#0, b:#1, c:#2, d:#3, e:#4, f:#5, g:#6, h:#7; if the input characteristic value is 4 bits abcd, the allocation relationship is a:#0, a:#1, b:#2, b:#3, c:#4, c:#5, d:#6, d:#7; if the input characteristic value is 2 bits ab, the allocation relationship is a:#0, a:#1, a:#2, a:#3, b:#4, b:#5, b:#6, b:#7; if the input characteristic value is 1 bit a, the allocation relationship is a:#0, a:#1, a:#2, a:#3, a:#4, a:#5, a:#6, a:#7.

[0011] Preferably, the configurable in-memory computing module includes eight identical in-memory computing core units, each of which stores a different bit of the weight and can dynamically map 1 bit of the weight.

[0012] Preferably, the multiplication and accumulation result shift addition module is provided with addition and splicing functions.

[0013] Preferably, the adding function is: adding the two outputs to obtain a result; the splicing function is: directly splicing the two outputs to obtain a result.

[0014] Preferably, the criterion for selecting addition or splicing is whether the two outputs belong to the same output channel of the neural network layer. If so, the addition function is selected; if not, the splicing function is selected.

[0015] According to the present invention, a method for in-memory calculation with dynamically adjustable precision is provided, comprising:

[0016] Step S1, mapping the neural network weight parameters to the configurable in-memory computing module according to the accuracy;

[0017] Step S2: sending the eigenvalues ​​to the eigenvalue bit dynamic division module for replication and division according to the precision;

[0018] Step S3: Send the multiplication and accumulation results calculated in the memory to the multiplication and accumulation result shift addition module for addition and splicing to obtain the output eigenvalue.

[0019] According to the present invention, a memory computing device with dynamically adjustable precision is provided, the device includes a processor and a memory, the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to execute the steps of the memory computing method with dynamically adjustable precision.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention provides an in-memory computing system and method with dynamically adjustable precision. By designing a module for dynamically dividing eigenvalue bits, a configurable in-memory computing module for dynamically dividing weight bits, and a multiplication and accumulation result shift and addition module, the system supports acceleration of neural network model inference with dynamically adjustable precision, thereby improving the efficiency of neural network inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0023] Figure 1 A schematic diagram of the structure of an in-memory computing system with dynamically adjustable precision provided by the present invention;

[0024] Figure 2 A schematic diagram of the structure of the eigenvalue bit dynamic division module provided by the present invention;

[0025] Figure 3 A schematic diagram of the structure of a configurable in-memory computing module with dynamic weight bit partitioning provided by the present invention;

[0026] Figure 4 A schematic diagram of the structure of the multiplication and accumulation result shift addition module provided by the present invention;

[0027] Figure 5 A schematic flow chart of an in-memory calculation method with dynamically adjustable precision provided by the present invention.

[0028] Description of reference numerals:

[0029] DETAILED DESCRIPTION

[0030] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0031] Figure 1 A schematic diagram of the structure of an in-memory computing system with dynamically adjustable precision provided by the present invention is shown in FIG. Figure 1 As shown, the present invention provides an in-memory computing system with dynamically adjustable precision, including a dynamic feature value bit division module 101, a configurable in-memory computing module 102 for dynamic weight bit division, and a multiplication and accumulation result shift addition module 103, wherein:

[0032] The eigenvalue bit dynamic division module 101 is used to dynamically and adjustably split the eigenvalue into different precisions and send the result to the configurable in-memory calculation module 102 .

[0033] The number of bits determines the accuracy of calculations. For example, INT8 precision indicates that calculations are performed using fixed-point 8-bit eigenvalues ​​and fixed-point 8-bit weights. INT4 precision indicates that calculations are performed using fixed-point 4-bit eigenvalues ​​and fixed-point 4-bit weights. INT8 and higher precision are generally considered high-precision calculations, while INT4 and lower precision are considered low-precision calculations. Higher precision means more bits for weights and eigenvalues, but requires more hardware storage and computation. Lower precision means fewer bits for weights and eigenvalues, but requires less hardware storage and computation.

[0034] In the present invention, the eigenvalue bit dynamic division module 101 is provided with bit copying and segmentation functions.

[0035] The configurable in-memory computing module 102 includes eight identical in-memory computing core units 1021 , each of which stores a different bit of the weight and can dynamically map 1 bit of the weight.

[0036] In the present invention, the configurable in-memory computing module 102 is provided with an in-memory computing function.

[0037] The multiplication-accumulation result shift-addition module 103 is used to perform shift-addition on the multiplication-accumulation results of the in-memory calculation core units 1021 with different weights to obtain a complete output eigenvalue.

[0038] In the present invention, the multiplication and accumulation result shift addition module 103 is provided with addition and splicing functions.

[0039] The present invention provides an in-memory computing system and method with dynamically adjustable precision. By designing a module for dynamically dividing eigenvalue bits, a configurable in-memory computing module for dynamically dividing weight bits, and a multiplication and accumulation result shift and addition module, the system supports acceleration of neural network model inference with dynamically adjustable precision, thereby improving the efficiency of neural network inference.

[0040] In the present invention, Figure 2 The structural diagram of the dynamic division module of the characteristic value bit provided by the present invention is as follows: Figure 2 As shown, the present invention provides the eigenvalue bit copying and segmentation functions. Figure 2 The black dot in the middle indicates the selection of the copy or split function. If it is the copy function, the 1-bit value is copied twice and sent to two forks; if it is the split function, the 2-bit value is split into two 1-bit values ​​and sent to two forks. If the input characteristic value is 8 bits abcdefgh, then its final distribution relationship is a:#0, b:#1, c:#2, d:#3, e:#4, f:#5, g:#6, h:#7; if the input characteristic value is 4 bits abcd, then its final distribution relationship is a:#0, a:#1, b:#2, b:#3, c:#4, c:#5, d:#6, d:#7; if the input characteristic value is 2 bits ab, then its final distribution relationship is a:#0, a:#1, a:#2, a:#3, b:#4, b:#5, b:#6, b:#7; if the input characteristic value is 1 bit a, then its final distribution relationship is a:#0, a:#1, a:#2, a:#3, a:#4, a:#5, a:#6, a:#7.

[0041] Specifically, the eigenvalue bit copying and splitting function is implemented by performing copying and splitting operations based on the number of consecutive zeros in the input eigenvalue, with the goal of removing consecutive zeros from the 8-bit input eigenvalue and then reproducing it to 8 bits through copying. For example, for the 8-bit eigenvalue "01001000," the consecutive zeros in bits 0 and 1 are removed, and the consecutive zeros in bits 4 and 5 are removed, leaving a remaining eigenvalue of 4 bits. This remaining eigenvalue is then copied to "00111100," which is then split into the 8-bit sequence of "0," "0," "1," "1," "1," "0," and transmitted to the in-memory computing core units 1021 #0 through #7. For example, consider the 8-bit characteristic value "00100000". If the consecutive 0s in the 0th, 1st, 2nd, and 3rd bits are removed, and the consecutive 0s in the 6th and 7th bits are removed, the remaining characteristic value is 2 bits. This is then copied to "11110000", which is then split into 8 bits of "1", "1", "1", "1", "0", "0", "0", and "0", and transmitted to the computing core units 1021 in the memory #0-#7. This automated copying and splitting function can automatically determine the location of the consecutive 0s and reduce the computational overhead of this portion, automatically performing the bit copying and splitting operations.

[0042] In the present invention, Figure 3 The structural diagram of the configurable in-memory computing module for dynamic weight bit partitioning provided by the present invention is as follows: Figure 3 As shown, the present invention provides an in-memory computing function. Different bits of the weight are dynamically divided into different in-memory computing core units 1021. If the weight is 8 bits ijklmnop, the final distribution relationship is p:#0, o:#1, n:#2, m:#3, l:#4, k:#5, j:#6, i:#7; if the input eigenvalue is 4 bits ijkl, the final distribution relationship is l:#0, k:#1, j:#2, i:#3, l:#4, k:#5, j:#6, i:#7; if the input eigenvalue is 2 bits ij, the final distribution relationship is j:#0, i:#1, j:#2, i:#3, j:#4, i:#5, j:#6, i:#7; if the input eigenvalue is 1 bit i, the final distribution relationship is i:#0, i:#1, i:#2, i:#3, i:#4, i:#5, i:#6, i:#7.

[0043] The structural diagram described above is merely illustrative. The in-memory computing module may be composed of an analog in-memory computing module or a digital in-memory computing module, and the selected memory may be composed of SRAM, RRAM, Flash, etc. Some or all of these units may be selected according to actual needs to achieve the purpose of the present embodiment. Persons of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0044] In the present invention, Figure 4 The schematic diagram of the structure of the multiplication and accumulation result shift addition module provided by the present invention is as follows: Figure 4 As shown, the present invention provides addition and splicing functions. Figure 4 The black dot in the figure indicates whether to select the addition or concatenation function. If the addition function is selected, the two outputs are added together; if the concatenation function is selected, the two outputs are directly concatenated. The criterion for selecting addition or concatenation is whether the two outputs belong to the same output channel of the neural network layer. If so, the addition function is selected; otherwise, the concatenation function is selected.

[0045] Figure 5 A flowchart of an in-memory calculation method with dynamically adjustable precision provided by the present invention is shown as follows: Figure 5 As shown, the present invention provides an in-memory computing method with dynamically adjustable precision, comprising:

[0046] Step S1: Map the neural network weight parameters to the configurable in-memory computing modules according to the precision. For example, 8-bit weights are mapped to the in-memory computing core units 1021 #0 to #7, 4-bit weights are mapped to two groups of in-memory computing core units 1021 #0 to #3 and #4 to #7, and so on.

[0047] Step S2: Send the eigenvalues ​​to the eigenvalue bit dynamic division module for replication and division according to the precision. For example, the 8-bit eigenvalues ​​are divided and sent to the #0~#7 in-memory computing core unit 1021. The 4-bit eigenvalues ​​are copied and divided, with the 0th bit sent to #0~#1, the 1st bit sent to #2~#3, the 2nd bit sent to #4~#5, the 3rd bit sent to #6~#7, and so on.

[0048] In step S3, the multiplication and accumulation results calculated in the memory are sent to the multiplication and accumulation result shift and addition module for addition and splicing. For example, the 8-bit * 8-bit multiplication and accumulation results of #0~#7 are shifted and added to obtain the output eigenvalue, and the 4-bit * 4-bit multiplication and accumulation results of #0~#3 and #4~#7 are shifted and added and spliced ​​to obtain the output eigenvalue.

[0049] The present invention also provides an in-memory computing device with dynamically adjustable precision, comprising: a processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to execute the steps of the in-memory computing method with dynamically adjustable precision as described above.

[0050] The present invention also provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of the in-memory calculation method with dynamically adjustable precision as described above.

[0051] It should be noted that those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as "circuits," "modules," or "platforms."

[0052] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When at least one processor of a user device executes the computer-executable instructions, the user device executes the various possible methods described above. The computer-readable medium includes a computer storage medium and a communication medium, wherein the communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in the user device. Of course, the processor and the storage medium can also exist as discrete components in the communication device.

[0053] The present application also provides a program product, which includes a computer program. The computer program is stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium. At least one processor executes the computer program so that the server implements any method of the above-mentioned embodiments of the present invention.

[0054] The program product may utilize any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0055] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. An in-memory computing system with dynamically adjustable precision, characterized in that: include: Eigenvalue bit dynamic division module, weight bit dynamic division configurable in-memory calculation module and multiplication and accumulation result shift addition module; The eigenvalue bit dynamic division module is used to dynamically and adjustably split the eigenvalue into different precisions and send it to the configurable in-memory calculation module, the configurable in-memory calculation module includes multiple in-memory calculation core units, each in-memory calculation core unit stores different bits of weights, and the multiplication and accumulation result shift addition module is used to shift and add the multiplication and accumulation results of the in-memory calculation core units with different weights to obtain a complete output eigenvalue; The bit copy and bit splitting functions are set in the characteristic value bit dynamic division module. The bit copy is to copy a 1-bit value into two copies and send them to two forks; the bit splitting is to split a 2-bit value into two 1-bit values ​​and send them to two forks; The method for implementing the eigenvalue bit copying and splitting function is as follows: copying and splitting operations are performed according to the number of consecutive 0s in the input eigenvalue, with the purpose of removing consecutive 0s in the 8-bit input eigenvalue and supplementing it to 8 bits through copying; for the 8-bit eigenvalue "01001000", the consecutive 0s in the 0th and 1st bits are removed, and the consecutive 0s in the 4th and 5th bits are removed. The remaining eigenvalue at this time is 4 bits, which is then copied to "00111100", and then split into 8 bits "0", "0", "1", "1". , "1", "1", "0", "0", and transmit it to the computing core units in the memory #0~#7; the 8-bit eigenvalue "00100000", remove the consecutive 0s in the 0th, 1st, 2nd, and 3rd bits, and remove the consecutive 0s in the 6th and 7th bits, the remaining eigenvalue is 2 bits, and then it is copied to "11110000", which is then split into 8 bits of "1", "1", "1", "1", "0", "0", "0", "0", and transmitted to the computing core units in the memory #0~#7; The automatic copy and split function can automatically determine the location of consecutive 0s and reduce the computational overhead of this part, automatically performing bit copy and split operations.

2. The in-memory computing system with dynamically adjustable precision according to claim 1, characterized in that: The bit copying and bit splitting functions include: If the input characteristic value is 8 bits abcdefgh, the allocation relationship is a:#0, b:#1, c:#2, d:#3, e:#4, f:#5, g:#6, h:#7; if the input characteristic value is 4 bits abcd, the allocation relationship is a:#0, a:#1, b:#2, b:#3, c:#4, c:#5, d:#6, d:#7; if the input characteristic value is 2 bits ab, the allocation relationship is a:#0, a:#1, a:#2, a:#3, b:#4, b:#5, b:#6, b:#7; if the input characteristic value is 1 bit a, the allocation relationship is a:#0, a:#1, a:#2, a:#3, a:#4, a:#5, a:#6, a:#7.

3. The in-memory computing system with dynamically adjustable precision according to claim 1, characterized in that: The configurable in-memory computing module includes eight identical in-memory computing core units, each of which stores different bits of weight and can dynamically map 1 bit of the weight.

4. The in-memory computing system with dynamically adjustable precision according to claim 1, characterized in that: The multiplication and accumulation result shift addition module is provided with addition and splicing functions.

5. The in-memory computing system with dynamically adjustable precision according to claim 4, characterized in that: The addition function is to add the two outputs to get the result; the splicing function is to directly splice the two outputs to get the result.

6. The in-memory computing system with dynamically adjustable precision according to claim 5, characterized in that: The criterion for selecting addition or splicing is whether the two outputs belong to the same output channel of the neural network layer. If so, the addition function is selected; if not, the splicing function is selected.

7. A method for in-memory computing with dynamically adjustable precision, based on the in-memory computing system with dynamically adjustable precision according to any one of claims 1 to 6, characterized in that: include: Step S1, mapping the neural network weight parameters to the configurable in-memory computing module according to the accuracy; Step S2: sending the eigenvalues ​​to the eigenvalue bit dynamic division module for replication and division according to the precision; Step S3: Send the multiplication and accumulation results calculated in the memory to the multiplication and accumulation result shift addition module for addition and splicing to obtain the output eigenvalue.

8. An in-memory computing device with dynamically adjustable precision, characterized in that: The device includes a processor and a memory, wherein the memory stores executable program instructions. When the processor calls the program instructions in the memory, the processor is used to execute the steps of the in-memory calculation method with dynamically adjustable precision as described in claim 7.

Images