Multiplication and addition operation device and control method for multiplication and addition operation

Abstract

The present invention discloses a multiply-accumulate operation device and a control method therefor. The multiply-accumulate operation device includes a feature information filter and an in-memory arithmetic unit. The feature information filter records multiple specified bits of multiple feature information, compares the received input information with the specified bits to generate a comparison result, and generates a screening address based on the comparison result. The in-memory arithmetic unit records all bits of the feature information. The in-memory arithmetic unit performs a multiply-accumulate operation on the feature information and the input information according to the screening address to generate an operation result.

Description

Technical Field

[0001] The present invention relates to a multiply-accumulate operation device and a control method thereof for the multiply-accumulate operation, and more particularly to a multiply-accumulate operation device and a control method thereof that can reduce power consumption. Background Art

[0002] With the progress of electronic technology, artificial intelligence has also developed vigorously. In order to provide a more efficient artificial intelligence computing tool, so-called in-memory computing units have been proposed.

[0003] In addition, in the technical field of graphic comparison, content-addressable memories play an important role. In the prior art, static memories are often used to implement content-addressable memories. However, in the current technical field, static memories cannot meet the high-density requirements of circuits.

[0004] Constructing in-memory computing units with non-volatile memories is also a feasible method. However, on the premise that the amount of data to be processed is increasing, how to reduce the power consumption required by in-memory computing units has become an important issue. Summary of the Invention

[0005] The present invention provides a multiply-accumulate operation device and a control method thereof for the multiply-accumulate operation, which can reduce unnecessary error operation actions to achieve the effect of reducing power consumption.

[0006] The multiply-accumulate operation device of the present invention includes a feature information filter and an in-memory computing unit. The feature information filter records multiple specified bits of multiple feature information, compares the received input information with the specified bits to generate a comparison result, and generates a screening address according to the comparison result. The in-memory computing unit is coupled to the feature information filter. The in-memory computing unit records all bits of the feature information. The in-memory computing unit performs a multiply-accumulate operation on the feature information and the input information according to the screening address to generate an operation result.

[0007] The control method of the multiply-accumulate operation of the present invention includes: providing a feature information filter to record multiple specified bits of multiple feature information; providing the feature information filter to compare the input information with the specified bits to generate a comparison result, and generating a screening address according to the comparison result; and activating the screening address of the in-memory computing unit to perform a multiply-accumulate operation on the feature information and the input information to generate an operation result, where the in-memory computing unit records all bits of the feature information.

[0008] Based on the above, the multiply-accumulate operation device of the present invention first performs a screening operation on the input information through the feature information filter and correspondingly generates a screening address. Then, the in-memory arithmetic unit enables a part of the screening address in the memory to perform a multiply-accumulate operation on the feature information and the input information to generate an operation result. In this way, the input information does not need to perform a multiply-accumulate operation with all the feature information. The in-memory arithmetic unit only needs to enable a part of the screening address to perform a multiply-accumulate operation on the corresponding feature information and the input information. This can effectively reduce the power consumed during the operation and improve the calculation efficiency. Brief Description of the Drawings

[0009] Figure 1 A schematic diagram showing the multiply-accumulate operation device according to an embodiment of the present invention.

[0010] Figure 2 A schematic diagram showing the multiply-accumulate operation device according to another embodiment of the present invention.

[0011] Figure 3 A schematic diagram showing the generation method of the feature information of the multiply-accumulate operation device according to an embodiment of the present invention.

[0012] Figure 4 A schematic diagram showing the implementation manner of the feature information filter according to an embodiment of the present invention.

[0013] Figure 5A A waveform diagram showing the sensing operation of the matching line of the feature information filter according to an embodiment of the present invention.

[0014] Figure 5B A characteristic curve diagram of the transistor in the three-state content addressable memory cell according to an embodiment of the present invention.

[0015] Figure 5C A waveform diagram showing the sensing operation of the matching line of the feature information filter according to an embodiment of the present invention.

[0016] Figure 6 A schematic diagram showing the implementation manner of the three-dimensional architecture multiply-accumulate operation device according to an embodiment of the present invention.

[0017] Figure 7 A flowchart showing the control method of the multiply-accumulate operation according to an embodiment of the present invention.

[0018]

Symbol Description

[0019] 100, 200, 600: Multiply-accumulate operation device

[0020] 110, 210, 320: Feature information filter

[0021] 120, 220, 330: In-memory arithmetic unit

[0022] 211: Memory array

[0023] 212: Source driver

[0024] 213: Buffer

[0025] 214: Sense amplifier and encoder

[0026] 215: Decoder and storage element

[0027] 221: Memory array

[0028] 222: Word line driver

[0029] 223: Controller

[0030] 224: Page buffer

[0031] 2241: Latch

[0032] 225: Current detector

[0033] 310: Feature extractor

[0034] 312: Input layer

[0035] 311: Output layer

[0036] 400: Polymorphic content addressable memory

[0037] 410: Memory cell

[0038] 420: Precharge circuit

[0039] 430: Sense amplifier and encoder

[0040] FG1, FG2, M1: Transistors

[0041] ML: Match line

[0042] SL: Source line

[0043] SELB: Reverse search signal

[0044] SEL: Search signal

[0045] ST: Start signal

[0046] VM: Precharge voltage

[0047] MSB: Most significant bit

[0048] LSB: Least significant bit

[0049] BDIN: Sample information

[0050] CR: Operation result

[0051] DIN1: Set bit of the input information

[0052] DIN2: All bits of the input information

[0053] READD: Encoded screening address

[0054] SADD: Screening address

[0055] VREF: Reference voltage

[0056] 510 - 540: Curve

[0057] IC1, IC2: Chips

[0058] S710 - S730: Control steps Detailed implementation manner

[0059] Please refer to Figure 1 , Figure 1 , which shows a schematic diagram of a multiply - add operation device according to an embodiment of the present invention. The multiply - add operation device 100 includes a feature information screener 110 and an in - memory arithmetic unit 120. The feature information screener 110 records multiple designated bits of multiple feature information. The feature information screener 110 receives the set bit DIN1 of the input information, and is used to compare the received set bit DIN1 of the input information with the designated bits of the stored feature information to generate a comparison result, and generate a screening address SADD according to the comparison result.

[0060] In detail, the above - mentioned feature information can be extracted from a sample information. In an artificial intelligence system, taking the application of image recognition as an example, the designer can establish sample information as the basis for recognition, and perform data extraction operations on the sample information to obtain feature information. In this embodiment, the feature information screener 110 does not store the complete feature information, but only stores multiple designated bits in the feature information. Among them, the designated bit can be the most significant bit (MSB) in the feature information.

[0061] The in - memory arithmetic unit 120 is coupled to the feature information screener 110. The in - memory arithmetic unit 120 records all bits of the feature information. The in - memory arithmetic unit 120 performs a multiply - add operation on the feature information and all bits DIN2 of the input information according to the screening address generated by the feature information screener 110 to generate an operation result CR.

[0062] In this embodiment, taking the application of image recognition as an example, the input information is the image information to be recognized. The feature information filter 110 can first compare the set bit DIN1 of the input information with the designated bit of the feature information to perform a pre-screening operation on the input information. Through this pre-screening operation, the feature information filter 110 can generate a screening address SADD according to the comparison result, where the screening address is the address in the in-memory arithmetic unit 120 that stores the feature information related to the input information.

[0063] Continuing the above description, when performing image recognition, the in-memory arithmetic unit 120 can receive all bits DIN2 of the input information and enable the feature information stored in the corresponding screening address SADD, so that the feature information stored in the corresponding screening address SADD performs a multiply-accumulate operation with all bits DIN2 of the input information and generates an operation result CR.

[0064] It can be easily understood from the above description that the multiply-accumulate operation device 100 according to the embodiment of the present invention can use the feature information filter 110 to perform a pre-screening operation on the input information. Then, through the screening address SADD obtained in the pre-screening operation, the in-memory arithmetic unit 120 is enabled to provide partial (homologous to the input information) feature information to perform a multiply-accumulate operation with all bits DIN2 of the input information. In this way, the in-memory arithmetic unit 120 does not need to perform a multiply-accumulate operation on all the feature information, which can effectively reduce the complexity of the operation and save unnecessary power consumption.

[0065] Please refer to the following Figure 2 , Figure 2 for a schematic diagram showing a multiply-accumulate operation device according to another embodiment of the present invention. The multiply-accumulate operation device 200 includes a feature information filter 210 and an in-memory arithmetic unit 220. In this embodiment, the feature information filter 210 is a polymorphic content-addressable memory (such as a binary content-addressable memory (BCAM) or a ternary content-addressable memory (TCAM)). The in-memory arithmetic unit 220 is a NAND flash memory.

[0066] The feature information filter 210 includes a memory array 211, a source driver 212, a buffer 213, and a sense amplifier and encoder 214. The buffer 213 is used to receive the set bit DIN1 of the input information and provide a search signal to the memory array 211 according to the set bit DIN1 of the input information. The memory array 211 is used to store multiple designated bits of multiple feature information and is used to compare the search signal with the designated bits of the feature information to generate a comparison result. In this embodiment, the memory array 211 can be composed of flash memory cells, resistive random-access memory (ReRAM) cells, phase-change memory (PCM) cells, magnetic random access memory (MRAM) cells, or ferroelectric field-effect transistor (FeFET) memory cells. Moreover, the memory array 211 can have a two-dimensional structure or a three-dimensional structure, without any specific limitation.

[0067] The sense amplifier and encoder 214 is used to sense the comparison result, perform an encoding operation on the sensed comparison result, and generate an encoded screening address READD accordingly. The encoding operation performed by the sense amplifier and encoder 214 is a priority encode operation. In terms of the hardware architecture, the sense amplifier and encoder 214 can be implemented by combining the match line sense amplifier and the priority encoder of the polymorphic content addressable memory well-known to those skilled in the art, without any special limitation.

[0068] In this embodiment, the feature information filter 210 further includes a decoder and a storage element 215. The decoder and the storage element 215 can decode the encoded information generated by the sense amplifier and encoder 214 to obtain a screening address SADD. The decoder and the storage element 215 can further temporarily store the screening address SADD and provide the screening address SADD to the in-memory arithmetic unit 220.

[0069] On the other hand, the in-memory arithmetic unit 220 can be a NAND Flash memory. The in-memory arithmetic unit 220 includes a memory array 221, a word line driver 222, a controller 223, a page buffer 224, and a current detector 225. All bits of the feature information are stored in the memory array 221. The page buffer 224 is used to receive the input information DIN and temporarily store all bits DIN2 of the input information through the latch 2241.

[0070] When performing the multiply-accumulate operation, the controller 223 receives the screening address SADD and causes the word line driver 222 to drive multiple word lines corresponding to the screening address SADD. At the same time, all bits DIN2 provided by the latch 2241 are sent to the bit lines of the memory array 221, and all bits DIN2 of the input information can perform a multiply-accumulate operation with the data stored in the memory cells corresponding to the activated word lines. On the other hand, the word line driver 222 can provide a demand signal RQ to the decoder and the storage element 215, so that the decoder and the storage element 215 provide the screening address SADD.

[0071] The current detector 2245 senses the operation result of the multiply-accumulate operation by receiving the current generated by the memory cell string on the source line and based on a preset reference current.

[0072] Next, please refer to Figure 3 , Figure 3 FIG. showing a schematic diagram of the generation method of the feature information of the multiply-accumulate operation device according to an embodiment of the present invention. In the embodiment of Figure 3 , the multiply-accumulate operation device can be provided with a feature extractor 310. The feature extractor 310 is used to perform a data extraction operation on the sample information BDIN to obtain the feature information. Among them, the feature extractor 310 can be a calculator of an artificial neural network (ANN), and the sample information BDIN can be data in the form of Mixed National Institute of Standards and Technology (MNIST). The feature extractor 310 can have multiple operation layers, where the input layer 312 can have a relatively large number of nodes, and the output layer 311 can have a relatively small number of nodes.

[0073] The feature extractor 310 can be implemented by using any form of circuit, such as digital circuits, in-memory arithmetic units, and other hardware circuits well-known to those skilled in the art that can perform neural network operations, without any specific limitations.

[0074] The feature extractor 310 can also write the most significant bit (MSB) of the feature information generated by the nodes of the output layer 311 into the feature information filter 320, and write all bits from the most significant bit (MSB) to the least significant bit (LSB) of the feature information into the in-memory arithmetic unit 330.

[0075] Next, please refer to Figure 4 , Figure 4 FIG. shows a schematic diagram of an implementation of the feature information filter according to an embodiment of the present invention. In this implementation, the feature information filter can be implemented using a polymorphic content-addressable memory 400. Among them, the polymorphic content-addressable memory 400 includes a storage unit 410, a precharge circuit 420, and a sense amplifier and encoder 430. The storage unit 410 includes transistors FG1 and FG2. The first end of the transistor FG1 is coupled to the match line ML, the second end of the transistor FG1 is coupled to the source line SL, and the control end of the transistor FG1 receives the reverse search signal SELB. The transistor FG2 is coupled in parallel with the transistor FG2. The first end of the transistor FG2 is coupled to the match line ML, the second end of the transistor FG2 is coupled to the source line SL, and the control end of the transistor FG2 receives the search signal SEL. Among them, the reverse search signal SELB and the search signal SEL are inverse signals to each other.

[0076] The transistors FG1 and FG2 are floating-gate transistors and store complementary binary data respectively.

[0077] On the other hand, the precharge circuit 420 is coupled to the match line ML. The precharge circuit 420 precharges a match voltage on the match line ML to the precharge voltage VM according to a start signal ST. The precharge circuit 420 is constructed by the transistor M1 and is turned on or off according to the start signal ST. The sense amplifier and encoder 430 are coupled to the match line ML. The sense amplifier and encoder 430 compare the match voltage on the match line ML with the reference voltage VREF during sensing to generate a comparison result, and encode the comparison result to generate a filter address SADD.

[0078] Regarding the action details, please refer to Figure 4 and Figure 5A simultaneously, where Figure 5A FIG. shows a waveform diagram of the sensing action of the match line of the feature information filter according to an embodiment of the present invention. Among them, when the data stored in the storage unit 410 is logic 0, the transistors FG1 and FG2 can be written with logic 0 and 1 respectively; when the data to be written is logic 1, the transistors FG1 and FG2 can be written with logic 1 and 0 respectively.

[0079] During an initial period before sensing, the precharge circuit 420 can be turned on according to the start signal ST, and the matching voltage on the matching line ML is precharged to the precharge voltage VM. Then, the transistor M1 is turned off, and the sensing period begins.

[0080] During the sensing period, when the data to be searched for is logic 0, the search signal SEL can be made equal to the search voltage VSR, and the reverse search signal SELB can be made equal to the 0 voltage. At this time, if the data stored in the memory cell 410 is logic 0, then neither the transistors FG1 nor FG2 are turned on. The matching voltage on the matching line ML can be maintained equal to the precharge voltage VM (such as curve 510), indicating a state where the comparison result is a match. In contrast, if the data stored in the memory cell 410 is logic 1, then the transistor FG2 is turned on, and a discharge path is formed between the matching line ML and the source line SL. The matching voltage on the matching line ML is then pulled down to be equal to the voltage on the source line SL (e.g., 0 voltage), such as curve 520, indicating a state where the comparison result is a mis-match.

[0081] On the other hand, if during the sensing period, when the data to be searched for is logic 1, the reverse search signal SELB can be made equal to the search voltage VSR, and the search signal SEL can be made equal to the 0 voltage. At this time, if the data stored in the memory cell 410 is logic 0, then the transistor FG1 can be turned on, and the matching voltage on the matching line ML is pulled down to 0 voltage (such as curve 520), indicating a state where the comparison result is a mis-match. In contrast, if the data stored in the memory cell 410 is logic 1, then neither the transistors FG1 nor FG2 are turned on, and the matching line ML is maintained equal to the precharge voltage VM (such as curve 510), indicating a state where the comparison result is a match.

[0082] It should be noted that the sense amplifier and encoder 430 can compare the matching voltage on the matching line ML with the reference voltage VREF to determine whether the comparison result is a match or a mis-match. Among them, the reference voltage VREF can be set to be between the precharge voltage VM and the 0 voltage.

[0083] Please refer to the following Figure 4 and Figure 5B , where Figure 5B shows the characteristic curves of the transistors in the three-state content addressable memory cell according to an embodiment of the present invention. Among them, curve 501 represents the characteristic curve when the transistor is written with data 0, and curve 502 represents the characteristic curve when the transistor is written with data 1. If a search voltage VSR equal to the voltage V1 (e.g., 1 volt) is applied to the gate of the transistor, the current ratio of the turned-on transistor to the turned-off transistor can be greater than 10 6When a search voltage VSR equal to voltage V2 (e.g., 1.5 volts) is applied to the gate of the transistor, the current ratio of the turned-on transistor to the turned-off transistor can be greater than 10 5 .

[0084] Please refer to the following Figure 4 and Figure 5C , where Figure 5C FIG. shows the sensing operation waveform diagram of the matching line of the feature information filter according to an embodiment of the present invention. Figure 5C The waveform of Figure 5A is a reverse search implementation compared to the waveform of

[0085] During the sensing period, when the data to be searched is logic 0, the reverse search signal SELB can be made equal to the search voltage VSR, and the search signal SEL can be made equal to 0 volts. At this time, if the data stored in the memory cell 410 is logic 0, the transistor FG1 is turned on at this time. The matching voltage on the matching line ML can be pulled down to 0 volts (such as curve 540), indicating a matching state of the comparison result. In contrast, if the data stored in the memory cell 410 is logic 1, the transistors FG1 and FG2 are not turned on at this time. The matching voltage on the matching line ML remains equal to the precharge voltage VM, such as curve 530, indicating a non-matching state of the comparison result.

[0086] Please refer to Figure 6 , Figure 6A schematic diagram showing an implementation manner of a multiply-accumulate operation device with a three-dimensional architecture according to an embodiment of the present invention. Among them, the multiply-accumulate operation device 600 can be composed of two different chips IC1 and IC2. The feature information filter in the multiply-accumulate operation device 600 can be set on the chip IC1. The in-memory arithmetic unit in the multiply-accumulate operation device 600 can be set on the chip IC2. The chip ICl and the chip IC2 can be arranged in an overlapping manner and integrated in a multi-chip package manner. Through the implementation manner of the present invention, the multiply-accumulate operation device 600 can achieve the advantages of low power consumption, high operation efficiency, low size, and high circuit density.

[0087] In this embodiment, the chip IC1 and the chip IC2 can be a NOR flash memory and a NAND flash memory respectively, and are respectively used to implement the feature information filter and the in-memory arithmetic unit.

[0088] Please refer to the following Figure 7 , Figure 7 A flowchart showing a control method for multiply-accumulate operations according to an embodiment of the present invention. Among them, in step S710, a feature information filter is provided to record multiple specified bits of multiple feature information. Then, in step S720, the feature information filter is provided to compare the input information with the specified bits to generate a comparison result, and a screening address is generated based on the comparison result. In step S730, the screening address of the in-memory arithmetic unit is activated, so that the feature information and the input information perform a multiply-accumulate operation to generate an operation result, where the in-memory arithmetic unit records all bits of the feature information.

[0089] In summary, the multiply-accumulate operation device of the present invention can first perform a screening operation on the input information by the feature information filter, and enable the in-memory arithmetic unit to perform a multiply-accumulate operation with the input information by enabling part of the feature information. In this way, the power consumed during the operation of the multiply-accumulate operation device can be effectively reduced, and the calculation efficiency can be improved.

[0090] So far, the embodiments of the present disclosure have been described in detail with reference to the accompanying drawings.

[0091] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention should be included within the protection scope of the present invention.

Claims

1. A multiply-add operation device, wherein, Comprising: A feature information filter that records multiple designated bits of multiple feature information, compares multiple set bits in an input information received with these designated bits to generate a comparison result, and generates a screening address based on the comparison result; And An in-memory arithmetic unit coupled to the feature information filter, the in-memory arithmetic unit records all bits of these feature information, and the in-memory arithmetic unit performs a multiply-accumulate operation on these feature information and all bits in the input information based on the screening address to generate an operation result; Wherein, the feature information filter determines the matching status of multiple set bits in the input information and these designated bits to generate the comparison result, and performs an encoding action based on the comparison result to generate the screening address.

2. The multiply-add operation device according to claim 1, wherein These designated bits are respectively the most significant bits of these feature information.

3. The multiply-add operation device according to claim 1, wherein, The feature information filter is a polymorphic content-addressable memory or an analog content-addressable memory.

4. The multiply-add operation device according to claim 1, wherein The feature information filter has a memory array, the memory array has a two-dimensional structure or a three-dimensional structure, and the memory array is composed of flash memory cells, resistive memory cells, phase change memory cells, magnetoresistive random access memory cells or ferroelectric field effect memory cells.

5. The multiply-add operation device according to claim 3, wherein, The polymorphic content-addressable memory includes a memory array composed of multiple memory cells, and each of these memory cells includes: A first transistor having a first end coupled to a match line, a second end of the first transistor coupled to a source line, and a control end of the first transistor receiving a reverse search signal; and A second transistor having a first end coupled to the match line, a second end of the second transistor coupled to the source line, and a control end of the second transistor receiving a search signal, Wherein the first transistor and the second transistor are floating gate transistors.

6. The multiply-add operation device according to claim 5, wherein, The polymorphic content-addressable memory further includes: A precharge circuit coupled to the match line, and precharges a match voltage of the match line to a precharge voltage according to a start signal; A sense amplifier and encoder coupled to the match line, compares the match voltage with a reference voltage during a sensing period to generate the comparison result, and encodes the comparison result to generate the screening address.

7. The multiply-add operation device according to claim 6, wherein, When the content stored in each of these memory cells matches the search signal, the match voltage is pulled down to a reference ground voltage during the sensing period, and when the content stored in each of these memory cells does not match the search signal, the match voltage is maintained equal to the precharge voltage during the sensing period.

8. The multiply-add operation device according to claim 6, wherein, When the content stored in each of these memory cells does not match the search signal, the match voltage is pulled down to a reference ground voltage during the sensing period, and when the content stored in each of these memory cells matches the search signal, the match voltage is maintained equal to the precharge voltage during the sensing period.

9. The multiply-add operation device according to claim 1, wherein, Further comprising: A feature extractor that performs a data extraction operation on a sample information to obtain these feature information, and writes these designated bits of these feature information to the feature information filter, and writes all bits of these feature information to the in-memory arithmetic unit.

10. The multiply-add operation device according to claim 1, wherein, The in-memory arithmetic unit is a NAND flash memory.

11. The multiply-add operation device according to claim 1, wherein, The feature information filter is provided on a first chip, and the in-memory arithmetic unit is provided on a second chip, and the first chip is different from the second chip.

12. A control method for multiplication and addition operations, wherein, It includes: Providing a feature information filter to record multiple specified bits of multiple feature information; Providing the feature information filter to compare multiple set bits in an input information with these specified bits to generate a comparison result, and generating a screening address according to the comparison result; and Activating the screening address of an in-memory arithmetic unit to perform a multiply-accumulate operation on these feature information and all bits in the input information to generate an operation result, wherein the in-memory arithmetic unit records all bits of these feature information; Wherein, the step of providing the feature information filter to compare multiple set bits in an input information with these specified bits to generate a comparison result, and generating a screening address according to the comparison result includes: judging the matching state of the multiple set bits in the input information and these specified bits to generate the comparison result, and performing an encoding action according to the comparison result to generate the screening address.

13. The control method according to claim 12, wherein, These specified bits are respectively the most significant bits of these feature information.

14. The control method according to claim 12, wherein, It further includes: Performing a data extraction action on a sample information to obtain these feature information, and writing these specified bits of these feature information into the feature information filter, and writing all bits of these feature information into the in-memory arithmetic unit.

Images