Neural network computation device and operational method thereof

TWI945885BActive Publication Date: 2026-11-11MACRONIX INTERNATIONAL CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
TW114122738
Authority / Receiving Office
TW · TW
Patent Type
Patents
Current Assignee / Owner
Filing Date
2025-06-17
Publication Date
2026-11-11
Estimated Expiration
2045-06-16

Smart Images

  • Figure TWG2TB001920637_001
    Figure TWG2TB001920637_001
  • Figure TWG2TB001920637_002
    Figure TWG2TB001920637_002
  • Figure TWG2TB001920637_003
    Figure TWG2TB001920637_003
Patent Text Reader

Abstract

A neural network computing device includes a processing unit and an in-memory computing unit. The processing unit compares input data with a threshold range to generate a first subset of input data and a second subset of input data. The first subset of input data and the second subset of input data are outside and within the threshold range, respectively. The processing unit performs vector-matrix operations based on the first subset of input data and first weight data to generate a first result. The in-memory computing unit performs vector-matrix operations based on a quantized second subset of input data corresponding to the second subset of input data and second weight data to generate a second result. The processing unit sums the first result and the second result to generate a computational result.
Need to check novelty before this filing date? Find Prior Art

Claims

1. A neural network computing device, comprising: a storage unit for storing first weight data; a processing unit for comparing input data with a threshold range to generate a first subset of input data and a second subset of input data, wherein the first subset of input data is outside the threshold range and the second subset of input data is within the threshold range, wherein the processing unit is configured to perform a vector matrix operation based on the first subset of input data and the first weight data to generate a first result; and an in-memory computing unit coupled to the processing unit, comprising: an in-memory computing device for storing second weight data and performing the vector matrix operation based on a quantized second subset of input data corresponding to the second subset of input data and the second weight data to generate a second result, wherein the processing unit is further configured to receive the second result from the in-memory computing unit and sum the first result and the second result to generate computing result data.

2. The neural network computing device as claimed in claim 1, wherein the processing unit comprises: a comparator circuit for comparing the input data with the threshold range and outputting the first subset of input data to an input buffer circuit, and outputting the second subset of input data to the computing unit in the memory.

3. The neural network computing device as claimed in claim 2, wherein the storage unit is integrated within the computing unit in the storage unit, wherein the input buffer circuit is used to transmit a request signal to the storage unit to obtain the first weight data, wherein the processing unit further includes: an arithmetic logic unit for receiving the first weight data and multiplying the first input data subset and the first weight data to perform the vector matrix operation to generate the first result.

4. The neural network computing device as claimed in claim 2, wherein the in-memory computing unit further includes: a quantization circuit for receiving the second subset of input data from the processing unit and quantizing the second subset of input data to generate the quantized second subset of input data.

5. The neural network computing device as claimed in claim 1, wherein the processing unit comprises: a comparator circuit for comparing the input data with the threshold range, outputting the first subset of input data to an input buffer circuit, and outputting the second subset of input data; and a quantization circuit coupled between the comparator circuit and the in-memory computing unit for receiving the second subset of input data from the comparator circuit, quantizing the second subset of input data to generate the quantized second subset of input data, and outputting the quantized second subset of input data to the in-memory computing unit.

6. The neural network computing device as claimed in claim 1, wherein the storage unit and the computing unit within the storage unit are separate from each other.

7. The neural network computing device as claimed in claim 1, wherein the in-memory computing unit further includes: an output buffer circuit for outputting the second result to the processing unit, wherein the amount of data in the second result is greater than the amount of data in the first result.

8. A method of operating a neural network computing device, comprising: dividing input data into a first input data subset and a second input data subset according to at least a threshold through a processing unit, wherein the absolute value of the data in the first input data subset is greater than the absolute value of the data in the second input data subset; performing a vector matrix operation on the first input data subset and the first weight data through the processing unit to generate a first result; receiving the second input data subset from the processing unit through a storage-based computing unit, and performing the vector matrix operation on the second input data subset and the second weight data to generate a second result; and receiving the second result from the storage-based computing unit through the processing unit, and adding the first result and the second result to generate a computation result data.

9. The method of operating a neural network computing device as claimed in claim 8, wherein the at least one threshold includes a first threshold and a second threshold, and the method of operating the neural network computing device further includes: analyzing a data distribution of the input data to obtain a first boundary value of the data distribution as the first threshold and a second boundary value as the second threshold.

10. A method of operating the neural network computing device as claimed in claim 8, wherein the absolute value of the data in the first subset of input data is greater than the at least one threshold, and the absolute value of the data in the second subset of input data is less than or equal to the at least one threshold.

11. The method of operating the neural network computing device as described in claim 8 further includes: transmitting a request signal through the processing unit to the computing unit in the memory to obtain the first weight data stored in the computing unit in the memory.

12. The method of operating the neural network computing device as described in claim 8 further comprises: quantizing the second input data subset before performing the vector matrix operation based on the second input data subset and the second weight data.

Citation Information

Patent Citations

  • Multi-threshold configuration device and method based on photoelectric storage and calculation integrated unit

    CN116049094A

  • Data processing method, device and equipment

    CN118862963A

  • Convolutional neural network, operational optimization method, device, electronic device and storage medium

    TW202223761A

  • System and method for designing and manufacturing optimized multi-core and / or multi-processor intergrated circuit architecture with static scheduling of multiple processing pipelines

    TW202501249A

  • In-memory analog neural cache

    US20190057304A1