Neural network computing device and method

By separating the fixed and unfixed computing layers of feature patterns in neural network operations, using higher and lower but more elastic neuronal circuits, the problem of von Neumann bottleneck is solved, and high efficiency and low cost are achieved AI hardware computing, suitable for edge and cloud devices.

CN113033792BActive Publication Date: 2024-11-01IND TECH RES INST
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202011535857.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-12-24
Filing Date
2020-12-23
Publication Date
2024-11-01
Estimated Expiration
2040-12-23

AI Technical Summary

Technical Problem

The existing von Neumann structures lead to the limitations of power consumption and computing rate in artificial intelligence hardware, called von Neumann bottleneck, and it is difficult to meet the rapidly developing artificial intelligence hardware efficiency needs.

Method used

A neural network computing device is adopted to divide neural network computing into two categories: fixed and non-fixed feature patterns, using higher-performance first neuron circuits and better performance respectively. The low but more elastic second neuron circuit performs calculations to realize the separation of the operation layer with fixed feature patterns and the non-fixed operation layer.

Benefits of technology

Among the limited computing resources, the recognition efficiency and accuracy of neural networks are improved, the hardware cost is reduced, and neural network accelerators are suitable for edge devices and cloud-based neural network accelerators, avoiding the limitations of von Neumann's bottleneck.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113033792B_ABST
    Figure CN113033792B_ABST
Patent Text Reader

Abstract

A neural network operation device and method. The neural network operation device includes a first neural network circuit and a second neural network circuit. The first neural network circuit is used to perform a neural network operation of at least one operation layer in a neural network algorithm with a fixed feature pattern. The second neural network circuit is used to perform a neural network operation of at least one operation layer in a neural network algorithm with a non-fixed feature pattern. The first neural network circuit has a greater performance than the second neural network circuit.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a computing device and method, and more particularly to a neural network computing device and method. Background Art

[0002] In today's life, artificial intelligence (AI) is widely used in different technical fields to achieve applications such as identification, warning, and operation assistance. However, due to the rapid development of AI, various new networks have been proposed, and the demand for hardware performance has been continuously raised. In order to meet the needs of the development of artificial intelligence, efficient artificial intelligence computing hardware has also become a major development goal.

[0003] Furthermore, the computing hardware for realizing artificial intelligence can be mainly realized through the Von Neumann structure, which mainly performs neuron operations by storing weight values ​​in memory and processing input signals through processing units and accessing memory weight values ​​to generate operation results. However, since the processing units need to access weight information from the memory to perform operations, it will consume a lot of power and cause operation delays. The Von Neumann structure will face the Von Neumann bottleneck, which will limit the power consumption and operation rate performance of the neuron hardware. Summary of the invention

[0004] An embodiment of the present invention provides a neural network computing device, which includes a first neural network circuit and a second neural network circuit. The first neural network circuit is used to perform a neural network computing of at least one computing layer with a fixed feature pattern in a neural network algorithm. The second neural network circuit is used to perform a neural network computing of at least one computing layer with a non-fixed feature pattern in a neural network algorithm. The first neural network circuit has a greater performance than the second neural network circuit.

[0005] An embodiment of the present invention provides a neural network operation method, which is applicable to a neural network operation device having a first neural network circuit and a second neural network circuit built therein, wherein the performance of the first neural network circuit is greater than that of the second neural network circuit. The method comprises the following steps: using the first neural network circuit to perform a neural network operation of at least one operation layer with a fixed feature pattern in a neural network algorithm; and using the second neural network circuit to perform a neural network operation of at least one operation layer with a non-fixed feature pattern in a neural network algorithm.

[0006] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1is a schematic diagram of a neural network architecture according to an embodiment of the present invention.

[0008] Figure 2 is a schematic diagram of transfer learning of a neural network according to an embodiment of the present invention.

[0009] Figure 3 is a block diagram of a neural network computing device according to an embodiment of the present invention.

[0010] Figure 4 is a flow chart of a neural network operation method according to an embodiment of the present invention.

[0011] Figure 5 is a schematic diagram of a neuron circuit according to an embodiment of the present invention.

[0012] Figure 6 is a flow chart of a neural network operation method according to an embodiment of the present invention.

[0013] Figure 7 is a flow chart of a neural network operation method according to an embodiment of the present invention.

[0014] Reference numerals

[0015] 1.1a,1b: Neural Network

[0016] 2: To identify the signal

[0017] 3.4: Training Data

[0018] 12, 14, 16: Operation layer set

[0019] 30: Neural network computing device

[0020] 32: First Neuron Circuit

[0021] 34: Second Neuron Circuit

[0022] I1~Im:Input

[0023] R11~Rmn:weight

[0024] O1~On:Output

[0025] SA1~SAn: Sense amplifier

[0026] S402~S404, S602~S608, S702~S708: Steps DETAILED DESCRIPTION

[0027] The structural principle and working principle of the present invention are described in detail below in conjunction with the accompanying drawings:

[0028] The embodiment of the present invention proposes a neural network computing device that realizes field-specific artificial intelligence (AI) in a semiconductor process. Among them, based on the computing in memory (CIM) and transfer learning methods, the embodiment of the present invention uses a higher-performance neuron circuit to perform neural network operations in a specific computing layer with fixed feature patterns in the neural network (for example, the structure and weights do not need to be updated), and then cooperates with a lower-performance but more flexible neuron circuit to perform neural network operations in other computing layers with unfixed feature patterns in the neural network (for example, the structure and weights need to be updated). Therefore, neural networks can be used to realize the recognition of signals such as images / sounds in edge devices with limited computing resources.

[0029] Figure 1 is a schematic diagram of a neural network architecture according to an embodiment of the present invention. Figure 1 The neural network 1 of this embodiment is a convolution neural network (CNN) as an example, and the processing of the convolution neural network on the signal to be identified 2 is divided into multiple stages such as pre-processing, adjustment, feature extraction, connection, and determination according to the function. Among them, the signal to be identified 2 is, for example, a sensing signal measured by a sensor such as a sound or image, and in the pre-processing stage, it will be pre-processed by a pre-processing circuit such as signal amplification, filtering, noise suppression, compensation, digital-analog conversion, analog feature extraction, etc., to generate the input of the neural network 1.

[0030] In the adjustment phase, the neural network 1 may, for example, apply various adaptation techniques to adjust or reduce the variation of features to make them suitable for subsequent feature extraction.

[0031] In the feature extraction stage, the neural network 1, for example, uses a feature extractor composed of multiple convolutional layers and pooling layers to extract features from the adjusted input data. The convolutional layer, for example, moves different convolution kernels on an input image composed of multiple inputs to perform convolution operations to obtain a set of feature maps. The pooling layer, for example, uses a nonlinear pooling function to downsample the input image to reduce the number of parameters and the amount of calculation. In some embodiments, the pooling layer can also reduce the sensitivity of the convolutional layer to the edges of objects.

[0032] In the first few layers of feature extraction, the neural network 1 uses fixed feature patterns such as points, lines, circles or polygons to extract features of the input data. These features are usually not specific to the object to be identified, but can serve as the basis for subsequent specific feature extraction of the identified object. Taking face recognition as an example, by first extracting simple features such as lines, contours or shapes in the face, it will be beneficial to the subsequent feature extraction of specific parts such as eyes, nose, mouth, etc.

[0033] In the connection stage, the neural network 1, for example, connects a classifier consisting of one or more fully connected layers to the previous operation layer (such as the convolutional layer of the last layer of the feature extractor) to receive the weights of all features captured by the previous feature extractor and perform subsequent classification.

[0034] In the decision stage, the neural network 1 is, for example, a classifier composed of the above-mentioned fully connected layers. For the feature weights received in the connection stage, a nonlinear function such as a normalized exponential function (softmax function) is used to calculate the probability of individual features (the output of each feature is between 0 and 1, and the sum of the outputs is 1). Finally, the output layer determines and outputs the final recognition result through voting based on the probability of each feature calculated by the classifier. The recognition result is, for example, the probability that the signal 2 to be recognized belongs to each category, which can indicate which category the signal 2 to be recognized belongs to (for example, the one with the largest probability).

[0035] Figure 2 is a schematic diagram of transfer learning of a neural network according to an embodiment of the present invention. Figure 2 , this embodiment is based on Figure 1 Take neural network 1 as an example to illustrate the concept of transfer learning.

[0036] First, the operation layer architecture of the neural network 1 is determined according to the type of input data and the classification to be identified (for example, including an input layer applied to the adjustment stage, multiple hidden layers applied to the feature extraction stage, the connection stage and the decision stage, and an output layer for outputting the result of the decision stage) to establish the neural network 1, and the weights of multiple nodes in each operation layer are generated in a random manner.

[0037] Next, multiple images obtained from an image recognition database (e.g., ImageNet) and their corresponding classification results are sent as training data 3 to the neural network 1 for training, thereby obtaining a trained neural network 1a, in which the weights of the nodes in each operation layer of the neural network 1a have been updated by learning the training data 3. The multiple operation layers of the neural network 1a can be divided into an operation layer set 12 for performing neural network operations with fixed feature patterns and an operation layer set 14 for performing neural network operations with non-fixed feature patterns according to the feature patterns they operate.

[0038] Then, the new training data 4 and the new classification are imported into the neural network 1a, and the neural network 1a is retrained to generate a new neural network 1b. During the training process, since the feature patterns recognized by each operation layer in the operation layer set 12 are fixed, there is no need to update the weights of the nodes therein, and only the structure adjustment and / or weight update are performed on the multiple operation layers of the operation layer set 14. For example, in the neural network 1b, corresponding to the new classification, the number of nodes in the output layer in the adjusted operation layer set 16 will increase from 3 to 5, and the weights of the nodes of each operation layer will also be updated by learning the new training data 3.

[0039] Taking the recognition of animal images as an example, the input of the input layer is all the pixels of the pattern, and the first convolution layer will judge the most basic graphics, such as horizontal lines, straight lines, circles, etc. The second layer will judge slightly more complex graphics, such as polygons, and the third layer will judge more complex graphics, and so on. It can be seen that the feature recognition of the first few layers has no direct relationship with the target to be recognized. Whether it is to recognize cats, dogs, birds, elephants, or tigers, they are not tiny graphics at the pixel level. However, the first few layers are also extremely important, because no matter what graphics, they are all pieced together by pixel-level graphics. Therefore, in an embodiment of the present invention, by retaining the first few layers, replacing or updating the latter layers, and retraining the parameters of the model, a new model that meets the requirements can be obtained. Based on the new model, only the latter layers need to be trained, and the parameters calculated are less, so the problem of over-fitting is less likely to occur.

[0040] Based on the above neural network architecture, an embodiment of the present invention proposes a neural network computing device architecture based on an in-memory computing method. Figure 3 is a block diagram of a neural network computing device according to an embodiment of the present invention. Figure 3The neural network computing device 30 of this embodiment is, for example, a neural network accelerator using a resistive memory architecture, or a processing engine (PE) in the accelerator, which is applied to deep learning (DL) to assist a processor (such as a central processing unit) in performing neural network computing. The neural network computing is, for example, a multiply-accumulate (MAC) computing.

[0041] The neural network computing device 30 includes a first neuron circuit 32 and a second neuron circuit 34, and the performance of the first neuron circuit 32 is greater than that of the second neuron circuit 34. In some embodiments, the first neuron circuit 32 is a one-time programming (OTP) memory, such as a read only memory (ROM) or an electronic fuse (Efuse) memory; the second neuron circuit 34 is a multi-time programming (MTP) memory, such as a static random access memory (SRAM) or a flash memory. In other embodiments, the first neuron circuit 32 and the second neuron circuit 34 can also be a combination of other types of memories, which is not limited here.

[0042] In some embodiments, the neural network computing device 30 further includes a buffer for temporarily storing input and output data and a computing circuit (not shown) for supporting nonlinear function and pooling function operations, which is not limited here.

[0043] Figure 4 is a flowchart of a neural network operation method according to an embodiment of the present invention. Figure 3 and Figure 4 The method of this embodiment is applicable to Figure 3 The neural network computing device 30 of the present invention is used. The following describes the detailed steps of the neural network computing method of the present invention in combination with the various components in the neural network computing device 30.

[0044] In step S402, the neural network operation device 30 uses the first neuron circuit 32 to perform a neural network operation of at least one operation layer with a fixed feature pattern in the neural network algorithm. The fixed feature pattern includes a point, line, circle or polygon that is not specific to the object to be identified, and is not limited here. The first neuron circuit 32 is used, for example, to perform the operation of the operation layer responsible for feature learning in the neural network algorithm or other operation layers that are not directly related to the object to be identified.

[0045] In some embodiments, the neural network computing device 30 may first perform pre-processing such as signal amplification, filtering, noise suppression, compensation, analog-to-digital conversion, and analog feature extraction on the input signal to generate the input of the first neuron circuit 30 (e.g., corresponding to Figure 1 processing in the pre-processing stage).

[0046] In some embodiments, the first neuron circuit 32 adopts a computing in memory (CIM) architecture, which includes a plurality of input lines and a plurality of output lines that are cross-configured with each other, a plurality of storage units that are respectively configured at a plurality of intersections of the input lines and the output lines and store a plurality of weights of neural network operations, and a plurality of read-out amplifiers that are respectively connected to the output lines.

[0047] Figure 5 is a schematic diagram of a neuron circuit according to an embodiment of the present invention. Figure 5 This embodiment illustrates Figure 3 The architecture and operation of the first neuron circuit 32 are described. The first neuron circuit 32 adopts a crossbar array architecture, which includes a plurality of input lines and a plurality of output lines in a crossbar configuration. These input lines and output lines are, for example, word lines and bit lines of a memory circuit, and the inputs I1 to Im of the neural network operation can be sent to the input lines through the control of a row decoder or a column decoder, and are not limited here.

[0048] A storage unit is configured at each intersection of the input line and the output line. This storage unit is, for example, a semiconductor device such as a read-only memory (ROM) or an electronic fuse (Efuse). It can, for example, store the weights of the neural network operation (the weights R11~Rmn as shown in the figure, where m and n are positive integers) through the control of the input line to generate a corresponding output current or equivalent resistance when enabled or turned on.

[0049] The storage cells can be divided into n rows and m columns, and the storage cells in each column are coupled to an input line to receive the input I1~Im of the neural network operation to control the current size or equivalent resistance value of the storage cells in the column. Each storage cell, for example, performs a multiplication operation on the input of the connected input line and outputs the product to the connected output line (for example, the input voltage is generated through the conductance corresponding to the stored resistance value to generate an output current). The storage cells in each row are coupled to an output line, and the output line accumulates the product of the outputs of the connected storage cells to generate outputs O1~On (for example, the current output by each storage cell is integrated to generate an output current).

[0050] The read amplifiers SA1 to SAn are connected to the output lines respectively, and are used to read out the output lines and the products of the connected storage unit outputs, which are obtained by multiplying the output lines and the outputs O1 to On, and use them as the inputs of the operation layer of the next layer.

[0051] In some embodiments, when the neural network operation device 30 performs neural network operations of multiple operation layers in the first neuron circuit 32, for example, an adjustment circuit (not shown) that can adjust the output result is added between adjacent operation layers. This adjustment circuit is respectively connected or configured to the readout amplifiers SA1~SAn, and can be used to adjust the gain and bias of the output read out by the readout amplifiers SA1~SAn, so that the adjusted output is suitable as the input of the next operation layer.

[0052] Figure 6 is a flowchart of a neural network operation method according to an embodiment of the present invention. Figure 5 and Figure 6 The method of this embodiment is applicable to Figure 5 The first neuron circuit 32 is used as the first neuron circuit. The following describes the detailed steps of the neural network operation method of the present invention in combination with the various components in the first neuron circuit 32.

[0053] In step S602 , the neural network operation device 30 inputs a plurality of inputs I1 to Im of the neural network operation into the input lines of the first neuron circuit 32 .

[0054] In step S604 , the storage unit of the first neuron circuit 32 performs a multiplication operation on the inputs I1 -Im of the connected input lines, and outputs the product to the connected output lines.

[0055] In step S606, the sense amplifiers SA1 to SAn of the first neuron circuit 32 sense output lines O1 to On which the products of the outputs of the connected storage cells are accumulated.

[0056] In step S608 , the adjustment circuit of the first neuron circuit 32 adjusts the gain and offset of the outputs read out by the sense amplifiers SA1 to SAn so that the adjusted outputs are suitable as inputs to the next operation layer.

[0057] By appropriately adjusting the output results to meet the characteristics or requirements of the next computing layer, better accuracy characteristics can be achieved without significantly changing the hardware architecture.

[0058] Back to Figure 4In step S404, the neural network computing device 30 uses the second neural network circuit 34 to perform the neural network computing of at least one computing layer in the neural network algorithm whose feature pattern is not fixed. That is, the second neural network circuit 34 is used to perform the computing of the computing layer responsible for classification in the neural network algorithm or other computing layers that need to be modified due to different recognition objects.

[0059] Similar to the first neuron circuit 32, the second neuron circuit 34 also adopts an in-memory computing architecture (eg Figure 5 As shown, however, unlike the first neuron circuit 32, the storage unit in the second neuron circuit 34 adopts a semiconductor device such as a static random access memory or a flash memory, which has lower performance but higher flexibility than the storage unit of the first neuron circuit 32. Therefore, the architecture of the operation layer can be adaptively adjusted according to new input data and the weight values ​​of the nodes in each operation layer can be updated.

[0060] Figure 7 is a flowchart of a neural network operation method according to an embodiment of the present invention. Figure 3 and Figure 7 The method of this embodiment is applicable to Figure 3 The second neuron circuit 34 is a circuit that uses Figure 5 The architecture of the first neuron circuit 32 is shown.

[0061] In step S702 , the neural network operation device 30 inputs a plurality of inputs of the neural network operation into the input lines of the second neuron circuit 34 .

[0062] In step S704 , the storage unit of the second neuron circuit 34 performs a multiplication operation on the inputs of the connected input lines, and outputs the product to the connected output lines.

[0063] In step S706, the sense amplifier of the second neuron circuit 34 senses the output line by accumulating the product of the connected storage unit outputs.

[0064] In step S708 , the second neuron circuit 34 adjusts the architecture of the operation layer or updates the weights stored in each storage unit according to the output read out by the readout amplifier, so that the adjusted operation layer can support new classification operations and has better recognition accuracy.

[0065] In some embodiments, the second neuron circuit 34 may perform post-processing such as analog-to-digital conversion, normalization, and activation function operation on the output to serve as the input of the operation layer of the next layer, which is not limited here.

[0066] In some embodiments, similar to the first neuron circuit 32, the second neuron circuit 34 may also add an adjustment circuit that can adjust the output result between adjacent operation layers to adjust the gain and bias of the output read out by the read-out amplifier, so that the adjusted output is suitable as the input of the next operation layer, thereby achieving better accuracy characteristics, which is not limited here.

[0067] In summary, the neural network computing device and method of the embodiment of the present invention combines the advantages of in-memory computing and transfer learning, divides the neural network into two parts according to the feature pattern of the operation it performs, and uses a higher-performance neuron circuit to execute the neural network operation of a specific computing layer with a fixed feature pattern, and uses a lower-performance but more flexible neuron circuit to execute other computing layers with unfixed feature patterns. Therefore, a high-performance, low-cost and flexible AI accelerator chip structure can be realized, which can be used in a neural network accelerator at the terminal (edge) or cloud.

[0068] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A neural network computing device, characterized in that: include: A first neuron circuit includes a first set of operation layers in a trained neural network to perform neural network operations with a fixed feature pattern; as well as a second neuron circuit comprising a second set of operation layers in the trained neural network to perform neural network operations with unfixed feature patterns, wherein the first neuron circuit has a greater performance than the second neuron circuit; The fixed characteristic pattern includes a point, a line, a circle or a polygon; The first neuron circuit is a one-time programmable memory, the second neuron circuit is a multi-time programmable memory, and the one-time programmable memory includes a read-only memory or an electronic fuse memory; The training process of the trained neural network includes: Build a neural network; Sending a training data to the neural network to obtain the trained neural network, and distinguishing the first computing layer set and the second computing layer set in the trained neural network; In response to receiving new training data, the new training data is sent to the trained neural network obtained in the previous training for retraining to generate a new trained neural network. In the process of retraining the trained neural network, the structure and weights of the first set of operation layers are not updated, and only at least one of the structure and weights of the second set of operation layers is updated.

2. The neural network computing device according to claim 1, characterized in that: The multi-time programmable memory includes a static random access memory or a flash memory.

3. The neural network computing device according to claim 1, characterized in that: Also includes: The pre-processing circuit performs pre-processing on the input signal to generate multiple inputs of the first neuron circuit, wherein the pre-processing includes one or a combination of signal amplification, filtering, noise suppression, compensation, digital-to-analog conversion, and analog feature extraction.

4. The neural network computing device according to claim 1, characterized in that: The first neuron circuit and the second neuron circuit respectively include: A plurality of input lines and a plurality of output lines arranged crosswise with each other, wherein the input lines respectively receive a plurality of inputs of the neural network operation; A plurality of storage units, respectively disposed at a plurality of intersections of the input line and the output line, storing a plurality of weights of the neural network operation, for performing multiplication operations on the inputs of the connected input lines and outputting the products to the connected output lines; and A plurality of readout amplifiers are respectively connected to the output lines and are used to read out outputs obtained by accumulating the products of the outputs of the connected storage units to the output lines.

5. The neural network computing device according to claim 4, characterized in that: The first neuron circuit and the second neuron circuit further include: A plurality of adjustment circuits are respectively connected or configured to the readout amplifiers to adjust the gain and bias of the outputs read out by the readout amplifiers so that the adjusted outputs are suitable as inputs of the operation layer of the next layer.

6. The neural network computing device according to claim 1, characterized in that: The neural network operation is a multiplication and addition operation.

7. A neural network computing method, applicable to a neural network computing device having a first neural network circuit and a second neural network circuit built therein, wherein the performance of the first neural network circuit is greater than that of the second neural network circuit, characterized in that: The method comprises the following steps: Utilizing a first set of operation layers in a trained neural network included in the first neuron circuit to perform neural network operations with fixed feature patterns; and Utilizing the second set of operation layers in the trained neural network included in the second neuron circuit to perform neural network operations with unfixed feature patterns; The fixed characteristic pattern includes a point, a line, a circle or a polygon; The first neuron circuit is a one-time programmable memory, the second neuron circuit is a multi-time programmable memory, and the one-time programmable memory includes a read-only memory or an electronic fuse memory; The training process of the trained neural network includes: Build a neural network; Sending a training data to the neural network to obtain the trained neural network, and distinguishing the first computing layer set and the second computing layer set in the trained neural network; In response to receiving new training data, the new training data is sent to the trained neural network obtained in the previous training for retraining to generate a new trained neural network. In the process of retraining the trained neural network, the structure and weights of the first set of operation layers are not updated, and only at least one of the structure and weights of the second set of operation layers is updated.

8. The method according to claim 7, characterized in that The first neuron circuit includes a plurality of input lines and a plurality of output lines that are arranged crosswise with each other, a plurality of storage units that are respectively arranged at a plurality of intersections of the input lines and the output lines and store a plurality of weights of the neural network operation, and a plurality of readout amplifiers that are respectively connected to the output lines. The step of using the first neuron circuit to perform the neural network operation with a fixed feature pattern includes: Inputting a plurality of inputs of the neural network operation into the input lines respectively; The storage unit performs a multiplication operation on the inputs of the connected input lines and outputs the product to the connected output lines; and The output obtained by accumulating the product of the outputs of the memory cells connected to the output line is sensed by the sense amplifier.

9. The method according to claim 7, characterized in that: Before the step of using the first neuron circuit to perform the neural network operation with the fixed feature pattern, the method further includes: The input signal is pre-processed to generate the input of the first neuron circuit, wherein the pre-processing includes one or a combination of signal amplification, filtering, noise suppression, compensation, analog-to-digital conversion, and analog feature extraction.

10. The method according to claim 8, characterized in that After the step of reading out the output obtained by accumulating the product of the outputs of the connected storage units through the output amplifier, the method further comprises: The output read out by the readout amplifier is post-processed to serve as the input of the operation layer of the next layer. The post-processing includes one or a combination of analog-to-digital conversion, normalization and activation function operation.

11. The method according to claim 10, characterized in that The post-processing further includes adjusting the gain and bias of the output read out by the readout amplifier, so that the adjusted output is suitable as the input of the operation layer of the next layer.

12. The method according to claim 7, characterized in that The second neuron circuit includes a plurality of input lines and a plurality of output lines that are arranged crosswise with each other, a plurality of storage units that are respectively arranged at a plurality of intersections of the input lines and the output lines and store a plurality of weights of the neural network operation, and a plurality of readout amplifiers that are respectively connected to the output lines. The step of using the second neuron circuit to perform the neural network operation with a non-fixed feature pattern includes: Inputting a plurality of inputs of the neural network operation into the input lines respectively; The storage unit performs a multiplication operation on the inputs of the connected input lines, and outputs the product to the connected output lines; The output obtained by accumulating the product of the outputs of the connected storage cells to the output line is sensed by the sense amplifier; and The architecture of the operation layer is adjusted or the weights stored in each of the storage units are updated according to the output read out.

13. The method according to claim 12, characterized in that After the step of reading out the output obtained by accumulating the product of the outputs of the connected storage units through the output amplifier, the method further comprises: The output is post-processed to serve as the input of the operation layer of the next layer, and the post-processing includes one or a combination of analog-to-digital conversion, normalization and activation function operation.

14. The method according to claim 13, characterized in that The post-processing further includes adjusting the gain and bias of the output read out by the readout amplifier, so that the adjusted output is suitable as the input of the operation layer of the next layer.

15. The method according to claim 7, characterized in that The multi-time programmable memory includes static random access memory or flash memory.

16. The method according to claim 7, characterized in that The neural network operation is a multiplication and addition operation.

Citation Information

Patent Citations

  • Neural network circuit

    JP1996022446A

  • Neural network controller with fixed long-term and adaptive short-term memory

    US20080172349A1

  • Scalable neural hardware for the noisy-or model of bayesian networks

    US20150286924A1

  • Systems and methods for deep learning processor

    US20170316312A1

  • Multitransmitter model of the neural network with an internal feedback

    WO2014060001A1