Memristor processing unit, storage and calculation integrated processor, electronic equipment and operation method

Abstract

A memristor processing unit, a memory-computing integrated processor, an electronic device, and an operation method, the memristor processing unit comprising a main memristor array, a redundant memristor array, and an on-chip learning control circuit, the main memristor array being configured to store a first weight combination of a last fully connected layer of a neural network model, the redundant memristor array being configured to store a second weight combination of the last fully connected layer of the neural network model, and the on-chip learning control circuit being configured to store a second weight combination of the last fully connected layer of the neural network model; the redundant memristor array is configured to store a second weight combination for the last fully connected layer; the on-chip learning control circuit is configured to perform incremental learning of the neural network model. The memristor processing unit can be applied to an incremental learning-oriented storage and calculation integrated system so as to efficiently realize on-chip incremental learning.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a memristor processing unit, a storage-computing integrated processor, an electronic device, and an operating method based on the memristor processing unit. Background Art

[0002] Incremental learning is a machine learning method that continuously learns and adapts to new data based on existing knowledge. It allows the neural network model to further learn new data after learning a part of the old data to improve the performance of the model on the new data; in the process of learning new data, no or only a small amount of old data is used. Incremental learning allows the neural network model to learn new data quickly, even in real time, based on new data (or adding a small amount of old data); at the same time, incremental learning only needs to process new data (or adding a small amount of old data), which can save a lot of computing resources and time. In summary, compared with traditional machine learning methods that have obtained all training data before training, incremental learning methods have the advantages of high flexibility, low resource overhead, and strong learning ability in specific application scenarios. These advantages make incremental learning methods suitable for building smart terminal devices with adaptive capabilities.

[0003] At present, mainstream neural network acceleration chips based on digital circuit technology can be divided into cloud training acceleration chips and edge inference acceleration chips. Cloud training acceleration chips have high computing power requirements and are relatively insensitive to chip power consumption and energy efficiency. They are deployed in the cloud or computing center to accelerate the training process of neural network models; edge inference acceleration chips are more sensitive to power consumption and energy efficiency and are deployed on smart terminal devices to accelerate the inference process of neural network models. At present, there is still a lack of efficient hardware platforms for incremental learning scenarios that can be deployed on smart terminal devices. Summary of the invention

[0004] At least some embodiments of the present disclosure provide a memristor processing unit, comprising a main memristor array, a redundant memristor array and an on-chip learning control circuit, wherein the main memristor array is configured to store a first weight combination of a last fully connected layer of a neural network model; the redundant memristor array is configured to store a second weight combination for the last fully connected layer; and the on-chip learning control circuit is configured to perform incremental learning of the neural network model.

[0005] For example, in a memristor processing unit provided in at least some embodiments of the present disclosure, the second weight combination includes a weight combination expanded in response to category incremental learning of the neural network model; or the second weight combination includes a weight combination obtained by copying the last fully connected layer in response to sample incremental learning of the neural network model.

[0006] For example, in a memristor processing unit provided by at least some embodiments of the present disclosure, the main memristor array and the redundant memristor array are independent of each other and each includes its own peripheral driving circuit.

[0007] For example, in a memristor processing unit provided by at least some embodiments of the present disclosure, the first weight combination of the last fully connected layer includes the initial weights of the last fully connected layer obtained by off-chip training of the neural network model.

[0008] At least some embodiments of the present disclosure also provide a storage-computing integrated processor, including a memristor computing module, wherein the memristor computing module includes at least one memristor processing unit.

[0009] For example, in a storage-computing integrated processor provided by at least some embodiments of the present disclosure, the memristor computing module also includes a local cache and a local digital computing circuit, and the memristor processing unit is signal-connected to the local cache and the local digital circuit; the local cache is configured to cache intermediate calculation results between the memristor processing units within the memristor computing module and / or cache the input and output of the memristor computing module; the local digital computing circuit is configured to implement activation functions, pooling or regularization.

[0010] For example, in a storage-computing integrated processor provided in at least some embodiments of the present disclosure, an on-chip interconnection unit is also included, wherein the storage-computing integrated processor includes multiple memristor computing modules, and the on-chip interconnection unit is configured to implement communication between the multiple memristor computing modules.

[0011] An electronic device provided by at least some embodiments of the present disclosure includes a storage-computing integrated processor provided by any embodiment of the present disclosure.

[0012] At least some embodiments of the present disclosure also provide an operating method for a memristor processing unit, comprising: storing a first weight combination of a last fully connected layer of a neural network model in a main memristor array; storing a second weight combination for the last fully connected layer in a redundant memristor array; and performing incremental learning of the neural network model by an on-chip learning control circuit.

[0013] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, a first weight combination of a final fully connected layer of a neural network model is stored in a main memristor array, including: storing the initial weights of the final fully connected layer of the neural network model obtained through off-chip training in the main memristor array.

[0014] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, the incremental learning is category incremental learning, and the second weight combination includes a weight combination expanded in response to the category incremental learning; or the incremental learning is sample incremental learning, and the second weight combination includes a weight combination obtained by copying the last fully connected layer in response to the sample incremental learning.

[0015] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, a second weight combination for the last fully connected layer is stored in a redundant memristor array, including: in response to the incremental learning being category incremental learning, initializing the memristors corresponding to the second weight combination in the redundant memristor array.

[0016] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, the memristors corresponding to the second weight combination are initialized in the redundant memristor array, including: initializing the memristors corresponding to the second weight combination based on a single-sided check write operation; or initializing the memristors corresponding to the second weight combination based on a gate voltage current limiting setting operation; or initializing the memristors corresponding to the second weight combination based on sampling of existing weights.

[0017] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, a second weight combination for a last fully connected layer is stored in a redundant memristor array, comprising: in response to the incremental learning being sample incremental learning, copying the weight combination of the last fully connected layer to obtain multiple copies, and storing at least one copy obtained by copying the last fully connected layer in the redundant memristor array.

[0018] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, incremental learning of a neural network model is performed by an on-chip learning control circuit, including: updating weights of multiple copies in incremental learning, and adjusting at least one weight in the last fully connected layer.

[0019] For example, in an operating method for a memristor processing unit provided by at least some embodiments of the present disclosure, the method further includes: during the execution of an inference task, using an average value of multiple copies as the weight output of the last fully connected layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0021] Figure 1A conceptual diagram of incremental learning;

[0022] Figure 2A A schematic diagram of the structure of a memristor processing unit provided in some embodiments of the present disclosure;

[0023] Figure 2B A schematic diagram of a memristor array and its peripheral driving circuit provided in some embodiments of the present disclosure;

[0024] Figure 3 A schematic diagram of the structure of a storage-computing integrated processor provided in some embodiments of the present disclosure;

[0025] Figure 4 A schematic flow chart of an operation method based on a memristor processing unit provided for some embodiments of the present disclosure;

[0026] Figure 5 is a schematic flow chart of an operation method corresponding to category incremental learning;

[0027] Figure 6 A schematic diagram of weight expansion in a category incremental learning process in some embodiments of the present disclosure;

[0028] Figure 7 A schematic flowchart of an operation method of sample incremental learning in some embodiments of the present disclosure;

[0029] Figure 8 To correspond to Figure 7 A schematic flowchart of an operation method performed by an on-chip learning control circuit provided in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0031] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0032] Incremental learning means learning new knowledge by learning new data on the basis of existing knowledge, while not forgetting existing knowledge. The new data used in incremental learning can be divided into new categories or new samples, corresponding to two modes of incremental learning: category incremental learning and sample incremental learning. Category incremental learning means that the neural network model learns to recognize new categories by learning new category data on the basis of being able to recognize some categories, so category incremental learning increases the number of categories that the model can recognize. Sample incremental learning means that the model enhances the adaptability of the model to new samples by learning new sample data of the recognizable categories on the basis of being able to recognize some categories, so sample incremental learning does not increase the number of categories that the model can recognize. In addition, sample incremental learning is not only applicable to the above-mentioned classification tasks, but also to regression tasks.

[0033] Figure 1 The concept of incremental learning is shown. Figure 1 As shown, incremental learning includes category incremental learning and sample incremental learning. For example, a neural network model has the prior knowledge of identifying dogs and cats, and learns to identify new categories by learning new category data (such as rabbits), which is category incremental learning. The neural network model enhances its adaptability to new sample data by learning new sample data (other types of dogs), which is sample incremental learning.

[0034] Compared with traditional machine learning methods that have obtained all training data before training, incremental learning methods have the advantages of high flexibility, low resource overhead, and strong learning ability in specific application scenarios. These advantages make incremental learning methods suitable for building smart terminal devices with adaptive capabilities. However, there is still a lack of efficient hardware platforms for incremental learning scenarios that can be deployed on smart terminal devices. This is because for hardware platforms based on digital circuit technology, the high energy consumption caused by the large amount of data transfer is the main factor limiting its application in incremental learning scenarios.

[0035] First, for processors based on the von Neumann architecture, such as the central processing unit (CPU) and graphics processing unit (GPU), the storage unit and the computing unit are physically separated; the frequent transfer of large amounts of data between the storage unit and the control unit leads to excessive energy consumption.

[0036] Secondly, some neural network acceleration chips, such as tensor processing units (TPUs), reduce the cost of data transfer by reusing weights and increasing on-chip static random access memory (SRAM). However, because the SRAM unit structure is large, its storage density is low, so the SRAM capacity may not be enough to store the weights of the entire neural network model. At this time, the neural network acceleration core still needs to exchange a large amount of data with off-chip memory (such as dynamic random access memory - DRAM, etc.); data transfer still leads to excessive energy consumption.

[0037] Finally, SRAM, DRAM, etc., which are usually used to store weights during the calculation process, are volatile memories, and data will be lost after power failure. However, due to energy consumption constraints, smart terminal devices often adopt intermittent working mode - turning off some circuits when there is no task to save energy, such as some computing circuits and volatile memories. In this case, every time new data arrives, the weights are first moved from the non-volatile memory to the on-chip or off-chip volatile memory, and the neural network model is trained to learn the new data; then, the updated weights are moved to the off-chip non-volatile memory, and some computing circuits and volatile memories are turned off to save energy. The data movement caused by repeatedly turning off and on the volatile memory further increases the energy consumption of the entire hardware platform.

[0038] In summary, in the hardware platform based on digital circuit technology, there is a large amount of data transfer between computing units and storage units, between on-chip memory and off-chip memory, and between volatile memory and non-volatile memory; the high energy consumption caused by the large amount of data transfer limits its application in incremental learning scenarios.

[0039] Memristors (such as resistive memory, phase change memory, conductive bridge memory, etc.) are non-volatile devices whose conductance state can be adjusted by applying external stimuli. For example, according to Kirchhoff's current law and Ohm's law, an array of such devices can perform multiplication and accumulation calculations in parallel, and both storage and calculation occur in each device in the array. Multiplication and accumulation is the core computing task required to run a neural network.

[0040] When mapping a neural network to a memristor array, the weight parameters can be directly programmed as the conductance of the memristor array, or the weight parameters (weights) can be mapped to the conductance of the memristor array according to a certain rule. The memristor array obtained in this way can realize storage-computing technology, which combines storage and computing, greatly reducing the overhead of data handling. In addition, the weight update process of the neural network model can also be efficiently implemented by modulating the conductance of the memristor in situ. At the same time, since the memristor is also non-volatile, it can avoid data handling caused by shutting down and turning on in the hardware platform based on digital circuit technology, which is very suitable for the mechanism of intermittent operation of smart terminal devices to reduce energy consumption.

[0041] The inventors of the present disclosure note that, as mentioned above, since the memristor storage-computing integrated technology can efficiently realize the forward reasoning, backpropagation and in-situ weight update of the neural network model (corresponding to the conductivity modulation process of the memristor), this storage-computing fusion and non-volatility characteristics make the memristor storage-computing integrated technology very suitable for developing chips for incremental learning scenarios.

[0042] The present disclosure proposes a memristor processing unit, a storage-computing integrated processor, and an operation method based on the memristor processing unit to efficiently implement an on-chip incremental learning process.

[0043] Figure 2A A schematic diagram of the structure of a memristor processing element (PE) provided in some embodiments of the present disclosure. The memristor processing element is based on a memristor array and includes a main memristor array, a redundant memristor array and an on-chip learning control circuit.

[0044] The main memristor array can be configured to store the first weight combination of the last fully connected layer of the neural network model. For example, the main memristor array can be configured to store the initial weights of the last fully connected layer obtained by the neural network model through off-chip training. This configuration can enhance the generalization ability of the neural network model and is conducive to subsequent on-chip incremental learning. For example, the neural network can be a network structure of various appropriate forms, such as a back propagation (BP) neural network, a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), etc. For example, the convolutional neural network can also be a fully convolutional network, a deep convolutional network, a U-net, etc. For example, the neural network includes an input layer, an intermediate layer, and an output layer, and the output layer can be, for example, a fully connected layer.

[0045] The redundant memristor array may be configured to store a second weight combination including a weight combination for the last fully connected layer. For example, the redundant memristor array may be configured to store a weight combination expanded in response to the incremental learning of categories of the neural network model, the second weight combination including the weight combination obtained by expansion, for example, for newly added categories in the incremental learning of categories, the last fully connected layer may be expanded in the row (or column) direction for the newly added categories, which corresponds to the increase in the output dimension of the last fully connected layer of the neural network model. For example, the redundant memristor array may also be configured to store a weight combination obtained by copying the last fully connected layer in response to the incremental learning of samples of the neural network model, the second weight combination including the weight combination obtained by copying, for the copy of the last fully connected layer, the weight update amount of each iteration in the incremental learning process of samples can be controlled within a suitable range to avoid over-learning of new samples and forgetting of existing knowledge, while also avoiding non-ideal weight updates that may be caused by a single conductivity modulation operation of the memristor.

[0046] For example, in at least one example, the main memristor array and the redundant memristor array can be provided independently, for example, the two can be distinguished in physical settings, for example, the main memristor array and the redundant memristor array can also include respective peripheral drive circuits, for example, the peripheral drive circuit includes a word line drive circuit, a bit line drive circuit, a source line drive circuit, a data reading circuit, etc. It should be noted that the peripheral drive circuit can be added and cut accordingly according to the different designs (such as row and column scales) of the main memristor array and the redundant memristor array, and the embodiments of the present disclosure do not limit the specific peripheral circuit implementation method.

[0047] The on-chip learning control circuit can be configured to perform incremental learning of the neural network model, for example, to implement various training algorithm designs, and for example, the on-chip learning control circuit can also be configured to implement an error back propagation (BP) algorithm. For example, the on-chip learning control circuit can be implemented by a digital circuit, such as a programmable digital circuit.

[0048] In the memristor processing unit for incremental learning of the disclosed embodiment, through the control of the on-chip learning control circuit, in the process of incremental learning, the memristor array can integrate storage and computing, greatly reducing the overhead of data handling, reducing the dependency of incremental learning on resources, and making incremental learning adaptable to more application scenarios. In addition, in at least one example, the memristor array can be divided to store neural network weights and efficiently implement matrix-vector multiplication, thereby efficiently implementing forward reasoning calculations and back propagation calculations without the need for weight data handling.

[0049] Furthermore, it should be understood that Figure 2A The exemplary structure of the memristor processing unit shown can be tailored and added according to different architectures and application scenarios, such as adding an on-chip storage module, a direct memory access module, etc., which is not limited in the embodiments of the present disclosure.

[0050] Figure 2B A schematic diagram of a memristor array and its peripheral driving circuits provided in at least one embodiment of the present disclosure, wherein the peripheral driving circuits are used to implement input and output of the memristor array. The memristor array can be used to implement a main memristor array and / or a redundant memristor array.

[0051] For example, Figure 2B As shown, the peripheral driving circuit of the memristor array includes a signal acquisition device, a word line driving circuit, a bit line driving circuit, a source line driving circuit, and a data output circuit.

[0052] For example, the signal acquisition device is configured to convert the digital signal into a plurality of first analog signals through a digital to analog converter (DAC) so as to be input to a plurality of column signal input terminals of the memristor array during convolution processing.

[0053] For example, the memristor array includes M source lines, M word lines, and N bit lines, and a plurality of memristor units arranged in an array of M rows and N columns. For example, each memristor unit is a 1T1R (one transistor one memristor) structure, or other types such as 2T2R (two transistors two memristors).

[0054] For example, the operation of the memristor array is realized through a word line driving circuit, a bit line driving circuit, and a source line driving circuit.

[0055] For example, the word line driving circuit includes a plurality of multiplexers (Multiplexer, Mux for short) for switching the word line input voltage; the bit line driving circuit includes a plurality of multiplexers for switching the bit line input voltage; the source line driving circuit also includes a plurality of multiplexers (Mux) for switching the source line input voltage. For example, the source line driving circuit also includes a plurality of ADCs for converting analog signals into digital signals. In addition, a trans-impedance amplifier (TIA for short) (not shown in the figure) can be further arranged between the Mux and the ADC in the source line driving circuit to complete the current-to-voltage conversion for ADC processing.

[0056] For example, the memristor array includes an operation mode and a calculation mode. When the memristor array is in the operation mode, the memristor unit is in an initialized state, and the values ​​of the parameter elements in the parameter matrix can be written into the memristor array. For example, the source line input voltage, the bit line input voltage, and the word line input voltage of the memristor are switched to the corresponding preset voltage interval through a multiplexer.

[0057] For example, the word line input voltage is switched to a corresponding voltage range by a control signal WL_sw[1:M] of a multiplexer in a word line driving circuit. For example, when a memristor is set, the word line input voltage is set to 2V (volts); for example, when a memristor is reset, the word line input voltage is set to 5V; for example, the word line input voltage can be switched to a corresponding voltage range by a control signal WL_sw[1:M] of a multiplexer in a word line driving circuit. Figure 2B The voltage signal V_WL[1:M] is obtained.

[0058] For example, the source line input voltage is switched to a corresponding voltage interval by a control signal SL_sw[1:M] of a multiplexer in a source line driving circuit. For example, when a memristor is set, the source line input voltage is set to 0V. For example, when a memristor is reset, the source line input voltage is set to 2V. For example, the source line input voltage can be switched to a corresponding voltage interval by a control signal SL_sw[1:M] of a multiplexer in a source line driving circuit. Figure 2B The voltage signal V_SL[1:M] is obtained.

[0059] For example, the bit line input voltage is switched to a corresponding voltage range by a control signal BL_sw[1:N] of a multiplexer in a bit line driving circuit. For example, when a memristor is set, the bit line input voltage is set to 2V. For example, when a memristor is reset, the bit line input voltage is set to 0V. For example, the bit line input voltage can be switched to a corresponding voltage range by a control signal BL_sw[1:N] of a multiplexer in a bit line driving circuit. Figure 2B The DAC in the

[0060] For example, when the memristor array is in computing mode, the memristors in the memristor array are in a conductive state that can be used for computing, and the bit line input voltage input to the column signal input terminal does not change the conductance value of the memristor. For example, multiplication and addition operations can be performed through the memristor array to complete the calculation. For example, the word line input voltage is switched to a corresponding voltage interval through the control signal WL_sw[1:M] of the multiplexer in the word line driving circuit, for example, when an on signal is applied, the word line input voltage of the corresponding row is set to 5V, and when no on signal is applied, the word line input voltage of the corresponding row is set to 0V, for example, the GND signal is connected; the source line input voltage is switched to a corresponding voltage interval through the control signal SL_sw[1:M] of the multiplexer in the source line driving circuit, for example, the source line input voltage is set to 0V, so that the current signals of the multiple row signal output ends can flow into the data output circuit; the bit line input voltage is switched to a corresponding voltage interval through the control signal BL_sw[1:N] of the multiplexer in the bit line driving circuit, for example, the bit line input voltage is set to 0.1V-0.3V, so as to complete the convolution operation by utilizing the characteristic that the memristor array can perform multiplication and addition operations.

[0061] For example, the data output circuit may include multiple TIAs and ADCs, which may convert current signals at multiple row signal output terminals into voltage signals, and then convert them into digital signals for subsequent processing.

[0062] At least one embodiment of the present disclosure also provides a memristor computing module, which includes at least one memristor processing unit provided by any embodiment of the present disclosure, and may also include a local cache and a local digital circuit as needed, and the memristor processing unit is signal-connected to the local cache and the local digital circuit.

[0063] It should be noted that the memristor computing module in the embodiments of the present disclosure can also be tailored and added according to different needs and scenarios, and the embodiments of the present disclosure are not limited to this.

[0064] For example, in some embodiments of the present disclosure, the local cache can be configured to cache intermediate calculation results between memristor processing units in a memristor computing module to achieve communication between memristor processing units in the memristor computing module. For example, the local cache can also be configured to cache the input and output of the memristor computing module. For example, the local digital computing circuit can be configured to implement operations such as activation functions, pooling or regularization for neural networks.

[0065] Figure 3 The structure of a storage-computing integrated processor provided by some embodiments of the present disclosure is shown. Figure 3As shown, the storage-computing integrated processor includes at least one memristor computing module, and may also include a network on chip (NOC). It should be noted that the embodiments of the present disclosure do not limit the number of memristor computing modules in the storage-computing integrated processor. For example, the storage-computing integrated processor may be implemented in the form of a chip, such as a system chip (SOC) together with a CPU / GPU, or may be implemented as a coprocessor of a CPU, etc.

[0066] The storage-computing integrated processor includes a plurality of memristor computing modules, among which the memristor computing modules have corresponding on-chip interconnection units, which are configured to realize communication between the memristor computing modules, and the communication includes the transmission of control signals and / or data signals. It should be understood that the on-chip interconnection units can be set according to the usage scenario, and the embodiments of the present disclosure are not limited to this.

[0067] It should be noted that in some embodiments of the present disclosure, the memristor computing module in the integrated computing processor may also include an input and output cache unit, for example, the output cache unit is configured to cache the calculation results between the memristor computing modules. For example, the output cache unit may also be configured to cache off-chip inputs and outputs.

[0068] For example, the storage-computing integrated processor in some embodiments of the present disclosure may also include a data flow control unit or a global storage.

[0069] exist Figure 3 In the storage-computing integrated processor provided by the illustrated embodiment, multiple memristor computing modules can communicate through an on-chip interconnect unit, and the storage and computing are integrated into the module, thereby greatly reducing the overhead of data transfer: the memristor array in the memristor computing module can store the weights of the neural network and efficiently implement matrix-vector multiplication, so that forward reasoning calculations and back-propagation calculations can be efficiently implemented without the need for weight data transfer; the weight update process of the neural network model can also be efficiently implemented by modulating the conductance of the memristor in situ; in addition, the non-volatility of the memristor avoids data transfer caused by shutting down and turning on in the hardware platform based on digital circuit technology, and is very suitable for the mechanism of intermittent operation of smart terminal devices to reduce energy consumption, thereby reducing power consumption during incremental learning.

[0070] At least one embodiment of the present disclosure further provides an electronic device, which includes a storage-computing integrated processor of any of the above embodiments. For example, the electronic device in the embodiments of the present disclosure may be a server device or a terminal device, for example, the terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.

[0071] Refer to the following Figure 4 The operating methods for the memristor processing unit of some embodiments of the present disclosure are described in detail.

[0072] like Figure 4 As shown, according to at least one embodiment of the present disclosure, the operating method for the above-mentioned memristor processing unit includes the following steps S10-S30.

[0073] Step S10, storing the first weight combination of the last fully connected layer of the neural network model in the main memristor array.

[0074] Each weight in the first weight combination may be, for example, a weight updated during the on-chip training process, or may be an initial weight at the beginning of the on-chip training.

[0075] For example, the acquired data can first be used to train a set of parameters of the neural network model outside the memristor storage and computing integrated processor (such as based on a general-purpose image processing unit (GPU), AI accelerator, etc.). During this off-chip training process, the generalization ability of the neural network model can also be enhanced by introducing noise, using data enhancement, or retraining based on a pre-trained model on a larger data set (such as ImageNet, etc.), which is beneficial to subsequent on-chip incremental learning. Then, the above-mentioned trained neural network model is deployed to the main memristor array, for example, the initial weights of the last fully connected layer of the neural network model obtained by off-chip training are stored in the main memristor array.

[0076] Step S20, storing a second weight combination for the last fully connected layer in the redundant memristor array.

[0077] Step S30: the on-chip learning control circuit performs incremental learning of the neural network model.

[0078] In some embodiments of the present disclosure, incremental learning is achieved by adjusting the weights of the last fully connected layer of the neural network model by an on-chip learning control circuit. For example, the incremental learning may be category incremental learning or sample incremental learning.

[0079] Regarding step S20, in some embodiments of the present disclosure, the specific implementation method of step S20 may be different for category incremental learning and sample incremental learning.

[0080] For example, combined with Figure 5 The operating methods of redundant memristor arrays for class incremental learning according to some embodiments of the present disclosure are described in detail.

[0081] Step S210 , storing in a redundant memristor array a weight combination expanded in response to category incremental learning.

[0082] For example, for category incremental learning, weight expansion is performed along the output dimension in the last fully connected layer of the neural network model for the newly added categories.

[0083] Figure 6 This is a schematic diagram of weight expansion in the process of incremental learning of categories. Figure 6 For example, for each new category added, the output dimension of the last fully connected layer of the neural network model increases by one. Figure 6 The newly added weights and outputs are indicated by dashed lines. For example, corresponding to the memristor processing unit, the main memristor array stores the weights corresponding to the categories before the expansion, and the redundant memristor array stores the weights obtained by expanding the newly added categories along the column (or row) direction; correspondingly, when the categories are added again, the main memristor array and the redundant memristor array store the weights corresponding to the categories before the expansion as a whole, and the redundant memristor array stores the weights obtained by expanding the newly added categories again along the column (or row) direction.

[0084] Step S2110 , in response to the incremental learning being class incremental learning, initializing the memristors corresponding to the second weight combination in the redundant memristor array.

[0085] When performing incremental category learning, first expand the weights of the newly added categories along the output dimension in the last fully connected layer of the neural network model. During the weight expansion process, for example, the existing weights of the last fully connected layer of the neural network model are within a certain range and satisfy a certain distribution law; therefore, if the weights expanded for the newly added categories also conform to the distribution law to a certain extent after being initialized, it can reduce the training time and improve the processing accuracy.

[0086] For example, in some embodiments of the present disclosure, a weight initialization strategy based on a unilateral verification write operation can be used to appropriately initialize the weights of the newly added category expansion. The unilateral verification write operation means that when programming the conductance of the memristor, it is only necessary to determine whether the conductance value of the memristor is greater than (or less than) a certain value; that is, the target conductance range has a boundary on only one side. If the conductance value of the memristor falls within the target range or the number of programming times reaches a preset upper limit, the programming operation is stopped; otherwise, the programming operation continues. When using the unilateral verification write operation to initialize the weights, first set a maximum allowable conductance value, such as the maximum conductance value in the existing weights; then use the unilateral verification write operation to initialize each memristor in the weights expanded for the newly added category: if the conductance value falls within the target range or the number of programming times reaches a preset upper limit, stop the reset (RESET) operation (the operation of reducing the conductance value of the memristor); otherwise, continue the reset operation. In this way, the write noise in the unilateral verification operation is used to achieve random initialization of the weights expanded for the newly added category.

[0087] For example, in some embodiments of the present disclosure, a weight initialization strategy based on a gate voltage current limiting setting operation can be used to appropriately initialize the newly added category expansion weights. When a memristor is set (SET) operated (an operation that increases the conductance value of the memristor), the gate voltage applied to the transistor connected in series with it plays a role of current limiting - the conductance value achieved by the memristor after being set has a clear correlation with the gate voltage of the above transistor. If multiple memristors are set at the same gate voltage, the conductance values ​​achieved by these memristors will show a certain random distribution.

[0088] For example, when performing weight initialization using a gate voltage current limiting setting operation, first set a gate voltage, for example, use the mean of the existing weighted conductance values ​​of the last fully connected layer of the neural network model to select the corresponding gate voltage according to the relationship between the gate voltage and the conductance; then perform a setting operation on each memristor in the weight expanded for the newly added category under the gate voltage current limiting (applying the gate voltage to the transistor connected in series with the memristor). In this way, the randomness in the gate voltage current limiting setting operation is used to achieve random initialization of the weight expanded for the newly added category.

[0089] For example, in some embodiments of the present disclosure, a weight initialization strategy based on sampling of existing weights may be used to appropriately initialize the newly added category extension weights. In the category incremental learning scenario, the purpose of randomly initializing the extended weights is to make them have a similar conductance range and weight distribution as the existing weights, so the extended weights can be initialized by sampling the existing weights.

[0090] For example, when randomly initializing by sampling existing weights, the following operations are performed on each memristor in the extended weights in sequence:

[0091] 1) Select a memristor with an existing weight according to a certain random algorithm;

[0092] 2) Read the conductance value of the selected memristor;

[0093] 3) Programming the memristor in the extended weight according to the read conductance value.

[0094] For example, when programming the memristor array, a bilateral check write operation can be used, that is, the memristor is programmed to a certain error range of the target conductance value (corresponding to the read conductance value) by repeated checking; for example, a gate voltage current limiting setting operation can also be used, that is, the read conductance value is used to select the corresponding gate voltage according to the relationship between the gate voltage and the conductance, and then the memristor is set under the gate voltage current limiting (the gate voltage is applied to the transistor connected in series with the memristor). In this way, the random initialization of the weights expanded for the newly added category is achieved by sampling the existing weights.

[0095] For example, combined with Figure 7 The operating methods of a redundant memristor array for sample incremental learning according to some embodiments of the present disclosure are described in detail.

[0096] Step S220, storing in the redundant memristor array a weight combination obtained by copying the last fully connected layer in response to sample incremental learning.

[0097] Since sample incremental learning requires the neural network model to enhance its adaptability to new samples while trying not to forget existing knowledge, the amount of weight update in each iteration (or for each training batch) cannot be too large during sample incremental learning, otherwise it will lead to over-learning of new samples and forgetting of existing knowledge. In some embodiments of the present disclosure, when implementing a neural network model based on a memristor array, the weight update is achieved by modulating the conductance of the memristor in situ. On the other hand, if the conductivity modulation of the memristor has non-ideal characteristics such as poor analog resistance switching characteristics and write noise, a single conductivity modulation operation may cause a large, non-ideal weight update. In this case, the non-ideal characteristics in the conductivity modulation will cause the memristor chip to perform poorly in the sample incremental learning scenario. To this end, in some embodiments of the present disclosure, a method such as Figure 7 The operating method shown is to copy the last fully connected layer to obtain a weight combination in response to sample incremental learning and store the weight combination in a redundant memristor array.

[0098] Step S2210, in response to the incremental learning being sample incremental learning, the weight combination of the last fully connected layer is copied to obtain multiple copies, and at least one copy obtained by copying the last fully connected layer is stored in a redundant memristor array.

[0099] In addition to one copy stored in a redundant memristor array, for example, other copies may be stored elsewhere outside the memristor processing unit, for example.

[0100] When performing weight replication for sample incremental learning, by copying the last fully connected layer of the neural network model multiple times, the impact of non-ideal characteristics in the conductance modulation of the memristor can be weakened, thereby improving the performance of the memristor storage and computing processor in the sample incremental learning scenario.

[0101] It should be understood that for category incremental learning and sample incremental learning, the operations of step S10 and step S30 may be the same, and will not be repeated here.

[0102] Figure 8 To correspond to Figure 7 A schematic flow chart of an operation method performed by an on-chip learning control circuit provided in some embodiments of the present disclosure. The on-chip learning control circuit performs incremental learning of a neural network model, including:

[0103] Step S310, updating the weights of multiple copies in incremental learning, and adjusting the weights in the last fully connected layer.

[0104] In incremental learning, whether due to an increase in categories or an increase in samples, when the weights of the last fully connected layer need to be adjusted, for example, when it is necessary to expand to increase the dimension and thus add a row or column of memristors or modify the resistance values ​​of memristors corresponding to certain weights, multiple copies are adjusted synchronously so that the multiple copies remain consistent with each other.

[0105] Step S320 (not shown in the figure), during the execution of the inference task, the average value of multiple copies is used as the weight output of the final fully connected layer.

[0106] For example, the average value may be an arithmetic mean or another type of average value.

[0107] Figure 8 In some embodiments of the present disclosure shown in , the method of making multiple copies of the last fully connected layer of the neural network model is used to overcome the impact of the non-ideal characteristics of the conductance modulation of the memristor in the storage-computing integrated processor on the incremental learning of samples. Compared with the overhead of the entire neural network model, the overhead introduced by only copying the last fully connected layer of the model is still small in most cases, and will not cause the storage-computing integrated processor to make excessively high power consumption requirements, and can take into account the advantages of high flexibility of incremental learning, low resource overhead, and strong learning ability.

[0108] There are a few points to note about this disclosure:

[0109] (1) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can refer to the general design.

[0110] (2) In the absence of conflict, features in the same embodiment or in different embodiments of the present disclosure may be combined with each other.

[0111] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A memristor processing unit, comprising a main memristor array, a redundant memristor array and an on-chip learning control circuit, wherein: The main memristor array is configured to store a first weight combination of a last fully connected layer of a neural network model; The redundant memristor array is configured to store a second weight combination for the last fully connected layer; The on-chip learning control circuit is configured to perform incremental learning of the neural network model.

2. The memristor processing unit according to claim 1, wherein: The second weight combination includes a weight combination expanded in response to category incremental learning of the neural network model; or The second weight combination includes a weight combination obtained by copying the last fully connected layer in response to sample incremental learning of the neural network model.

3. The memristor processing unit according to claim 1, wherein: The main memristor array and the redundant memristor array are independent of each other and respectively include respective peripheral driving circuits.

4. The memristor processing unit according to claim 1, wherein: The first weight combination of the last fully connected layer includes the initial weights of the last fully connected layer obtained by off-chip training of the neural network model.

5. A storage and computing integrated processor, comprising a memristor computing module, wherein: The memristor computing module includes at least one memristor processing unit as described in any one of claims 1-4.

6. The integrated storage and computing processor according to claim 5, wherein: The memristor computing module also includes a local cache and a local digital computing circuit. The memristor processing unit is signal-connected to the local cache and the local digital circuit; The local cache is configured to cache intermediate calculation results between memristor processing units in the memristor computing module and / or cache input and output of the memristor computing module; The local digital computing circuit is configured to implement an activation function, pooling or regularization.

7. The integrated storage and computing processor according to claim 6 further comprises an on-chip interconnect unit, wherein: The storage-computing integrated processor includes a plurality of memristor computing modules. The on-chip interconnect unit is configured to implement communication between a plurality of the memristor computing modules.

8. An electronic device comprising the integrated storage and computing processor as described in any one of claims 5 to 7.

9. A method for operating the memristor processing unit of claim 1, comprising: Storing a first weight combination of a last fully connected layer of a neural network model in the main memristor array; storing a second weight combination for the last fully connected layer in the redundant memristor array; Incremental learning of the neural network model is performed by the on-chip learning control circuit.

10. The operating method according to claim 9, wherein: Storing a first weight combination of the last fully connected layer of the neural network model in the main memristor array, comprising: The initial weights of the last fully connected layer of the neural network model obtained through off-chip training are stored in the main memristor array.

11. The operating method according to claim 10, wherein: The incremental learning is category incremental learning, and the second weight combination includes a weight combination expanded in response to the category incremental learning; or The incremental learning is sample incremental learning, and the second weight combination includes a weight combination obtained by copying the last fully connected layer in response to the sample incremental learning.

12. The operating method according to claim 11, wherein: Storing a second weight combination for the last fully connected layer in the redundant memristor array comprises: In response to the incremental learning being class incremental learning, the memristors corresponding to the second weight combination are initialized in the redundant memristor array.

13. The operating method according to claim 12, wherein: Initializing the memristor corresponding to the second weight combination in the redundant memristor array includes: Initializing the memristor corresponding to the second weight combination based on a unilateral verification write operation; or Initializing the memristor corresponding to the second weight combination based on a gate voltage current limiting setting operation; or The memristor corresponding to the second weight combination is initialized based on sampling the existing weights.

14. The operating method according to claim 11, wherein: Storing a second weight combination for the last fully connected layer in the redundant memristor array comprises: In response to the incremental learning being sample incremental learning, the weight combination of the last fully connected layer is copied to obtain multiple copies, and At least one copy of the last fully connected layer is stored in the redundant memristor array.

15. The operating method according to claim 14, wherein: The on-chip learning control circuit performs incremental learning of the neural network model, including: In the incremental learning, weights of the multiple copies are updated to adjust at least one weight in the last fully connected layer.

16. The operating method according to claim 14 or 15, further comprising: During the execution of the inference task, an average value of the multiple copies is used as the weight output of the last fully connected layer.

Images