Plastic neuron circuit, neuron time constant calculation method and electronic equipment

Abstract

The invention relates to the technical field of artificial neural networks, in particular to a plastic neuron circuit, a neuron time constant calculation method and electronic equipment, and the circuit comprises a storage sub-circuit which is used for storing an input voltage; the shared sub-circuit comprises a plurality of capacitors, the plurality of capacitors switch capacitors sharing charges with the storage sub-circuit in each period, and a time constant of a target neuron is shaped by sharing the charges; and the conversion sub-circuit is used for carrying out nonlinear conversion on the voltage generated by the shared charge and storing the voltage after nonlinear conversion to the capacitor of the shared sub-circuit, and the capacitor of the shared charge in the plurality of capacitors is different from the capacitor of the voltage after nonlinear conversion in each period. Therefore, the problems of limitation on adjustment and application range of neuron time constants, high hardware resource consumption, high cost and the like in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the technical field of artificial neural networks, and in particular to a plastic neuron circuit, a method for calculating a neuron time constant, and an electronic device. Background Art

[0002] With the rapid development of AI (Artificial Intelligence) technology, artificial neural network models are constantly being updated and iterated, and their scale is increasing. However, general-purpose hardware systems face significant challenges in cost and power consumption when dealing with ultra-large-scale neural networks, making it difficult to meet the needs of efficient computing. Therefore, neural network chips specifically designed to accelerate AI tasks have emerged. These chips aim to achieve high-speed computing through highly integrated neurons, while simultaneously approaching the large-scale and low-power characteristics of biological neural networks as closely as possible.

[0003] Although current artificial neural network research draws on the top-level architecture of biological neural networks, existing large-scale neural network hardware systems remain insufficient in achieving the fundamental properties of biological neurons. Current hardware implementations mostly use unified neurons for integration and struggle to achieve time constant plasticity. Many methods require extensive hardware resources to support complex time constant adjustments.

[0004] For example, the traditional method of using digital circuits to implement state storage requires a large number of storage units and frequent state updates. When the time constant needs to be adjustable, it is necessary to add configured peripheral circuits, which increases the power consumption and cost of the system; the method of using capacitors to implement analog state storage and attenuation is also relatively common, but different applications usually require consideration of different time constants. The area overhead of the capacitors used is too large and difficult to adjust; new materials and new devices are used to simulate the characteristics of biological neurons, but they are limited in integration capabilities, and are very limited in the consistency between neurons, the adjustable range of neurons, and the scope of application. Summary of the Invention

[0005] The present application provides a plastic neuron circuit, a method for calculating a neuron time constant, and an electronic device to address the problems in related technologies such as limitations on the adjustment and application scope of the neuron time constant, as well as high hardware resource consumption and cost.

[0006] In a first aspect, an embodiment of the present application provides a plastic neuron circuit, comprising: a storage subcircuit for storing an input voltage; a sharing subcircuit, wherein the sharing subcircuit includes a plurality of capacitors, and the plurality of capacitors switches the capacitors sharing charge with the storage subcircuit in each cycle, thereby shaping the time constant of the target neuron through the shared charge; and a conversion subcircuit for performing nonlinear conversion on the voltage generated by the shared charge, and storing the nonlinearly converted voltage in the capacitors of the sharing subcircuit, wherein in each cycle, the capacitors sharing charge among the plurality of capacitors are different from the capacitors storing the nonlinearly converted voltage.

[0007] Optionally, the storage subcircuit includes a first switch and a first capacitor, and the first capacitor is used to store the input voltage after the first switch is closed.

[0008] Optionally, the shared sub-circuit further includes multiple groups of second switches and third switches, each capacitor of the shared sub-circuit is connected to a group of second switches and third switches, and the capacitor sharing charge with the first capacitor is determined by closing and / or opening the second switch and the third switch.

[0009] Optionally, a fourth switch is provided between the storage sub-circuit and the sharing sub-circuit, for controlling the conduction or disconnection of the storage sub-circuit and the sharing sub-circuit.

[0010] Optionally, the first switch, the second switch, the third switch and the fourth switch are closed and / or opened a target number of times in each cycle, so that the multiple capacitors switch to share charge with the storage subcircuit in each cycle, thereby shaping the time constant of the target neuron by sharing charge.

[0011] A second aspect of the present application provides a method for calculating a neuron time constant, which is applied to a plastic neuron circuit as in the above embodiment, and includes the following steps: building a target neural network model on target simulation software, wherein the target neural network model includes multiple plastic neuron circuits as in the above embodiment, and each plastic neuron circuit corresponds to a neuron; obtaining the expected effect of the target neural network model in processing the target task; adjusting the number of shared charges in the plastic neuron circuit corresponding to each neuron so that the target neural network model achieves the expected effect, and determining the time constant of each neuron based on the adjusted number of shared charges.

[0012] Optionally, before adjusting the number of times the plastic neuron circuit corresponding to each neuron shares charges, the method includes: obtaining a nonlinear conversion relationship of each plastic neuron circuit; fitting the nonlinear conversion relationship to obtain a nonlinear function, and adjusting the number of times the plastic neuron circuit corresponding to each neuron shares charges based on the nonlinear function.

[0013] Optionally, adjusting the number of times that charges are shared in the plastic neuron circuit corresponding to each neuron includes calculating the number of times that charges are shared in the plastic neuron circuit corresponding to each neuron according to a nonlinear function of each neuron and a covariance evolution algorithm.

[0014] A third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform a method for calculating a neuron time constant as described in the above embodiment.

[0015] The fourth aspect of the present application provides a computer-readable storage medium having a computer program or instructions stored thereon, and the computer program or instructions are executed by a processor to perform the method for calculating the neuron time constant as described in the above embodiment.

[0016] Therefore, this application has at least the following beneficial effects:

[0017] The present application discloses a plastic neuron circuit, comprising a storage subcircuit, a sharing subcircuit, and a conversion subcircuit. The storage subcircuit stores an input voltage, and the shared charge is periodically switched between multiple capacitors in the shared subcircuit to share charge with the storage subcircuit. The time constant of the target neuron is shaped by the shared charge, eliminating the need for complex hardware resources to achieve time regulation of the neuron. The time constant of the neuron can be shaped by simply sharing the charge, reducing hardware resource requirements and enabling a wide range of time-scale adjustments. Furthermore, the conversion subcircuit performs a nonlinear conversion on the voltage generated by the shared charge to simulate the signal transmission process of the neuron. By implementing the nonlinear conversion at the hardware level, the need for an external processor is reduced, resulting in greater adaptability and a wider range of applications. This solves technical problems in related technologies, such as limitations on the regulation and application scope of the neuron time constant, and high hardware resource consumption and costs.

[0018] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of a framework of a plastic neuron circuit provided according to an embodiment of the present application;

[0021] Figure 2 A specific circuit diagram of a plastic neuron circuit provided according to an embodiment of the present application;

[0022] Figure 3A schematic diagram of implementing a ReLU (Rectified Linear Unit) activation function in a common-drain amplifier circuit according to an embodiment of the present application;

[0023] Figure 4 A flowchart of a method for calculating a neuron time constant according to an embodiment of the present application;

[0024] Figure 5 A flowchart of a time constant training algorithm combining software and hardware according to an embodiment of the present application;

[0025] Figure 6 Schematic diagram of a rotating neuron reservoir system provided according to an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0028] In the era of rapid AI development, artificial neural networks, as its fundamental model, are constantly being updated and iterated, and are constantly expanding towards ever-larger scales. General-purpose hardware systems are gradually becoming insufficient to keep up with the demands of ultra-large-scale neural networks, and the resulting surge in cost and power consumption is non-negligible. This has led to the emergence of a number of neural network accelerator chips dedicated to AI. These hardware systems are specialized for specific neural networks, integrate a large number of neurons, and enable high-speed computing. Trends indicate that the ultra-high-speed development of AI will require ever-more efficient neural network hardware systems to support it, even aiming to approach the ultra-large scale and ultra-low power consumption of biological neural networks while maintaining high computing power.

[0029] Current research on artificial neural networks is based on biological neural networks, learning from their unique mechanisms at the top level to achieve specific applications. However, current large-scale neural network hardware systems generally neglect the implementation of neurons, the fundamental building blocks of biological neural networks. Neurons in living organisms are individually plastic and are implemented in the analog domain, enabling the neural system to adapt to varying time constants in complex environments. The diverse interactions between neurons, across different brain regions, and at diverse timescales are crucial for cognitive functions, enabling the neural system to process a wide range of stimuli, from rapid reflexes to long-term memory recall. However, current hardware implementations mostly utilize unified neurons for integration, and most are digital. This presents challenges in implementing analog neuromorphic hardware and realizing neuronal plasticity. Common hardware solutions often require significant hardware resources, such as large capacitor storage requiring significant area overhead; digital storage requiring constant state updates, which creates trade-offs between power consumption and high accuracy and low cost; and the poor consistency and tunability of new devices. This leads to higher power consumption and area overhead, while also hindering wide timescale adjustment, limiting their application in diverse tasks and large-scale integration.

[0030] In the most common digital technology solutions, achieving hardware system adjustability requires external circuit configuration. This requires the initial configuration of storage units with sufficient capacity and state update circuits with sufficient speed to enable adjustment. For capacitor and resistor circuits and new devices, achieving adjustability requires the addition of numerous redundant units and the need to fully consider the matching of time constants in the application domain, resulting in significant overhead. Furthermore, there is a lack of relevant optimization algorithms for specific hardware configurations, and models for solutions other than digital implementations are difficult to construct, making the optimization process even more difficult.

[0031] Existing hardware neurons have difficulty achieving time constant plasticity, and many methods require a large amount of hardware resources to support complex time constant adjustments. For example, the traditional method of using digital circuits to implement state storage requires a large number of storage units and frequent state updates. When time constant adjustment is required, it is necessary to add configured peripheral circuits, which increases the power consumption and cost of the system. The method of using capacitors to implement analog state storage and decay is also relatively common, but different time constants are usually considered for different applications, and the area overhead of the capacitors used is too large and difficult to adjust. There are also many studies that use new materials and new devices to simulate the characteristics of biological neurons, but their integration capabilities are limited, and the consistency between neurons, the adjustable range of neurons, and the scope of application are very limited.

[0032] To this end, in response to the problems of high cost, low integration, poor adjustability, and high power consumption of neuron implementation solutions in related technologies, an embodiment of the present application provides a plastic neuron circuit, which stores input voltage through a storage subcircuit, and periodically switches multiple capacitors in a shared subcircuit to share charge with the storage subcircuit. The time constant of the target neuron is shaped by the shared charge. There is no need to use complex hardware resources to achieve time regulation of neurons. The time constant of the neuron can be shaped only by sharing the charge, which reduces the hardware resource requirements and can achieve a wide range of time size adjustment. The voltage generated by the shared charge is nonlinearly converted by the conversion subcircuit to simulate the signal transmission process of the neuron. By realizing nonlinear conversion at the hardware level, the demand for external processors is reduced, and the adaptability is stronger and the application range is wider.

[0033] Specifically, Figure 1 A schematic diagram of a plastic neuron circuit provided in an embodiment of the present application.

[0034] like Figure 1 As shown, the plastic neuron circuit 10 includes a storage subcircuit 11 , a sharing subcircuit 12 and a conversion subcircuit 13 .

[0035] Among them, the storage subcircuit 11 is used to store the input voltage; the sharing subcircuit 12 includes multiple capacitors, and the multiple capacitors switch the capacitors that share charges with the storage subcircuit 11 in each cycle, so as to shape the time constant of the target neuron by sharing the charges; the conversion subcircuit 13 is used to perform nonlinear conversion on the voltage generated by the shared charges, and store the nonlinearly converted voltage in the capacitors of the sharing subcircuit 12, wherein in each cycle, the capacitors that share charges among the multiple capacitors are different from the capacitors that store the nonlinearly converted voltage.

[0036] It can be understood that the embodiment of the present application constructs a plastic neuron circuit 10, including a storage subcircuit 11, a sharing subcircuit 12 and a conversion subcircuit 13. The input voltage is stored by the storage subcircuit 11, and the capacitors sharing the charge with the storage subcircuit 11 are periodically switched by multiple capacitors in the sharing subcircuit 11. The time constant of the target neuron is shaped by the shared charge. There is no need to use complex hardware resources to achieve time regulation of the neuron. The time constant of the neuron can be shaped only by sharing the charge, which reduces the hardware resource requirements and can achieve a wide range of time size adjustment. The voltage generated by the shared charge is nonlinearly converted by the conversion subcircuit 13 to simulate the signal transmission process of the neuron. By realizing nonlinear conversion at the hardware level, the demand for external processors is reduced, and the adaptability is stronger and the application range is wider.

[0037] The conversion subcircuit 13 of the embodiment of the present application can also be called a nonlinear activation function circuit, such as a common-drain amplifier circuit, which can better simulate the signal transmission process of neurons and reduce the demand for external processors by realizing nonlinear conversion at the hardware level.

[0038] In the embodiment of the present application, the storage sub-circuit 11 includes a first switch and a first capacitor.

[0039] The first switch is represented by W1 , the first capacitor is represented by C1 , and the first capacitor C1 is used to store the input voltage after the first switch W1 is closed.

[0040] In the embodiment of the present application, the shared sub-circuit 12 includes multiple groups of second switches and third switches.

[0041] The second switch is represented by W3 and the third switch is represented by W4.

[0042] In an embodiment of the present application, the shared subcircuit 12 includes multiple capacitors, each of which is connected to a set of second switches and third switches. The capacitor that shares charge with the first capacitor is determined by closing and / or opening the second switch and the third switch.

[0043] For example, the shared sub-circuit 12 includes a second capacitor C2 and a third capacitor C3. The second capacitor C2 is connected to a set of second switches W3 and third switches W4. The third capacitor C3 is connected to a set of second switches W3 and third switches W4. The capacitor that shares charge with the first capacitor C1 is determined by closing and opening the second switch W3 and the third switch W4.

[0044] For example, when the second switch W3 is closed and the third switch W4 is open, the capacitor that shares charge with the first capacitor C1 is the third capacitor C3. Conversely, when the second switch W3 is open and the third switch W4 is closed, the capacitor that shares charge with the first capacitor C1 is the second capacitor C2.

[0045] In the embodiment of the present application, a fourth switch W2 is provided between the storage sub-circuit 11 and the sharing sub-circuit 12 for controlling conduction or disconnection between the storage sub-circuit 11 and the sharing sub-circuit 12 .

[0046] In an embodiment of the present application, the first switch W1, the second switch W3, the third switch W4 and the fourth switch W2 are closed and / or opened a target number of times in each cycle, so that multiple capacitors switch to capacitors that share charges with the storage subcircuit 11 in each cycle, thereby shaping the time constant of the target neuron by sharing the charge.

[0047] It can be understood that the first to fourth switches in the embodiment of the present application are closed and opened according to the target number of times in each cycle, so that multiple capacitors can switch to capacitors sharing charges with the storage sub-circuit 11 in each cycle. That is to say, the number of switches of the control switch group in one cycle can adjust the number of times the shared charges are shared, and the shared charges can realize the control of the neuron decay speed, that is, realize the plasticity of the neuron time constant.

[0048] The following describes the plastic neuron circuit constructed in the embodiment of the present application through a specific embodiment. Figure 2 As shown, the weighted multiplication and addition calculation process is realized through the charge sharing principle of the switched capacitor circuit, and the time constant plasticity is achieved on this basis.

[0049] exist Figure 2 The circuit shown includes three capacitors, C1, C2, and C3. C1 stores the input voltage state. C2 and C3 are equivalent, each performing two functions: sharing charge with C1 and storing the output voltage after passing through the nonlinear function circuit. C2 and C3 maintain different operating states during each cycle, switching between them once per cycle. Figure 2 There is a nonlinear activation function circuit between C2 and C3. The common drain amplifier circuit is used here. The input and output voltage curves of the circuit can be approximated as a positive shift of V th (threshold voltage) ReLU function, Figure 3 Transfer relation for the approximate ReLU function implemented for this circuit.

[0050] It should be noted that the neuron setting of an artificial neural network usually includes multiplication and addition operations and nonlinear operations. The embodiment of the present application introduces nonlinearity in the calculation process in order to simulate the calculation process of biological neurons. There are many algorithms for implementing nonlinear activation functions, including tanh, ReLU, etc. The input-output relationship of the function can be nonlinear. Different nonlinear activation functions need to be implemented through different circuit designs and have different applicable application scenarios. For example, ReLU is suitable for classification tasks, tanh is suitable for prediction tasks, etc. The common drain amplifier circuit used in the embodiment of the present application to implement the ReLU function has a relatively simple circuit and can basically meet the application requirements.

[0051] Based on this circuit, one weight multiplication and addition calculation can be realized through the process of charge sharing, and multiple multiplication and addition calculations can be realized through multiple charge sharing, which is equivalent to changing the time constant of the neuron and realizing the ReLU function through the common-drain amplifier circuit.

[0052] For example, Figure 2As shown, in the first cycle, the initial state of the switch group is W1, W2, and W3 open, and W4 closed. The capacitance ratio of C1, C2, and C3 is set to 1:9:9. The initial voltage on C2 is U2, and the voltage on C1 is ReLU(U2). The operation process is to first close W1, inject the input voltage into C1, recorded as U1, and after C1 stores the voltage value, W1 is opened and then W2 is closed. Since W2 and W4 are both on, C2 and C1 share charge. After charge sharing is completed, W2 is opened. At this time, after one charge sharing, the voltages on C1 and C2 are equal. According to the charge sharing mechanism, the voltage is 0.1×U1+0.9×U2. The voltage on C3 is ReLU(0.1×U1+0.9×U2). In this process, a charge sharing process is completed.

[0053] Repeating the switch group operation of W1, W2, W3, and W4 twice in the same cycle, the voltage on C1 and C2 is equal, and equal to 0.1×U1+0.9×(0.1×U1+0.9×U2)=(1-0.9 2 )×U1+0.9 2 × U2, after n switching operations, the voltage between C1 and C2 can be (1-0.9 n )×U1+0.9 n ×U2, at this time the voltage on C3 is ReLU[(1-0.9 n )×U1+0.9 n × U2]. It can be seen that the more times the switch is shared, the more the stored states of C2 and C3 are affected by the input. Assuming U1 = 0 and U2 = 3, after one share, the voltage on U2 is equal to 2.7. After multiple shares, the voltage on U2 gradually approaches 0. Therefore, it can be roughly assumed that the neuron decay rate can be controlled by controlling the number of times the switch group switches in a cycle, that is, achieving plasticity of the time constant.

[0054] It should be noted that in the field of integrated circuits, it is possible to achieve precise proportions of capacitance, as well as sufficiently low leakage and low on-resistance of the switch group. Therefore, it can be basically assumed that hardware neurons can maintain high efficiency and stability during long-term operation, without significant performance degradation due to long-term use, and meet f[(1-0.9 n )×U1+0.9 n ×U2], the non-ideal factors have little impact.

[0055] In addition, the plastic neuron circuit of the embodiment of the present application can be developed based on CMOS technology, the area overhead of a single neuron is extremely small, large-scale integration can be achieved, and the needs of large-scale neural networks can be met.

[0056] According to the plastic neuron circuit proposed in the embodiment of the present application, it includes a storage subcircuit, a sharing subcircuit and a conversion subcircuit. The input voltage is stored by the storage subcircuit, and the capacitors sharing the charge with the storage subcircuit are periodically switched by multiple capacitors in the sharing subcircuit. The time constant of the target neuron is shaped by the shared charge. There is no need to use complex hardware resources to achieve time regulation of neurons. The shaping of the time constant of the neuron is achieved only by sharing the charge, which reduces the hardware resource requirements and can achieve a wide range of time size adjustment. The voltage generated by the shared charge is nonlinearly converted by the conversion subcircuit to simulate the signal transmission process of the neuron. By realizing nonlinear conversion at the hardware level, the demand for external processors is reduced, and the adaptability is stronger and the application range is wider.

[0057] Next, a method for calculating a neuron time constant according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0058] It should be noted that the embodiments of the present application can map the above-constructed plastic neuron circuit, that is, the working mechanism of the hardware, into software for training, thereby obtaining the time constants that multiple neurons should be configured with so that the neural network can work effectively.

[0059] Figure 4 4 is a flow chart of a method for calculating a neuron time constant according to an embodiment of the present application.

[0060] like Figure 4 As shown, the method for calculating the neuron time constant is applied to the above-mentioned plastic neuron circuit, comprising the following steps:

[0061] In step S101, a target neural network model is built on the target simulation software, wherein the target neural network model includes a plurality of plastic neuron circuits as described above, and each plastic neuron circuit corresponds to a neuron.

[0062] Among them, the target simulation software can be selected according to the specific situation, and there is no specific limitation on this. The target neural network model is a neural network model that needs to perform tasks and includes a network model of multiple neurons, such as a reserve pool model.

[0063] It can be understood that the embodiment of the present application can build a target neural network model on the target simulation software, and the target neural network model includes multiple plastic neuron circuits mentioned above, each plastic neuron circuit corresponds to a neuron, mapping the working mechanism of the hardware to the software.

[0064] In step S102, the expected effect of the target neural network model in processing the target task is obtained.

[0065] In step S103, the number of times the charge is shared in the plastic neuron circuit corresponding to each neuron is adjusted so that the target neural network model achieves the expected effect, and the time constant of each neuron is determined based on the adjusted number of times the charge is shared.

[0066] It can be understood that the embodiments of the present application can adjust the number of shared charges in the plastic neuron circuit corresponding to each neuron in software so that the target neural network model achieves the expected effect, and determine the time constant of each neuron based on the adjusted number of shared charges. By mapping the working mechanism on the hardware to the software for training and adjustment, the optimal time constant can be obtained without affecting the actual hardware performance to meet the needs of specific tasks, and the stability and reliability of the hardware implementation can be guaranteed. In addition, the ability of the target neural network model to adapt to different input signals can be enhanced by adjusting the time constant.

[0067] In an embodiment of the present application, before adjusting the number of times the plastic neuron circuit corresponding to each neuron shares charges, the method includes: obtaining a nonlinear conversion relationship of each plastic neuron circuit; fitting the nonlinear conversion relationship to obtain a nonlinear function, and adjusting the number of times the plastic neuron circuit corresponding to each neuron shares charges based on the nonlinear function.

[0068] It can be understood that the embodiment of the present application tests the nonlinear function used in the hardware (i.e., the plastic neuron circuit) to obtain a curve, and fits the curve in the software to obtain the nonlinear function, and adjusts the number of times the charge is shared in the plastic neuron circuit corresponding to each neuron based on the nonlinear function.

[0069] Specifically, the input voltage and output voltage of the converter circuit in the above plastic neuron circuit are fitted to obtain a nonlinear function. For example, the nonlinear function of a neuron is the above f[(1-0.9 n )×U1+0.9 n ×U2], that is, the working mechanism or calculation process of the neuron is f[(1-0.9 n )×U1+0.9 n ×U2].

[0070] In an embodiment of the present application, adjusting the number of times that charges are shared in the plastic neuron circuit corresponding to each neuron includes calculating the number of times that charges are shared in the plastic neuron circuit corresponding to each neuron according to a nonlinear function of each neuron and a covariance evolution algorithm.

[0071] It can be understood that the embodiments of the present application can calculate the number of shared charges in the plastic neuron circuit corresponding to each neuron based on the nonlinear function of each neuron and use the covariance evolution algorithm. By combining software and hardware for joint training, it is ensured that the hardware working mechanism can be accurately mapped to the software model, and the covariance evolution algorithm is used to optimize the number of shared charges of each neuron, which can improve the overall performance of the neural network and ensure the stability and reliability of the hardware implementation.

[0072] In an embodiment of the present application, after the time constant of each neuron is determined, it is mapped to hardware for execution, and the number of switching times of the switch group of each neuron is set in each cycle, thereby changing the equivalent time constant of the neuron.

[0073] In the embodiment of the present application, before mapping the working mechanism of the plastic neuron circuit to the software, the ratio of the capacitors in the plastic neuron circuit can also be adjusted. The ratios of C1:C2 and C1:C3 should be as small as possible.

[0074] Specifically, the embodiment of the present application trains the neuron time constant by combining software and hardware, and the process is as follows: Figure 5 shown.

[0075] First, the nonlinear function used in the hardware is tested to obtain the curve, which is then fitted in the software. Then the covariance evolution algorithm is introduced for training. The neural network architecture is built in the software through simulation software. The f[(1-0.9 n )×U1+0.9 n ×U2] as the calculation process for neuron operation, where f is the expression of the fitted nonlinear function. The covariance evolution algorithm is then updated by setting the neural network's performance indicator for a specific task as the basis for adjusting the value of n in each neuron to optimize the neural network's performance. The n value for each neuron calculated by the software is then mapped to the hardware. The specific mapping method is to set the number of on-off cycles of each neuron's switching group in each cycle, thereby changing the neuron's equivalent time constant.

[0076] Among them, n is the number of switches in a cycle, that is, the number of charge sharing in a cycle, and also the number of switches of the switch group in a cycle. The size of n directly determines the equivalent time constant of the neuron. If n is relatively large, then the equivalent time constant is small, the number of charge sharing times is equal to the number of switches of the switch group, and the direction of change of the time constant is opposite to the previous number of switches.

[0077] It should be noted that the aforementioned explanation of the embodiment of the plastic neuron circuit is also applicable to the calculation method of the neuron time constant of this embodiment, and will not be repeated here.

[0078] According to the calculation method of neuron time constant proposed in the embodiment of the present application, the working mechanism of the hardware's plastic neuron circuit can be mapped to the software, and the working mechanism of each neuron can be trained based on the expected effect of the target neural network model to obtain the optimal time constant. By combining software and hardware for joint training, it is ensured that the hardware working mechanism can be accurately mapped to the software model, and the time constant of each neuron is optimized by the covariance evolution algorithm, which can improve the overall performance of the neural network and ensure the stability and reliability of the hardware implementation.

[0079] The following describes the application of the plastic neurons in the reservoir network according to the embodiment of the present application through a specific embodiment.

[0080] In the reserve pool neural network, there is a calculation process similar to f[ax+bs], where f is the activation function, a and b are coefficients, x is the input voltage of the plastic neuron circuit (i.e., neuron), and s is the storage voltage state of the plastic neuron circuit (i.e., neuron) at the previous moment, which is consistent with the proposed neuron calculation process. Therefore, the neurons proposed in this application can be used in the reserve pool hardware network.

[0081] The rotating neuron reservoir system in the reservoir field is used as a specific embodiment, such as Figure 6 The figure shows a rotating neuron reservoir system. In this system, there are three layers: input layer, reservoir layer and output layer. The input layer upgrades the input dimension through the mask vector. The reservoir layer realizes neuron connection through the front and rear rotors, and realizes linear and nonlinear transformation calculation process through neurons. The output layer collects the output of the reservoir layer and obtains the final result through full connection calculation.

[0082] The NARMA2 task, a second-order nonlinear dynamics task, was used as a test task for the reservoir system. The training process involves providing the input sequence from the task to the reservoir network. The reservoir layer calculates a matrix of state vectors. The weight distribution of the reservoir system's output layer is calculated using the least squares method, using the known expected output from the task and this state matrix. During inference, the input sequence is fed to the reservoir network to obtain a state matrix. The output is calculated using the known output layer weights. The error between this output and the expected output is compared to determine the network's performance in achieving the task. During training, a covariance evolution algorithm is introduced. While training the output layer, the output error is used as a training guide. Simultaneously, the n value of each neuron is trained, resulting in a reservoir network with improved optimization performance. During inference, the reservoir system outperforms networks composed of non-plastic neurons.

[0083] In the specific application of hardware neural network and environment interaction, it is necessary to determine the range of neuron time constant according to the actual time constant of the application. The time constant of the neuron is mainly related to the length of the calculation cycle, the number of switching groups n, and the ratio of capacitors C1 to C2 and C3. Therefore, before training, it is necessary to determine how long the neural network needs to complete a calculation in the slowest case under the specific application, so as to determine the size of the calculation cycle, C 1: C2 and the ratio of C1:C3 should be as small as possible, so that more time constant values can be achieved through the number of switching groups. On this basis, the n value of each neuron is obtained through a combination of software and hardware algorithms.

[0084] In summary, the neurons of the embodiments of the present application are close to the characteristics of biological neurons in terms of plasticity and threshold triggering. (1) The wide range of plasticity of time constants achieved by neurons can meet the diverse application requirements from fast signal processing to real-time environment interaction; (2) The reserve pool computing network combined with the software and hardware joint training method enriches the internal state, thereby improving task performance; (3) In addition, neurons store analog states and realize storage and calculation at the same time, eliminating the storage and transmission overhead in the digital field; (4) The implementation of neurons can be developed based on CMOS technology, and the area overhead of single neurons is extremely small, which can achieve large-scale integration and meet the needs of large-scale neural networks.

[0085] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0086] Memory 701 , processor 702 , and computer programs stored in the memory 701 and executable on the processor 702 .

[0087] When the processor 702 executes the program, the method for calculating the neuron time constant provided in the above embodiment is implemented.

[0088] Furthermore, the electronic device further includes:

[0089] The communication interface 703 is used for communication between the memory 701 and the processor 702 .

[0090] The memory 701 is used to store computer programs that can be run on the processor 702 .

[0091] The memory 701 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0092] If the memory 701, processor 702, and communication interface 703 are implemented independently, the communication interface 703, memory 701, and processor 702 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0093] Optionally, in a specific implementation, if the memory 701, the processor 702 and the communication interface 703 are integrated on a chip, the memory 701, the processor 702 and the communication interface 703 can communicate with each other through an internal interface.

[0094] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0095] An embodiment of the present application further provides a computer-readable storage medium having a computer program or instruction stored thereon. When the computer program or instruction is executed by a processor, the method for calculating the neuron time constant as described above is implemented.

[0096] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0097] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0098] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing a custom logical function or process step, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed in a different order than shown or discussed, including performing functions in a substantially simultaneous manner or in a reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0099] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, it can be implemented using any one or a combination of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0100] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

Claims

1. A plastic neuron circuit, characterized in that: include: a storage subcircuit, for storing an input voltage; a sharing subcircuit, wherein the sharing subcircuit comprises a plurality of capacitors, wherein the plurality of capacitors switches the capacitors that share charge with the storage subcircuit in each cycle, thereby shaping the time constant of the target neuron by sharing the charge; The conversion subcircuit is configured to perform nonlinear conversion on a voltage generated by the shared charge and store the nonlinearly converted voltage in a capacitor of the shared subcircuit, wherein in each cycle, the capacitor for sharing the charge is different from the capacitor for storing the nonlinearly converted voltage among the plurality of capacitors.

2. The plastic neuron circuit according to claim 1, wherein: The storage subcircuit includes a first switch and a first capacitor, wherein the first capacitor is used to store an input voltage after the first switch is closed.

3. The plastic neuron circuit according to claim 2, wherein: The shared sub-circuit also includes multiple groups of second switches and third switches. Each capacitor in the shared sub-circuit is connected to a group of second switches and third switches. The capacitor that shares charge with the first capacitor is determined by closing and / or opening the second switch and the third switch.

4. The plastic neuron circuit according to claim 3, characterized in that A fourth switch is provided between the storage sub-circuit and the sharing sub-circuit, and is used to control the conduction or disconnection of the storage sub-circuit and the sharing sub-circuit.

5. The plastic neuron circuit according to claim 4, characterized in that: The first switch, the second switch, the third switch, and the fourth switch are closed and / or opened a target number of times in each cycle, so that the multiple capacitors switch to the capacitor sharing charge with the storage subcircuit in each cycle, thereby shaping the time constant of the target neuron by sharing the charge.

6. A method for calculating a neuron time constant, characterized in that: The method is applied to the plastic neuron circuit according to any one of claims 1 to 5, comprising the following steps: Building a target neural network model on a target simulation software, wherein the target neural network model includes a plurality of plastic neuron circuits according to any one of claims 1 to 5, each plastic neuron circuit corresponding to a neuron; Obtaining the expected effect of the target neural network model on processing the target task; The number of times that charges are shared in the plastic neuron circuit corresponding to each neuron is adjusted so that the target neural network model achieves the expected effect, and the time constant of each neuron is determined based on the adjusted number of times that charges are shared.

7. The method for calculating the neuron time constant according to claim 6, characterized in that: Before adjusting the number of times each neuron shares charge in the plastic neuronal circuit, including: Obtaining a nonlinear conversion relationship of each plastic neuron circuit; The nonlinear conversion relationship is fitted to obtain a nonlinear function, and the number of times the charge is shared in the plastic neuron circuit corresponding to each neuron is adjusted based on the nonlinear function.

8. The method for calculating the neuron time constant according to claim 7, characterized in that: The adjusting the number of times of shared charges in the plastic neuron circuit corresponding to each neuron comprises: The number of times the charge is shared in the plastic neuron circuit corresponding to each neuron is calculated according to the nonlinear function of each neuron and the covariance evolution algorithm.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for calculating the neuron time constant according to any one of claims 6 to 8.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instruction is executed by a processor to implement the method for calculating the neuron time constant according to any one of claims 6 to 8.

Images