Dynamic synaptic brain-like calculation training method and application

Abstract

The invention discloses a dynamic synaptic brain-like calculation training method and application. The method comprises the following steps: selecting a data set of handwritten number recognition of a classical task MNIST; the control of neuromodulation represented by dopamine on the movement of a transmitter receptor is regarded as a nonlinear dynamic process, and a dynamic synaptic model is constructed; establishing three networks, namely a multilayer artificial neural network, a multilayer spiking neural network and a multilayer brain-like network formed by artificial neurons and spiking neurons; and applying the dynamic highlighting model to the three networks, performing corresponding task training in combination with the data set, and updating network weight parameters to obtain a trained task model. By using the method, the existing model training method can accommodate the artificial neural network and various computational neurology models, so that the classification or recognition task can be completed more accurately. The method can be widely applied to the field of network training.

Description

Technical Field

[0001] The present invention relates to the field of network training, and in particular to a dynamic synaptic brain-like computing training method and application. Background Art

[0002] Artificial neural networks (ANNs) originated from limited observations, understanding, and conjectures of biological neural circuits in the mid-20th century. However, due to computing power limitations, ANNs have undergone significant simplifications in their application, such as treating neurons as static and replacing neural impulses with nonlinear, differentiable functions.

[0003] Compared to artificial neural networks, computational neuroscience focuses more on reproducing the characteristics of neural circuits themselves. Existing computational neuroscience or brain-inspired networks based on synaptic plasticity typically rely on the spiking discharges of neurons. These methods are ineffective in the absence of spiking discharges. Furthermore, even for neural network models with spiking discharges, their reliance on gradient descent has limitations in terms of training effectiveness. This means that existing training methods struggle to effectively achieve optimal results for brain-inspired models.

[0004] In classic image classification tasks, computational neuroscience or brain-like networks cannot achieve accurate classification and recognition due to the limitations of training effects. Summary of the Invention

[0005] In view of this, in order to solve the technical problem that existing model training methods rely on gradient descent and are incompatible with artificial neural networks and various computational neuroscience models, which in turn leads to the inability to accurately complete the classification tasks of the corresponding networks, the present invention proposes a dynamic synaptic brain-like computing training method, which includes the following steps:

[0006] Use the classic task MNIST handwritten digit recognition dataset;

[0007] The control of neuromodulators represented by dopamine on the movement of transmitter receptors is regarded as a nonlinear dynamic process, and a dynamic synaptic model is constructed;

[0008] Build three types of networks: a multi-layer artificial neural network, a multi-layer spiking neural network, and a multi-layer brain-like network consisting of artificial neurons and spiking neurons;

[0009] The dynamic synaptic model is applied to the above three networks, and the corresponding task training is performed in combination with the data set. The network weight parameters are updated to obtain the trained task model.

[0010] Based on the above scheme, the present invention provides a dynamic synaptic brain-like computing training method and application. By drawing on the mechanism of synaptic plasticity in animal neural circuits, an optimization algorithm suitable for brain-like computing is constructed, which makes it biologically reasonable and compatible with artificial neural networks and various computational neurological models, so that it can complete classification or recognition tasks more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flowchart of the steps of a dynamic synaptic brain-like computing training method of the present invention;

[0012] Figure 2 It is a structural diagram of param_groups in a specific embodiment of the present invention;

[0013] Figure 3 It is a structural diagram of a specific embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of the reward changes at each step during the multi-layer artificial neural network training process according to a specific embodiment of the present invention;

[0015] Figure 5 This is a schematic diagram of the change of loss at each step during the training of a multi-layer artificial neural network according to a specific embodiment of the present invention;

[0016] Figure 6 This is a schematic diagram of the accuracy of each epoch on the MNIST test set during the training of a multi-layer artificial neural network according to a specific embodiment of the present invention;

[0017] Figure 7 Schematic diagram of the exploration-convergence process of a certain weight during the training of a multi-layer artificial neural network according to a specific embodiment of the present invention;

[0018] Figure 8 This is a schematic diagram of the accuracy of each epoch of a multi-layer artificial neural network trained using the stochastic gradient descent method on a test set in a specific embodiment of the present invention;

[0019] Figure 9 This is a schematic diagram of the reward changes at each step during the multi-layer pulse network training process according to a specific embodiment of the present invention;

[0020] Figure 10 This is a schematic diagram of the change in loss at each step during the multi-layer spike network training process according to a specific embodiment of the present invention;

[0021] Figure 11 This is a schematic diagram of the accuracy of each epoch on the MNIST test set during the multi-layer spike network training process according to a specific embodiment of the present invention;

[0022] Figure 12Schematic diagram of the exploration-convergence process of a certain weight during the multi-layer spike network training process according to a specific embodiment of the present invention;

[0023] Figure 13 Schematic diagram of the change of β(τ) inside a LIF neuron in a multi-layer spiking network according to a specific embodiment of the present invention;

[0024] Figure 14 This is a schematic diagram of the accuracy of a multi-layer spiking network per epoch optimized using the stochastic gradient descent method in a specific embodiment of the present invention. The yellow line shows the accuracy of the training set, and the blue line shows the accuracy of the test set.

[0025] Figure 15 Schematic diagram of the structure of a brain-like network composed of an artificial neural network and a spiking neural network connected in series according to a specific embodiment of the present invention;

[0026] Figure 16 This is a schematic diagram of the reward changes at each step during the brain-like network training process according to a specific embodiment of the present invention;

[0027] Figure 17 This is a schematic diagram of the change in loss at each step during the brain-like network training process according to a specific embodiment of the present invention;

[0028] Figure 18 This is a schematic diagram of the accuracy of each epoch on the MNIST test set during the brain-like network training process of a specific embodiment of the present invention;

[0029] Figure 19 This is a schematic diagram of the exploration-convergence process of a certain weight during the brain-like network training process in a specific embodiment of the present invention;

[0030] Figure 20 This is a schematic diagram of the changes in β(τ) inside a certain LIF neuron during the brain-like network training process in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0032] It should be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0033] It should be understood that the terms "system," "device," "unit," and / or "module" used in this application are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0034] As used in this application and the claims, unless the context clearly indicates an exception, the terms "a," "an," "an," and / or "the" are not intended to refer to the singular and may include the plural, unless the context clearly indicates otherwise. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements. The phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, product, or apparatus that includes the elements.

[0035] In the description of the embodiments of this application, "plurality" refers to two or more than two. The terms "first" and "second" below are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0036] In addition, flow charts are used in this application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0037] Reference Figure 1 , which is a flow chart of an optional example of the dynamic synaptic brain-like computing training method proposed in the present invention. This method can be applied to computer devices. The training method proposed in this embodiment may include but is not limited to the following steps:

[0038] Step S1, obtaining an image dataset;

[0039] Step S2: Considering the control of neuromodulators represented by dopamine on the movement of transmitter receptors as a nonlinear dynamic process, a dynamic synaptic model is constructed;

[0040] Step S3: Using the dynamic synaptic model as a network optimizer, the preset network is trained in combination with the image data set to obtain a trained classification network.

[0041] In some feasible embodiments, the dynamic synapse model specifically includes:

[0042] Biological principles of synaptic plasticity:

[0043] Real biological neurons rely on synaptic connections to transmit signals, and the strength of these signals is influenced by factors within the synapse, such as modulators and receptors. For example, on the postsynaptic membrane, there are multiple receptor proteins, including the N-methyl-D-aspartate receptor (NMDA receptor) and the α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPA receptor). These receptors migrate between the postsynaptic membrane and other regions through processes such as lateral diffusion, endocytosis, and exocytosis. The presence of these receptors in the postsynaptic membrane corresponds to an increase in synaptic strength, while their presence in other regions indicates a decrease in synaptic strength. At the same time, neuromodulators represented by dopamine affect the motor properties of these receptor proteins through biochemical cascade reactions such as Ca2+ / calmodulin dependent protein kinase II (CaMKII), cyclic adenosine monophosphate (cAMP) and Protein Kinase A (PKA).

[0044] This complex biochemical cascade results in instability and spontaneous fluctuations in synaptic strength in real organisms, even in the absence of learning (Choquet & Triller, 2013). These fluctuations are primarily caused by the spontaneous movement of transmitter receptors on the postsynaptic membrane and the constant changes in synaptic size. As mentioned above, the former is caused by receptor drift on the cell membrane or endocytosis / efflux, while the latter indirectly affects synaptic strength by affecting the number of transmitter receptors available for synaptic accommodation. Both lead to fluctuations in the number of postsynaptic receptors receiving neurotransmitter signals, thereby affecting synaptic strength. This process is more pronounced in newborn neurons or dendritic spines than in mature neurons or synapses.

[0045] Taking into account the computational cost and implementation complexity, the proposed dynamic synaptic model simplifies the actual biological process and regards the control of the movement of transmitter receptors by neuromodulators represented by dopamine as a nonlinear dynamic process. Among them, the influence of neuromodulators on the damping term is a first-order integral, that is, positive rewards correspond to an increase in damping, and negative rewards correspond to a decrease in damping, which leads to an increase and decrease in the exploration range of synaptic strength, respectively. Here it is regarded as a high-order process, so that the weight of the influence of neuromodulators on the damping term is no longer a constant, but a function of the reward statistics. Therefore, the dynamic process of synaptic strength can be divided into two main parts: 1. The fluctuation center C of the number of receptors; 2. The fluctuation amplitude A of the number of receptors. Use a sine function to approximate the fluctuation process. For a certain synapse, its synaptic strength changes with time t:

[0046]

[0047] Where A is the amplitude of the fluctuation, C is the center of the fluctuation, and T i is the ith period of the fluctuation. Period T i It is not fixed. When the instantaneous synaptic strength W passes through the fluctuation center at a certain angle, a new cycle will be generated from a Gaussian distribution:

[0048]

[0049] where μ is the mean of the distribution and σ 2 is the variance.

[0050] This fluctuation approximates the chaotic phenomenon of synaptic strength. Related theoretical simulations have demonstrated that this change can enable the network to search within the parameter space defined by synaptic strength and, under the influence of neuromodulators, shift toward reward-rich regions, thereby optimizing various artificial neural networks or computational neuroscience models (Wei & Webb, 2018b, 2018a). The core mechanism can be simplified as follows: for a synapse:

[0051]

[0052] in is the rate of change of the synaptic strength fluctuation center, α is the learning rate of the strength fluctuation center, R can represent the reward or punishment generated after the model interacts with the outside world, and β is the convergence rate of the strength fluctuation amplitude.

[0053] As can be seen, during the process of synaptic fluctuations, if the model's behavior generates a reward (or punishment), the center of the synaptic strength fluctuation will shift toward (or away from) the current strength that caused the reward, while the amplitude of the fluctuation decreases (or expands). As a result, the synaptic strength will gradually approach the area where higher rewards can be obtained. Furthermore, the chaotic nature of the fluctuations ensures the traversal and unpredictability of the network's search through the entire parameter space, enabling it to find the global optimal solution and ultimately converge the synaptic strength under the continuous regulation of rewards.

[0054] It is not difficult to find that the changes in the center and amplitude of synaptic fluctuations are very dependent on the current reward R(t), that is, the synapse will not consider the trend of increase or decrease in the overall reward over a longer period of time, it is only regulated by the reward caused by the current behavior. In specific tasks, fluctuations in synaptic strength may cause fluctuations in the overall reward of the network, thereby affecting the convergence of the synapse in the later stage. Therefore, a dynamic mechanism for promoting and inhibiting the exploration of spontaneous synaptic fluctuations is added. It is manifested in the addition of a long-term statistic α about the reward to the change in amplitude A, which is used to consider the overall performance of the task process model over a period of time, rather than being limited to short-term or instantaneous good or bad behavior. The specific strategy is as follows:

[0055] ① During the period when the overall reward is on the rise, dynamic synapses explore as usual;

[0056] ② During the period when the overall reward does not change much (the change is close to 0), it means that the network may be stuck in a local optimal solution and needs to strengthen exploration;

[0057] ③ During periods when overall rewards tend to decline, dynamic synapses weaken exploration.

[0058] The update of the final strategy amplitude A can be expressed as:

[0059]

[0060] In this strategy, A t The derivation of the differential form is described later. In actual tasks, α is designed to be a nonlinear function of reward, such as the difference between short-term and long-term rewards. The specific values of c, α0, and α1 need to be determined based on the task and are provided here for illustrative purposes only.

[0061] In some feasible embodiments, step S3 specifically includes:

[0062] The dynamic synaptic model will be implemented using PyTorch, a mature GPU parallel platform, as a network optimizer. Its invocation method is similar to other official PyTorch optimizers (such as Adam). Leveraging PyTorch's support for CUDA, the model (i.e., optimizer) will support preliminary optimization of network parameters of various sizes and types (including neuron models with complex dynamics), and will utilize GPUs to accelerate weight training in large-scale networks.

[0063] Class definition:

[0064] The dynamic synapse model is defined as the DynamicSynapse class, which inherits from the Optimizer class in torch.optim.optimizer and contains:

[0065] ① Initialization function __init__(). Initializes training parameters such as learning rate and step size, and gives default values for parameters that are not passed in but required by the model;

[0066] ② Optimizer state initialization function _init_states(). The dynamic synaptic model itself is a dynamic system, as shown in Equation (1), with dynamic characteristics. It needs to store system state parameters for normal iteration. The specific details of key parameters will be introduced later;

[0067] ③ Step function step(). After obtaining rewards or penalties, the dynamic synaptic model will optimize the synaptic strength that needs to be trained, that is, the weight. The specific process is described in formulas (3) and (4).

[0068] Status key parameters:

[0069] Period: The fluctuation period of the network connection weight, i.e., the synaptic strength, corresponds to T in formula (1) i This parameter usually needs to be set specifically according to the task characteristics or optimization requirements;

[0070] period_centre: The average period of the fluctuation of the connection weight, i.e., the synaptic strength. Because the period of each synaptic fluctuation is not fixed, but satisfies Equation (2), it corresponds to μ. Note that when initializing this parameter, the passed period is used as the initial value of period_centre.

[0071] period_var: variance σ in control equation (2) 2 The period of weight fluctuation is controlled by formula (2). This parameter is multiplied by period_centre to obtain the standard deviation σ in formula (2). If not specifically set, the default value is 0.1.

[0072] t_in_period: The moment when the weight fluctuation is currently in the cycle, which will be reset to 0 with the cycle update. It is essentially the accumulation of time steps dt, corresponding to formula (1)

[0073] time: The weight has been trained by the dynamic synapse for a long time, which is not reset with the cycle update. It is essentially the accumulation of time steps dt, corresponding to t in formula (1);

[0074] weight: current weight, i.e. current synaptic strength, corresponding to W in formula (1);

[0075] weight_centre: weight fluctuation center, corresponding to C in equations (1) and (3);

[0076] weight_centre_update_rate: update rate of weight fluctuation center, corresponding to α in formula (3);

[0077] amp: weight fluctuation amplitude, corresponding to A in formula (1), usually needs to be set specifically according to the characteristics of the task;

[0078] weight_oscilate_decay: the convergence rate of the weight fluctuation amplitude, corresponding to β in equation (4);

[0079] zero_cross: marks whether the weight has gone through a complete cycle, which is used to update the weight fluctuation cycle.

[0080] Call implementation:

[0081] Compared with general neural networks implemented based on PyTorch, the dynamic synaptic model itself does not rely on gradient descent at all, so there is no need for backpropagation steps. The specific construction and operation methods are different from general networks.

[0082] (1) Network structure

[0083] Since backpropagation is not required, the dynamic synaptic model can reduce a large number of derivative operations. The network has the following main differences in the PyTorch-based implementation:

[0084] ① No need for optimizer.zero_grad()

[0085] For all parameters that need to be optimized, you can specify their requires_grad attribute as false to save computation and memory overhead.

[0086] ②No need for loss.backward()

[0087] The dynamic synaptic model does not have a back-propagation process. The calculation of loss or reward can use the loss function in torch.nn or can be customized.

[0088] ③ Need to explicitly pass loss into optimizer.step()

[0089] Unlike optimizers that use the loss.backward() method, dynamic synaptic models do not record gradients. Instead, they require loss or reward to regulate the fluctuation of synaptic strength, that is, weights. Therefore, the loss or reward must be explicitly passed to optimizer.step().

[0090] (2) Operation mode

[0091] The specific operation mode and implementation process of the dynamic synaptic model optimizer are as follows:

[0092] ①Optimizer definition and initialization

[0093] When using PyTorch to define a network structure (inherited from the torch.nn.Module class), specify the optimizer as a DynamicSynapses instance, and the first initialization parameter must be the parameter self.parameters() of the current network structure class;

[0094] Then, based on the specific task requirements, specify the initial learning rate lr, time step dt, period period, scaling factor period_var, time t_in_period, weight fluctuation center update rate weight_centre_update_rate, weight amplitude amp, and weight fluctuation amplitude convergence rate weight_oscilate_decay. Note that there are no period_centre, weight, or weight_centre. This is because the first item is initialized with period, and the second item, weight, is the current synaptic strength of the dynamic synapse, i.e., the current weight of the network. It is generated by weight_centre and does not need to be input externally. The third item, weight_centre, can be initialized using PyTorch properties, and subsequent updates are performed only within the optimizer.

[0095] ②Weight transfer

[0096] When defining a network structure using PyTorch, the network layer weights defined using torch.nn will be automatically added to the Optimizer's list of parameters to be optimized; and custom network layer weights will also be added to the Optimizer's list of parameters to be optimized after being created as torch.nn.Parameter instances.

[0097] Therefore, the initial network weights can be defined or created as torch.nn.Parameter instances using torch.nn. The initialization method of the parent class Optimizer is then used to form a dictionary with the key params and the key value being a list of weights in the order they are passed in. The dictionary is stored in a list called param_groups. The structure of the param_groups is referenced. Figure 2 ,. The list with the key params is the list of parameters to be optimized by the Optimizer mentioned above (hereinafter referred to as params). The network weights are stored here and updated with the training iterations. The structure can be seen in the figure above. Note that the weights passed in for initialization are the initialization values of the weight center weight_centre. In subsequent training, the actual current value of the network weight (W)weight will be generated using (C)weight_centre according to formula (1), and the network weights in params will be updated synchronously based on the value assigned by tensor.data. And weight_centre is updated according to formula (3) as the internal state parameter of the optimizer.

[0098] ③Hierarchical control

[0099] After the network weights are added to params, the optimizer initialization state parameters passed in (1) are already in param_groups due to the parent class Optimizer method, and are also stored in a dictionary. However, only the initial state or some hyperparameters (such as lr, dt, etc.) are passed in. Some dynamic state quantities in the dynamic synaptic model are continuously updated based on iterations (such as weight, weight_centre, t_in_peroid, zero_cross, etc.). Subsequently, based on the specific changes in the model during task training, the optimization of each layer of the network weights is controlled separately. Therefore, it is necessary to record the key parameter states corresponding to each layer of weights. In other words, for each layer of weights, the model's dynamic parameters can be adjusted to regulate it based on its current state.

[0100] Use the state property of the parent class Optimizer to complete the above functions. Figure 3As shown, this attribute is a DefaultDict. Its characteristic is that if the index key name does not exist, the current index key name is created and the given key value is assigned to form a dictionary. The current weight of the weight network is used as the key name, and the key parameters of the dynamic synapse (also in the form of a dict) are used as the key value to form a dict and placed in the state of the parent class Optimizer. Because this index is the address of the weight tensor rather than the value, it will not be modified as the weight is updated. Therefore, the weight of each layer can be bound to the key parameters of the dynamic synapse. In fact, the key parameters stored in the state also include the weight of each layer itself (weight).

[0101] Before the optimizer begins training weights, it initializes the optimizer (i.e., the dynamic synaptic model) state parameters using _init_states_ and places them in the state attribute of the parent class Optimizer. Subsequent steps use the key parameters corresponding to the current layer weights to optimize the weights, thus achieving hierarchical control.

[0102] ④Optimization

[0103] After a time step, the network generates output, which interacts with the environment to generate rewards. Then the network's optimizer, the dynamic synaptic model, receives external losses or rewards R, and then:

[0104] i. Time cumulative update, t_in_period + = dt, time + = dt

[0105] ii. The current weight of the network is updated, and weight_centre generates weight according to formula (1)

[0106] iii. Weight fluctuation center update, generated by the difference between weight_centre and current weight and reward R according to formula (3), specifically implemented as the differential form of formula (3):

[0107] C t+1 =C t +ΔC t ,#7)

[0108] ΔC t =α(W(t)-C t )R t Δt.#8)

[0109] iv. Weight fluctuation amplitude update, generated by rewards according to formula (4). The specific implementation is the difference form of formula (4):

[0110] A t+1 =A t +ΔA t ,#9)

[0111] ΔA t =-βR t Δt.#10)

[0112] Since the left term ΔA of formula (10) t Very small (β is small, resulting in the change of A in one time step is usually around 10 -6 Therefore, the exponential approximation is adopted in the actual difference implementation of formula (4), that is,

[0113]

[0114] Combining equations (9), (10), and (11), we can obtain the final difference approximation:

[0115]

[0116] v. Period update: calculate whether each weight fluctuation has completed a period, use zero_cross to record the weight index that completes the entire period, and generate a new period according to formula (2), where μ = period_centre, σ = period_var*period_centre

[0117] vi. Weight return, use the tensor.data method to return the weight data to params

[0118] At this point, a complete dynamic synapse model optimization step is completed.

[0119] The dynamic synaptic optimization method does not rely on gradient descent and is compatible with artificial neural networks and a variety of computational neuroscience models. Therefore, the present invention selects the classic task MNIST handwritten digit recognition and builds three networks to apply and verify the optimization method: a multi-layer artificial neural network, a multi-layer spiking neural network, and a multi-layer brain-like network composed of artificial neurons and spiking neurons.

[0120] The MNIST (Mixed National Institute of Standards and Technology) dataset consists of 70,000 images, 60,000 of which are training images and 10,000 of which are test images. The training set consists of handwritten digits from 250 different individuals, 50% of whom are high school students and 50% are Census Bureau employees. The test set maintains the same proportion of handwritten digits, ensuring that the author sets for the test and training sets are disjoint.

[0121] The dataset contains 28×28 pixel images of handwritten digits from 0 to 9, with white handwritten digits on a black background. The black background is represented by 0, and the white text is represented by a floating-point number between 0 and 1, with the closer to 1, the whiter the text.

[0122] The following examples use the classic classification task MNIST and compare the optimization results of the two networks with the baseline results optimized using the stochastic gradient descent method.

[0123] Example 1: Multi-layer artificial neural network

[0124] A 784×128×10 multi-layer artificial neural network was built, and the cross entropy between the network output and the correct label was taken as the loss at each step. The loss within a certain period was low-pass filtered and recorded as the long-term loss. avg .pass:

[0125]

[0126] Generate rewards for each step, where loss is calculated avg The iterative formula is defined as:

[0127]

[0128] The training initialization hyperparameters are set to lr = 0.001, dt = 10, period = 1000, amp = 0.05, weight_oscilate_decay = 0.0025, weight_centre_update_rate = 1, and the rest remain default. The above low-pass filter parameter γ = 0.9.

[0129] The experimental results are as follows Figure 4-Figure 8 As shown in the figure, the results show that the multi-layer artificial neural network converged to a good value, achieving an MNIST classification accuracy of 88%. Furthermore, the dynamic synaptic model successfully explored a region with higher rewards, driving the entire model parameters toward that region. It is worth noting that in the later stages of network training, although the model parameters were already at a relatively optimal level, the rewards still fluctuated significantly due to the synaptic exploration mechanism. This volatility slowed the model's convergence rate. However, from another perspective, this volatility effectively prevented the model from falling into a local optimum.

[0130] Example 2: Multi-layer spiking neural network

[0131] Unlike the multi-layer artificial neural network in Example 1, the neurons in the spike network are replaced with leaky integrate-and-fire (LIF) neurons. This is a simplification of biological neurons, but compared to artificial neurons, the information they provide about external input is stored in the pulse firing frequency rather than the pulse intensity. It contains a parameter β that defines the integration and leakage rate of the LIF neuron:

[0132] U t+1 =βU t +W t X t ,#15)

[0133] Where U is the membrane potential of the LIF neuron, W is the connection weight matrix between the LIF neurons in this layer and the neurons in the previous layer, X is the output of the neurons in the previous layer, and W t X t It constitutes the input of the current LIF neuron at time t. As WX continues, U will continue to increase and then reach a certain threshold V threshold After that, the neuron fires a spike and resets the membrane potential.

[0134] Compared to normal networks, spiking networks require the ability of dynamic synaptic models to jointly optimize the internal parameters of neurons and the weights of the links between neurons. Generally speaking, existing optimizations for spiking neural networks rely on substitute gradient methods, using substitute gradient functions instead of the discharge of spiking neurons during backpropagation to avoid the vanishing or exploding gradients. However, this only works for simpler brain-like neurons such as LIF and is not effective for more complex neurons. Therefore, the key to the task is to optimize network weights and internal parameters of neurons by relying solely on dynamic synaptic models and rewards without using substitute gradient methods. This is the basis for the subsequent construction of more complex brain-like networks, especially those containing artificial neurons and brain-like neurons.

[0135] This example uses snnTorch to build a 784×1000×10 multi-layer spiking neural network. Because LIF neurons rely on integrated discharge, which requires an integration process, compared to artificial neural networks that input one image at a time, this network inputs one image at a time, stacked over time. In other words, an image is continuously fed into the network over a period of time, causing neurons to fire and produce output.

[0136] This embodiment takes the number of discharges of the last layer of LIF neurons in a period, and after low-pass filtering, takes the cross entropy with the correct label as the loss for each step. The same reward generation strategy as the artificial neural network is adopted (see formula (b) (c)). Note that for the internal parameter β of the LIF neuron, directly fluctuating β by using the exploration parameter consistent with the network connection weight will make the overall network output extremely unstable. Therefore, referring to the simplified process of the LIF neuron, according to:

[0137]

[0138] The internal parameter β in Equation 15) is converted, where Δt is the time conversion coefficient and τ is the actual optimized parameter after conversion.

[0139] This embodiment uses a dynamic synaptic model to optimize τ during training. In actual implementation, Δt = 0.01, and the initial value of τ is set to -0.6 (in this case, β ≈ 0.96).

[0140] The training initialization hyperparameters are set to lr = 0.01, dt = 10, period = 1000, amp = 0.05 or 0.03 (for τ), weight_oscilate_decay = 0.001, weight_centre_update_rate = 1 or 0.1 (for τ), and the rest remain default. The low-pass filter parameter γ = 0.9.

[0141] The experimental results are as follows Figures 9-14 From the results, we can see that the performance of the multi-layer spike network exceeds 70% in MNIST classification accuracy; and from the process point of view, the dynamic synaptic model successfully explored the area with higher rewards and drove the network weights and neuron internal hyperparameters to shift towards this area.

[0142] Example 3: Larger Brain-like Networks Composed of Artificial Neurons and Spiking Neurons

[0143] These applications demonstrate the ability of dynamic synaptic models to optimize large-scale artificial neural networks and spiking neural networks, respectively. While there are currently different methods for training these two types of networks separately, such as gradient descent for artificial neural networks, STDP for spiking neural networks, and alternative gradient methods for LIF neurons, there is no biologically plausible method for simultaneously training neural networks composed of these two types of neuron models.

[0144] Dynamic synapses are capable of synchronously training a hybrid model composed of these two types of neurons. A four-layer feedforward brain-like network was built using snnTorch, where the first two layers are artificial neurons and the last two layers are pulse neurons, or LIF neurons. The former are activated by functions such as RELU, while the latter rely on the neuron's own integrated discharge. The entire network is optimized using a dynamic synaptic model, including network weights and LIF neuron internal parameters. The network structure is as follows: Figure 15 shown.

[0145] Keep the reward generation strategy unchanged. Initialize the training hyperparameters to lr = 0.01, dt = 10, period = 1200, amp = 0.05, weight_oscilate_decay = 0.01, weight_centre_update_rate = 1, and keep the rest as default or the same as the previous task. The low-pass filter parameter γ = 0.9.

[0146] Some training results are as follows Figures 16-20 As shown in the figure, from the results, the performance of the brain-like network composed of artificial neural networks and pulse neural networks reaches about 65% in MNIST classification accuracy; and from the process point of view, the convergence of the network is similar to that of the previous task, and the final performance is also similar, indicating the preliminary feasibility of the dynamic synaptic model in compatibility between artificial neural networks and brain-like neural networks.

[0147] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A dynamic synaptic brain-like computing training method, characterized in that: The following steps are involved: Get image dataset; The control of neuromodulators represented by dopamine on the movement of transmitter receptors is regarded as a nonlinear dynamic process, and a dynamic synaptic model is constructed; The dynamic synaptic model is used as a network optimizer, and the preset network is trained in combination with the image data set to obtain a trained classification network.

2. A dynamic synaptic brain-like computing training method according to claim 1, characterized in that: For a certain synapse, its synaptic strength changes with time t: Where A is the amplitude of the fluctuation, C is the center of the fluctuation, and T i is the ith period of the fluctuation, and W(t) represents the instantaneous synaptic strength.

3. A dynamic synaptic brain-like computing training method according to claim 2, characterized in that: It also includes the addition of dynamic mechanisms to promote and inhibit the exploration of spontaneous synaptic fluctuations. The specific strategies are as follows: Among them, A t+1 represents the strategy amplitude at step t+1, A t represents the strategy amplitude at step t, Δt represents the step length from step t to step t+1, β represents the convergence rate of the amplitude, and R t represents the reward of step t, α0 and α1 represent the corresponding preset thresholds, and α represents the long-term statistics of the reward.

4. A dynamic synaptic brain-like computing training method according to claim 1, characterized in that: The optimization process of the dynamic synaptic model as a network optimizer is specifically as follows: Update time, network current weight, weight fluctuation center, weight fluctuation amplitude, and period according to the corresponding update formula; Return the current weights of the network.

5. A dynamic synaptic brain-like computing training method according to claim 1, characterized in that: The preset network is a multi-layer artificial neural network, wherein: The reward formula for the training process is as follows: in, Represents the long-term loss of the t-th step, loss t represents the cross entropy loss at step t; The iterative formula is defined as: Wherein, γ represents a preset parameter.

6. A dynamic synaptic brain-like computing training method according to claim 1, characterized in that: The preset network is a multi-layer pulse neural network, wherein: The conversion formula of the internal parameters of the LIF neuron is as follows: Here, β is an internal parameter of the LIF neuron, representing the neuron's integration and leakage rate, Δt is the conversion time coefficient, and τ is the converted parameter. This means that optimizing β translates to optimizing τ, which is used to maintain stable network output.

7. A dynamic synaptic brain-like computing training method according to claim 1, characterized in that: The preset network is a hybrid model composed of artificial neurons and spiking neurons, where: The artificial neuron is activated using a RELU function; The spiking neuron is dependent on the neuron's own integrated discharge; Optimization is performed using a dynamic synaptic model, including network weights as well as internal parameters of spiking neurons.

8. A method for applying a dynamic synaptic model, characterized in that: The following steps are involved: The dynamic synaptic model as claimed in claim 1 is applied to a multi-layer artificial neural network, a multi-layer spiking neural network, and a multi-layer brain-like network composed of artificial neurons and spiking neurons, and is trained in combination with a recognition data set to obtain a trained recognition model.

Images