A balanced propagation optimization method and system based on a deep convolutional neural network

Abstract

The application provides a balanced propagation optimization method and system based on a deep convolutional neural network, and belongs to the technical field of neural network optimization. The method comprises the following steps: constructing a general network structure form based on the gradient and energy distribution of energy calculation in an original network; defining an additional connection on the last node of the general network structure form, determining the projection of the last node through the additional connection, thereby determining an embedding space, and obtaining a complete network structure; defining an adjacency matrix between each node in the complete network structure according to the node topological relationship of the original network, and the adjacency matrix is used to establish the connection relationship between each network layer in the complete network structure; taking a preprocessed image as the input of the complete network structure, calculating the stable point of the complete network structure at different energy descent stages, determining the lowest energy point with the stable point, and thereby completing the balanced propagation optimization of the deep convolutional neural network.

Description

Technical Field

[0001] This invention belongs to the field of neural network optimization technology, and particularly relates to a balanced propagation optimization method and system based on deep convolutional neural networks. Background Technology

[0002] Backpropagation (BP) enables state-of-the-art deep neural networks to learn useful representations from structured data such as speech, vision, and text. BP is currently the most successful algorithm for training artificial neural networks, and a crucial question arises during training: how should synaptic connections be modified to make the network fit the indicative signal given by the objective function? This is a difficult problem because a single synapse can have complex effects on downstream processing. BP solves the error propagation problem using the chain rule. While BP has been effectively implemented in practical applications, it is difficult to imagine how it could be implemented in biological systems. This is because BP requires two separate phases in the connections between neurons: a nonlinear forward propagation that propagates neuronal activity and a linear backward propagation that carries a signed gradient signal. Both types of processing are also inconvenient for training physical neural networks, as both should be handled by the same circuit. Despite its unreliability, representations learned using BP yield better embeddings and stronger representational capabilities compared to networks trained using purely biological learning rules such as STDP.

[0003] Equilibrium propagation (EP) is an alternative to backpropagation (BP) algorithms that perform state and weight updates in dynamic systems converging to a fixed point, such as models based on continuous Hopfield energy functions. EP is also performed in two phases: In the first phase, the dynamic system is presented with static input data until the states throughout the network stabilize at an equilibrium or fixed point. This state is called free equilibrium. In the second phase, a teacher signal slightly pushes the specified output unit toward the target value until the dynamics stabilize to a second equilibrium, called fine-tuned equilibrium. EP estimates the loss gradient by comparing neuron activity between the two different equilibrium phases. EP is attractive because the resulting learning rule is spatially local when the energy function consists of two interaction forces, as is the case in continuous Hopfield networks. Furthermore, EP has been shown to approximate the true gradient produced by backpropagation learning when the action factor of the push approaches zero infinitely. More generally, the implicit differentiation performed by EP makes it more suitable for meta-learning, where explicitly backpropagating errors through internal optimization of loops becomes impractical due to the high memory requirements of storing intermediate time steps used for regular automatic differentiation.

[0004] However, the classic EP (Easy Gradient Estimation) has several limitations. First, the gradient estimation of the EP only approximates the actual loss gradient when the scaling factor of the push approaches zero infinitely. This requirement makes it impractical for small-amplitude instruction signals to be obfuscated by noise, which is unrealistic in neurodynamic systems and also impractical for brain learning models that treat feedback as a strong modulatory process. Furthermore, the mechanisms by which biological circuits and human brain circuits learn better remain elusive, and it is difficult to definitively conclude whether the operating mechanisms of the real biological brain match the currently assumed mechanisms. Finally, whether stronger and deeper extensions can be made to complex visual tasks, and whether its characteristics can be further extended to text and video tasks, are also pressing issues that need to be addressed in the future.

[0005] How synapses in hierarchical neural circuits adjust during learning remains a challenging problem known as the credit assignment problem. Balanced propagation (EP) offers a biologically more plausible solution to this problem in artificial neural networks. EP is an algorithm for convergent recurrent neural networks (RNNs), which, by definition, are given static inputs and recursively converge to a steady state corresponding to the network's predictions. EP proceeds in two phases. In the first phase, the network reaches a first steady state by a free fall of energy, and then the network's output layer is pushed toward the true prediction target until a second steady state is reached. In the second phase, perturbations originating from the output layer propagate forward in time to the upstream layers, producing a local error signal that perfectly matches the signal computed via backpropagation in time (BPTT) (the classic method for training RNNs). A comparison between EP and recurrent backpropagation can be explored; due to this strong theoretical guarantee, EP can provide clues for understanding biological learning. Furthermore, the spatial locality of the learning rules prescribed by EP, and the possibility that they are also temporally local, is highly attractive for designing energy-efficient neuromorphic hardware implementations of gradient-based learning algorithms.

[0006] However, to meet these expectations, EP should be able to scale to complex tasks. To date, work on EP has limited its experiments to the MNIST classification task and shallow network architectures. Despite theoretical guarantees for EP, the literature shows that, to date, EP implementations have not successfully matched the performance of standard deep learning methods for training deep networks on more challenging vision tasks. This problem becomes even more challenging when using more biologically plausible topologies where the synaptic connections in the network are unidirectional. In such cases, some existing improvements to EP have resulted in a decrease in MNIST accuracy compared to standard EP.

[0007] Other biologically viable alternatives to backpropagation (BP) have been attempted to extend to complex visual tasks. The use of feedback alignment and target propagation variants on CIFAR-10 and ImageNet demonstrates that their performance is significantly lower than backpropagation. Feedback alignment performs better than symbolic symmetry on ImageNet when the alignment between forward and backward weights is enhanced by additional mechanisms, where feedback weights are treated as signs of forward weights, and is almost identical to backpropagation. However, in feedback alignment and target propagation, error feedback does not affect forward neural activity but is linked through different backward paths, a problem that EP avoids. Related learning rules also address this issue, based on rate-based equivalence rules and relying on specialized synapses and complex network topologies, and have been compared with CIFAR-10 and ImageNet. Related work on implicit models shows that training deep networks can be structured to solve fixed-point (steady-state) equations, enabling more accurate backpropagation. This framework has been shown to solve challenging visual tasks. While EP typically uses steady-state, the process of reaching steady-state and the learning rules differ. Compared to these methods, EP offers minimal circuitry requirements for inference and gradient computation, making it an excellent candidate for energy-efficient neuromorphic learning hardware design. However, its application in deep convolutional neural networks and complex vision tasks remains a challenge.

[0008] As the application requirements for balanced propagation increase, the development of network models with strong representation and generalization capabilities becomes particularly important. Existing work on balanced propagation focuses too much on the design of neural circuits to improve the computational speed of the algorithm; however, work addressing the theoretical issues is scarce.

[0009] Classical balance propagation uses the error between the gradient differences of two steady-state phases to update weights. This provides a novel strategy for weight updates, allowing for a relatively close estimation of weights based on existing gradients. However, this method introduces a first-order gradient bias, which hinders the effectiveness of balance propagation on more complex visual tasks and significantly increases the difficulty of learning and training deep neural networks. Three-stage balance propagation alleviates the first-order gradient bias problem by introducing a negative scaling factor. Theoretically, as the scaling factor approaches zero, this calculation approximates the gradient obtained through backpropagation. However, the dynamical system still struggles to find suitable embeddings for the last layer. Relying solely on the energy function to update the state of nodes in the network not only makes it difficult to find suitable embeddings for each neuron but also to find the energy minimum point quickly. This directly affects the efficiency of the balance propagation algorithm and limits its application to more difficult tasks. Furthermore, the training process consumes significant resources.

[0010] In deep neural networks trained using the backpropagation algorithm, gradient diminishing or even vanishing can occur. However, this problem is particularly pronounced when training deep neural networks using the balanced propagation algorithm, due to inappropriate learning rates and network structures. These issues directly hinder the convergence of deep neural networks based on the balanced propagation algorithm, further affecting the network's ability to reach the minimum energy point. The energy function needs to be adjusted according to different network structures to ensure that the network always converges to the minimum energy point in each energy iteration. However, this design is both inconvenient and impractical; designing a new energy function for each new network structure makes the balanced propagation algorithm unusable.

[0011] In the classical equilibrium propagation method for calculating energy, for each batch of data, the minimum energy point may not be reached within a limited number of iterations. Furthermore, during training, energy changes are in a dynamic oscillation process, making it difficult to quickly find the minimum energy point. This results in high time costs. Summary of the Invention

[0012] This invention aims to address the problems of poor training performance of the classic balanced propagation algorithm in deep convolutional neural networks and difficulty in learning on complex visual datasets; and proposes an optimized balanced propagation scheme based on deep convolutional neural networks.

[0013] The first aspect of this invention discloses a method for balance propagation optimization based on a deep convolutional neural network. The method includes: Step S1, constructing a general network structure based on the gradient and energy distribution calculated in the original network; Step S2, defining additional connections on the last node of the general network structure, determining the projection of the last node through these additional connections, thereby determining the embedding space to obtain a complete network structure; Step S3, defining an adjacency matrix between nodes in the complete network structure according to the node topology of the original network, the adjacency matrix being used to establish the connection relationships between various network layers in the complete network structure; Step S4, using a preprocessed image as input to the complete network structure, calculating the stable points of the complete network structure at different energy descent stages, determining the lowest energy point with a stable point, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0014] According to the method of the first aspect of the present invention, in step S1: the number of network layers L and the number of network channels of the first K layers of the general network structure are set based on the gradient and energy distribution of energy calculation in the original network, and the first K layers of the network are built in order of increasing number of network channels; for the last LK layers of the network, a convolutional layer with a kernel size of 1×1 is used, and the number of channels is consistent with that of the Kth layer of the network, where L and K are both positive integers and L>K.

[0015] According to the method of the first aspect of the present invention, step S2 specifically includes: taking the network node of the Lth layer network as the last node, obtaining the number of channels of the Lth layer network, adding a mapping layer to the Lth layer network after the Lth layer network based on the number of channels of the Lth layer network, as the L+1th layer network, the mapping layer being a fully connected layer or the convolutional layer; adding a fully connected layer after the L+1th layer network, as the L+2th layer network, the number of channels of the L+2th layer network being consistent with the number of target classifiers, encapsulating the L+1th layer network and the L+2th layer network in an additional list; obtaining the node state of the last node located in the Lth layer network through the additional connection, and passing the node state to the additional list to calculate the projection of the last node in the two layers of the network included in the additional list, so as to establish the embedding space, thereby obtaining the complete network structure.

[0016] According to the method of the first aspect of the present invention, in step S3, an adjacency matrix between each node in the complete network structure is defined based on the node topology of the original network, and a mapping type for each adjacency matrix is ​​determined. The connection relationship between each network layer in the complete network structure is established based on the adjacency matrix and the corresponding mapping type.

[0017] According to the method of the first aspect of the present invention, in step S4, the energy is gradually reduced. For each reduction in energy, the preprocessed image is used as the input of the complete network structure to analyze the convergence state of the complete network structure as the stable point of the current energy reduction stage, until the lowest energy point with a stable point is determined, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0018] According to the method of the first aspect of the present invention, in step S4, the preprocessing of the image includes data augmentation, image flipping, edge cropping and type conversion, wherein the preprocessed image is of tensor type.

[0019] A second aspect of this invention discloses a balance propagation optimization system based on a deep convolutional neural network. The system includes: a first processing unit configured to: construct a general network structure based on the gradient and energy distribution calculated in the original network; a second processing unit configured to: define additional connections on the last node of the general network structure, determine the projection of the last node through the additional connections, thereby determining the embedding space to obtain a complete network structure; a third processing unit configured to: define an adjacency matrix between nodes in the complete network structure according to the node topology of the original network, the adjacency matrix being used to establish the connection relationships between network layers in the complete network structure; and a fourth processing unit configured to: use a preprocessed image as input to the complete network structure, calculate the stable point of the complete network structure at different energy descent stages, determine the lowest energy point with a stable point, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0020] According to the system of the second aspect of the present invention, the first processing unit is specifically configured to: set the number of network layers L and the number of network channels of the first K layers of the general network structure based on the gradient and energy distribution of energy calculation in the original network, and build the first K layers of the network in an order of increasing number of network channels; for the last LK layers of the network, all use convolutional layers with a kernel size of 1×1, and the number of channels is consistent with that of the Kth layer of the network, where L and K are both positive integers and L>K.

[0021] According to the system of the second aspect of the present invention, the second processing unit is specifically configured to: take the network node of the Lth layer network as the last node, obtain the number of channels of the Lth layer network, add a mapping layer to the Lth layer network after the Lth layer network based on the number of channels of the Lth layer network, as the L+1th layer network, the mapping layer being a fully connected layer or the convolutional layer; add a fully connected layer after the L+1th layer network, as the L+2th layer network, the number of channels of the L+2th layer network being consistent with the number of target classifiers, encapsulate the L+1th layer network and the L+2th layer network in an additional list; obtain the node state of the last node located in the Lth layer network through the additional connection, and pass the node state to the additional list to calculate the projection of the last node in the two layers of the network included in the additional list, so as to establish the embedding space, thereby obtaining the complete network structure.

[0022] According to the system of the second aspect of the present invention, the third processing unit is specifically configured to: define an adjacency matrix between each node in the complete network structure based on the node topology of the original network, determine the mapping type of each adjacency matrix, and establish the connection relationship between each network layer in the complete network structure based on the adjacency matrix and the corresponding mapping type.

[0023] According to the system of the second aspect of the present invention, the fourth processing unit is specifically configured to: gradually reduce the energy, and for each energy reduction, use the preprocessed image as the input of the complete network structure, analyze the convergence state of the complete network structure as the stable point of the current energy reduction stage, until the lowest energy point with a stable point is determined, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0024] According to the system of the second aspect of the present invention, the fourth processing unit is specifically configured to: preprocess the image including data augmentation, image flipping, edge cropping and type conversion, wherein the preprocessed image is of tensor type.

[0025] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the balanced propagation optimization method based on a deep convolutional neural network described in the first aspect of this disclosure.

[0026] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the balanced propagation optimization method based on a deep convolutional neural network as described in the first aspect of this disclosure.

[0027] In summary, the technical solution proposed in this invention extends energy calculation to a three-stage process and further seeks the minimum value using the equilibrium state. It adds an extra mapping outside the network structure involved in energy calculation to accelerate the search for suitable embeddings. Connections are established between low and high network layers to support gradient transfer between networks. Based on classical equilibrium propagation, this invention fully explores the gradient and energy distribution between network layers and combines the information obtained from skipping layers to effectively find stable points. The model has higher accuracy and stronger generalization than the classical equilibrium propagation algorithm, and strongly supports the development of the equilibrium propagation algorithm in complex task scenarios. Attached Figure Description

[0028] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating the operation of an embodiment of the present invention;

[0030] Figure 2 This is a network structure diagram according to an embodiment of the present invention;

[0031] Figure 3 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] The first aspect of this invention discloses a method for balance propagation optimization based on a deep convolutional neural network. The method includes: Step S1, constructing a general network structure based on the gradient and energy distribution calculated in the original network; Step S2, defining additional connections on the last node of the general network structure, determining the projection of the last node through these additional connections, thereby determining the embedding space to obtain a complete network structure; Step S3, defining an adjacency matrix between nodes in the complete network structure according to the node topology of the original network, the adjacency matrix being used to establish the connection relationships between various network layers in the complete network structure; Step S4, using a preprocessed image as input to the complete network structure, calculating the stable points of the complete network structure at different energy descent stages, determining the lowest energy point with a stable point, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0034] In some embodiments, in step S1: based on the gradient and energy distribution of energy calculation in the original network, the number of network layers L and the number of network channels of the first K layers of the general network structure are set, and the first K layers of the network are built in order of increasing number of network channels; for the last LK layers of the network, convolutional layers with a kernel size of 1×1 are used, and the number of channels is consistent with that of the Kth layer of the network, where L and K are both positive integers and L>K.

[0035] In some embodiments, step S2 specifically includes: taking the network node of the Lth layer network as the last node, obtaining the number of channels of the Lth layer network, adding a mapping layer to the Lth layer network after the Lth layer network based on the number of channels of the Lth layer network, as the L+1th layer network, the mapping layer being a fully connected layer or the convolutional layer; adding a fully connected layer after the L+1th layer network, as the L+2th layer network, the number of channels of the L+2th layer network being consistent with the number of target classifiers, encapsulating the L+1th layer network and the L+2th layer network in an additional list; obtaining the node state of the last node located in the Lth layer network through the additional connection, and passing the node state to the additional list to calculate the projection of the last node in the two layers of the network included in the additional list, so as to establish the embedding space and thereby obtain the complete network structure.

[0036] In some embodiments, in step S3, adjacency matrices between nodes in the complete network structure are defined according to the node topology of the original network, and the mapping type of each adjacency matrix is ​​determined. The connection relationships between network layers in the complete network structure are established according to the adjacency matrices and the corresponding mapping types.

[0037] In some embodiments, in step S4, the energy is gradually reduced. For each reduction in energy, the preprocessed image is used as the input to the complete network structure to analyze the convergence state of the complete network structure as the stable point of the current energy reduction stage, until the lowest energy point with a stable point is determined, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0038] In some embodiments, in step S4, the preprocessing of the image includes data augmentation, image flipping, edge cropping, and type conversion, wherein the preprocessed image is of type tensor.

[0039] Specifically, this study explores gradients and energy distributions during energy computation within the network, designing a general network structure. It defines external links at the final node, reads the state of the final node, uses these links to find its projection, and identifies a suitable embedding space. A large-scale topological adjacency matrix is ​​defined, and skip-layer connections are established based on this matrix. Stable point information at different stages of energy computation is obtained, and this information is repeatedly used to further search for the lowest energy point. Figure 1 As shown.

[0040] The algorithm explores the gradient and energy distribution in the energy calculation process of the network and designs a general network structure. Specifically, it includes building a convolutional neural network with the number of channels increasing from few to many, upsampling the data to avoid loss of information in each calculation, and then adding convolutional layers with a kernel size of 1×1 while maintaining the same number of output channels as the last convolutional layer. This process continues until all layers are added.

[0041] Defining additional connections at the last node of the network and reading the state of the last node specifically involves: reading the number of output channels of the last layer of the network structure, using it as the number of input channels, and establishing a mapping, which includes, but is not limited to, fully connected layers and convolutional layers. Adding fully connected layers with the desired mapping dimension as the target, mapping the number of channels in the last layer of the network to the number of classifiers, and encapsulating these mapping layers in an additional list, separate from the network layers used to calculate the energy function. Reading the state of the last output node after the last layer and passing this state to the additional list for computation, finding a suitable embedding for the last layer.

[0042] Defining a large-scale topological adjacency matrix for a network, and establishing interconnections between network layers based on this matrix, specifically includes: establishing adjacency relationships based on the desired topology, defining the adjacency matrix, and defining the mapping type between each relationship. Based on the established adjacency matrix, and according to the mappings and mapping types between each connection, the network structure is built, passing information from lower layers to higher layers. An example of the completed structure is shown below. Figure 2 As shown.

[0043] The process involves acquiring stable point information at different stages of energy calculation, repeatedly utilizing this stable point information, and then further searching for the minimum energy point. Specifically, this includes: acquiring the input image, performing data augmentation on the input image, obtaining the flipped and cropped image, and converting it to a tensor. The network is then trained according to a prescribed procedure to obtain and save steady-state information. Using this steady-state information, the energy function is repeatedly calculated to allow the model to move towards the minimum energy point more quickly, thus accelerating convergence.

[0044] The technical effects of this invention include: (1) avoiding gradient vanishing and accelerating training speed. Deep BP convolutional networks suffer from gradient reduction, a problem first solved in ResNet with skip connections. Skip connections send information from lower layers closer to the input directly to higher layers closer to the output, and help to quickly calculate gradients to support parameter updates. (2) Regarding external connections, it automatically finds the projection that minimizes the loss function. This is equivalent to the update rule of the BP network, which helps the network quickly find a suitable embedding space for the last layer. The inference result is not read directly from the last node, but has a linear mapping as read-out. (3) Utilizing information from stable points, it further finds the energy minimum point, accelerating the convergence speed.

[0045] Example: Load the CIFAR10 and MNIST datasets, process the datasets, and perform ablation experiments on each dataset. Compare the results without external-link, without skip-layer, and with them. The experiments also demonstrate that the effect of adding them is completely due to the effect of not adding or adding less of them.

[0046] Experiments were conducted on the MNIST dataset to verify the training and testing accuracy with and without external links and skip layers. It was found that adding them resulted in faster convergence and higher accuracy. Clearly, the MNIST dataset presents a significant challenge for achieving balanced propagation in deep convolutional networks.

[0047] Experiments were conducted on the CIFAR-10 dataset to verify the training and testing accuracy with and without a skip-layer. It was found that adding a skip-layer resulted in faster convergence and higher accuracy. Clearly, the CIFAR-10 dataset presents a significant learning challenge for achieving balanced propagation in deep convolutional networks.

[0048] A second aspect of this invention discloses a balance propagation optimization system based on a deep convolutional neural network. The system includes: a first processing unit configured to: construct a general network structure based on the gradient and energy distribution calculated in the original network; a second processing unit configured to: define additional connections on the last node of the general network structure, determine the projection of the last node through the additional connections, thereby determining the embedding space to obtain a complete network structure; a third processing unit configured to: define an adjacency matrix between nodes in the complete network structure according to the node topology of the original network, the adjacency matrix being used to establish the connection relationships between network layers in the complete network structure; and a fourth processing unit configured to: use a preprocessed image as input to the complete network structure, calculate the stable point of the complete network structure at different energy descent stages, determine the lowest energy point with a stable point, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0049] In some embodiments, the first processing unit is specifically configured to: set the number of network layers L and the number of network channels of the first K layers of the general network structure based on the gradient and energy distribution of energy calculation in the original network, and build the first K layers of the network in an order of increasing number of network channels; for the last LK layers of the network, all use convolutional layers with a kernel size of 1×1, and the number of channels is consistent with that of the Kth layer of the network, where L and K are both positive integers and L>K.

[0050] In some embodiments, the second processing unit is specifically configured to: take the network node of the Lth layer network as the last node, obtain the number of channels of the Lth layer network, add a mapping layer to the Lth layer network after the Lth layer network based on the number of channels of the Lth layer network, as the L+1th layer network, the mapping layer being a fully connected layer or the convolutional layer; add a fully connected layer after the L+1th layer network, as the L+2th layer network, the number of channels of the L+2th layer network being consistent with the number of target classifiers, encapsulate the L+1th layer network and the L+2th layer network in an additional list; obtain the node state of the last node located in the Lth layer network through the additional connection, and pass the node state to the additional list to calculate the projection of the last node in the two layers of the network included in the additional list, so as to establish the embedding space and thereby obtain the complete network structure.

[0051] In some embodiments, the third processing unit is specifically configured to: define an adjacency matrix between each node in the complete network structure according to the node topology of the original network, determine the mapping type of each adjacency matrix, and establish the connection relationship between each network layer in the complete network structure according to the adjacency matrix and the corresponding mapping type.

[0052] In some embodiments, the fourth processing unit is specifically configured to: gradually reduce the energy, and for each energy reduction, use the preprocessed image as input to the complete network structure, analyze the convergence state of the complete network structure as the stable point of the current energy reduction stage, until the lowest energy point with a stable point is determined, thereby completing the balance propagation optimization of the deep convolutional neural network.

[0053] In some embodiments, the fourth processing unit is specifically configured to preprocess the image, including data augmentation, image flipping, edge cropping, and type conversion, wherein the preprocessed image is of type tensor.

[0054] A third aspect of this invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the balanced propagation optimization method based on a deep convolutional neural network described in the first aspect of this disclosure.

[0055] Figure 3 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 3 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0056] Those skilled in the art will understand that Figure 3The structure shown is merely a structural diagram of the part related to the technical solution of this disclosure and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0057] A fourth aspect of this invention discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the balanced propagation optimization method based on a deep convolutional neural network as described in the first aspect of this disclosure.

[0058] In summary, the technical solution proposed in this invention extends energy calculation to a three-stage process and further seeks the minimum value using the equilibrium state. It adds an extra mapping outside the network structure involved in energy calculation to accelerate the search for suitable embeddings. Connections are established between low and high network layers to support gradient transfer between networks. Based on classical equilibrium propagation, this invention fully explores the gradient and energy distribution between network layers and combines the information obtained from skipping layers to effectively find stable points. The model has higher accuracy and stronger generalization than the classical equilibrium propagation algorithm, and strongly supports the development of the equilibrium propagation algorithm in complex task scenarios.

[0059] In deep convolutional neural networks, existing techniques struggle to perform well on complex visual tasks. First, the depth of the network increases the difficulty of gradient propagation and computation, easily leading to gradient diminishing or vanishing problems. This results in parameters of each network layer not updating, making it difficult for the network to learn new patterns. Second, calculating the energy function is challenging for complex tasks. Dynamical systems struggle to find the energy minimum in high-dimensional space, and finding suitable embeddings for node states is also difficult. Under these conditions, it's difficult for the dynamical system to reach a steady state through energy descent. Finally, for each batch of data, the energy function cannot reach the energy minimum within a limited number of iterations. The energy descent process is oscillating and jittery, resulting in long convergence times and training difficulties. To address these issues, this invention proposes a follow-up extra connection that does not participate in energy function calculation. During the update process, it is equivalent to using a backpropagation (BP) update rule, accelerating the system to find the most suitable embedding for the last layer node. This directly promotes the search for the steady-state energy minimum, speeding up convergence. This invention establishes connections between layers, passing information from lower layers to higher layers and simultaneously aiding in gradient propagation during backpropagation. This invention analyzes the energy calculation values ​​in each round and further utilizes steady-state information to accelerate the reaching of the lowest energy point from the steady state, thereby further shortening the training and learning time and speeding up network convergence.

[0060] This invention avoids gradient vanishing by using a Skip-layer, thus accelerating training and facilitating rapid gradient calculation to support parameter updates. By using an external link, it automatically finds the projection that minimizes the loss function, helping the network quickly find a suitable embedding space for the final layer. Furthermore, by using Repeat, it leverages information from stable points to further locate the energy minimum, accelerating convergence.

[0061] Please note that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A balanced propagation optimization method based on deep convolutional neural networks, characterized in that, The method includes: Step S1: Based on the gradient and energy distribution calculated in the original network, construct a general network structure. Step S2: Define additional connections on the last node of the general network structure, and determine the projection of the last node through the additional connections, thereby determining the embedding space to obtain the complete network structure; Step S3: Based on the node topology of the original network, define the adjacency matrix between each node in the complete network structure. The adjacency matrix is ​​used to establish the connection relationship between each network layer in the complete network structure. Step S4: Use the preprocessed image as input to the complete network structure, calculate the stable point of the complete network structure at different energy descent stages, determine the lowest energy point with a stable point, and thus complete the balance propagation optimization of the deep convolutional neural network.

2. The balanced propagation optimization method based on a deep convolutional neural network according to claim 1, characterized in that, In step S1: Based on the gradient and energy distribution of energy calculation in the original network, the number of network layers L and the number of network channels of the first K layers of the general network structure are set, and the first K layers of the network are built in order of increasing number of network channels; For the subsequent LK layers, all use convolutional layers with a kernel size of 1×1, and the number of channels is consistent with that of the Kth layer. L and K are both positive integers and L > K.

3. The balanced propagation optimization method based on a deep convolutional neural network according to claim 2, characterized in that, Step S2 specifically includes: Using the network node of the Lth layer network as the last node, obtain the number of channels of the Lth layer network, and add a mapping layer of the last node after the Lth layer network based on the number of channels of the Lth layer network, as the L+1th layer network. The mapping layer is a fully connected layer or the convolutional layer. A fully connected layer is added after the (L+1)th layer network to form the (L+2)th layer network. The number of channels in the (L+2)th layer network is consistent with the number of target classifiers. The (L+1)th layer network and the (L+2)th layer network are encapsulated in an additional list. The node state of the last node in the Lth layer network is obtained through the additional connection, and the node state is passed to the additional list to calculate the projection of the last node in the two layers of the network contained in the additional list, so as to establish the embedding space and obtain the complete network structure.

4. The balanced propagation optimization method based on a deep convolutional neural network according to claim 3, characterized in that, In step S3, based on the node topology of the original network, an adjacency matrix between each node in the complete network structure is defined, and the mapping type of each adjacency matrix is ​​determined. Based on the adjacency matrix and the corresponding mapping type, the connection relationship between each network layer in the complete network structure is established.

5. The balanced propagation optimization method based on a deep convolutional neural network according to claim 4, characterized in that, In step S4, the energy is gradually reduced. For each energy reduction, the preprocessed image is used as the input to the complete network structure to analyze the convergence state of the complete network structure and use it as the stable point of the current energy reduction stage. This process continues until the lowest energy point with a stable point is determined, thereby completing the balance propagation optimization of the deep convolutional neural network.

6. The balanced propagation optimization method based on a deep convolutional neural network according to claim 5, characterized in that, In step S4, the preprocessing of the image includes data augmentation, image flipping, edge cropping, and type conversion, and the preprocessed image is of tensor type.

7. A balanced propagation optimization system based on a deep convolutional neural network, characterized in that, The system includes: The first processing unit is configured to: construct a general network structure based on the gradient and energy distribution calculated in the original network; The second processing unit is configured to: define additional connections on the last node of the general network structure, determine the projection of the last node through the additional connections, thereby determining the embedding space to obtain the complete network structure; The third processing unit is configured to: define an adjacency matrix between the nodes in the complete network structure based on the node topology of the original network, wherein the adjacency matrix is ​​used to establish the connection relationship between the network layers in the complete network structure; The fourth processing unit is configured to: take the preprocessed image as input to the complete network structure, calculate the stable point of the complete network structure at different energy descent stages, determine the lowest energy point with a stable point, thereby completing the balance propagation optimization of the deep convolutional neural network.

8. The balanced propagation optimization system based on a deep convolutional neural network according to claim 7, characterized in that, The first processing unit is specifically configured as follows: Based on the gradient and energy distribution of energy calculation in the original network, the number of network layers L and the number of network channels of the first K layers of the general network structure are set, and the first K layers of the network are built in order of increasing number of network channels; For the subsequent LK layers, all use convolutional layers with a kernel size of 1×1, and the number of channels is consistent with that of the Kth layer. L and K are both positive integers and L > K. The second processing unit is specifically configured as follows: Using the network node of the Lth layer network as the last node, obtain the number of channels of the Lth layer network, and add a mapping layer of the last node after the Lth layer network based on the number of channels of the Lth layer network, as the L+1th layer network. The mapping layer is a fully connected layer or the convolutional layer. A fully connected layer is added after the (L+1)th layer network to form the (L+2)th layer network. The number of channels in the (L+2)th layer network is consistent with the number of target classifiers. The (L+1)th layer network and the (L+2)th layer network are encapsulated in an additional list. The node state of the last node in the Lth layer network is obtained through the additional connection, and the node state is passed to the additional list to calculate the projection of the last node in the two layers of the network contained in the additional list, so as to establish the embedding space and obtain the complete network structure.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the balanced propagation optimization method based on a deep convolutional neural network as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the balanced propagation optimization method based on a deep convolutional neural network as described in any one of claims 1-6.

Images