Image recognition system based on gradient evaluation and enhancement system
By introducing gradient evaluation and enhancement modules into the image recognition system, the gradient disappearance and explosion problems are solved, the model performance and training stability are improved, and more efficient image recognition effect is achieved.
Patent Information
- Application Number
- CN202310582759.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, gradients are easily disturbed during artificial neural network training, resulting in gradient vanishing and gradient explosion. There is a lack of effective methods to evaluate and improve gradient quality and affect model performance.
Using a gradient evaluation and enhancement system, the weight gradient evaluation and correction module and feature gradient enhancement module are used to evaluate and optimize the network weight gradient and feature gradient to reduce noise and improve model performance.
Effectively evaluate and enhance the quality of model gradients, improve training efficiency and stability, improve model performance, and can be combined with a variety of training optimization algorithms to significantly improve the recognition rate.
Smart Images

Figure CN119027704A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a technology in the field of image recognition, and in particular to an image recognition system based on a gradient evaluation and enhancement system. Background Art
[0002] At present, artificial neural network image recognition models based on deep learning can achieve a high degree of accuracy. However, recent studies have shown that in the process of artificial neural network training, adding some continuous small perturbations to the gradient can cause a high probability of misjudgment of the final model. Many factors can interfere with the gradient, such as calculation errors and system cumulative errors, which will introduce noise in the calculation process of the gradient; gradient vanishing and gradient explosion during network training; low-quality, out-of-distribution data will produce noisy gradients, etc. The quality of the gradient generated during model training directly affects the performance of the final model. At present, there is still no standard evaluation method for the quality of model gradients, and there are few methods that can directly reduce the noise of model gradients, improve the quality of gradients, and thus improve the performance of the final model. Summary of the invention
[0003] In view of the above-mentioned defects and shortcomings of the prior art, the present invention proposes an image recognition system based on a gradient evaluation and enhancement system. Through two gradient optimization modules, the spatial information and temporal information of the gradient are combined to evaluate and enhance the gradient of the network weight gradient and the network feature gradient respectively, which can effectively solve the problems of gradient disappearance and gradient explosion, and improve the training efficiency, performance and training stability of the network.
[0004] The present invention is achieved through the following technical solutions:
[0005] The invention relates to an image recognition system based on a gradient evaluation and enhancement system, comprising: a backbone network, a weight gradient evaluation and correction module (GC-W) and a feature gradient enhancement module (GC-ODE), wherein: the weight gradient evaluation and correction module calculates the correction amount of the weight gradient according to the original weight gradient, the historical weight gradient and the reference weight gradient from the backbone network, and performs gradient evaluation and gradient correction on the network weight gradient; the feature gradient enhancement module calculates the feature gradient enhancement data according to the image features extracted by the backbone network, and then injects the feature gradient enhancement data into the backbone network to realize the optimization and enhancement of the network feature gradient. Technical Effects
[0006] The gradient evaluation and enhancement system during model training proposed by the present invention uses a learning-based module to evaluate and enhance the neural network training gradient. It can well evaluate the quality of the model gradient, and at the same time, correct and enhance it, and can be used in combination with a variety of model training optimization algorithms to achieve better model performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 It is a schematic diagram of the present invention;
[0008] Figure 2 Schematic diagram of the weight gradient evaluation and correction module;
[0009] Figure 3 Schematic diagram of the feature gradient enhancement module GC-ODE;
[0010] Figure 4 It is the schematic diagram of the correction module GC-W;
[0011] Figure 5 Schematic diagram of the training curve of the present invention on the image recognition task. DETAILED DESCRIPTION
[0012] like Figure 1 and Figure 2 As shown, an image recognition system based on a gradient evaluation and enhancement system involved in this embodiment includes: a backbone network and a weight gradient evaluation and correction module GC-W and a feature gradient enhancement module GC-ODE connected thereto, wherein: the weight gradient evaluation and correction module calculates the correction amount of the weight gradient according to the original weight gradient, the historical weight gradient and the reference weight gradient from the backbone network, and performs gradient evaluation and gradient correction on the network weight gradient; the feature gradient enhancement module calculates the feature gradient enhancement data according to the image features extracted by the backbone network and injects it into the backbone network to optimize and enhance the network feature gradient.
[0013] The backbone network structure is based on a residual network structure, including four basic structural modules, each of which contains two residual structures.
[0014] like Figure 3 As shown, the weight gradient evaluation and correction module includes: a gradient acquisition unit, a gradient reference unit and a gradient correction amount processing unit, wherein: the gradient acquisition unit acquires the current network weight gradient and calculates the spatial distribution information of the current gradient; the gradient reference unit calculates the time distribution information based on the gradient history according to the historical reference gradient, and then fuses the spatial distribution information of the current gradient with the time distribution information based on the gradient history through self-attention to obtain the correction amount of the weight gradient; the gradient correction amount processing unit adds the correction amount of the weight gradient to the original weight gradient to obtain the optimized weight correction gradient.
[0015] The self-attention method, i.e., the self-attention calculation mechanism, calculates the correlation between the weight gradient and the historical weight gradient at each position, and performs weighted averaging on each position in the sequence according to these correlations, thereby obtaining the correction amount of the weight gradient.
[0016] like Figure 4 As shown, the feature gradient enhancement module includes: a feature acquisition unit, a neural differential equation unit and a feature injection unit, wherein: the feature acquisition unit extracts the current features according to the input image; the neural differential equation unit obtains the feature gradient enhancement data according to the current feature calculation; the feature injection unit injects the feature gradient enhancement data into the backbone network to realize auxiliary training.
[0017] This embodiment relates to an image recognition method based on the above system, including:
[0018] Step 1) Build and train the backbone network:
[0019] Step 1.1) According to the weight gradient at time t and the historical average gradient at time t Calculate the corrected gradient at step t Where: Q t Depend on Obtained through 1×1 convolution. K t and Depend on They are obtained by 1×1 convolution respectively;
[0020] Step 1.2) Calculate the updated weight gradient: Where: f e x(·) and f ex (·) are all 1×1 convolutions;
[0021] Step 1.3) Use the updated weight gradient Update weight w;
[0022] Step 2) Build and train the weight gradient enhancement module GC-W, which specifically includes:
[0023] Step 2.1) Collect gradient data at time t-τ: weight gradient Historical average gradient Reference gradient
[0024] Step 2.2) According to the weight gradient and reference gradient For each feature dimension (traversed by k), calculate the angle loss function
[0025] Step 2.3) According to the weight gradient and reference gradient Calculating the oscillating loss function
[0026] Step 2.4) According to the angle loss function Update the parameters of the weight gradient evaluation and correction modules;
[0027] Step 2.5) According to the shock loss function Update SE structure parameters;
[0028] Step 3) Based on the image features extracted by the backbone network, a feature gradient enhancement module GC-ODE is constructed, feature gradient enhancement data is calculated and injected into the backbone network to optimize and enhance the network feature gradient, specifically including:
[0029] 3.1) Obtain the current image features and construct the neural differential equation branch: in: xm(t)(0≤t≤1) is the image feature from h m , i.e., time t = 0) to h m+1 , i.e. the change process at time t = 1). φ(x, t, θ) is the network branching function, and θ is the network parameter of network φ;
[0030] 3.2) Use the general differential equation solver to forward propagate the currently constructed neural differential equation branch: x m (1) = Solver(x m (t0),φ,t0,t1,θ), where: t0=0, t1=1, x m (t0) = h m ;
[0031] 3.3) Use the RMD algorithm (i.e. reverse mode differentiation algorithm) to backpropagate the currently constructed neural differential equation branch: Where: L is the loss function of the backbone network, and Respectively represent the network loss function for weight θ and feature x m The gradient of (0);
[0032] 3.4) According to the gradient of the loss function of the backbone network to the weight θ Update neural differential equation branch parameters;
[0033] 3.5) According to the loss function of the backbone network, feature x m Gradient of (0) Update feature gradients.
[0034] Step 4) completing the training and updating of the parameters of the backbone neural network according to the above-mentioned construction of GC-W and GC-ODE modules;
[0035] Step 5) Obtain the backbone neural network according to the above steps, and perform reasoning and testing on the image recognition task, specifically including:
[0036] 5.1) Obtain the trained backbone neural network: x is the input image, y is the predicted label;
[0037] 5.2) Calculate the image recognition accuracy based on the predicted label and the real label of the image, that is, calculate the proportion of correctly predicted samples to the total number of samples:
[0038] The supervised experiment is as follows: In the experimental setting of python 3.6 software environment and GeForce GTX 1080 hardware environment, the ResNet network structure is trained using the gradient evaluation and enhancement system in the image recognition task, and finally the model used in the experiment can achieve a high recognition rate in each task. The corresponding experimental data is as follows:
[0039] Under the setting of supervised learning, the weight gradient evaluation and correction module GC-W and the feature gradient enhancement module GC-ODE can improve the performance of the basic network. The effectiveness of the training gradient evaluation and enhancement system has been verified on multiple datasets, including the Cifar dataset and the ImageNet dataset.
[0040] The unsupervised experiment is as follows: In the experimental setting of python 3.6 software environment and GeForce GTX 1080 hardware environment, the ResNet network and VIT-B network structure are trained using the gradient evaluation and enhancement system in the unsupervised image recognition task, and finally the model used in the experiment can achieve a high recognition rate in each task. The corresponding experimental data is as follows:
[0041] Under the setting of unsupervised learning, the weight gradient evaluation and correction module GC-W and the feature gradient enhancement module GC-ODE can improve the performance of the basic network. The effect of the training gradient evaluation and enhancement system has been verified on a variety of network structures, including ResNet network and VIT network.
[0042] like Figure 5 As shown, the training effect and stability of the present invention are significantly improved compared with the SGD training method. The present invention is not limited to the optimization of the learning rate and gradient calculation method of the model, but uses a learning-based module to evaluate and enhance the neural network training gradient. As an embedded method, the present invention can be used in combination with a variety of model training optimization algorithms to achieve better model performance.
[0043] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.
Claims
1. An image recognition system based on a gradient evaluation and enhancement system, characterized in that: include: A backbone network, a weight gradient evaluation and correction module, and a feature gradient enhancement module, wherein: the weight gradient evaluation and correction module calculates the correction amount of the weight gradient according to the original weight gradient from the backbone network, the historical weight gradient, and the reference weight gradient, and performs gradient evaluation and gradient correction on the network weight gradient; The feature gradient enhancement module calculates the feature gradient enhancement data based on the image features extracted by the backbone network and then injects it into the backbone network to optimize and enhance the network feature gradient.
2. The image recognition system based on the gradient evaluation and enhancement system according to claim 1, characterized in that: The weight gradient evaluation and correction module includes: a gradient acquisition unit, a gradient reference unit and a gradient correction amount processing unit, wherein: the gradient acquisition unit acquires the current network weight gradient and calculates the spatial distribution information of the current gradient; the gradient reference unit calculates the time distribution information based on the gradient history according to the historical reference gradient, and then fuses the spatial distribution information of the current gradient with the time distribution information based on the gradient history through self-attention to obtain the correction amount of the weight gradient; the gradient correction amount processing unit adds the correction amount of the weight gradient to the original weight gradient to obtain the optimized weight correction gradient.
3. The image recognition system based on the gradient evaluation and enhancement system according to claim 2, characterized in that: The self-attention method, i.e., the self-attention calculation mechanism, calculates the correlation between the weight gradient and the historical weight gradient at each position, and performs weighted averaging on each position in the sequence according to these correlations, thereby obtaining the correction amount of the weight gradient.
4. The image recognition system based on the gradient evaluation and enhancement system according to claim 1, characterized in that: The feature gradient enhancement module includes: a feature acquisition unit, a neural differential equation unit and a feature injection unit, wherein: the feature acquisition unit extracts the current features according to the input image; the neural differential equation unit calculates the feature gradient enhancement data according to the current features; the feature injection unit injects the feature gradient enhancement data into the backbone network to realize auxiliary training.
5. An image recognition method based on a gradient evaluation and enhancement system based on the system of any one of claims 1 to 4, characterized in that: include: Step 1) Build and train the backbone network: Step 2) construct and train the weight gradient enhancement module GC-W; Step 3) Based on the image features extracted by the backbone network, a feature gradient enhancement module GC-ODE is constructed, feature gradient enhancement data is calculated and injected into the backbone network to optimize and enhance the network feature gradient; Step 4) completing the training and updating of the parameters of the backbone neural network according to the above-mentioned construction of GC-W and GC-ODE modules; Step 5) Obtain the backbone neural network according to the above steps, and perform reasoning and testing on the image recognition task.
6. The image recognition method based on the gradient evaluation and enhancement system according to claim 5, characterized in that: The step 1 specifically includes: Step 1.1) According to the weight gradient at time t and the historical average gradient at time t Calculate the corrected gradient at step t Where: Q t Depend on After 1×1 convolution, K t and Depend on They are obtained by 1×1 convolution respectively; Step 1.2) Calculate the updated weight gradient: Where: f ex (·) and f ex (·) are all 1×1 convolutions; Step 1.3) Use the updated weight gradient Update the weight w.
7. The image recognition method based on the gradient evaluation and enhancement system according to claim 5 is characterized in that: The step 2 specifically includes: Step 2.1) Collect gradient data at time t-τ: weight gradient Historical average gradient Reference gradient Step 2.2) According to the weight gradient and reference gradient For each feature dimension, calculate the angle loss function Among them: k is the traversal parameter; Step 2.3) According to the weight gradient and reference gradient Calculating the oscillating loss function Step 2.4) According to the angle loss function Update the parameters of the weight gradient evaluation and correction modules; Step 2.5) According to the shock loss function Update SE structure parameters.
8. The image recognition method based on the gradient evaluation and enhancement system according to claim 5, characterized in that: The step 3 specifically includes: 3.1) Obtain the current image features and construct the neural differential equation branch: Where: 0≤t≤1, x m (t) is the image feature from h m , that is, from t = 0 to h m+1 , that is, the change process at time t = 1, φ(x, t, θ) is the network branch function, and θ is the network parameter; 3.2) Use a general differential equation solver to forward propagate the currently constructed neural differential equation branch: x m (1) = Solver(x m (t0), φ, t0, t1, θ), where: t0 = 0, t1 = 1, x m (t0) = h m ; 3.3) Use the reverse mode differentiation algorithm to backpropagate the currently constructed neural differential equation branch: Where: L is the loss function of the backbone network, and The network loss function is the weight θ and feature x respectively. m The gradient of (0); 3.4) According to the gradient of the loss function of the backbone network to the weight θ Update neural differential equation branch parameters; 3.5) According to the loss function of the backbone network, feature x m Gradient of (0) Update feature gradients.
9. The image recognition method based on the gradient evaluation and enhancement system according to claim 5, characterized in that: The step 5 specifically includes: 5.1) Obtain the trained backbone neural network: x is the input image, y is the predicted label; 5.2) Calculate the image recognition accuracy based on the predicted label and the real label of the image, that is, calculate the proportion of correctly predicted samples to the total number of samples
