A two-stream neural network deep learning method combining gradient ascent and descent algorithms

Through the dual-stream neural network method combining gradient rise and fall algorithms in deep learning, the problem that deep learning methods are prone to fall into local optimal solutions when parameter adjustment is performed, the robustness and recognition rate of the model are improved, and faster convergence and higher accuracy.

CN115062774BActive Publication Date: 2024-11-22UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210710920.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-22
Publication Date
2024-11-22
Estimated Expiration
2042-06-22

AI Technical Summary

Technical Problem

Existing deep learning methods are prone to fall into local optimal solutions when parameter adjustments, and are not robust enough, making them difficult to effectively apply in real scenarios.

Method used

Using a dual-stream neural network method combining gradient rise and descent algorithm, the gradient rise flow is used to update parameters at a specified probability through the fusion of gradient fall flow and gradient rise flow during the training stage, to avoid local optimal solutions, and to sum them through feature weighting. Or convolutional fusion to achieve result fusion.

Benefits of technology

It improves the convergence speed and final recognition rate of the model, increases the robustness of the neural network, and can effectively jump out of local minimum values ​​and find better solutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115062774B_ABST
    Figure CN115062774B_ABST
Patent Text Reader

Abstract

The present invention discloses a dual-stream neural network deep learning method combining gradient ascent and descent algorithms, specifically: adding a gradient ascent flow to the original network and modifying the optimizer source code; then starting to train the network, first training the gradient descent flow for multiple rounds, judging whether the dual-stream fusion conditions are met at the end of each round, and starting to train the gradient ascent flow when the conditions for integrating into the gradient ascent flow are met; inputting the training results into the discriminator to obtain the loss, and if it is dual-stream training, the input is the result after the dual-stream fusion; updating the parameter weights in reverse according to the loss; performing dual-stream training until the training termination conditions are met, such as reaching the maximum number of iterations; then starting to test the network, downloading the dual-stream weights, using the same fusion method, and inputting the fusion results into the discriminator to obtain the prediction results. This method avoids the problem of blindly adjusting the parameters of the existing deep learning method in the direction of loss reduction when adjusting the parameters, and avoids the problem of the neural network falling into the local optimal solution.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of neural network and deep learning technology, and in particular to a dual-stream neural network deep learning method combining gradient ascent and descent algorithms. Background Art

[0002] Deep neural networks have demonstrated very powerful application capabilities in many fields. However, the effect of applying them to real scenarios is often affected by various noises. The gradient descent algorithm is the most commonly used optimization algorithm for neural network model training. It usually uses the gradient to perform back propagation based on the loss function. Through continuous iteration, the weights of the network's learnable parameters are updated to find the lowest loss point, and finally the model achieves a better effect. Traditional deep neural networks often only use the gradient descent algorithm to update parameter weights. When the gradient drops to a certain range, it is easy to fall into a local optimal solution, which greatly reduces the application effect in actual scenarios.

[0003] At present, regarding the optimization of neural network robustness, patent CN112784915A proposes an algorithm for optimizing decision boundaries. The main steps of this method are: first, build a network and input pictures; second, train the network and build a decision domain model based on the parameters obtained after training; then use the built decision domain model to generate adversarial samples; finally, use the adversarial sample to train the network built in step 1, so that the decision boundary of the network moves in the direction of the adversarial sample, so that the network's sensitivity to disturbances is reduced, and the neural network model obtained by inputting the image to be classified is classified. Although this method can effectively improve the performance of the neural network, it is difficult to train. During the training process, it is difficult to distinguish whether progress is being made, and it is easy to crash, that is, the generator begins to degenerate and cannot continue to learn. At present, there is no efficient and stable solution in China.

[0004] In order to improve the robustness of the neural network, patent CN112329935A proposes an image generation method. First, an original image data set for training a neural network model is obtained, and a specified number of images are selected as the data set to be mutated. Second, important pixels that can affect the classification decision are screened out from the pixels of each image in the data set to be mutated. During the screening, all pixels of the corresponding images in the data set to be mutated are used as pixels to be screened. The linear coefficients of the output of each pixel to be screened in the propagation of the neural network model are obtained, and the influence parameter index for characterizing the influence of the pixel is calculated for each pixel to be screened using the linear coefficient. The important pixels are screened out according to the influence parameter index calculated, and a specified number of the important pixels are taken as target pixels. Third, the target pixels of each image in the data set to be mutated are mutated to generate a mutated data set. Finally, the original image data set and the mutated data set finally constitute a new image data set for training the neural network model. The method used by this technology to handle the variation of some data sets is not stable enough. In the second step, there is no specified standard for the selection of important pixels. When the selection of important pixels that affect the classification decision is inaccurate, the data set after the variation processing may not be much different from the original data set, making the effect unable to achieve the expected effect or even having no effect. Summary of the invention

[0005] The purpose of the present invention is to provide a dual-stream neural network deep learning method that combines gradient ascent and descent algorithms to address the problem that when adjusting parameters of existing deep learning methods, the parameters are always adjusted in the direction of reducing loss, and when the gradient drops to a certain range, it is easy to fall into a local optimal solution.

[0006] The dual-stream neural network deep learning method combining gradient ascent and descent algorithms provided by the present invention is designed as follows: in the training phase: the network first updates the parameter weights in reverse with the conventional gradient descent algorithm. After multiple rounds of training with such gradient descent flow, the gradient ascent flow is integrated when the evaluation index (loss / accuracy) of the network tends to be stable; the gradient ascent flow performs parameter updates (gradient ascent / gradient descent) according to the specified probability; then the results generated by the dual streams are integrated and input into the discriminator (classifier / target detector) until the training termination condition is reached. In the testing phase: load the weights of the gradient ascent flow and descent flow obtained by training, input test data, integrate the results generated by the dual streams and input them into the discriminator to obtain the prediction results.

[0007] The specific steps of this method are as follows:

[0008] S1. Add a gradient ascent flow to the original network. Split the original network model into a feature extraction part and a discriminator part; use the original feature extraction part as a gradient descent flow, and add a gradient ascent flow with a network structure consistent with the gradient descent flow.

[0009] S2. Modify the optimizer source code:

[0010] The mathematical expression of the gradient descent algorithm is:

[0011] θ1=θ0-α·▽J(θ)

[0012] The mathematical expression of the gradient ascent algorithm is:

[0013] θ1=θ0+α·▽J(θ)

[0014] In the formula, θ1 represents the updated value of the parameter to be learned in the deep network, θ0 represents the current value of the parameter to be learned, α represents the step size, and ▽J(θ) represents the partial differential of the loss function J with respect to the current parameter θ, that is, the gradient at the current position.

[0015] Modify the optimizer source code according to the difference between the gradient ascent algorithm and the descent algorithm, and add a function to update the network parameters using the gradient ascent algorithm.

[0016] S3. Start training the network. First, train the gradient descent flow for multiple rounds. At the end of each round, determine whether the conditions for dual-flow fusion are met. The judgment criteria are: when the evaluation indicators of the current network tend to be stable, that is, when the network no longer converges further, the conditions for integrating into the gradient ascent flow are met, and start training the gradient ascent flow; when training the dual streams for the first time, the initial parameter value of the gradient ascent flow is the parameter value of the current gradient descent flow.

[0017] The method for judging whether the dual-stream fusion conditions are met can be manual judgment or automatic judgment. Manual judgment means that after the original network is fully trained, the network indicators are observed to no longer change significantly after which iteration, that is, the network performance is no longer significantly improved. Then, the number of epochs for integrating the gradient ascent flow is specified in the new network. Automatic judgment means that the best indicators in the training process are recorded at the beginning of the training. When the training indicators of several consecutive epochs do not exceed the best indicators recorded, the conditions for integrating the gradient ascent flow are considered to have been met.

[0018] S4. Input the training results into the discriminator to obtain loss. If it is two-stream training, the input is the result of two-stream fusion.

[0019] The feature fusion method is to perform weighted summation of features or use convolution to fuse features.

[0020] The formula for the weighted sum of features is as follows:

[0021] y sum =x down *γ+x up *(1-γ)

[0022] In the formula, γ∈[0,1] represents the proportion of the results produced by the gradient descent flow in the fusion, and x down 、x up ,y sum ∈R H×W×D The larger γ is, the more the fusion result tends to be gradient descent flow. The key point of this method is that the sum of the weights of the two flows must be 1.

[0023] Using convolutional fusion features is to first concatenate the two feature tensors in terms of the number of channels, and then implement a set of filters through convolution to reduce the dimension by 2 times, that is,

[0024] y cat =f cat (x down ,x up )

[0025] y conv =y cat *f+b

[0026] Among them, y cat ∈R H×W×2D , f represents the filter, b represents the bias, y conv ∈R H×W×D ; The filter is implemented through a layer of convolution, the number of input channels is 2D, the number of output channels is D, the convolution kernel size is 1, and the bias term is set to True.

[0027] S5. Update parameter weights in reverse according to loss;

[0028] Both the gradient descent flow and the ascent flow have their own optimizers, and update the network parameters in their own way. In the gradient descent flow, the gradient descent algorithm is used to update the parameters each time; the gradient ascent flow selects the gradient descent algorithm or the gradient ascent algorithm under the specified probability. The probability selection can be realized by generating random numbers. In order to ensure that the random numbers are the same for each training, a random seed can be used. In this way, different parameter update methods are realized for the two flows. For the parameter update of the discriminator, the general gradient descent algorithm is still used for update.

[0029] S6, dual-stream training until the training termination condition is reached, such as reaching the maximum number of iterations;

[0030] S7. Start testing the network, download the dual-stream weights, use the same fusion method, and input the fusion result into the discriminator to obtain the prediction result.

[0031] Compared with the prior art, the present invention is beneficial in that:

[0032] The current deep learning methods are all based on gradient descent, and the consensus formed is that the smaller the error, the better, so the prior art uses the gradient descent method to reduce the error. The present invention designs a dual-stream neural network that combines a gradient ascent algorithm with a gradient descent algorithm. The present invention adds gradient ascent to have the following effects: First, it converges faster. When the number of iterations is small during training, the effect is significantly better than before the improvement; second, after the number of iterations increases and finally stabilizes, the recognition rate is better than the original one; third, in principle, it can help the existing model escape from the local minimum and fall into the problem of local minimum. This is a common problem in machine learning. If you fall into the local minimum and cannot escape, you cannot find the optimal solution.

[0033] The gradient ascent algorithm combined in the present invention avoids blindly adjusting the parameters of existing deep learning methods in the direction of reducing loss, avoids the problem of neural networks falling into local optimal solutions, increases the robustness of neural networks to a certain extent, and can be widely applied to any neural network structure, with good practical significance and application prospects.

[0034] The consensus in this field is that gradient ascent means training in the direction of increasing error. In principle, the effect of the model should be getting worse and worse, so gradient ascent is usually avoided as much as possible. The present invention innovatively adds gradient ascent, and in some cases there is a certain probability of triggering gradient ascent, which does not always rise, and achieves unexpected technical effects.

[0035] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 , some source code screenshots of the pytorch-sgd algorithm.

[0037] Figure 2 , explanation of the meaning of some parameters of the pytorch-sgd algorithm.

[0038] Figure 3 , the step_up method of the pytorch-sgd algorithm.

[0039] Figure 4 , schematic diagram showing an example of the judgment criteria for dual-stream fusion.

[0040] Figure 5 , training phase flow chart.

[0041] Figure 6, the results of the method of the present invention and the existing method in the Resnet34 network classification Cifar10 test (20 epochs).

[0042] Figure 7 , the results of the Cifar10 test (100 epochs) of the method of the present invention and the existing method in Resnet34 network classification.

[0043] Figure 8 , the results of the method of the present invention and the existing method in the Resnet34 network classification Cifar10 test (200 epochs). DETAILED DESCRIPTION

[0044] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0045] like Figure 1-8 As shown, the dual-stream neural network deep learning method provided by the present invention combines the gradient ascent and descent algorithms, and the specific steps are as follows:

[0046] S1. Modify the original network model and add gradient ascent flow to the original network.

[0047] This method is not limited to a specific network and can be combined with any existing network. The original network model is split into a feature extraction part and a discriminator (classifier / target detector) part; the original feature extraction part after the split is used as a gradient descent flow as a whole, and a gradient ascent flow is added. The basic structure of the model is consistent with the gradient descent flow, but it has an independent optimizer.

[0048] S2. Modify the optimizer source code:

[0049] The mathematical expression of the gradient descent algorithm is:

[0050] θ1=θ0-α·▽J(θ) (1)

[0051] The mathematical expression of the gradient ascent algorithm is:

[0052] θ1=θ0+α·▽J(θ) (2)

[0053] In the formula, θ1 represents the updated value of the parameter to be learned in the deep network, θ0 represents the current value of the parameter to be learned, α represents the step size, and ▽J(θ) represents the partial differential of the loss function J with respect to the current parameter θ, that is, the gradient at the current position.

[0054] The difference between the two algorithms lies in the updates along different directions of the gradient, one updates along the direction of gradient descent, and the other updates along the direction of gradient ascent.

[0055] There are three main steps to implement reverse parameter update of neural network in Pytorch framework:

[0056] Step 1: optimizer.zero_grad(), to achieve gradient zeroing, that is, the gradient record of the previous batch is cleared;

[0057] Step 2: loss.backward(), back propagation calculates the gradient value of each parameter and stores it in the grad attribute of the tensor;

[0058] Step 3: optimizer.step() performs a step parameter update via gradient descent.

[0059] By viewing the optimizer's initialization code, you can know the specific gradient descent algorithm used by the original network, such as whether it is the stochastic gradient descent method SGD or the Adam gradient descent method. Common gradient descent algorithms are generally provided by third-party packages. For example, if you use the SGD algorithm under the Pytorch framework, the parameter update step method is in the "optim / sgd.py" file under the torch package installation path.

[0060] Figure 1 The source code of the parameter update step method of the gradient descent SGD algorithm in the Pytorch framework is shown. There are four boxes. The code of box a is mainly to obtain some parameters. These parameters are mainly used in different ways to calculate the gradient. The specific meanings are as follows Figure 2 As shown. Box b indicates that the gradient d_p is obtained from the grad attribute of tensor p. Box c indicates that the gradient d_p is recalculated in different ways according to the input parameters. The statements in box d are the core statements for implementing the gradient descent algorithm, and their meanings are:

[0061] Tensor p = tensor p - gradient d_p * learning rate lr (3)

[0062] Satisfying the formula (1) of the above gradient descent algorithm, modifying the above statement to p.data.add_(group['lr'],d_p) means:

[0063] Tensor p = tensor p + gradient d_p * learning rate lr (4)

[0064] The formula (2) of the gradient ascent algorithm is satisfied, so a step_up method that updates the parameters using the gradient ascent algorithm can be added to the file, as follows: Figure 3 shown.

[0065] S3. Start training the network. First, train the gradient descent flow for multiple rounds. At the end of each round, determine whether the conditions for dual-flow fusion are met. The judgment criteria are: when the evaluation indicators (loss / accuracy) of the current network tend to be stable, that is, when the network no longer converges further, the conditions for integrating into the gradient ascent flow are met, and start training the gradient ascent flow. When training the dual flow for the first time, the initial parameter value of the gradient ascent flow is the parameter value of the current gradient descent flow. Specific judgment criteria are given as examples: Figure 4 As shown, Figure 4 It is a graph drawn based on the data. The horizontal axis is the number of iterations, and the vertical axis is the accuracy. When the accuracy no longer increases significantly, the curve gradually tends to be horizontal, accompanied by small fluctuations, that is, the evaluation index (loss / accuracy) tends to be stable (the position indicated by the arrow in the figure). Therefore, the timing of incorporating the gradient ascent flow can be judged based on the changes in the evaluation index. For example, if the highest accuracy that the model can currently achieve is 90%, then during the training process, when the accuracy is close to 90%, the accuracy curve will gradually become horizontal, that is, the number of iterations when the model first reaches the highest accuracy can be selected as the timing of incorporating the gradient ascent flow.

[0066] The method for judging whether the dual-stream fusion conditions are met can be manual judgment or automatic judgment. Manual judgment means that after the original network is fully trained, the network indicators are observed to no longer change significantly after which iteration, that is, the network performance is no longer significantly improved. Then, the number of epochs for integrating the gradient ascent flow is specified in the new network. Automatic judgment means that the best indicators in the training process are recorded at the beginning of the training. When the training indicators of several consecutive epochs do not exceed the best indicators recorded, the conditions for integrating the gradient ascent flow are considered to have been met.

[0067] S4. Input the training results into the discriminator to obtain loss. If it is two-stream training, the input is the result of two-stream fusion.

[0068] The feature fusion method is to perform weighted summation of features or use convolution to fuse features.

[0069] The formula for the weighted sum of features is as follows:

[0070] y sum =x down *γ+x up *(1-γ) (5)

[0071] In the formula, γ∈[0,1] represents the proportion of the results produced by the gradient descent flow in the fusion, and x down 、x up ,y sum ∈R H×W×DThe larger γ is, the more the fusion result tends to be gradient descent flow. The key point of this method is that the sum of the weights of the two flows must be 1.

[0072] Using convolutional fusion features is to first concatenate the two feature tensors in terms of the number of channels, and then implement a set of filters through convolution to reduce the dimension by 2 times, that is,

[0073] y cat =f cat (x down ,x up ) (6)

[0074] y conv =y cat *f+b (7)

[0075] Among them, y cat ∈R H×W×2D , f represents the filter, b represents the bias, y conv ∈R H×W×D ; The filter is implemented through a layer of convolution, the number of input channels is 2D, the number of output channels is D, the convolution kernel size is 1, and the bias term is set to True.

[0076] S5. Update parameter weights in reverse according to loss;

[0077] Both the gradient descent flow and the ascent flow have their own optimizers, and update the network parameters in their own way. In the gradient descent flow, the gradient descent algorithm is used to update the parameters each time; the gradient ascent flow selects the gradient descent algorithm or the gradient ascent algorithm under the specified probability. The probability selection can be realized by generating random numbers. In order to ensure that the random numbers are the same for each training, a random seed can be used. In this way, different parameter update methods are realized for the two flows. For the parameter update of the discriminator, the general gradient descent algorithm is still used for update.

[0078] S6: Dual-stream training until the training termination condition is reached, such as reaching the maximum number of iterations. The entire training process is as follows: Figure 5 shown.

[0079] S7. Start testing the network, download the dual-stream weights, use the same fusion method, and input the fusion result into the discriminator to obtain the prediction result.

[0080] The method of the present invention was tested on the instance Resnet34 network classification Cifar10. Figure 6-8 Some test results are shown. Figure 6 This is the result of the Cifar10 test (20 epochs) of the method of the present invention and the existing method in Resnet34 network classification. Figure 7This is the test result (100 epochs) of the method of the present invention and the existing method in the Resnet34 network classification Cifar10. Figure 8 The results of the Cifar10 test (200 epochs) of the method of the present invention and the existing method in the Resnet34 network classification. Among them, (a), (b), and (c) are the test results of the method of the present invention, the test results of the existing method, and the comparison diagram of the effects of the two methods. It can be seen that the method of the present invention is to set the gradient ascent flow to use the gradient ascent algorithm to reversely update the parameters with a probability of 0.3, and the gradient descent flow ratio is 0.8 and the gradient ascent flow ratio is 0.2 during feature weighted fusion.

[0081] Through comparative experiments, it can be found that the method of the present invention converges faster than the original solution most of the time and can reach stability with fewer iterations; at the same time, the method of the present invention significantly improves the final classification accuracy, further illustrating that the method can find a way out of the local optimal solution and find a better solution.

[0082] In summary, the present invention designs a dual-stream neural network that combines a gradient ascent algorithm with a gradient descent algorithm. The gradient ascent algorithm combined in this scheme avoids blindly adjusting the parameters of existing deep learning methods in the direction of reducing loss, avoids the problem of the neural network falling into a local optimal solution, increases the robustness of the neural network to a certain extent, and can be widely applied to any neural network structure, with good practical significance and application prospects.

[0083] The above description is only a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A two-stream neural network deep learning method combining gradient ascent and descent algorithms, characterized in that: This method is used for image classification and the steps are as follows: S1. Add gradient upflow to the original network; S2. Modify the optimizer source code; S3, start training the network, first train the gradient descent flow for multiple rounds, and judge whether the conditions for dual-flow fusion are met at the end of each round. The judgment criteria are: when the evaluation index of the current network tends to be stable, that is, when the network no longer converges further, the conditions for integrating into the gradient ascending flow are met, and start training the gradient ascending flow; when training the dual flow for the first time, the initial parameter value of the gradient ascending flow is the parameter value of the current gradient descending flow; S4. Input the training result into the discriminator to get the loss. If it is a two-stream training, the input is the result after the two-stream fusion. S5. Update parameter weights in reverse according to loss; S6, dual-stream training until the training termination condition is reached; S7. Start testing the network, download the dual-stream weights, use the same fusion method, and input the fusion result into the discriminator to obtain the prediction result.

2. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 1, characterized in that: In step S3, the method for judging whether the dual-stream fusion condition is met adopts manual judgment. Manual judgment means observing the evaluation index of the network after which the evaluation index no longer changes significantly after the original network is fully trained, that is, the network performance no longer improves significantly. Then, the number of epochs of the gradient upflow is specified in the new network.

3. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 1, characterized in that: In step S3, the method for judging whether the dual-stream fusion condition is met adopts an automatic judgment method. Automatic judgment means recording the best indicators in the training process at the beginning of training. When the training indicators of several consecutive epochs do not exceed the recorded best indicators, it is considered that the conditions for integrating into the gradient upflow are met.

4. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 1, characterized in that: The step S1 is: splitting the original network model into a feature extraction part and a discriminator part; taking the original feature extraction part after splitting as a whole as a gradient descent flow, and adding a gradient ascent flow whose network structure is consistent with the gradient descent flow.

5. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 1, characterized in that: In step S2, the optimizer source code is modified, and the mathematical expression of the gradient descent algorithm is: The mathematical expression of the gradient ascent algorithm is: In the formula, θ1 represents the updated value of the parameter to be learned in the deep network, θ0 represents the current value of the parameter to be learned, and α represents the step size. Represents the partial differential of the loss function J with respect to the current parameter θ, that is, the gradient at the current position; Modify the optimizer source code according to the difference between the gradient ascent algorithm and the descent algorithm, and add a function to update the network parameters using the gradient ascent algorithm.

6. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 1, characterized in that: In step S4, the feature fusion method is feature weighted summation or convolution fusion feature.

7. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 6 is characterized in that the feature The formula for weighted summation is as follows: y sum =x down *c+x up *(1-c) In the formula, γ∈[0,1] represents the proportion of the results produced by the gradient descent flow in the fusion. x down 、x up ,y sum ∈R H×W×D The larger γ is, the more the fusion result tends to be towards the gradient descent flow. The key point of this method is that the sum of the weights of the two flows must be 1.

8. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 6, characterized in that: Using convolutional fusion features is to first concatenate the two feature tensors in terms of the number of channels, and then implement a set of filters through convolution to reduce the dimension by 2 times, that is, y cat =f cat (x down ,x up ) y conv =y cat *f+b Among them, y cat ∈R H×W×2D , f represents the filter, b represents the bias, y conv ∈R H×W×D ; The filter is implemented through a layer of convolution, the number of input channels is 2D, the number of output channels is D, the convolution kernel size is 1, and the bias term is set to True.

9. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 1, characterized in that: In step S5, the gradient descent flow and the ascent flow are both set with their own optimizers to update the network parameters in their own ways; the gradient descent algorithm is used to update the parameters each time in the gradient descent flow; the gradient ascent flow selects the gradient descent algorithm or the gradient ascent algorithm under the specified probability.

10. The dual-stream neural network deep learning method combining gradient ascent and descent algorithms as claimed in claim 9, characterized in that: The parameters of the discriminator are updated using the gradient descent algorithm.