Unlock instant, AI-driven research and patent intelligence for your innovation.

Regularization-based method for solving problem of gradient disappearance of adversarial residual transform network

A network and residual technology, applied in instruments, character and pattern recognition, computing models, etc., can solve the problem of anti-residual transformation network gradient disappearance, and achieve the effect of reducing adverse effects, easy training, and ensuring stability

Inactive Publication Date: 2020-05-22
TONGJI UNIV
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a method based on regularization to solve the problem of gradient disappearance of the residual transformation network in order to overcome the defect that the gradient disappearance in the above-mentioned prior art has a great influence on the optimization of model parameters

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Regularization-based method for solving problem of gradient disappearance of adversarial residual transform network
  • Regularization-based method for solving problem of gradient disappearance of adversarial residual transform network
  • Regularization-based method for solving problem of gradient disappearance of adversarial residual transform network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0075] Set the regularization parameter β=0 of the anti-residual network model (ARTN) on Office-31, that is, no regularization. Except for the regularization term, the experimental parameters have the same settings as the Bayesian image classification experiments. Experimental results such as figure 2 As shown, in D→A, W→A, A→W and A→D, the accuracy of the unregularized model is 59.5%, 59.8%, 76.0% and 75.9%, respectively, which is lower than that of the regularized model 1.4%, 1.2%, 0.2%, and 0.2%. The model with regularization is superior to DANN and the model without regularization in all tasks, indicating that regularization has high effectiveness, that is, the regularization term strengthens the generalization ability of the model.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a regularization-based method for solving a problem of gradient disappearance of an adversarial residual transform network. The method comprises the following steps that S1, afeature extractor extracts feature distribution of source data of a source domain and target data of a target domain; S2, a conversion network maps the feature distribution of the source data to a target domain to generate forged target data; S3, a label classifier is established for forged target data through training, and the label value of the target data is predicted through the label classifier; S4, a domain classifier compares the forged target data with the target data, a result is fed back to a feature extractor, the feature extractor improves an extraction process according to the result, and mapping of network optimization source data to a target domain is converted; S5, the steps S1-S4 are executed circularly until the value of a loss function of the adversarial residual errortransformation network model converges, and a regularization item is set in the loss function. Compared with the prior art, the method has the advantages that the influence of gradient disappearance on model parameter optimization is reduced, and the stability of the model is improved.

Description

technical field [0001] The invention relates to the field of computer machine learning, in particular to a regularization-based method for solving the gradient disappearance problem of an adversarial residual transformation network. Background technique [0002] Deep neural networks trained on large-scale labeled datasets achieve excellent performance in different tasks, however, since the transferability of feature distributions decreases with the distance between the base task and the target task, domain adaptation ( DA) to solve this problem, can leverage the knowledge learned in one specific domain to effectively improve the performance in related but different domains. Early methods of DA aimed at learning domain-invariant features, representing data by jointly minimizing the distance, and measuring the fitness index between a pair of source and target domains. In recent years, the concept of Generative Adversarial Networks (GANS) has been widely used, The application ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06N20/00G06K9/62
CPCG06N20/00G06F18/241
Inventor 何良华蔡冠羽
Owner TONGJI UNIV