Image recognition method of confrontation domain self-adaption and conjugate gradient kernel extreme learning machine

Abstract

The invention discloses an image recognition method for confrontation domain self-adaption and a conjugate gradient kernel extreme learning machine. The method is characterized by comprising the following steps: firstly, constructing a feature extractor to obtain initial features of a source domain and a target domain; and then constructing an adversarial domain adaptive network to optimize two domain features, enabling a source domain feature extractor and a target domain feature extractor to play a game with a discriminator so as to reduce inter-domain differences between a target domain and a source domain, projecting the inter-domain differences to a common feature space, and finding an optimal target domain feature. And finally, constructing a classifier to recognize the image, and constructing a conjugate gradient kernel extreme learning machine as a classifier to classify the optimal target domain features to obtain a classification result. The invention aims to provide a lightweight network, namely an adversarial domain adaptive and conjugate gradient kernel extreme learning machine, which solves the problem of dependence of existing deep learning on a large amount of annotation data, reduces sample annotation cost in image recognition, reduces hardware implementation cost, realizes cross-domain knowledge migration, and completes unsupervised and high-precision image recognition of a target domain.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition and domain adaptive learning, and relates to an image recognition method against domain adaptive and conjugate gradient kernel extreme learning machines. Background Art

[0002] Deep learning has become a research hotspot in machine learning [1][2] , deep learning, especially deep convolutional neural networks, can automatically learn to extract rich features from image big data, and its research results have been widely applied to various fields. Although deep learning methods have achieved great results, they have the following problems: 1) Deep learning requires a large amount of labeled training sample data, and many domain experts are very expensive to label, and some labeled samples are scarce; 2) Deep learning requires a large amount of memory and computing resources, otherwise the training process will be extremely time-consuming; 3) Training deep neural networks often becomes very complicated due to overfitting and convergence problems, and it is often necessary to repeatedly adjust the learning parameters in the framework or network. The high performance of deep learning is closely related to a large number of labeled samples. The high requirements for the hardware configuration lead to high hardware costs. At the same time, the large amount of labeled data requires a high cost of labeling. Therefore, how to use the existing labeled domain data knowledge to label the unlabeled domain data for classification is of great significance in various application fields.

[0003] Domain Adaptation is one of the main methods to solve the above domain knowledge migration. It can overcome the differences in data distribution and label space between the two domains, migrate the labeled source domain knowledge to the unlabeled target domain, and complete the learning task of the target domain. [3] ) can learn a model with good performance on the source domain data, adapt to the target domain data with different distribution or even different category label space, and solve the learning task of the target domain data with no labeled samples or a small number of labeled samples. At present, domain adaptation can be roughly divided into two categories. One method is based on statistical moment alignment, which uses the distance function to minimize the distribution difference between the source domain and the target domain, including the maximum mean deviation (MMD) [4] , Center moment deviation (CMD) [5] and second-order statistical alignment. The other is an adversarial approach, which shortens the distance between the two domains through adversarial training and obtains effective cross-domain features. Existing adversarial domain adaptation methods are mainly divided into three categories: [3] : Gradient reversal-based methods, minimax optimization-based methods, and generative adversarial network-based methods. The first two methods minimize the domain distribution difference through feature-level domain confusion supervised by a domain discriminator, and the latter method achieves pixel-level domain transfer by synthesizing indistinguishable target domain images.

[0004] In order to solve the above problems of deep learning, an unsupervised domain adaptive image recognition method is proposed: adversarial domain adaptation and conjugate gradient kernel extreme learning machine, which can not only solve the problem of insufficient image annotation samples in image recognition applications, but also realize cross-domain knowledge transfer and high-precision image classification.

[0005] [1] K.He,

[0006] [2] C. Szegedy, S. Ioffe, V. Vanhoucke, A. Alemi. Inception-v4, inception-resNet and the impact of residual connections on learning, in: AAAI Conference on Artificial Intelligence, 2016.

[0007] [3] L. Zhang,

[0008] [4]Tzeng, E., Hoffman, J., Zhang, et al. Deep domain confusion: Maximizing for domain invariance[J]. Computing Research Repository, 2014.

[0009] [5]Zellinger W, Grubinger T, Lughofer, E, et al. Central momentdiscrepancy (CMD) for domain-invariant representation learning [C]. International Conference on Learning Representations, 2017. Summary of the invention

[0010] The purpose of the present invention is to provide a lightweight network: adversarial domain adaptation and conjugate gradient kernel extreme learning machine, which solves the dependence of existing deep learning on a large amount of labeled data, reduces the sample labeling cost in image recognition, reduces the hardware implementation cost, realizes cross-domain knowledge transfer, and completes unsupervised and high-precision image recognition in the target domain.

[0011] The technical solution adopted by the present invention is

[0012] The image recognition method of adversarial domain adaptation and conjugate gradient kernel extreme learning machine is characterized in that a feature extractor is first constructed to obtain preliminary features of the source domain and the target domain; then an adversarial domain adaptation network is constructed to optimize the features of the two domains, and the source domain and target domain feature extractors and discriminators are allowed to compete with each other to reduce the domain difference between the target domain and the source domain, and project them into a common feature space to find the optimal target domain features. Finally, a classifier is constructed to recognize the image, and a conjugate gradient kernel extreme learning machine is constructed as a classifier to classify the target domain features and obtain the classification results.

[0013] The beneficial effects of the present invention are

[0014] 1. In adversarial domain adaptation and conjugate gradient kernel extreme learning machine, a manifold adversarial domain adaptation model is designed to transfer labeled source domain knowledge to unlabeled target domain data, achieving highly cross-domain knowledge transfer.

[0015] 2. Adversarial domain adaptation and conjugate gradient kernel extreme learning machines can achieve high-precision unsupervised image recognition when there are no labeled samples for image recognition.

[0016] 3. Adversarial domain adaptation and conjugate gradient kernel extreme learning machine is a lightweight network. The conjugate gradient kernel extreme learning machine classifier is designed to achieve cross-domain knowledge transfer and high-precision image recognition classification at a low hardware cost (even without a GPU). BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is the overall structural framework of the method of the present invention;

[0018] like Figure 1 As shown, X S is the source data, X T is the target data. S ,F T are the feature extractors of the source domain and the target domain respectively. S ,L S F S The node numbers of the input layer and hidden layer. T ,L T F T The node numbers of the input layer and hidden layer. S ,b S F S The weights and biases of T ,b T F T The weights and biases of H(X S ),H(X T ) are the feature maps of the source domain and the target domain, respectively, as shown in the blue part of the figure. The orange part is the adversarial domain adaptation network. T ) is the feature map of the adversarial domain adaptation network after the loss reaches the minimum value. S is the label of the source data. T is the label of the target data. DETAILED DESCRIPTION

[0019] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] The whole process of the method is divided into three stages: constructing a feature extractor to obtain preliminary features of the source domain and the target domain, constructing an adversarial domain adaptive network to optimize the features of the two domains, and constructing a classifier to recognize images.

[0021] First, we construct a feature extractor to obtain the preliminary features of the source domain and the target domain. We design the feature extractors F for the source domain and the target domain respectively. S ,F T , the two feature extractors have similar structures. The feature extractor consists of a feedforward neural network with one hidden layer. Assuming that the number of input layers of the feature extractor is n, the number of hidden layers of the feature extractor node is L, the activation function of the hidden layer in the feature extractor F is:

[0022] H=G(a ij ,b j ,x i ),a ij ∈R n×L ,b j ∈R 1×L (1)

[0023] Where G(aij ,b j ,x i ) is a nonlinear function, a ij and b j are weight and bias respectively.

[0024] Hidden layer output H∈R n×L Defined as:

[0025]

[0026] Then in the second step, we build an adversarial domain adaptation network to optimize the features of the two domains. In this stage, the target domain feature extractor F T Continuously updated, while the source domain feature extractor F S The feature extraction node remains unchanged and is not affected by the input data. Therefore, the optimized feature mapping of the source domain and the target domain can be learned. By making the feature extractor and the discriminator continuously conduct adversarial learning, the source domain data and the target domain data can be projected into a common feature space, and the domain difference between the data sets is continuously reduced. Finally, when the adversarial loss function reaches the minimum value, the model can learn an optimal feature space, and at this time, the optimal feature H′(X T ). Among them, the discriminator loss is defined as:

[0027]

[0028] The training loss function of the target domain in this process is defined as:

[0029]

[0030] The final third step is to build a classifier: the conjugate gradient kernel extreme learning machine recognizes images. This step at least includes the following.

[0031] (1) Input source domain X S , select the linear kernel function K(μ,υ)=μ·υ as the kernel function of the conjugate gradient kernel extreme learning machine, determine the conjugate gradient method accuracy error value ε, penalty coefficient λ according to actual conditions, and the activation function is g(x).

[0032] (2) Calculate the kernel function Ω of the source domain data S as follows.

[0033]

[0034] (3) Use the conjugate gradient method to calculate the output weight β of the conjugate gradient kernel extreme learning machine ADAELM , where ε=1,n=1

[0035] (4) Input target domain X T, select the linear kernel function K(μ,υ)=μ·v as the kernel function of the conjugate gradient kernel extreme learning machine, determine the conjugate gradient method accuracy error value ε, penalty coefficient λ according to actual conditions, and the activation function is g(x).

[0036] (5) Calculate the kernel function Ω of the target domain data T As follows, where the kernel parameter is 1.

[0037] Ω T =F T ′(X T )F T ′(X T ) T ,X T =[x1 T ,x2 T ,…,x n T ] T (6)

[0038] (6) Classify and predict the target domain data, and get the predicted label:

[0039] Y T =Ω T β ADAELM (7)

[0040] Finally, the output of the adversarial domain adaptation and conjugate gradient kernel extreme learning machine is:

[0041]

[0042] Where h(x i )=[h1(x i )…h L (x i )] is the hidden layer output,

[0043] The classification loss function of this part is defined as:

[0044]

[0045] In the whole method, the loss function mainly consists of three parts. The first is the loss caused by adversarial adaptation, the second is the loss of target model training, and the last is the classification loss caused by the classifier. Therefore, the total loss function is defined as follows.

[0046]

Claims

1. An image recognition method based on adversarial domain adaptation and conjugate gradient kernel extreme learning machine, characterized in that First, construct a feature extractor to obtain the preliminary features of the source domain and the target domain; design the feature extractors F for the source domain and the target domain respectively. S ,F T , the two feature extractors have similar structures. The feature extractor consists of a feedforward neural network with one hidden layer. Assuming that the number of input layers of the feature extractor is n, the number of hidden layers of the feature extractor node is L, the activation function of the hidden layer in the feature extractor F is: H=G(a ij ,b j ,x i ),a ij ∈R n×L ,b j ∈R 1×L (1) Where G(a ij ,b j ,x i ) is a nonlinear function, a ij and b j They are weight and bias respectively; Hidden layer output H∈R n×L Defined as:

2. The image recognition method based on adversarial domain adaptation and conjugate gradient kernel extreme learning machine is characterized in that the adversarial domain adaptation network is constructed to optimize the two domain features; at this stage, the target domain feature extractor F T Continuously updated, while the source domain feature extractor F S The feature extraction node remains unchanged and is not affected by the input data. Therefore, the optimized feature mapping of the source domain and the target domain can be learned. By making the feature extractor and the discriminator continuously conduct adversarial learning, the source domain data and the target domain data can be projected into a common feature space, and the domain difference between the data sets is continuously reduced. Finally, when the adversarial loss function reaches the minimum value, the model can learn an optimal feature space, and at this time, the optimal feature H′(X T ), where the discriminator loss is defined as: The training loss function of the target domain in this process is defined as:

3. A tumor diagnosis method based on manifold adversarial domain adaptation, characterized by Finally, a classifier is constructed: the conjugate gradient kernel extreme learning machine recognizes the image. This step at least includes the following: (1) Input source domain X S , select the linear kernel function K(μ,v)=μ·v as the kernel function of the conjugate gradient kernel extreme learning machine, determine the conjugate gradient method accuracy error value ε, penalty coefficient λ according to actual conditions, and the activation function is g(x); (2) Calculate the kernel function Ω of the source domain data S as follows: (3) Use the conjugate gradient method to calculate the output weight β of the conjugate gradient kernel extreme learning machine ADAELM , where ε=1, n=1; (4) Input target domain X T , select the linear kernel function K(μ,v)=μ·v as the kernel function of the conjugate gradient kernel extreme learning machine, determine the conjugate gradient method accuracy error value ε, penalty coefficient λ according to actual conditions, and the activation function is g(x); (5) Calculate the kernel function Ω of the target domain data T As follows, where the kernel parameter is 1; Ω T =F T ′(X T )F T ′(X T ) T ,X T [x1 T ,x2 T ,…,x n T ] T (6) (6) Classify and predict the target domain data, and get the predicted label: Y T =Oh T b ADAELM (7) Finally, the output of the adversarial domain adaptation and conjugate gradient kernel extreme learning machine is: Where h(x i )=[h1(x i )…h L (x i )] is the hidden layer output, The classification loss function of this part is defined as: In the whole method, the loss function mainly consists of three parts: the first is the loss caused by adversarial adaptation, the second is the loss of target model training, and the last is the classification loss caused by the classifier. Therefore, the total loss function is defined as follows:

Images