Zero sample learning method based on stream generation and non-redundancy features

Abstract

The invention relates to a zero sample learning method based on generated streams and non-redundancy features, which is characterized in that firstly, the generated streams are used for generating zero sample learning visual features, and the generated streams can effectively relieve the feature confusion problem; then, a mapping network M is trained through mutual information to learn non-redundancy features, redundant information is removed from the visual features through the mapping network, and discrimination information cannot be lost; and compared with other de-entanglement models, the mapping network is simple in structure, and the network optimization time can be further shortened. The invention aims to provide a zero sample learning method based on a generated stream and non-redundancy features, and solves the problem that features extracted by a backbone network have no good correlation with semantic attributes due to the fact that most existing ZSL methods use the deep backbone network. Meanwhile, the network structure is further optimized, so that the model is relatively simple and stable and is easy to train.

Description

Technical Field The invention belongs to the fields of zero-sample learning, transfer learning and image recognition, and relates to a zero-sample learning method for generating flow and non-redundant features. Background Art Most machine learning tasks usually require a large amount of labeled data, which is very expensive, time-consuming, and even impossible to obtain. Therefore, we propose Zero-shot Learning (ZSL) to solve these problems. In ZSL, the visual features of unseen classes are not available during the training phase, but the visual features of unseen classes need to be identified during the testing phase. ZSL is a recent [1-3] It is a topic of extensive research and rapid development. Most ZSL methods use deep backbone networks such as ResNet101 or Transformer to extract features in images. These networks are usually pre-trained on the ImageNet dataset and are not specifically designed for ZSL. Compared with traditional image recognition, ZSL adds semantic attributes to each class. The features extracted by the backbone network are not well correlated with the semantic attributes. In addition, the manually annotated semantic attributes inevitably contain redundant information, which also affects the visual and semantic alignment. DLFZR [4] First, feature disentanglement was introduced in ZSL, pointing out that each dimension of the features extracted by the deep backbone network does not correspond to a semantic attribute one-to-one. Therefore, DLFZRL decomposes image features into three potential representations: semantic features, non-semantic features, and non-discriminative features, and uses semantic features for classification. The paper [5] believes that non-semantic visual features are completely irrelevant to semantic attributes, and it is difficult to transfer the feature extraction function from the visible class to the unseen class, so the SDGZSL method is proposed. [5] The DLFZRL was improved by only decomposing the original features into two semantically consistent and semantically irrelevant latent representations, and achieved better performance. However, the semantic disentanglement network they designed was too complex, resulting in unstable training. Therefore, a zero-shot learning method based on generative flow and non-redundant features is proposed here. The network used to learn non-redundant features in this method is very simple, very easy to learn, and reduces the complexity of the model, saving memory space.

[0001] FEYang,YHLee,CCLin and YCFWang.Semantics-guided intra-category knowledge transfer for generalized zero-shot learning.InternationalJournal of Computer Vision,2023,pp.1331-1345.

[0002] Y. Zhang, Y. Tian, S. Zhang and Y. Huang. Dual-uncertainty Guided Cycle-consistent Network for Zero-shot Learning. IEEE Transactions on Circuits and Systems for Video Technology, 2023.

[0003] Z. Liu, Y. Li, L. Yao, X. Wang and G. Long. Task aligned generative meta-learning for zero-shot learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, pp. 8723 - 8731.

[0004] B. Tong, C. Wang, M. Klinkigt, Y. Kobayashi and Y. Nonaka. Hierarchical Disentanglement of Discriminative Latent Features for Zero-Shot Learning. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 11467 - 11476.

[0005] Z. Chen, Y. Luo, S. Wang, R. Qiu, J. Li, Z. Zhang and Z. Huang. Semantics Disentangling for Generalized Zero-Shot Learning. Proceedings of the IEEE / CVF International Conference on Computer Vision, 2021, pp. 8712 - 8720. Summary of the Invention The purpose of the present invention is to provide a zero-shot learning method based on generative flow and non-redundant features, which solves the problem that most existing ZSL methods use deep backbone networks, resulting in poor correlation between the features extracted by the backbone network and the semantic attributes. At the same time, the network structure is further optimized to make the model relatively simple, stable and easy to train. The technical solution adopted by the present invention is The zero-shot learning method based on generative flow and non-redundant features is characterized in that the generative flow is first used to generate zero-shot learning visual features, and the generative flow can effectively alleviate the feature confusion problem. Then a mapping network M is trained to learn non-redundant features using mutual information. The mapping network removes redundant information from visual features without losing discriminant information. Compared with other disentanglement models, the mapping network has a simple structure and can further reduce the time of optimizing the network. The beneficial effects of the present invention are 1. The new algorithm uses generative flow to effectively alleviate the feature confusion problem. 2. Use mutual information to train a mapping network to learn non-redundant features. 3. Compared with the disentanglement model, the mapping network structure of this method is simple, which further reduces the time for optimizing the network. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 It is the network framework diagram of the method; Figure 2 It is the conditional affine coupling layer structure of the generated flow, where the left picture shows the structure when the generated flow undergoes forward transformation, and the right picture shows the structure when the generated flow undergoes inverse transformation. DETAILED DESCRIPTION The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The network framework of the method of the present invention is as follows Figure 2 As shown, the specific implementation steps of the method are as follows: Step 1: Train the generative flow network and construct the forward and reverse processes of the generative flow. The formula is as follows: z=f(x;a,θ),z~N(0,1),(1) Among them, x represents the original visual features, a represents the semantic features, and θ represents the network parameters. Represents the generated visual features, z is sampled from a standard Gaussian distribution. The forward process can be trained using the maximum log-likelihood method: in, represents the logarithm of the determinant of the Jacobian matrix, p X and p Zdenote the distribution of visual feature x and the standard Gaussian distribution respectively. According to Bayes’ theorem, the posterior distribution is expressed as p X (θ; a d ,c)∝p X (x; a d ,θ)·p θ (θ). Finally, the generative flow loss is expressed as: L flow =E[-logp X (x; a d ,θ)]-logp θ (θ).(4) Step 2: Learn the mapping function M to map the real visual features of the visible class and the synthetic visual features of the invisible class to a new non-redundant feature space. In order to exploit the non-redundant information in the semantic embedding method, we simply regard the semantic descriptor space as a new feature space and require the function M to map the original visual features into the semantic descriptor space. Therefore, we only need to constrain the structured objective with bounded variational mutual information, resulting in the total loss of the network as follows: Where b is the upper limit of the increase.

Claims

1. A zero-shot learning method based on generative flow and non-redundant features, characterized by First, use the generation flow to generate zero-shot learning visual features. The generation flow can effectively alleviate the problem of feature confusion. The general process of this step is as follows: Step 1: Train the generative flow network and construct the forward and reverse processes of the generative flow. The formula is as follows: z=f(x;a,θ),z~N(0,1), (1) Among them, x represents the original visual features, a represents the semantic features, and θ represents the network parameters. Represents the generated visual features, z is sampled from a standard Gaussian distribution; The forward process can be trained using the maximum log-likelihood method: in, represents the logarithm of the determinant of the Jacobian matrix, p X and p Z represent the distribution of visual feature x and the standard Gaussian distribution respectively; according to Bayes’ theorem, the posterior distribution is expressed as p X (θ; a d ,c)∝p X (x; a d ,θ)·p θ (θ); Finally, the generated flow loss is expressed as: L flow =E[-log p X (x;a d ,θ)]-log p θ (i)。 (4) 2. A zero-shot learning method based on generative flow and non-redundant features, characterized by Then, the mapping network M is trained by mutual information learning to map the real visual features of the visible class and the synthetic visual features of the invisible class to a new non-redundant feature space. The mapping network removes redundant information from the visual features without losing discriminant information. Compared with other disentanglement models, the mapping network has a simple structure and can further reduce the time of optimizing the network. The general process of this step is as follows: In order to exploit the non-redundant information in the semantic embedding method, we simply regard the semantic descriptor space as a new feature space and require the function M to map the original visual features into the semantic descriptor space. Therefore, we only need to constrain the structured objective with bounded variational mutual information, resulting in the total loss of the network as follows: s.t.E p(x,a) [D KL [p M (z|x)||r(z)]]≤b, (5) Where b is the upper limit of the increase.

Images