Zero-sample visual classification method for cross-modal semantic enhancement generative adversarial network

Abstract

The invention discloses a zero-sample visual classification method for a cross-modal semantic enhancement generative adversarial network. The method comprises the following steps: firstly downloading a plurality of sample images and a tag and semantic features corresponding to each sample image, and extracting the visual features of each sample image through a residual neural network; and then constructing and training a cross-modal semantic enhanced generative adversarial network, and training a cross-modal feature classifier based on zero sample classification based on the generative adversarial network, thereby completing visual classification of the to-be-detected image.

Description

technical field [0001] The invention belongs to the technical field of zero-sample learning, and more specifically relates to a zero-sample visual classification method of a cross-modal semantically enhanced generative confrontation network. Background technique [0002] The main goal of zero-shot learning is to train the model with visual information of some visible categories and other auxiliary information (such as text descriptions), so that the learned model can correctly correct the visual information of invisible categories. Classification. Zero-shot learning can be divided into two sub-tasks: Conventional Zero-shot Learning and Generalized Zero-shot Learning. Traditional zero-shot learning only requires the model to complete the classification of invisible categories. Zero-shot learning requires the model to recognize both visible and unseen categories. A typical type of method in the existing methods is based on the embedding space method, which maps auxiliary inf...

AI Technical Summary

Problems solved by technology

[0003] However, the existing technology has the following objective shortcomings: 1. Embedding-based methods measure the similarity between visual information and auxiliary information by learning the embedding space, but because only visible categories of visual information are used to train the embedding space, the generalized zero When learning tasks from samples, this type of method has a bias problem: samples of unseen categories are easily misidentified as visible categories during the training phase.

This type of method is mainly limited by the quality of the generated information and the instability of the generated model. It is often difficult for the generated model to generate fine-grained visual features rich in identification information and semantic information, so that the trained model cannot solve the problem well. Bias problem leads to limited classification performance

In addition, the stability of the generative model is also one of the limitations of this type of method, requiring the generative model to generate finer features will make the generative model more likely to collapse

Images