Zero sample image recognition method and recognition device thereof, medium and computer terminal

Abstract

The invention relates to a zero sample image recognition method based on prototype domain alignment and cross-modal reconstruction. The method comprises the following main steps: extracting visual features of visible samples; obtaining visual distribution information of the visible images through a feature distribution encoder; obtaining generated visual features through a feature generator; training and optimizing parameters of the model of the feature generator by designing distribution regularization loss, domain consistency loss, visual reconstruction loss, adversarial loss and semantic reconstruction loss to obtain a trained feature generator; inputting the unseen class semantic features into a trained feature generator to obtain generated unseen class visual features; training an unseen class classifier by using the generated unseen class visual features; and utilizing the trained unseen class classifier to predict the unseen class image. According to the method, through prototype domain consistency alignment and cross-modal reconstruction, the generated visual features are closer to real visual features, and more category discrimination features are included.

Description

technical field [0001] The invention relates to a zero-sample image recognition method in the field of computer vision image recognition, in particular to a zero-sample image recognition method based on prototype domain alignment and cross-modal reconstruction, and a zero-sample image recognition method corresponding to the zero-sample image recognition method. A zero-sample image recognition device, a computer-readable storage medium and a computer terminal using the zero-sample image recognition method. Background technique [0002] Existing image recognition methods need to collect a large number of images with category labels to train the model in the model training stage, and the recognition stage can only identify the categories that have appeared in the training stage. However, in practical scenarios, it is often necessary to identify categories that lack images in the training phase, such as images of endangered species, medical tumor images, etc. In the above appli...

AI Technical Summary

Problems solved by technology

[0007] In order to solve the technical problems of inconsistent sample domain distribution and incomplete generated sample category information in the generated model, the present invention provides a zero-sample image recognition method based on prototype domain alignment and cross-modal reconstruction, which is similar to the zero-sample image recognition method Corresponding zero-sample image recognition device, computer-readable storage medium and computer terminal using the zero-sample image recognition method

Images