Zero sample sketch retrieval method based on semantic adversarial network

Abstract

The invention provides a zero sample sketch retrieval method based on a semantic adversarial network, which mainly solves the problems that in the prior art, the sketch intra-class variance is larger,and the visual knowledge is difficult to migrate from a known class to a non-seen class under the zero sample setting. The method comprises the steps of obtaining a training sample set, constructinga semantic adversarial network, and extracting the RGB image features through a VGG16 network, constructing a generation network to generate the RGB image features with discriminability, inputting theto-be-retrieved sketch into a semantic confrontation network to generate the semantic features, inputting the semantic features and the random Gaussian noise into the generation network to generate the RGB image features, and searching the first 200 images most similar to the RGB image features in an image retrieval library to obtain a retrieval result. According to the method, the intra-class variance of the sketch image features is reduced, the RGB image features generated according to the sketch image in each class can be ensured, the retrieval performance of zero sample sketch retrieval is improved, and the method can be used for the electronic commerce, medical diagnosis and remote sensing imaging.

Description

technical field [0001] The invention belongs to the technical field of image processing, in particular to a zero-sample sketch retrieval method, which can be used in e-commerce, medical diagnosis and remote sensing imaging. Background technique [0002] Sketch retrieval refers to the retrieval of real natural images based on hand-drawn sketches. The zero-shot sketch retrieval method is a method for real-world image retrieval of hand-drawn sketches of unknown categories. Existing sketch retrieval methods are mainly divided into two categories: artificially designed features and deep learning-based methods. Among them, methods based on artificially designed features include gradient field HOG descriptors and SIFT descriptors, while methods based on deep learning include twin networks, triplet networks, deep sketch hashes, etc., and their main ideas are to extract image or text information. The discriminative features are then projected into a common feature space for similar...

AI Technical Summary

Problems solved by technology

Although the above methods have achieved good performance, the two methods have not considered the problem of large variance within the sketch class, so the discriminative ability of the semantic information extracted by the pre-trained convolutional neural network is weak, and it is difficult to Accurately transfer visual knowledge of sketches from known to unseen classes

Images