Zero-sample sketch image retrieval method and system based on graph convolutional neural network

A convolutional neural network and image retrieval technology, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve the problems that the model is difficult to get the best results, the model is unstable, etc.

Active Publication Date: 2020-06-16
FUDAN UNIV
View PDF4 Cites 24 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Coupled with the instability of the generated model, it is difficult for the model to get the best results

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Zero-sample sketch image retrieval method and system based on graph convolutional neural network
  • Zero-sample sketch image retrieval method and system based on graph convolutional neural network
  • Zero-sample sketch image retrieval method and system based on graph convolutional neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0068] The present invention first proposes a novel zero-sample sketch image retrieval technology model, which effectively utilizes the visual information of sketches and images and the semantic information of their class labels to model cross-modal correlations between sketches and images to obtain a unified space The underlying deep feature representation leverages knowledge learned based on seen category labels to infer correlations between sketches and images of unseen categories. Using the constructed model can effectively promote the improvement of zero-sample sketch image retrieval accuracy and improve user experience. The model mainly includes the following parts:

[0069] (1) Feature encoding network (Encoding Network): The feature encoding network of the present invention adopts a twin network structure, and learns two mappings f( ) and g( ) from sketches to feature vectors and from images to feature vectors respectively. Two networks map sketches and images to the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of multimedia information retrieval, and particularly relates to a zero-sample sketch image retrieval method and system based on a graph convolutional neural network. The zero-sample sketch image retrieval system architecture provided by the invention comprises three important components: a feature coding network, a semantic maintenance network and a semantic reconstruction network. The method comprises the steps ofextracing sketches and image visual features through a feature extraction network; processing the visual information of the sketch and the image and the label semantic information of the sketch and the image at the same time through a graph convolution network; establishing a relationship between unseen categories and seen categories;enhancing the generalization ability of the model through a semantic reconstruction network;and finally, the model taking the sketch of which the category is not seen as an input and performing retrieval to find an image similar to the sketch. According to the invention, the variational auto-encoder is adopted to generate semantic information from visual information, so that the generalization ability of the model is further enhanced.

Description

technical field [0001] The invention belongs to the technical field of multimedia information retrieval, and in particular relates to a zero-sample sketch image retrieval method and system based on a graph convolutional neural network. Background technique [0002] Sketch-based image retrieval (Sketch-based Image Retrieval, SBIR) aims to retrieve the image that the user wants through hand-drawn sketches, and has been studied by academic circles for many years. Compared with traditional text-based image retrieval technology (Text-based Image Retrieval, TBIR), sketch-based image retrieval is more convenient and intuitive. Because, sometimes it is easier for a person to draw a detail of an image he / she wants than to describe it in words. While sketches are often very abstract, containing only a few lines, images are very specific, containing details such as color and texture. Therefore, there is a huge difference between sketches and images, and this difference is often calle...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/583G06K9/62G06N3/04G06N3/08
CPCG06F16/583G06N3/08G06N3/045G06F18/253
Inventor 张玥杰张兆龙
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap