Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Fine-grained freehand sketch image retrieval method based on attention enhancement

An attention and fine-grained technology, applied in the field of cross-media retrieval, it can solve the problems of low image accuracy and inconsistent target objects, and achieve the effect of excellent performance.

Active Publication Date: 2020-08-04
FUDAN UNIV
View PDF3 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The problems solved by the present invention include: the accuracy rate of images retrieved by the existing hand-drawn sketch image retrieval model in the hand-drawn sketch image retrieval results is low; the retrieved image is inconsistent with the target object in the query sketch; Research

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fine-grained freehand sketch image retrieval method based on attention enhancement
  • Fine-grained freehand sketch image retrieval method based on attention enhancement
  • Fine-grained freehand sketch image retrieval method based on attention enhancement

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0051] The specific implementation details of the present invention will be introduced in detail below.

[0052] (1) Image semantic feature extraction

[0053] A CNN is used to extract visual features of each hand-drawn sketch and color image. Compared with traditional feature extraction methods, CNN has more powerful feature learning and extraction capabilities; ResNet network with attention mechanism is used as image semantic extractor, and the output features of the last layer represent visual global features. Thus, for each input image, the network outputs its global visual feature representation.

[0054] In the present invention, for the input hand-drawn sketches and color images, the corresponding modal network branches are used to extract semantic features, that is, for the input query sketches, the sketch network branches are used to extract the semantic features of the sketches; Color images, using image network branches to extract semantic features of images.

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of cross-media retrieval, and particularly relates to a fine-grained freehand sketch image retrieval method based on attention enhancement. According to the deep fine-grained freehand sketch image retrieval FG-SBIR model, different attention mechanisms are used to further pay attention to fine-grained details between a sketch and an image. The new modelnot only pays attention to correlation information between a sketch mode and an image mode, but also pays attention to distinctiveness information in a single mode. According to the invention, a mutual loss method is provided to enhance the traditional triple loss and improve the discrimination capability of the model on the fine-grained characteristics in a single mode. For a given query sketch,related images with fine-grained instance-level similarity in a specific category can be returned, and the strict requirement for instance-level retrieval of fine-grained freehand sketch image retrieval is met.

Description

technical field [0001] The invention belongs to the technical field of cross-media retrieval, and in particular relates to a fine-grained hand-drawn sketch image retrieval method based on enhanced attention. Background technique [0002] With the ubiquity of mobile devices these days, it's becoming ever more convenient for people to sketch on-screen. Especially when searching for a specific image, people need to give a long textual description, but in fact they can also draw a sketch containing fine-grained visual features to express the retrieval needs. Therefore, Fine-grained Sketch-based Image Retrieval (FG-SBIR) has aroused extensive research interest of scholars. In recent years, although fruitful research results have been obtained in this field, it still faces many challenging problems, and more in-depth research work is needed to obtain better solutions. These challenging issues include, how to reduce the semantic difference between sketches and images from differe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/53G06N3/04
CPCG06F16/53G06N3/045
Inventor 张玥杰王燕飞
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com