Deep cross-mode correlation learning-based image retrieval method for free-hand sketch

An image retrieval and correlation technology, applied in image enhancement, image analysis, image data processing and other directions, can solve problems such as affecting user experience, high ranking position, etc.

Inactive Publication Date: 2018-09-28
FUDAN UNIV
View PDF5 Cites 55 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in most retrievals, the ranking results are often accompanied by images that are not relevant to the query, and they sometimes occupy a higher ranking position, which greatly affects the user experience
The simple distance measurement method cannot avoid the above problems

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Deep cross-mode correlation learning-based image retrieval method for free-hand sketch
  • Deep cross-mode correlation learning-based image retrieval method for free-hand sketch
  • Deep cross-mode correlation learning-based image retrieval method for free-hand sketch

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0071] The present invention firstly proposes a novel sketch-based image retrieval technology model, which deeply explores the deep visual and semantic features of multi-modal data in retrieval, performs cross-modal correlation modeling on query sketches, retrieved images and texts, and obtains The deep feature representation in the unified space, and then use the sampling in the retrieval results of the training set to obtain positive and negative samples to optimize the similarity function, and obtain a more discriminative similarity function. Utilizing the constructed model can effectively improve the accuracy of sketch retrieval and improve user experience. The model mainly includes the following parts:

[0072](1) Deep Multimodal Feature Generation. Since image retrieval based on sketches involves cross-modal matching of sketches, color pictures and texts, the original data needs to be converted into feature vectors first. In order to better extract the discriminative i...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of cross-media correlation learning, and particularly discloses a deep cross-mode correlation learning-based image retrieval method for a free-hand sketch.The method comprises three main algorithms of deep multi-mode feature generation, multi-mode correlation learning modeling and similarity sorting optimization. By utilizing a deep learning technology, a depth semantic feature and a depth visual feature are constructed for describing a text tagging part and an image/sketch part in a multi-mode document. Based on a multi-mode document representation, a cross-mode correlation model is built for modeling a whole multi-mode document set, thereby describing correlation among different modes of the multi-mode document. Based on correlation featuresobtained after correlation modeling, retrieval results are sorted and optimized, and color images and texts with the maximum similarity with the queried sketch are returned.

Description

technical field [0001] The invention belongs to the technical field of multimedia information retrieval, and in particular relates to a hand-drawn sketch retrieval method based on deep cross-modal correlation learning. Background technique [0002] With the popularity of image acquisition devices such as mobile phones and digital cameras and the development of Internet technology, digital images have exploded in the past few decades. Some image sharing websites, such as Flickr, upload millions of images every day. How to effectively search for images has become a hot research object in academia and industry, and many image retrieval systems have emerged as a result. Early image retrieval technologies are mainly divided into two categories according to different input types, the first is Text-based Image Retrieval (TBIR), and the second is Content-based Image Retrieval (Content-based Image Retrieval). , CBIR). [0003] Text-based image retrieval technology refers to the rea...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06T7/13G06T5/00G06K9/62
CPCG06T5/002G06T7/13G06T2207/20192G06T2207/10024G06T2207/20084G06T2207/20081G06F18/22G06F18/214
Inventor 张玥杰黄飞王燕飞张涛
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap