Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Click and vision fusion based weak supervision bilinear deep learning method

A deep learning and weakly supervised technology, applied in the field of weakly supervised bilinear deep learning, can solve the problems of high labor cost and lack of prospects, and achieve the effect of improving the effect.

Active Publication Date: 2017-07-04
HANGZHOU DIANZI UNIV
View PDF4 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Many researchers hope to make up for this by manually labeling attributes, but this method lacks prospects due to excessive labor costs

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Click and vision fusion based weak supervision bilinear deep learning method
  • Click and vision fusion based weak supervision bilinear deep learning method
  • Click and vision fusion based weak supervision bilinear deep learning method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0054] Such as figure 1 As shown, a weakly supervised bilinear deep learning method based on click and visual fusion, specifically includes the following steps:

[0055] Step (1) extracts the click feature corresponding to the image from the click data set and merges it according to semantic clustering, as follows:

[0056] 1-1. In order to meet the experimental needs, we separately extract all dog-related samples from the click data set Clickture provided by Microsoft to form a new data set Clickture-Dog. The data set has 344 pictures of dogs, and we filter the categories with less than 5 pictures, and finally get 283 sets of pictures. Then, the data set is split into training, validation, and testing in the form of 5:3:2. In order to improve the imbalance of the number of pictures in each class during training, we will select a class with more than 300 p...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a click and vision fusion based weak supervision bilinear deep learning method, which comprises the steps of 1, extracting click features of text composition of each image from a click dataset, and building new low-dimensional compact clock features in a combined text space through combining texts with similar semantics; 2, building a deep module with click and visual features being fused; 3, performing BP learning on network model parameters; 4, calculating the model prediction loss of each training sample, building a similarity matrix of the sample set, learning the sample reliability by using the sample loss and the similarity matrix at the same time, and weighting the samples by using the reliability; and 5, repeating the step 3 and the step 4, iteratively a neural network model and sample weights so as to train the whole network model until convergence. According to the method, click data and visual features are fused so as to construct a new bilinear convolution neural network framework which can be used for better performing recognition on a fine-grained image.

Description

technical field [0001] The invention relates to a fine-grained image classification method, in particular to a weakly supervised bilinear deep learning method based on click and visual fusion. Background technique [0002] As a research direction, Fine-Grained Visual Categorization (FGVC) is a sub-problem of target recognition. It is to distinguish different subcategories of the same type of objects. The objects involved are very similar in appearance, and require certain prior knowledge to distinguish them. For inexperienced people It is not easy to do this, but it is even more challenging for computers to automatically classify. [0003] In the research task of fine-grained image recognition, Tsung-Yu Lin et al. proposed a bilinear convolution neural network model (Bilinear Convolution neutral networks, BCNN), by applying it to fine-grained image recognition In the task, it is found that very good results have been achieved. This model is based on the popular deep learn...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/62G06F17/30
CPCG06F16/5846G06F18/23213G06F18/253G06F18/214
Inventor 俞俊谭敏郑光剑
Owner HANGZHOU DIANZI UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com