Multi-modal small sample learning method based on significance

A learning method and multi-modal technology, applied in character and pattern recognition, biological neural network models, instruments, etc., can solve problems such as costly manpower and financial resources, limited model applicability, data difficulties, etc., to enhance classification ability, enrich Feature representation, the effect of enhancing usability
CN111881997AActive Publication Date: 2020-11-03TIANJIN UNIV
2 Cites 4 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
TIANJIN UNIV
Publication Date
2020-11-03

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a multi-modal small sample learning method based on significance, which comprises a multi-modal combination part and a label propagation part, and specifically comprises the following steps: in a multi-modal combination process, firstly, carrying out significance map extraction on sample images of a support set through a pre-trained significance detection network, and separating out foregrounds and backgrounds of the sample images; secondly, acquiring word embedding of support set sample image foreground and background region semantics through a GloVe model as semanticinformation to assist visual information classification; and finally, performing adaptive combination on foreground, background and semantic information of the acquired support set sample image through a modal mixing mechanism to obtain sample feature representation with multi-modal information; in a label propagation process, firstly, subjecting a support set sample and a query set sample after modal combination to graph construction according to a K-nearest neighbor method; and finally, predicting the category of the query set sample without the label through the support set sample with thelabel.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to a small-sample image classification method, in particular to a small-sample learning method based on salient multimodal data processing. Background technique

[0002] With the introduction of convolutional neural networks, deep learning has made breakthroughs in image classification, speech recognition, and object detection. However, these studies usually require a large amount of labeled data for training, such as ImageNet, etc., but in reality In daily life, it is very difficult to obtain a large amount of data, such as photos of endangered species, medical images, etc., which seriously limits the applicability of the model in the real world, and the labeling of images will also consume a lot of manpower and financial resources. It can identify a new object with a very small number of samples, use previously learned knowledge to help quickly learn new content, and integrate new concepts into existing concept networks to learn...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More