Unlock instant, AI-driven research and patent intelligence for your innovation.

A Method of Image Description Generation Based on Neural Network and Image Concern

A neural network and image description technology, applied in the field of computer vision, which can solve the problems of inability to pay attention between image objects or attributes and high computational complexity

Active Publication Date: 2020-10-23
SYSU CMU SHUNDE INT JOINT RES INST +1
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, RNNs rely on good semantic expression input. For the field of image understanding, the degree of attention between image objects or attributes cannot be reflected by RNNs, and for neural networks, it is a high-dimensional information processing with high computational complexity.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Method of Image Description Generation Based on Neural Network and Image Concern
  • A Method of Image Description Generation Based on Neural Network and Image Concern
  • A Method of Image Description Generation Based on Neural Network and Image Concern

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0043] Such as figure 1 As shown, an image description generation method based on neural network and image concerns, including the following steps:

[0044]S1: Construct a multimodal model of the image of frame t at each moment:

[0045] 1) The text description information of the marked images in the training set is divided into a single word set, and the corresponding word is represented by a one-hot vector, which is used as the input of the text module of the model, and is projected into a dense word expression space through two embedding layers to become a semantic The word expression vector W t ;

[0046] 2) The word expression vector is used for the input of the circular convolutional neural network RNN ​​at a certain time frame t to perform the calculation of the circular convolutional neural network RNN, and the circular layer of the frame t at this time activates R t is the word expression vector of the current time frame and the recurrent layer R of the previous ti...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an image description generating method based on a neural network and image focuses. The method is characterized in that an original one-layer word embedding structure is replaced by a two-layer word embedding structure, and accordingly word expression can be learned effectively; image feature expression is directly used as the input of an m-RNN model, and accordingly the capacity of a circulating layer can be fully utilized, and the small-dimension circulating layer can be used; by a decision soft attention mechanism, the attention of an image salient region is reflected and used as one input of a multi-modal layer. By the method, the focus relation between targets or scenes is utilized effectively, and the semantic features of the image is described in a targeted manner.

Description

technical field [0001] The present invention relates to the field of computer vision, and more specifically, to an image description generation method based on a neural network and image attention points. Background technique [0002] Obtaining text-level image descriptions has become an important research topic in the current field of computer vision, and in real life, it has many application scenarios. Examples include early childhood education, image retrieval, and navigation for the blind. With the rapid development of computer vision and natural language processing technology, a large number of effective works on this topic appear, many of which regard it as a retrieval problem. The researchers project the features of text sentences and images into the same semantic space by learning a node embedding layer. These methods generate image descriptions by retrieving similar descriptions from text sentence datasets, but they lack image descriptions that can effectively com...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/62G06K9/46G06N3/04G06N3/08
CPCG06N3/084G06F16/51G06F16/5866
Inventor 胡海峰杨梁王腾张俊轩王伟轩
Owner SYSU CMU SHUNDE INT JOINT RES INST