Unlock instant, AI-driven research and patent intelligence for your innovation.

An image description method of multi-level connection recurrent neural network

A cyclic neural network and image description technology, applied in the field of image description with multi-level connection cyclic neural network, can solve the problem of ignoring the attention information of deep semantic concepts of images and so on.

Active Publication Date: 2021-04-06
SYSU CMU SHUNDE INT JOINT RES INST +1
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The above methods have achieved good results, but the current image description research often ignores the deep semantic concept of the image and the attention information of the region.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • An image description method of multi-level connection recurrent neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] Such as figure 1 As shown, an image description method of a multi-level connection recurrent neural network includes the following steps:

[0031] (1) Semantic attributes are extracted from the training set of tagged sentences, and an attribute vocabulary is constructed.

[0032] (2) The VGGNet model is used as the initial model of CNN, the single-label ImageNet dataset is used for pre-training of CNN parameters, and then the multi-label dataset MS COCO is used for fine-tuning of CNN parameters.

[0033] (3) Input the image to be described, divide it into different regions, input it into the trained CNN, express the image information into high-level semantic information, and obtain the prediction probability of semantic attributes.

[0034] (4) Send the image into the CNN network to extract the paraphrase vectors describing different regions.

[0035] (5) Calculate the weight corresponding to each interpretation according to the hidden variable information of the prev...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides an image description method of a multi-level connection cyclic neural network. The method constructs an attribute vocabulary from a training set of labeled sentences, adopts a VGGNet model as a CNN model, and uses a labeled data set to perform parameter training and adjustment of the CNN. Input For the image to be described, the semantic attribute prediction probability is obtained, and the image is sent to the CNN network to extract the description interpretation vector and calculate the weight corresponding to each interpretation, and then calculate the context vector according to the interpretation vector and its corresponding weight, and predict the semantic attribute The probability and context vectors are input into the multi-level connected cyclic neural network, and the combination of the output results is the natural language description of the image.

Description

technical field [0001] The present invention relates to the field of computer vision, and more specifically, relates to an image description method of a multi-level connection cyclic neural network. Background technique [0002] At present, the rapid development of science and technology and the Internet has greatly increased the amount of image data. At the same time, the demand for image information extraction is also increasing. It is a research hotspot in the field of computer vision to give natural language sentences that can describe the content of the image according to the image. [0003] With the development of computer vision technology and natural language processing technology, the topic of image description has been a hot topic for several years, and many methods have been proposed. For example: methods based on local region features, methods based on multimodal recurrent neural networks, methods based on convolutional neural networks (CNN) combined with recur...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/58G06F16/51G06N3/08
CPCG06N3/08G06F16/51G06F16/5866
Inventor 胡海峰吴捷张俊轩杨梁王伟轩
Owner SYSU CMU SHUNDE INT JOINT RES INST