Social network image description generation method based on attention feature extraction network

A feature extraction and social network technology, applied in the field of computer vision, can solve the problems of unclear image subject and background division, insufficient use of image semantic features, low resolution, etc., to eliminate the interference of resolution and surrounding background, and improve accuracy. With naturalness, the effect of improving precision

Inactive Publication Date: 2019-11-19
BEIJING UNIV OF POSTS & TELECOMM
View PDF3 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Due to the limitations of shooting equipment and shooting scenes, social network image data often have problems such as low resolution, unclear division of image subject and background, etc.
The research status at home and abroad shows that the existing image description generation methods, including multi-modal recurrent neural network, translation model based on attention mechanism, etc., all have the problem of insufficient utilization of semantic features of images.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Social network image description generation method based on attention feature extraction network
  • Social network image description generation method based on attention feature extraction network
  • Social network image description generation method based on attention feature extraction network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0016] In order to make the purpose, algorithm calculation and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings. The concrete realization of the algorithm of the present invention is divided into the following steps:

[0017] 1. Convolutional Neural Network Image Feature Extraction Based on Attention Mechanism

[0018] The present invention builds image features by stacking multiple attention structures, and each attention structure consists of two branches: a sampling branch and a backbone branch. The main branch can adapt to a variety of cutting-edge network structures. The purpose of the sampling branch is to calculate the attention weight of each pixel for the currently input feature map. The depth features of the image can reflect areas that are highly correlated with important targets in the image. Therefore, the sampling branch first needs to be extracted through ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to the field of image understanding, and provides a social network image description generation method based on an attention feature extraction network. The method comprises twoparts: an attention mechanism-based image feature extraction network: calculating attention regions of interest of images of different scales through high-level image features and language model contexts; and a language generation model based on a long-term and short-term memory network: generating description words by inputting image features of different scales and outputting the description words with a previous layer of language model. Context output of a language model is innovatively used for guiding extraction of a region of interest of image features in the description generation process, a theoretical system is complete, innovativeness is outstanding, and the method is mainly used for automatically generating text description for an image and has high practical value in the fieldof image understanding.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and specifically relates to various deep learning technologies, such as image feature extraction based on a convolutional neural network, and a long-short-term memory network language model based on a recurrent network. A Method for Image Caption Generation by Building an Attentional Feature Extraction Network. Background technique [0002] With the development of mobile Internet, mobile social platforms have enriched people's daily life. These social platforms have brought about the rapid growth of image data. Hot topic data in social networks contains a large amount of image data, and the cost of using purely manual methods to label the content of each image has also increased. Therefore, using intelligent methods to automatically extract image features and describe image expression content has become a research hotspot in the field of computer vision. Due to the limitations of shooti...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T11/00G06N3/04
CPCG06T11/00G06N3/049G06N3/045
Inventor杜军平薛哲李金轩周南
OwnerBEIJING UNIV OF POSTS & TELECOMM