A Description Method Based on Regional Image Feature Attention Network and Nearest Neighbor Ranking

An image feature and nearest neighbor technology, applied in the field of computer vision, can solve problems such as ignoring image feature information, and achieve the effect of reasonable sentence description

Active Publication Date: 2020-12-29
SHANTOU UNIV
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The above methods have achieved good results in image description, but the current image description research often ignores the attention image feature information of the region

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Description Method Based on Regional Image Feature Attention Network and Nearest Neighbor Ranking

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0027] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0028] A description method based on regional image feature attention network and nearest neighbor sorting in an embodiment of the present invention includes the following steps:

[0029] S1: Use the Googlenet model as the initial model of CNN (convolutional neural network), and take the feature parameter vector of the image from the fully connected layer as the global image feature vector of the image;

[0030] S2: The image is input into the VGG net CNN (deep convolutional neural network) model, and the regional image convolution feature attention map is taken out from the third layer of the fifth convolution layer of VGG net;

[0031] S3: Input the global image feature vector and depth semantic representation into the stacked gate loop unit, and decode the sema...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiments of the invention disclose a description method based on a regional image feature concerning network and nearest neighbor ordering. By combining global and regional image feature information, the image description can output overall information and highlight detail information; a double-layer semantic layer for mining deep semantic information and a stack type portal circulation unit for increasing the vertical depth of a recurrent neural network are designed to learn deeper semantic mapping between images and words; and the generated candidate descriptions are reordered by adopting a nearest neighbor algorithm and a semantic similarity, so that the finally output sentence description is more reasonable.

Description

technical field [0001] The invention relates to the field of computer vision, in particular to an image description method based on regional image feature attention network and nearest neighbor sorting. Background technique [0002] The difficulty of image description far exceeds the object detection and image classification that have been studied in the field of image understanding for many years. First, it not only needs to capture the existing objects in the image, but also needs to explain the relationship between objects in the image, the attributes of the objects and the activities they are participating in. What is even more difficult is that image description involves the combination and conversion of visual and language modalities, and a large amount of visual information needs to be compressed into a natural language description that satisfies syntactic and semantic rules. This has greatly tested the ability of computer algorithms to fuse multimodal information su...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/62G06N3/04
Inventor 陈耀文吴捷谢斯雅史新宝
Owner SHANTOU UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap