Image description method

A technology of image description and partial image, which is applied in the fields of instrumentation, computing, and electrical digital data processing, etc., can solve problems such as no effective correction, description target error, and computer difficulty, so as to enhance fault tolerance and generalization ability, reduce impact, The effect of increasing accuracy

Inactive Publication Date: 2019-10-29
SHANGHAI MARITIME UNIVERSITY
View PDF3 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Image description is relatively convenient for humans, but it is quite difficult for computers, because computers not only need to find the target and background in the picture, but also need to understand the relationship between them, which is a relatively complicated thing
[0004] Most of the existing image description m

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image description method
  • Image description method
  • Image description method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0067] based on the following Figure 1 to Figure 5 , specifically explain the preferred embodiment of the present invention.

[0068] Such as figure 1 As shown, the present invention provides an image description method, comprising the following steps:

[0069] Step 1. Resize the image, and scale the input images of different sizes to a uniform size.

[0070] Step 2, using the VGG convolutional neural network to extract the global image features in the picture;

[0071] Step 3, using the Faster R-CNN network to extract local image features in the picture;

[0072] Step 4. Fusing the global image feature and the local image feature through the global-local feature fusion algorithm to obtain the image fusion feature;

[0073] Step 5, processing image fusion features through a two-way long-short-term memory network with an attention mechanism, and generating preliminary image description sentences;

[0074] Step 6. Using the image target information obtained during partial ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an image description method, and the method comprises the steps: extracting global image features in a picture through a VGG convolution neural network, and extracting local image features in picture by using Faster R-CNN network, fusing global image features and local image features through a global-local feature fusion algorithm to acquire the image fusion features; processing image fusion features through a bidirectional long-short-term memory network with an attention mechanism; and generating a preliminary image description statement, performing WordNet word vectorsimilarity calculation based on the image target information obtained during local image feature extraction and nouns in the preliminary image description statement, correcting the image descriptionstatement, and generating a final image description statement. According to the method, the influence of useless information is reduced, the expression of key information is enhanced, the fault tolerance and generalization ability of the model are enhanced, and the accuracy of statement description is improved.

Description

technical field [0001] The invention relates to the field of image recognition processing, in particular to a method for object detection and description sentence generation in an image based on a codec and multiple feature fusion processing. Background technique [0002] With the rapid development of technology, smart phones are becoming more and more popular among people, and selfies and casual photos have gradually become a mainstream way of socializing, so images are growing at an exponential rate. As of 2014, Facebook alone has more than 250 billion images. Conventional image retrieval methods, such as manually labeling images and performing brief image descriptions, are no longer able to bear pictures of this magnitude. It becomes impossible to deal with it in a certain way, so it is conceived to use machines to automatically label and describe images. [0003] Image description thrives against the background of the rapid development of machine learning and deep learn...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62G06F17/27
CPCG06F40/247G06F40/289G06F18/22G06F18/2415G06F18/253
Inventor 吕诗奇刘晋
Owner SHANGHAI MARITIME UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap