Image subtitle generating method based on novel attention model

A technique of attention model and caption, applied in the field of image understanding

Inactive Publication Date: 2017-11-24
SHENZHEN WEITESHI TECH
View PDF0 Cites 37 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There are many studies on image subtitle generation, but there are still some challenges in generating subtitles by combining image saliency and context

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image subtitle generating method based on novel attention model
  • Image subtitle generating method based on novel attention model
  • Image subtitle generating method based on novel attention model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0042] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other if there is no conflict. The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0043] figure 1 It is a system flowchart of a method for generating image captions based on a new attention model. It mainly includes a saliency prediction model, selected image salient areas, saliency and text perception attention, caption generation, and analysis of attention state.

[0044] Among them, the saliency prediction model, based on the new attention model, proposes a new subtitle structure, which focuses on different parts of the input image during the subtitle generation process. Which parts of the specific image are significant and which parts need to be combined with context Yes, given by the saliency prediction model, the image is extracted by the convolutional neural network, and the re...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an image subtitle generating method based on a novel attention model. The image subtitle generating method mainly includes the steps of establishing a significance forecasting model; selecting a significant image region; forming significance and text perception attention, generating subtitles; analyzing the attention state, wherein different parts of input images are focused in the subtitle generating process through a novel subtitle structure, particularly, significant parts and parts needing to be combined with the context in the images are given through the significance forecasting model; images are extracted through a convolutional neural network, and corresponding subtitles are generated through the recurrent neural network; through attention model extension, two attention paths are created in the significance forecasting model, wherein one attention path importantly concerns the significant region, and another attention path importantly concerns the context region; the two paths jointly cooperate in the subtitle generating process, excellent subtitles are generated step by step, and a further contribution is provided for the innovative solution of image subtitle generating.

Description

Technical field [0001] The present invention relates to the field of image understanding, in particular to a method for generating image captions based on a novel attention model. Background technique [0002] Image captioning is an auxiliary tool for understanding image content. With the development of the Internet, image caption generation technology has received more and more attention. Image caption generation is often used in machine translation, human-computer interaction, artificial intelligence, video processing, and medical fields. Specifically, in the field of machine translation, the content of the input image can be explained, which improves the translation quality. In the field of artificial intelligence, intelligent assistants can recognize and process images, and then generate subtitles, making daily life more convenient. In the field of video processing, subtitles are automatically generated based on video images, reducing manpower consumption. In the medical f...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06K9/46G06N3/08
CPCG06F16/5866G06N3/08G06V10/462
Inventor 夏春秋
Owner SHENZHEN WEITESHI TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap