Image description generation method and device capable of generating dependency tree

An image description and image feature technology, applied in character and pattern recognition, biological neural network model, word processing, etc., can solve the problem of not taking into account the natural language text hierarchy

Pending Publication Date: 2020-09-22
SHENZHEN GRADUATE SCHOOL TSINGHUA UNIV
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Existing deep learning-based methods generate text as a sequence of words one by one, without taking into

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image description generation method and device capable of generating dependency tree
  • Image description generation method and device capable of generating dependency tree
  • Image description generation method and device capable of generating dependency tree

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] Embodiments of the present invention will be described in detail below. It should be emphasized that the following description is only exemplary and not intended to limit the scope of the invention and its application.

[0060] An embodiment of the present invention provides an image description generation method that can generate a dependency tree, the method includes modeling the ternary tree of the dependency tree of the image description text, extracting image features from the input image, and then based on the image The feature generates the root node of the ternary tree, and then generates three child nodes of the root node, and so on, recursively generates three child nodes of each node, thereby generating a ternary tree, and then the generated ternary tree Converted into a dependency tree of image description text; preferably, dynamically assign weights to each local image feature when generating each node through the attention module.

[0061] The image descr...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an image description generation method capable of generating a dependency tree. The method comprises the following steps: modeling a ternary tree of a dependency tree of an image description text, extracting image features from an input image, generating a root node of the ternary tree based on the image features, generating three child nodes of the root node, generating three child nodes of each node in a recursive manner in the same manner, generating the ternary tree, and converting the generated ternary tree into a dependency tree of an image description text; preferably, each local image feature is dynamically assigned a weight by an attention module when each node is generated. According to the method and device, the dependency path generated by the nodes is shorter than that generated by a sequence model, important words can be generated preferentially in the generation process, the generation mode is closer to human beings, and the performance higher than that of a reference model is achieved.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to an image description generating method and device capable of generating a dependency tree. Background technique [0002] Automatically generating natural language text describing the content of the image from a given image is a basic task in the field of artificial intelligence. Existing deep learning-based methods generate text as a sequence of words one by one, without considering the inherent hierarchical structure of natural language text, and can only generate sequential text. [0003] The disclosure of the above background technical content is only used to assist in understanding the inventive concepts and technical solutions of the present invention, and it does not necessarily belong to the prior art of this patent application. There is no clear evidence that the above content has been disclosed on the filing date of this patent application. Under the circumstances...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/46G06K9/62G06F40/14G06F40/279G06N3/04
CPCG06F40/14G06F40/279G06N3/049G06V10/44G06N3/044G06F18/253
Inventor 马志明袁春
Owner SHENZHEN GRADUATE SCHOOL TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap