Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Image description generation method and device based on scene graph

A technology of image description and scene graph, which is applied in the computer field, can solve problems such as wrong connection of things, and achieve the effect of improving accuracy, increasing structural information, and increasing information

Pending Publication Date: 2020-09-01
TSINGHUA UNIV
View PDF5 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The lack of structural information can easily lead to wrong connections when decoding to generate image descriptions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Image description generation method and device based on scene graph
  • Image description generation method and device based on scene graph
  • Image description generation method and device based on scene graph

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0039] The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

[0040] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, componen...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to an image description generation method and device based on a scene graph, and the method comprises the steps: inputting a to-be-processed image into a scene graph generation network, and obtaining a visual feature vector of each target object in the to-be-processed image and a relation type probability distribution vector between the target objects; performing weight prediction processing on the relationship category probability distribution vector to obtain a relationship weight vector between the target objects; performing context coding processing on the relationship weight vector and the visual feature vector to obtain an object feature vector of each target object; and performing language decoding processing on the object feature vector to obtain description information of the to-be-processed image. According to the image description generation method based on the scene graph, the scene graph can be used as intermediate representation, the content in one picture can be described in a more organized and structured mode, and therefore more accurate and richer image description can be generated.

Description

technical field [0001] The present disclosure relates to the field of computers, and in particular to a scene graph-based image description generation method and device. Background technique [0002] Image description is a task combining computer vision and natural language processing, that is, given a picture, the machine is asked to describe it in words. This technology has a wide range of reference scenarios. It can be used to assist the blind to perceive the surrounding environment, it can be used to help index massive images, and it can be used to establish richer forms of human-computer automatic dialogue. There are a lot of challenging problems in the image description task, such as how to decide which content in the image is worth describing, how to increase the richness of image description, etc. The most critical issue is how to better establish the correspondence between images and language. Typically, such models are split into an image encoder and language de...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06N3/04G06N3/08G06F40/284
CPCG06N3/08G06F40/284G06N3/045G06F18/2415
Inventor 季向阳江河
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com