Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A visual scene graph generation system and method based on relation regularization

A technology of scene graph and relationship, applied in the field of visual scene graph generation

Active Publication Date: 2019-05-07
UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF10 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the previous work only used the detection results of Faster R-CNN to judge the relationship between objects. Few people considered the influence of the relationship between objects on object detection.
And the previous work also revealed a phenomenon: if it is known that there is indeed a relationship between two objects, it will be much easier to judge what the relationship is

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A visual scene graph generation system and method based on relation regularization
  • A visual scene graph generation system and method based on relation regularization
  • A visual scene graph generation system and method based on relation regularization

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0064] The present invention aims to propose a system and method for generating a visual scene graph based on relational regularization, which can quickly and effectively judge whether there is a relation between objects, and is conducive to enhancing the detection effect of an object detection model.

[0065] In order to achieve the above purpose, the present invention designs a network based on relational regularization to generate a visual scene graph. Since previous work has proved that the label of the object has a great influence on the final scene graph, we propose the object label refinement module to improve the object label generated by Faster R-CNN. A relation generation module is then used to generate the final visual scene graph. Each module is composed of two-way long-short-term memory model (Bi-LSTMs) and graph convolutional network (GCNs). Bi-LSTMs are used to obtain features containing global context information, and then use this feature to obtain a relations...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a visual scene graph generation technology, and discloses a visual scene graph generation system and method based on relation regularization, which can quickly and effectivelyjudge whether a relation exists between objects, and is beneficial to enhancing the detection effect of an object detection model. The system comprises an object detector, an object label refiner andan object relation generator. An object in the image is detected through an object detector to botain a label, an object frame feature and a joint frame feature of the object; And the labels of the objects is refined by using an object label refiner, the relationship between the objects is acquired by using an object relationship generator, and generating a final visual scene graph. The system and methods are suitable for generating the visual scene graph.

Description

technical field [0001] The invention relates to a visual scene graph generation technology, in particular to a system and method for generating a visual scene graph based on relational regularization. Background technique [0002] Visual Scene Graph (Visual Scene Graph) is a high-level summary of image content, which consists of a series of nodes (entities in the image) and edges (relationships between entities). The task of visual scene graph generation refers to inputting a picture, and the model not only detects the objects (borders and categories) contained in the image, but also detects the relationship between objects. [0003] Because visual scene graph generation needs to detect objects contained in the image, most methods use a very effective object detection model - Faster R-CNN to detect the borders and categories of objects. However, the previous work only used the detection results of Faster R-CNN to judge the relationship between objects. Few people considered...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/32G06N3/04
Inventor 宋井宽郭昱宇高联丽
Owner UNIV OF ELECTRONICS SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com