Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and system for generating image scene graph based on anti-fact multi-agent learning

A multi-agent, agent technology, applied in the direction of ensemble learning, character and pattern recognition, biological neural network model, etc., can solve the problem of ignoring the prediction contribution of a single node

Inactive Publication Date: 2021-02-05
ZHEJIANG UNIV
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since the global evaluation index of the scene graph is an evaluation value of the overall generation quality, the prediction contribution of a single node is ignored

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and system for generating image scene graph based on anti-fact multi-agent learning
  • Method and system for generating image scene graph based on anti-fact multi-agent learning
  • Method and system for generating image scene graph based on anti-fact multi-agent learning

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0073] The public large-scale image scene graph generation dataset Visual Genome is used to train and test the image scene graph generation ability of this method. The data set contains a total of 108,077 image data, which contains 3.8 million object instances (Object Instances), and corresponds to 2.3 million relationship annotations (Relationships). figure 2 It is an example of the Visual Genome dataset. The image is composed of object instances (marked by unfilled rectangles), there are relationships between object instances (marked by colored filled rectangles), and the relationship between object instances and instances together constitutes a scene graph.

[0074] figure 1 is a counterfactual multi-agent algorithm frame diagram of the present invention, and its corresponding specific steps are as follows:

[0075] 1) First, split the data set into training set and test set according to the official recommended ratio. Use the pre-trained target detector to perform objec...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a system for generating an image scene graph based on anti-fact multi-agent learning. According to the method, an image scene graph generation task is converted into a multi-agent collaborative decision-making task. Each object is regarded as an agent, and the action space of each agent is all selectable object categories. All the agents can communicate with one another to encode surrounding visual elements, and feature expression in the agents is improved. And after multiple rounds of agent communication, the visual relationship between the agents is predicted by using a visual relationship prediction model to obtain a final scene graph prediction result. According to the method, a brand-new anti-fact multi-agent learning model is provided, evaluationindexes generated by the scene graph are used as optimization objectives of the model, the anti-fact multi-agent learning model comprises an anti-fact reference model, and the generation quality of the scene graph can be remarkably improved by improving the accuracy of object categories.

Description

technical field [0001] The invention relates to the main fields of deep neural network, target detection and scene graph generation in machine learning and computer vision research, and specifically relates to a method and system for generating an image scene graph based on counterfactual multi-agent learning. Background technique [0002] Visual scene understanding is an important research field in the field of computer vision research. It not only needs to predict the category and position of all objects in the scene, but also needs to predict the visual relationship between two objects. With the maturity of target detection and object segmentation technology, computers have been able to accurately identify the category, location and attributes of objects. However, visual scene understanding is not only about the recognition of individual objects, but also needs to further recognize the visual relationship between objects. All objects and visual relationships are combine...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/62G06N20/20G06N3/04
CPCG06N20/20G06N3/049G06V2201/07G06F18/2415G06F18/214
Inventor 庄越挺肖俊汤斯亮吴飞杨易
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com