Unified graph analysis network based on object detector and recursive neural network

Abstract

The invention provides a unified graph analysis network based on an object detector and a recursive neural network. The main content comprises the composition of an analytical graph, a dynamic graph generation network, multi-task training and cascade reasoning. The process is that firstly an object detector is used to detect an object in an image, then the existence of a side between vertexes is predicted through graph deduction, a binary graph is constructed online by using the dynamic graph generation network, information is aggregated from adjacent sides, then the network is trained in an end-to-end mode, and finally the generated relational information is processed so as to further generate a knowledge sentence. The unified graph analysis network solves problems such as error accumulation and loss of context in the graph because the input-output path is too long. Meanwhile, the unified graph analysis network can fully process information in the image, and can also be used for solving language-based questions such as question answering after the optimization.

Description

technical field [0001] The invention relates to the field of computer vision, in particular to a unified graph analysis network based on an object detector and a recursive neural network. Background technique [0002] Within a decade, performance on classical vision problems, such as image classification, object detection, and segmentation, has greatly improved due to the use of deep learning frameworks. Given the tremendous success of deep learning for such low-level vision problems, the next step may be to understand images and generate semantics, such as the relationship between objects, etc. Image understanding can be used in video telephony, video conferencing and other applications that need to transmit images, and compression based on understanding can greatly reduce the image data to be transmitted. The analysis and understanding of aerial remote sensing and satellite remote sensing images can be used for the investigation and research of resources such as geology, ...

AI Technical Summary

Problems solved by technology

[0004] Aiming at the problem that the existing technology is easy to make mistakes or lose context in image understanding and cannot fully process image information, the present invention proposes a unified graph analysis network based on object detector and recurrent neural network. First, an object detector is used Detect objects in images, then predict the existence of edges between vertices through graph inference, and use a dynamic graph generation network to construct bidirectional graphs online and aggregate information from adjacent edges, and then train the network in an end-to-end manner , and finally process the generated relational information to further generate knowledge sentences

Images