Relational Network Attention Blocks for Entity Relationship Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Reinforcement learning systems face challenges in effectively selecting actions in complex environments with multiple entities, particularly when state data is defined by arrays of pixels, as they struggle to capture non-local relationships and interactions between entities.
Innovation Solution
A neural network system that includes an input network for extracting entity data from image state data, a relational network with attention blocks to transform and emphasize salient relationships between entities, and an output network to select actions based on these relationships, utilizing attention mechanisms and adaptive networks to generate modified entity data and attention weights.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If traditional neural networks are used to process pixel-based state data, then the system can handle the input data, but it fails to capture non-local relationships and interactions between entities
Solution Approach 1:
The patent segments the state data by identifying and extracting individual entities from the pixel-based input, representing each entity as a separate data structure with its own features. This segmentation allows the system to process entities independently while still capturing their relationships through the relational network architecture.
Solution Approach 2:
The patent introduces an intermediary relational network layer that sits between the input network and output network. This relational network explicitly models relationships between entities by computing attention weights and relationship features, serving as a mediator that transforms entity representations into relationship-aware representations before passing them to the output network.
2Reliability
If the neural network learns relationships between multiple entities, then action selection performance improves, but training time and memory requirements increase
Solution Approach 1:
The patent changes the parameter representation by using attention mechanisms that dynamically compute relationship weights based on entity pairs. Instead of learning fixed relationship parameters for all entity pairs, the system computes adaptive attention weights during forward propagation, reducing the number of trainable parameters while maintaining the ability to capture complex relationships.
Solution Approach 2:
The patent implements partial relationship modeling by focusing attention on the most relevant entity pairs rather than computing all possible relationships. The attention mechanism selectively processes only the most salient relationships, avoiding the computational burden of modeling all possible entity interactions while still capturing the critical ones needed for effective action selection.
3Measurement precision
If the neural network uses attention mechanisms to emphasize salient relationships, then the system can identify important entity interactions, but the computational complexity increases
Solution Approach 1:
The attention mechanism computes relationship weights for all entity pairs but only uses the most salient ones for decision-making. By ranking and selecting only the top attention-weighted relationships, the system achieves precise relationship identification while reducing the effective computational load by ignoring less important relationships.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A neural network system for reinforcement learning is proposed, including an input network for extracting, from state data, respective entity data for each a plurality of entities which are present, or at least potentially present, in the environment. The entity data describes the entity. The neural network contains a relational network for parsing this data, which includes one or more attention blocks which may be stacked to perform successive actions on the entity data. The attention blocks each include a respective transform network for each of the entities. The transform network for each entity is able to transform data which the transform network receives for the entity into modified entity data for the entity, based on data for a plurality of the other entities. An output network is arranged to receive data output by the relational network, and use the received data to select a respective action (e.g. an action from a predefined space of possible actions.