Graph-Based Multi-Entity Event Recognition with Explainable Language Functions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-entity event recognition systems face challenges in accurately and efficiently identifying complex activities in video data, particularly in safety-critical situations, where automated decisions need to be explainable and understandable by humans.
Innovation Solution
An image processing system that uses a trained reinforcement learning policy and domain-specific language functions to represent video data as a graph, recognizing entities and attributes, and providing human-understandable explanations for predicted multi-entity events through the selection and assignment of domain-specific language functions to edges in the graph.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional multi-entity event recognition systems are used, then automated decisions can be made, but the explanations for these decisions are not human-understandable
Solution Approach 1:
The patent introduces an intermediary explanation generation module that translates the internal representation and decision-making process into human-understandable explanations. This mediator component bridges the gap between automated decision systems and human users, providing interpretable narratives about why certain multi-entity events were detected without compromising the automated decision-making capability.
2Measurement precision
If complex machine learning models are used to recognize multi-entity activities, then recognition accuracy improves, but system complexity increases
Solution Approach 1:
The patent segments the complex multi-entity event recognition task into distinct components: entity detection, attribute recognition, relationship identification, and event classification. By dividing the overall system into these modular segments, the patent maintains high recognition accuracy through specialized sub-components while reducing overall system complexity through manageable, independent modules that can be developed and validated separately.
3Measurement precision
If detailed analysis of video data is performed to recognize multi-entity events, then recognition accuracy improves, but processing time increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing video data to extract key features, pre-identifying entities and their attributes before the main event recognition process. This preliminary analysis prepares the data in advance, allowing the subsequent multi-entity event recognition to proceed more efficiently with reduced processing time while maintaining high accuracy through the use of pre-extracted relevant features.
Data Source
AI summary
An image processing system has a memory storing a video depicting a multi-entity event, a trained reinforcement learning policy and a plurality of domain specific language functions. A graph formation module computes a representation of the video as a graph of nodes connected by edges. A trained machine learning system recognizes entities depicted in the video and recognizes attributes of the entities. Labels are added to the nodes of the graph according to the recognized entities and attributes. The trained machine learning system computes a predicted multi-entity event depicted in the video. For individual ones of the edges of the graph, select a domain specific language function from the plurality of domain specific language functions and assign it to the edge, the selection being made at least according to the reinforcement learning policy. An explanation is formed from the domain specific language functions.