Natural Language Query Processing via Detector DAG Planning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image media searching technologies rely on captions or image-based queries, which are inadequate for live video applications like surveillance, as captions are often absent and image-based queries require users to sketch queries, limiting their effectiveness.
Innovation Solution
A system comprising a detector manager with a detector API, an ontology manager, and a reasoning engine that transforms natural queries into directed acyclic graphs (DAGs) to plan an execution list of detectors, enabling the system to answer queries on image data without requiring captions or manual sketching, using external detectors and ontologies to provide a flexible and expandable probabilistic logic framework.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If captions are used for searching image media, then query accuracy is improved, but applicability to live video is worsened due to absence of captions
Solution Approach 1:
The patent introduces an intermediary system that includes a query parser, detector manager, and reasoning engine. This intermediary translates natural language queries into detector execution plans, bridging the gap between user intent and visual content analysis without requiring captions or manual sketching.
Solution Approach 2:
The patent replaces the mechanical requirement of manual sketching or caption dependency with an automated natural language processing system. The reasoning engine automatically plans detector execution based on parsed queries, substituting manual or caption-based approaches with intelligent automation.
2Manufacturing precision
If image-based queries requiring sketching are used, then query specificity is improved, but ease of operation is worsened due to user burden
Solution Approach 1:
The patent replaces the mechanical action of manual sketching with natural language text input. The query parser processes textual descriptions and the reasoning engine translates them into detector execution plans, eliminating the need for users to draw or sketch while maintaining query specificity.
Solution Approach 2:
The system performs self-service by automatically parsing natural language queries and generating detector execution plans without requiring manual intervention for query formulation. The reasoning engine autonomously translates user intent into actionable detection tasks.
3Adaptability or versatility
If external detectors are integrated via detector API, then system adaptability is improved, but device complexity is worsened
Solution Approach 1:
The detector API serves as an intermediary layer between the reasoning engine and external detectors. It provides standardized interfaces for detector registration, execution, and result retrieval, managing complexity through abstraction while enabling flexible integration of multiple detectors.
Solution Approach 2:
The detector API provides universal functionality for integrating various types of detectors through a common interface. It handles detector registration, capability exposure, and execution planning uniformly, allowing the system to work with multiple detector types without increasing architectural complexity.
4Productivity
If natural language queries are transformed into query DAGs, then query processing efficiency is improved, but computational complexity is worsened
Solution Approach 1:
The query processing system segments the natural language query into a directed acyclic graph (DAG) structure, breaking down complex queries into smaller, manageable sub-queries and detection tasks. This segmentation enables parallel execution of independent detectors while maintaining logical dependencies.
Solution Approach 2:
The query parser performs preliminary action by transforming the natural language query into a structured DAG representation before detector execution. This preprocessing step organizes detection tasks and their dependencies in advance, enabling efficient execution planning and reducing computational overhead during actual detection.
Data Source
AI summary
Systems and methods for solving queries on image data are provided. The system includes a processor device coupled to a memory device. The system includes a detector manager with a detector application programming interface (API) to allow external detectors to be inserted into the system by exposing capabilities of the external detectors and providing a predetermined way to execute the external detectors. An ontology manager exposes knowledge bases regarding ontologies to a reasoning engine. A query parser transforms a natural query into query directed acyclic graph (DAG). The system includes a reasoning engine that uses the query DAG, the ontology manager and the detector API to plan an execution list of detectors. The reasoning engine uses the query DAG, a scene representation DAG produced by the external detectors and the ontology manager to answer the natural query.