Chart Analysis Models for Natural Language Querying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional chart-grounded question answering systems are inefficient and inaccurate due to their reliance on pixel-level information and hard-coded rules, leading to high response times and decreased accuracy in processing charts for query responses.
Innovation Solution
A natural language processing system that includes an answer model and a description model, trained via a two-phase method, to generate responses to queries based on chart data, using chart specifications to provide visual context and handle language variations, thereby improving efficiency, accuracy, and speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional models treat charts as images and rely on pixel-level information, then the system can process any chart format, but the response time is high and accuracy is low
Solution Approach 1:
The patent segments the chart processing task into two distinct models: an answer model that processes chart data and specifications to generate answers, and a description model that generates visual descriptions. This segmentation allows each model to specialize in specific aspects of chart understanding, improving processing efficiency while maintaining versatility across different chart formats.
Solution Approach 2:
The patent introduces chart specifications as an intermediary representation that bridges the gap between raw chart data and natural language queries. These specifications serve as a structured intermediate format that enables faster and more accurate processing compared to direct pixel-level analysis, resolving the contradiction between versatility and response time.
2Device complexity
If conventional models use hard-coded rules, then the system structure is simple, but the accuracy of answers is low
Solution Approach 1:
The patent transforms the system from using fixed hard-coded rules to using learnable parameters through machine learning models. The answer model and description model are trained on chart data and specifications, allowing them to adapt to different chart types and query patterns, significantly improving answer accuracy while maintaining reasonable system complexity through modular architecture.
3Loss of information
If the system generates both answer and visual explanation, then the completeness of response is improved, but the processing time increases
Solution Approach 1:
The patent divides the response generation process into two specialized models: the answer model that efficiently generates concise answers, and the description model that generates visual explanations. This segmentation allows parallel processing of answer and explanation generation, maintaining response completeness while optimizing processing time through specialized task allocation.
Solution Approach 2:
The system performs preliminary processing by generating chart specifications and extracting key features before answer and explanation generation. This preliminary action prepares the data in an optimized format that accelerates subsequent processing by both models, reducing overall processing time while maintaining complete and accurate responses.
Data Source
AI summary
Systems and methods for natural language processing are described. Embodiments of the present disclosure obtain a chart and a query via a user interface. An answer model generates an answer to the query based on the chart, wherein the answer model comprises a machine learning model trained based on chart data for the chart. A description model generates a visual description based on the answer and the chart, wherein the description model comprises a machine learning model trained based on a chart specification for the chart. A response component transmits a response to the query based on the answer and the visual description.