Image-Guided Chart Template Selection for Reliable Data Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, particularly large language models (LLMs), are prone to hallucinations when generating code for data visualizations, leading to inaccurate or misleading charts, complicating software development and increasing the risk of errors.
Innovation Solution
A computing system uses machine learning models to automate data visualization by applying an image encoding model to determine chart templates and query parameters, reducing the potential for hallucinations by relying on predetermined program instructions and actual database queries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are used to generate code for data visualizations, then development efficiency is improved, but accuracy deteriorates due to hallucinations
Solution Approach 1:
The system segments the code generation process into distinct components: a chart template model that selects from predetermined templates and a query parameter model that extracts parameters from images. This segmentation allows each component to specialize, reducing hallucinations while maintaining automation benefits.
Solution Approach 2:
The system performs preliminary action by pre-defining chart templates with correct code structures before generation is needed. This allows the model to select from validated templates rather than generating code from scratch, improving accuracy while maintaining efficiency.
2Extent of automation
If machine learning models generate chart code from scratch, then automation is improved, but reliability deteriorates due to model hallucinations
Solution Approach 1:
The system uses copying by selecting from predetermined chart templates that contain pre-validated code structures. Instead of generating code from scratch, the model copies and adapts proven templates, maintaining automation while improving reliability through template validation.
Solution Approach 2:
The chart template acts as an intermediary between the image input and the final code output. This intermediary layer provides a structured, validated intermediate representation that reduces direct hallucination risks while maintaining automation flow.
3Manufacturing precision
If traditional software development methods are used, then accuracy is maintained, but productivity deteriorates due to manual effort
Solution Approach 1:
Chart templates are prepared in advance with correct code structures and validated syntax. This preliminary preparation allows rapid assembly of accurate code during development, maintaining precision while improving productivity through template-based reuse.
Solution Approach 2:
The system changes parameters by extracting specific values from images (such as data values, labels, and configurations) and inserting them into predefined templates. This parameter substitution approach maintains code accuracy while automating the adaptation process for different charts.
Data Source
AI summary
Techniques are disclosed relating to automating data visualization using machine learning. In some embodiments, a computer system receives a request for program instructions to render a graphical chart from data stored in a database of the computing system. The request includes an image of a desired graphical chart. The computer system applies a machine learning model to the image to determine one or more query parameters associated with the desired graphical chart. The computer system provides the requested program instructions to render the graphical chart. The requested program instructions include a database query specifying one or more query parameters to retrieve the data from the database. The computer system may render the graphical chart based on the provided program instructions.