AI Chart De-Rendering With Separate Meta and Data Decoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional generative models for chart de-rendering fail to distinguish between meta and data information, leading to complex data representation, format errors, and inefficient token usage due to unnecessary tokenization of numbers.
Innovation Solution
An AI-based system that separates meta and data information using an image encoder, meta decoder, and data decoder, tokenizes data into single tokens, and uses a multi-layer perceptron (MLP) to extract numerical information efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If meta information and data information are included in a single data format, then the model can process all chart elements uniformly, but the data representation becomes complex and format errors increase
Solution Approach 1:
The patent segments chart information into two distinct data formats: meta information (chart title, axis labels, legend) and data information (numerical values). This segmentation allows each format to be optimized separately, reducing format errors while maintaining uniform processing through the unified model architecture.
2Adaptability or versatility
If numbers are treated as text and divided into irregular units, then the language model can process them, but token usage becomes excessive and inference speed decreases
Solution Approach 1:
The patent changes the parameter representation of numerical data by introducing a dedicated numeric embedding layer that directly processes numerical values as continuous parameters rather than tokenizing them as text. This allows the language model to process numbers efficiently without excessive tokenization, maintaining compatibility while improving inference speed.
3Measurement precision
If a rule-based model is used for chart de-rendering, then extraction accuracy is excellent, but scalability is poor due to requirement of separate models for each chart type
Solution Approach 1:
The patent merges the advantages of rule-based models and generative models by combining a unified transformer model architecture with specialized processing components (image encoder for visual patterns, numeric embedding layer for numerical data). This integration allows a single model to handle multiple chart types while maintaining high extraction accuracy through targeted processing mechanisms.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A chart de-rendering system, method, and program for extracting meta information and data information from a chart using artificial intelligence are disclosed. The system includes a memory in which an image encoder and an AI model are included, and a processor configured to execute or train the AI model, wherein the image encoder receives a chart, converts the chart into a first embedding processable by the AI model, and outputs the first embedding, and the AI model includes a meta decoder that outputs a second embedding including meta information from the first embedding, and a data decoder that outputs a fourth embedding including data information from a third embedding including information about an entity included in the second embedding. The data information included in the fourth embedding may be tokenized into a single token for each piece of data information, and a predefined repetitive template may be used in extracting the data information from the single tokens.