Synthetic Business Document Generation for Domain-Specific LLM Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large language models require extensive domain-specific training data for accurate document understanding, but in many domains, the availability of such data is limited, leading to reduced accuracy in performance.
Innovation Solution
A method to generate synthetic business documents by creating domain-agnostic models of spatial and content distributions of key value pairs using multivariate Gaussian distributions, deriving domain-specific models, and synthesizing documents to augment training data for large language models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If domain-specific training data is used for large language models, then model accuracy in document understanding tasks is improved, but data availability is limited in many domains
Solution Approach 1:
The patent creates synthetic copies of real business documents by extracting key-value pairs and regenerating documents that preserve the statistical properties (spatial distribution, content distribution, key relationships) of original documents. This copying approach generates unlimited domain-specific training data without requiring additional real-world documents, directly resolving the contradiction between needing accurate domain-specific data and facing data scarcity.
Solution Approach 2:
The patent transforms the training data generation process by changing from using only real documents to using synthetically generated documents that match the statistical parameters of real documents. By modeling and reproducing key parameters such as spatial distribution of elements, content distribution patterns, and key-value relationships, the system maintains data quality while expanding data quantity through synthetic generation.
2Measurement precision
If extensive domain-specific training data is collected, then model performance is improved, but time and resources for data collection and preparation increase
Solution Approach 1:
The patent performs preliminary analysis of real business documents to extract and model their statistical properties (spatial distribution, content distribution, key relationships) before generating synthetic data. By pre-establishing these models from a small set of real documents, the system eliminates the need for time-consuming collection and preparation of large amounts of real domain-specific data, while still achieving high model performance through realistic synthetic data generation.
Solution Approach 2:
Instead of manually collecting and preparing extensive real domain-specific data, the system creates synthetic copies that inherit the statistical characteristics of real documents. This copying mechanism automatically generates large volumes of training data with proper domain-specific properties, dramatically reducing the time and resources required for data preparation while maintaining model performance.
Data Source
AI summary
According to one embodiment, a method, computer system, and computer program product for generating synthetic business documents for data augmentation is provided. The embodiment may include identifying a first set of key value pairs (KVPs) within a set of business documents spanning multiple domains. The embodiment may include creating domain-agnostic models of spatial distribution and content distribution of KVPs within the set. The embodiment may include grounding the domain-agnostic models of spatial distribution and content distribution using a second set of domain-specific KVPs to derive domain-specific models of spatial distribution and content distribution of KVPs within the second set. The embodiment may include generating a set of synthetic domain-specific business documents using the derived domain-specific models of spatial distribution and content distribution. The embodiment may include augmenting a training data set of a large language model (LLM) with the set of synthetic domain-specific business documents.

