Chemical Table Extraction Using LLM Schemas and Knowledge Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques struggle to accurately and efficiently extract chemical molecules and their properties from complex tables in research publications and patents due to varied formats and lack of specialized chemical composition schemas, leading to incomplete or erroneous data extraction and challenges in integrating data into a unified knowledge graph.
Innovation Solution
A method and system using Large Language Models (LLMs) to identify structured tables, generate chemical composition instances, and reconcile them into a unified knowledge graph, leveraging deep learning-based boundary detection and chemical composition schemas for precise extraction and integration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to extract data from complex chemical tables, then the process is simpler to implement, but the extraction accuracy and completeness deteriorate due to inability to handle diverse table formats
Solution Approach 1:
The system segments the complex table extraction task into multiple specialized components: boundary detection module identifies table regions, table parsing module converts visual layouts to structured data, and LLM-based extraction module retrieves specific chemical information. This segmentation allows each component to specialize in handling diverse table formats, improving extraction accuracy without requiring a single monolithic complex system.
Solution Approach 2:
The patent introduces intermediate representations and schemas as mediators between the diverse table formats and the target chemical information. Table schemas define expected structures for different chemical tables, acting as intermediaries that standardize varied formats. The structured table representation serves as an intermediate form that bridges visual complexity and information extraction requirements.
2Reliability
If specialized processing is applied to handle diverse table formats, then extraction completeness improves, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-defining table schemas for different chemical table formats and pre-training LLMs on chemical domain knowledge. Boundary detection models are pre-trained to recognize table structures. This preliminary preparation enables the system to quickly process new tables without extensive real-time analysis, improving extraction completeness while controlling processing time.
Solution Approach 2:
The system employs dynamic adaptation mechanisms where the table parsing process adjusts to different table formats encountered during extraction. The LLM dynamically selects appropriate extraction strategies based on the detected table type and schema, allowing flexible handling of diverse formats without requiring exhaustive preprocessing of all possible table variations.
3Measurement precision
If manual verification is performed to ensure extraction accuracy, then data quality improves, but productivity and scalability deteriorate
Solution Approach 1:
The system implements feedback mechanisms where extraction results are validated against predefined chemical domain constraints and schemas. The LLM provides confidence scores for extracted information, and inconsistent results trigger re-extraction or flagging for review. This automated feedback loop maintains high data quality without requiring manual verification of every extraction, preserving productivity while ensuring accuracy.
Solution Approach 2:
The extraction system performs self-verification by checking extracted chemical information against domain knowledge and consistency rules. The LLM autonomously validates whether extracted molecules, properties, and relationships make chemical sense, reducing reliance on manual verification. This self-service capability maintains data quality while enabling scalable automated processing.
4Loss of information
If comprehensive table parsing is applied to extract all possible information, then information completeness improves, but data integration and reconciliation difficulty increases
Solution Approach 1:
The system employs a universal chemical information schema that can represent multiple types of chemical data (molecules, properties, reactions, relationships) in a unified structure. This multi-functional schema allows comprehensive information extraction from diverse table formats while providing a consistent representation for integration. The knowledge graph structure universally accommodates different information types, simplifying reconciliation despite information completeness.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Conventional models extract chemical data through querying tables by decomposing complex user queries. This disclosure relates generally to a method and system for extraction of chemical molecules and associated target properties from text and complex tables using Large Language Models (LLMs). The disclosed method extracts a plurality of molecular property values associated with the chemical molecules, from a plurality data sources, via generating a chemical composition schema utilizing LLMs. The chemical composition schema acts as a lens to view tabular information and a text. Relevant tables are identified and tabular information comprising chemical composition instances are extracted from a plurality of documents utilizing the chemical composition schema, along with LLMs and prompting techniques. The chemical composition instances are curated and reconciled into a unified knowledge graph, which is then used for querying. The disclosed method ensures precision in tabular information extraction without the need for extensive model training.