AI OCR Data Management for Text and Table Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional OCR-based unstructured data conversion devices fail to efficiently extract, identify, and utilize texts and tables in image and PDF files, lack data management systems, and require manual data collection and analysis, leading to increased worker hours and difficulty in managing big data.
Innovation Solution
An integrated management device using AI OCR, preprocessing, and a big data platform to extract and identify texts and tables, perform data management, and automate data collection and analysis, enabling efficient data storage, search, and conversion into desired formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional OCR-based devices store extracted texts only in relational databases, then data storage is achieved, but the ability to extract, identify, and utilize texts and tables in image and PDF files is lost
Solution Approach 1:
The patent segments the data processing pipeline into distinct modules: an extraction module that separates text and table data from image and PDF files, a conversion module that transforms unstructured data into structured formats, and a storage module that saves processed data in relational databases. This segmentation enables the system to maintain storage capability while adding extraction and identification functions.
Solution Approach 2:
The patent merges multiple functions into a single integrated device: OCR text recognition, table structure identification, data extraction from various file formats, and relational database storage. By combining these previously separate functions into one system, the device achieves both reliable data storage and versatile text/table extraction capabilities.
2Ease of manufacture
If manual work is used for data collection and analysis, then data processing can be performed, but worker hours increase
Solution Approach 1:
The patent implements self-service automation where the system automatically performs data collection from multiple sources, extracts text and tables using AI OCR, converts unstructured data to structured formats, and loads processed data into databases without human intervention. This eliminates the need for manual data processing work while maintaining comprehensive data processing capabilities.
Solution Approach 2:
The system performs preliminary automated actions including pre-processing of raw data, pre-extraction of text and tables before storage, and pre-conversion of unstructured data formats. By performing these actions automatically before human review or use, the system reduces the time workers would otherwise spend on repetitive data collection and preparation tasks.
3Adaptability or versatility
If experts are hired to process data into desired forms, then data can be managed, but it becomes difficult to hire and systematically manage big data
Solution Approach 1:
The patent replaces the mechanical system of hiring and managing expert human processors with an automated AI-based processing system. The AI OCR and machine learning models automatically perform data formatting, extraction, and conversion tasks that previously required expert knowledge, thereby reducing hiring difficulty while maintaining high-quality data processing capabilities.
Solution Approach 2:
The system changes the operational parameters from human expert judgment to automated algorithmic processing. By transforming data processing from a human-centric activity dependent on expert availability to an automated process driven by configurable parameters and AI models, the system reduces management complexity while preserving adaptability to different data formats and requirements.
4Measurement precision
If AI OCR and preprocessing modules are added to extract and process data, then extraction accuracy improves, but device complexity increases
Solution Approach 1:
The patent implements a universal AI OCR engine that handles multiple file types (images, PDFs, various document formats) and performs multiple functions (text recognition, table structure identification, data extraction) through a single integrated system. This multi-functionality improves extraction accuracy across diverse data sources while avoiding the complexity of maintaining separate specialized tools for each function.
Solution Approach 2:
The patent introduces an intermediary preprocessing module that acts as a bridge between raw data input and final storage. This intermediary layer standardizes data from various sources into a common format, improving extraction accuracy through consistent processing while encapsulating complexity within the preprocessing module, thereby keeping the overall system architecture manageable.
Data Source
AI summary
The present disclosure relates to an integrated management device on a big data platform. The integrated management device includes a processor that extracts a text and table in an image file and in a Portable Document Format (PDF) file, inputs unstructured data related to the text and table in the image file and in the PDF file into an Artificial Intelligence (AI) model, and stores the structured data that is outputted from the AI model in a relational database in a key and value form, perform an integrated search on the unstructured data and the structured data in a requested public data when a search request for the requested public data among the pieces of public data is received from a user terminal, obtains found public data in response to the integrated search being performed, and displays the found public data through the user terminal.