Segmented On-Device Knowledge Extraction from Visually Rich Documents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing knowledge extraction systems from unstructured data sources, particularly visually rich documents, require significant computational resources and are often implemented as server-based or cloud-based systems, exposing sensitive information to potential attacks and vulnerabilities.
Innovation Solution
Implementing a knowledge extraction system locally on edge devices, such as smartphones, using parallel processing of image segments to detect and classify entities within visually rich documents, reducing the need for remote access and minimizing computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If knowledge extraction is implemented as server-based or cloud-based systems, then computational resources are sufficient to process unstructured data sources, but sensitive information becomes vulnerable to attacks and transmission over public channels
Solution Approach 1:
The patent segments the knowledge extraction system into modular components that can run locally on edge devices. By dividing the system into smaller units (image processing module, text recognition module, entity extraction module), it enables distributed processing without requiring centralized cloud infrastructure, thus improving security while managing complexity.
Solution Approach 2:
The patent implements self-service by enabling edge devices to perform knowledge extraction autonomously without requiring external server assistance. The local processing capability allows devices to extract entities from images independently, eliminating the need to transmit sensitive data over public channels while maintaining extraction functionality.
2Reliability
If knowledge extraction is performed locally on edge devices, then data security is improved and transmission over public channels is eliminated, but computational resources on the device are limited
Solution Approach 1:
The patent divides the image processing task into segments that can be handled by limited edge device resources. By processing images in blocks and extracting features locally, the system reduces the computational burden on individual devices while maintaining security.
Solution Approach 2:
The patent applies partial action by extracting only the necessary entities and features from images rather than performing complete processing. This selective extraction approach reduces computational requirements on edge devices while still achieving the security benefit of local processing.
3Measurement precision
If complete images are transmitted to remote systems for processing, then accurate entity extraction can be achieved, but memory consumption and transmission time increase
Solution Approach 1:
The patent extracts only the essential features and entities from images rather than transmitting or processing complete images. By taking out only the necessary information (entity classes, key features), it reduces memory consumption and transmission requirements while maintaining extraction accuracy.
Solution Approach 2:
The patent performs partial processing by focusing only on extracting entities and their classes rather than analyzing all image data. This selective approach reduces the quantity of data that needs to be stored and transmitted while preserving the accuracy of entity extraction.
Data Source
AI summary
Computer-based content understanding can include segmenting an image into a plurality of blocks, wherein each block includes textual information from the image. For each block of the plurality of blocks, encoded feature data is generated by encoding visual information of the block and visual information of one or more neighboring blocks from the plurality of blocks and encoded textual data is generated by encoding the textual information of the block and the textual information of the one or more neighboring blocks. Further, using an entity class prediction model, one or more tokens of the block are classified into one or more entity classes based on a combination of the encoded textual data and the encoded feature data. A plurality of entities can be extracted from the image based on the entity classes of the plurality of blocks.