Edge Entity Extraction Using Visual-Text Models for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing entity extraction models face challenges in achieving accurate, real-time performance on client devices due to resource constraints, language scarcity, and the need to handle diverse web document layouts, leading to high latency and privacy concerns.
Innovation Solution
A multi-stage training process involving visual and text joint pretraining, cross-lingual fine-tuning, and model compression is employed to create a lightweight entity extraction model that considers both text and visual features, using edge computing to deploy on client devices, ensuring accuracy and privacy protection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If entity extraction is performed using traditional machine learning models on client devices, then extraction accuracy can be maintained, but device resources are excessively consumed and latency increases
Solution Approach 1:
The patent segments the entity extraction model into multiple specialized components (e.g., separate models for different entity types or language-specific models) that can be selectively loaded and executed based on the input document's requirements. This allows the system to maintain high extraction accuracy for specific tasks while consuming fewer device resources by not loading entire large-scale models unnecessarily.
Solution Approach 2:
The patent employs parameter changes by adapting model complexity and configuration based on the specific extraction task requirements. For example, using lighter model architectures for simple extraction tasks while deploying more complex models only when higher accuracy is needed, thereby optimizing the balance between extraction accuracy and device resource consumption.
2Reliability
If entity extraction models are deployed on client devices for privacy protection, then data privacy is enhanced, but model size and resource occupation increase
Solution Approach 1:
The patent divides large entity extraction models into smaller modular components that can be stored and executed on client devices. This segmentation reduces the immediate memory footprint and resource occupation while maintaining the capability to perform accurate entity extraction locally, thereby protecting user privacy without requiring excessively large model deployments.
Solution Approach 2:
The patent introduces intermediary mechanisms such as model distillation or knowledge transfer layers that enable compact model representations on client devices while preserving extraction capabilities. These intermediaries allow the system to maintain privacy-protecting local processing with reduced model size requirements.
3Loss of time
If entity extraction is performed in real-time on client devices, then latency is reduced, but extraction accuracy deteriorates due to resource constraints
Solution Approach 1:
The patent implements dynamic model selection and adaptation mechanisms that adjust the extraction process in real-time based on available device resources and task requirements. This allows the system to switch between different model configurations or processing strategies to maintain acceptable accuracy while achieving real-time performance constraints.
Solution Approach 2:
The patent performs preliminary actions by pre-processing input documents or pre-loading necessary model components into device memory before actual extraction occurs. This preparation work reduces the computational burden during real-time extraction, enabling both low latency and maintained accuracy through advance resource allocation.
Data Source
AI summary
The present disclosure proposes a method, an apparatus and a computer program product for entity extraction based on edge computing. A web document may be obtained. A text feature of the web document may be identified. A visual feature corresponding to the text feature may be identified. An entity type sequence corresponding to the web document may be extracted based on the text feature and the visual feature.