Data Table Entity Recognition With Table-Specific Query Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for entity recognition in data tables using large language models or deep learning models are inaccurate when the training sample coverage is insufficient, failing to accurately recognize corresponding entities across different data tables from a single query statement.
Innovation Solution
A method and apparatus for entity recognition in data tables that involves encoding query statements using different encoding manners for completely and partially matched word segments, matching the query statement with candidate data tables, and utilizing an entity recognition model to predict entities based on encoded results, ensuring accuracy by referencing entity information specific to each data table.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single entity recognition model is used for multiple data tables, then device complexity is reduced, but measurement precision deteriorates due to insufficient training sample coverage
Solution Approach 1:
The patent divides the entity recognition task into multiple specialized models, each trained on a specific data table. This segmentation allows each model to achieve high precision for its designated table without requiring a single complex model to handle all tables, thus resolving the contradiction between model complexity and recognition precision.
Solution Approach 2:
Each entity recognition model is optimized with local quality specific to its trained data table, including table-specific entity features and characteristics. This enables each model to achieve high precision for its specific table while keeping individual model complexity manageable, addressing the trade-off between precision and overall system complexity.
2Measurement precision
If multiple specialized entity recognition models are used for different data tables, then entity recognition precision is improved, but device complexity increases
Solution Approach 1:
The system segments entity recognition into multiple specialized models, each handling a specific data table. This segmentation improves precision for each table while managing complexity by creating simpler, focused models rather than one large complex model.
Solution Approach 2:
The patent implements a universal entity recognition framework that can accommodate multiple specialized models. The system provides multi-functionality by handling different data tables through a unified interface, reducing the operational complexity despite having multiple specialized models.
3Measurement precision
If encoding is performed for each data table separately, then entity recognition precision is improved, but processing time increases
Solution Approach 1:
The patent performs preliminary encoding of query statements for each data table before entity recognition. This preliminary action prepares the input data in advance with table-specific encoding, improving recognition precision while allowing the actual recognition process to be faster and more efficient.
Data Source
AI summary
A method of entity recognition for a data table, an electronic device and storage medium are provided. The method includes: obtaining a to-be-processed query statement described in a natural language; encoding the to-be-processed query statement according to entity information of a to-be-query data table for each to-be-query data table in at least one to-be-query data table, to acquire at least one encoded result; respectively inputting the at least one encoded result into an entity recognition model, to acquire at least one entity prediction result; and determining a target entity prediction result corresponding to the to-be-processed query statement according to the at least one entity prediction result, wherein the target entity prediction result indicates the entity included in the to-be-processed query statement.