BERT Model Input Noise Reduction via Entity Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing methods using the BERT model are hindered by excessive noise from knowledge graphs, which affects the accuracy and efficiency of the model's output.
Innovation Solution
A method that identifies and adds extended information to the input data of the BERT model by determining the second entity in maximum correlation with the first entity within a preset entity set, thereby reducing noise and improving data relevance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If knowledge graph information is added to BERT model input to improve task processing accuracy, then the model can access more contextual information, but excessive noise from low-relevance information increases input data amount and reduces processing efficiency
Solution Approach 1:
The patent extracts only the most relevant knowledge graph entities and relations for each input sentence by calculating correlation scores between sentence entities and knowledge graph entities. This selective extraction removes excessive noise while preserving useful information, thereby maintaining processing accuracy while improving efficiency.
Solution Approach 2:
The patent applies different processing strategies to different parts of the knowledge graph based on their relevance to the input sentence. High-correlation entities receive detailed processing while low-correlation entities are filtered out, creating a locally optimized information structure that balances accuracy and efficiency.
2Loss of information
If all knowledge graph entities are included in BERT model input to ensure comprehensive information, then information completeness is improved, but the input data amount increases and affects BERT model accuracy
Solution Approach 1:
The patent extracts only the most relevant knowledge graph entities and relations for each input sentence by calculating correlation scores between sentence entities and knowledge graph entities. This selective extraction removes excessive noise while preserving useful information, thereby maintaining processing accuracy while improving efficiency.
Solution Approach 2:
The patent changes the parameter of information inclusion from binary (include/exclude) to continuous by using correlation scores. Entities are included or excluded based on their correlation score thresholds, allowing dynamic adjustment of information completeness while maintaining model accuracy.
3Reliability
If extensive knowledge graph data is processed to reduce noise, then information quality improves, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary filtering of knowledge graph entities by calculating correlation scores before feeding data to the BERT model. This preliminary action identifies and removes low-quality information in advance, ensuring high information quality while avoiding the computational overhead of processing all possible entities.
Solution Approach 2:
The patent applies partial processing to the knowledge graph by focusing computational resources only on high-correlation entities rather than processing all entities equally. This partial action achieves high information quality for relevant entities while minimizing overall processing time.
Data Source
AI summary
Embodiments of the present application disclose a natural language processing method and apparatus, a device, and a readable storage medium. The method includes: obtaining a target sentence to be processed, and determining each first entity in the target sentence; for each first entity, in response to the first entity being present in a preset entity set, determining, in the preset entity set, a second entity in maximum correlation with the first entity, generating extended information based on the determined second entity, and adding the extended information after a location of the first entity in the target sentence, to obtain an updated target sentence, where the second entity is any entity in the preset entity set other than the first entity; and inputting the updated target sentence to a bidirectional encoder representations from transformer (BERT) model, such that the BERT model performs a natural language processing task.