AI Bad Case Discovery in NER Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for discovering bad cases in Named Entity Recognition (NER) systems are labor-intensive and inefficient, relying on manual operations and random sampling, which results in a low discovery rate and affects subsequent optimization and updates.
Innovation Solution
An AI-based method that performs named entity recognition, calculates confidence levels and probability values for character combinations, and determines bad cases using a trained probability value evaluating model, reducing manual effort and improving processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual operations and random sampling are used to discover bad cases, then the process is simple to implement, but the discovery rate of bad cases is very low and manpower costs are very high
Solution Approach 1:
The patent replaces manual mechanical operations with an automated computing system that performs named entity recognition, calculates confidence levels, evaluates probability values using word embedding models, and automatically identifies bad cases through computational analysis rather than human review
Solution Approach 2:
The system performs self-diagnosis by automatically evaluating its own recognition results through confidence level thresholds and probability value assessments, enabling it to identify its own bad cases without external manual intervention
2Productivity
If manual operations are used to review and analyze sampled recognition results, then the process is easy to understand, but the processing efficiency is low
Solution Approach 1:
The patent replaces manual review and analysis processes with automated computational procedures including named entity recognition algorithms, confidence level calculations, and probability value evaluations performed by computer processors
Solution Approach 2:
The system performs preliminary automated filtering and evaluation of recognition results using confidence level thresholds and probability value assessments before final bad case identification, preprocessing the data to efficiently identify problematic cases
Data Source
AI summary
The present disclosure provides a method and apparatus of discovering a bad case based on artificial intelligence, a device and a storage medium, wherein the method comprises: performing named entity recognition for a to-be-recognized query, and respectively obtaining a confidence level of each character in the query; respectively obtaining a probability value of each character of forming a word with a neighboring character in the query; determining whether there is a bad case according to the confidence level and the probability value. The solution of the present disclosure may be applied to save man power costs, and improve the processing efficiency and enhance a discovery rate of bad cases.