CRF Named Entity Recognition Using Sememe Library Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in efficiently extracting key information such as named entities and their classifications from large volumes of natural language text, particularly in medical contexts, where manual extraction is impractical due to the scale and continuous updates of data, and deep learning algorithms require substantial annotated data for effective performance.
Innovation Solution
A recognition method utilizing a conditional random field model that includes scoring functions and a sememe library to classify words or phrases based on part-of-speech, enabling the identification of named entities by matching text segments with sememes and determining their classifications, thereby reducing the need for extensive annotated data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning algorithms are used for named entity recognition, then recognition accuracy can be improved, but substantial annotated data is required which increases the complexity and cost of data preparation
Solution Approach 1:
The patent introduces a conditional random field model as an intermediary between the input text and the final named entity recognition results. This CRF model acts as a mediator that combines multiple scoring functions (including sememe matching and template-based scoring) to produce accurate classifications without requiring extensive annotated training data, thus resolving the contradiction between high accuracy and data preparation complexity
Solution Approach 2:
The patent segments the named entity recognition task into multiple independent scoring functions: sememe matching function, template function, and conditional random field model. Each function handles a specific aspect of the recognition task, allowing the system to achieve high accuracy through coordinated multiple simple functions rather than requiring a complex deep learning model with extensive training data
2Reliability
If manual extraction methods are used for named entities, then data quality can be maintained, but operational efficiency decreases due to large data volume and continuous updates
Solution Approach 1:
The patent enables the system to automatically process and classify named entities through the conditional random field model and sememe library, making the extraction process self-service without requiring manual intervention. The model autonomously handles large volumes of continuously updating data while maintaining quality through its multi-function scoring mechanism, thus resolving the contradiction between data quality and operational efficiency
3Measurement precision
If extensive annotated data is collected for training, then model performance can be improved, but time and resource consumption increase
Solution Approach 1:
The patent performs preliminary action by pre-building a sememe library and pre-defining multiple scoring functions and templates before the actual named entity recognition task. This preliminary preparation allows the conditional random field model to achieve high performance during inference without requiring time-consuming annotated data collection and model training, thus resolving the contradiction between model performance and annotation time
Data Source
AI summary
A recognition method and device of named entity, a creation method of dictionary of named entity, an electronic device, and a computer-readable medium. The method includes: obtaining target text; determining to-be-classified word and part-of-speech classification thereof in the target text according to CRF model, the CRF model including semantic function and template function; each template function being configured to provide a score for each character corresponding to each part-of-speech classification in the target text; the semantic function being configured to match at least part of word in the target text with a sememe in sememe library, and give a score of part-of-speech classification corresponding to a type attribute of the sememe in the sememe library for each character of the word when the word has a matched sememe; and determining the named entity in the target text according to the part-of-speech classification of the to-be-classified word.