Chinese named entity recognition retrieval enhancement framework based on uncertain components
By using an enhanced framework for Chinese named entity recognition and retrieval based on uncertain components, and employing Monte Carlo dropout or Top-K label sequence methods for uncertain component sampling and knowledge fusion, the problems of dictionary dependence and low retrieval efficiency are solved, achieving efficient and accurate Chinese named entity recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2022-02-17
- Publication Date
- 2026-04-28
AI Technical Summary
Among existing Chinese named entity recognition technologies, dictionary-based methods are expensive and cannot take into account both dynamic maintenance and semantic information, while retrieval-based methods are inefficient and introduce interference into pre-trained models, making it difficult to efficiently and accurately recognize Chinese named entities.
An enhanced framework for Chinese named entity recognition and retrieval based on uncertain components is adopted. Uncertain components are sampled and retrieved through a named entity recognition model, and prediction is performed by combining a knowledge fusion model. Uncertain components are sampled using Monte Carlo dropout or Top-K label sequence methods to obtain the uncertain components of the entity set and perform knowledge retrieval. Finally, the BERT model is used for recognition.
It achieves efficient and accurate Chinese named entity recognition, saving the cost of building and dynamically maintaining a high-quality dictionary, eliminating recognition ambiguity, and improving recognition efficiency.
Smart Images
Figure CN116663559B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of Chinese named entity recognition technology, specifically relating to an enhanced framework for Chinese named entity recognition retrieval based on uncertain components. Background Technology
[0002] Named entity recognition (NER) is a task that extracts specific types of text fragments from a given text, such as extracting people, places, and organizations. For Chinese NER, due to the lack of word segmentation symbols and the polysemy of Chinese characters, the task relies on dictionaries to eliminate out-of-vocabulary words during prediction. However, building and dynamically maintaining a high-quality dictionary is costly for any given application. Furthermore, dictionary-based methods cannot simultaneously address both the dynamic nature of the dictionary and the semantic information of the words. One approach uses word embeddings to incorporate word information, but updating the dictionary requires retraining the model; another approach utilizes only word boundary information, failing to incorporate semantic information.
[0003] Retrieval-based methods can utilize more general knowledge resources (such as search engines), and the retrieved knowledge can be used to eliminate ambiguity in the identification process. However, past retrieval methods employed a traversal strategy, which is highly inefficient, especially in online retrieval scenarios. Furthermore, with the development of pre-trained model technology, neural network models are now capable of accurately identifying most entity samples. Supplementing these samples with additional knowledge through retrieval is ineffective and may even introduce interference. Summary of the Invention
[0004] This invention is made to solve the above-mentioned problems, and its purpose is to provide a high-precision, efficient, and easily dynamically maintainable framework for Chinese named entity recognition and retrieval. The technical solution adopted in this invention is as follows:
[0005] This invention provides an enhanced framework for Chinese named entity recognition and retrieval based on uncertain components. It is characterized by comprising: a named entity recognition model for sampling uncertain components of an input sequence to obtain an entity set of uncertain components, and performing retrieval based on the entity set of uncertain components to obtain a knowledge sequence; and a knowledge fusion model for predicting the input sequence based on the knowledge sequence to obtain an entity recognition result, wherein the entity recognition result is a set of several Chinese names.
[0006] The Chinese named entity recognition and retrieval enhancement framework based on uncertain components provided by this invention may also have the following technical features, wherein the named entity recognition model, for the input sequence X = [x1, x2, ..., x...], ... nPrediction is performed to obtain the predicted label sequence Lp and the candidate sequence Lc. The uncertain component U = [x] of the entity set is obtained by parsing the predicted label sequence Lp and the candidate sequence Lc. i ,x i+1 ,…,x i+o ].
[0007] The Chinese named entity recognition and retrieval enhancement framework based on uncertain components provided by this invention may also have the following technical features, wherein the named entity recognition model uses a Monte Carlo dropout-based method to sample the uncertain components, including the following steps:
[0008] Step S1-1: Input the input sequence X into the named entity recognition model for prediction and perform Viterbi decoding to obtain the predicted label sequence Lp, where the predicted label sequence Lp is the predicted entity set;
[0009] Step S1-2: Enable dropout, then input the input sequence X into the named entity recognition model for prediction, and perform Viterbi decoding to obtain the candidate sequence Lc, where the candidate sequence Lc is the difference between the predicted entity set;
[0010] Steps S1-3: The difference between the entity set and the entity set is counted as an uncertain entity. All overlapping or adjacent uncertain entities are merged to obtain the uncertain component U of the entity set.
[0011] The Chinese named entity recognition and retrieval enhancement framework based on uncertain components provided by this invention may also have the following technical features, wherein the named entity recognition model uses a Top-K label sequence-based method to sample the uncertain components, including the following steps:
[0012] Step S2-1: The named entity recognition model predicts the input sequence X to obtain the label probability distribution;
[0013] Step S2-2: Apply a Viterbi decoding variant to the label probability distribution to obtain the Top-K label sequence, which is used as the predicted label sequence Lp. The remaining part of the input sequence X is used as the candidate sequence Lc. The predicted label sequence Lp is the predicted entity set, and the candidate sequence Lc is the difference between the predicted entity sets.
[0014] Step S2-3: Filter out sequences in the predicted label sequence Lp that differ from the candidate sequence Lc within one label;
[0015] Step S2-4: The difference between the entity set and the entity set is counted as an uncertain entity. All overlapping or adjacent uncertain entities are merged to obtain the uncertain component U of the entity set.
[0016] The Chinese named entity recognition and retrieval enhancement framework based on uncertain components provided by this invention can also have the following technical features: when the uncertain component U of the entity set is empty, the predicted label sequence Lp is used as the entity recognition result; when the uncertain component U of the entity set is not empty, the uncertain component U of the entity set is used as a keyword for retrieval and the retrieved content is organized to obtain the knowledge sequence K = [k1, k2, ..., k n ].
[0017] The Chinese named entity recognition and retrieval enhancement framework based on uncertain components provided by this invention may also have the following technical feature: the knowledge sequence K is concatenated with the input sequence X to obtain an input sequence with fused knowledge. and the input sequence of the fused knowledge The predicted label sequence Lp is embedded into a dense vector and added to it before being input into the knowledge fusion model to obtain the entity recognition result.
[0018] The Chinese named entity recognition and retrieval enhancement framework based on uncertain components provided by this invention may also have the following technical features, wherein the knowledge fusion model is a BERT model based on Transformer.
[0019] The Chinese Named Entity Recognition and Retrieval Enhancement Framework based on uncertain components provided by this invention may also have the following technical features, wherein the Named Entity Recognition Model is a basic NER model.
[0020] Invention Function and Effect
[0021] The Chinese Named Entity Recognition and Retrieval Enhancement Framework based on uncertain components according to the present invention, by including a named entity recognition model, can sample uncertain components from the input Chinese sequence and perform retrieval based on the uncertain components of the sampled entity set. Therefore, compared with traditional dictionary-dependent methods, it can effectively retrieve the required knowledge sequence without incurring the high costs of constructing and dynamically maintaining a high-quality dictionary, saving a significant amount of computing power. Furthermore, by including a knowledge fusion model, it can perform Chinese named entity recognition and prediction based on the retrieved knowledge sequence, thus eliminating ambiguity in the recognition process and obtaining more accurate prediction results. Compared with traditional detection methods using traversal strategies, it is more efficient. In summary, the enhancement framework of the present invention can efficiently obtain accurate Chinese named entity recognition results and save a significant amount of computing power. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of the structure of the Chinese named entity recognition and retrieval enhancement framework based on uncertain components in an embodiment of the present invention;
[0023] Figure 2 This is an example diagram illustrating the process of parsing the predicted label sequence and the candidate sequence set to obtain the uncertain components of the entity set in an embodiment of the present invention. Detailed Implementation
[0024] To make the technical means, creative features, objectives and effects of this invention easy to understand, the following describes in detail the Chinese named entity recognition and retrieval enhancement framework based on uncertain components of this invention with reference to embodiments and accompanying drawings.
[0025] <Example 1>
[0026] Figure 1 This is a schematic diagram of the structure of the Chinese named entity recognition and retrieval enhancement framework based on uncertain components in this embodiment.
[0027] like Figure 1 As shown, the Chinese named entity recognition retrieval enhancement framework based on uncertain components consists of a two-stage model, which combines uncertain component sampling and retrieval methods.
[0028] In the first stage, a named entity recognition model (i.e. Figure 1 The Base Model in the algorithm samples the uncertain components of the input sequence to obtain the uncertain components of the entity set, and then retrieves the knowledge sequence based on the sampled uncertain components of the entity set.
[0029] Specifically, the input sequence X = [x1, x2, ..., x... nThe input sequence X contains several Chinese characters. The input sequence X is fed into the basic NER model for prediction, resulting in the predicted label sequence Lp and candidate sequences Lc. Furthermore, by analyzing the predicted label sequence Lp and candidate sequences Lc, the uncertain component U = [x] can be obtained. i ,x i+1 ,…,x i+o If the uncertain component U is empty, it means that the input sequence X has low difficulty and does not contain uncertain components. The predicted label sequence Lp is the final prediction result (entity recognition result). If the uncertain component U is not empty, it means that the input sequence X contains uncertain components. In this case, the uncertain component U is used as a keyword for retrieval, and the retrieved content is organized to obtain the knowledge sequence K = [k1, k2, ..., k n ].
[0030] Among them, the named entity recognition model uses the Monte Carlo dropout method to sample uncertain components. Monte Carlo dropout keeps dropout on during prediction and estimates uncertainty by summarizing the differences in multiple forward propagations. It is a commonly used uncertainty estimation method in classification tasks.
[0031] Figure 2 This is an example diagram illustrating the uncertain components of the entity set obtained by parsing the predicted label sequence and the candidate sequence set in this embodiment.
[0032] The following will combine Figure 2 Explain the specific steps for sampling uncertain components using the Monte Carlo dropout method:
[0033] Step S1-1: Input the input sequence X into the named entity recognition model for prediction and perform Viterbi decoding to obtain the predicted label sequence Lp, where the predicted label sequence Lp is the predicted entity set.
[0034] Step S1-2: Enable dropout, then input the input sequence X into the named entity recognition model for prediction K times, and perform corresponding Viterbi decoding K times to obtain the candidate sequence Lc, where the candidate sequence Lc is the difference between the predicted entity sets;
[0035] Steps S1-3: The difference between the entity set and the entity set is counted as an uncertain entity. All overlapping or adjacent uncertain entities are merged to obtain the uncertain component U of the entity set.
[0036] Through the above steps, uncertain components were sampled using the Monte Carlo dropout method, resulting in uncertain components of the entity set "Asia-Pacific", "OECD", "APEC", and "organization". These uncertain components of the entity set can be used to retrieve the required knowledge more effectively.
[0037] In the second phase, a knowledge fusion model (i.e.) is used. Figure 1 The Knowledge Fusion Model in the first stage is used to make predictions by combining the knowledge (knowledge sequence) obtained in the first stage.
[0038] Specifically, the knowledge sequence K is concatenated with the input sequence X to obtain the input sequence that integrates the knowledge. Then integrate the input sequence of knowledge The predicted label sequence Lp is embedded into a dense vector and added together before being input into the knowledge fusion model for prediction, thus obtaining the final entity recognition result, which is a set of several Chinese names. In this embodiment, the knowledge fusion model is a pre-trained model based on Transformer (e.g., BERT).
[0039] <Example 2>
[0040] This embodiment provides an enhanced framework for Chinese named entity recognition and retrieval based on uncertain components, and its structure is similar to that of... Figure 1 As shown, compared with Example 1, the difference lies in that, in the first stage, the named entity recognition model uses a method based on Top-K label sequences to sample uncertain components, specifically including the following steps:
[0041] Step S2-1: The named entity recognition model predicts the input sequence X to obtain the label probability distribution;
[0042] Step S2-2: Apply a Viterbi decoding variant to the label probability distribution to obtain the Top-K label sequence, which is used as the predicted label sequence Lp. The remaining part of the input sequence X is used as the candidate sequence Lc. Similarly, the predicted label sequence Lp is the predicted entity set, and the candidate sequence Lc is the difference between the predicted entity sets.
[0043] Step S2-3: Filter out sequences within one label that differ from the predicted label sequence Lp and the candidate sequence Lc.
[0044] Step S2-4: The difference between the entity set and the entity set is counted as an uncertain entity. All overlapping or adjacent uncertain entities are merged to obtain the uncertain component U of the entity set mentioned above.
[0045] Through the above steps, uncertain components were sampled using the Top-K label sequence method. Similarly, the required knowledge can be retrieved more effectively through the uncertain components of the entity set. Compared with the Monte Carlo Dropout method in Example 1, it consumes less computing power because it does not require multiple forward propagations.
[0046] In this embodiment, the remaining structure and working steps are the same as in Embodiment 1, so they will not be described again.
[0047] Functions and effects of the embodiments
[0048] The Chinese Named Entity Recognition and Retrieval Enhancement Framework based on Uncertain Components provided by embodiments of the present invention, due to its inclusion of a named entity recognition model, can sample uncertain components from the input Chinese sequence and perform retrieval based on the uncertain components of the sampled entity set. Therefore, compared to traditional dictionary-dependent methods, it can effectively retrieve the required knowledge sequence without incurring the high costs of constructing and dynamically maintaining a high-quality dictionary, saving significant computational resources. Furthermore, due to the inclusion of a knowledge fusion model, it can perform Chinese named entity recognition and prediction based on the retrieved knowledge sequence, thus eliminating ambiguity in the recognition process and obtaining more accurate prediction results. Moreover, it is more efficient than traditional detection methods using traversal strategies. In summary, the enhancement framework of the embodiments of the present invention can efficiently obtain accurate Chinese named entity recognition results and save significant computational resources.
[0049] Specifically, in Example 1, the named entity recognition model uses a Monte Carlo dropout-based method to sample uncertain components. Since multiple predictions of Monte Carlo dropout can be executed in parallel, it takes less time and can quickly obtain uncertain components, thereby effectively retrieving the required knowledge based on uncertain costs.
[0050] In Example 2, the named entity recognition model uses the Top-K label sequence method for uncertain cost sampling. Since the Monte Carlo Dropout method requires multiple forward propagations, it consumes a lot of computing power. However, the Top-K label sequence method can save a lot of computing power, making the added framework in this example applicable to environments with relatively limited computing resources, thus expanding its application scope.
[0051] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the scope of the description of the above embodiments.
Claims
1. A method for enhancing Chinese named entity recognition and retrieval based on uncertain components, characterized in that, include: The input sequence is sampled for uncertain components by a named entity recognition model to obtain uncertain components of the entity set, and a knowledge sequence is obtained by searching based on the uncertain components of the entity set. as well as The input sequence is predicted based on the knowledge sequence using a knowledge fusion model to obtain entity recognition results, which are a set of several Chinese names. The named entity recognition model uses a Monte Carlo dropout-based method to sample the uncertain components, including the following steps: Step S1-1, the input sequence The named entity recognition model is input for prediction and Viterbi decoding is performed to obtain the predicted label sequence Lp, where the predicted label sequence Lp is the predicted set of entities; Step S1-2: Enable dropout, and then process the input sequence. The named entity recognition model is input for prediction and Viterbi decoding is performed to obtain a candidate sequence Lc, where the candidate sequence Lc is the difference between the predicted entity set; Steps S1-3: The difference between the entity set and the entity set is counted as an uncertain entity; all overlapping or adjacent uncertain entities are merged to obtain the uncertain component of the entity set. , The named entity recognition model uses a Top-K label sequence-based method to sample the uncertain components, including the following steps: Step S2-1, the named entity recognition model processes the input sequence Make predictions to obtain the label probability distribution; Step S2-2: Apply a Viterbi decoding variant to the label probability distribution to obtain the Top-K label sequence, which is used as the predicted label sequence Lp. Then, use the input sequence... The remaining part is used as the candidate sequence Lc, the predicted label sequence Lp is the predicted entity set, and the candidate sequence Lc is the difference between the predicted entity set; Step S2-3: Filter out sequences in the predicted label sequence Lp that differ from the candidate sequence Lc within one label; Steps S2-4: The difference between the entity set and the entity set is counted as an uncertain entity; all overlapping or adjacent uncertain entities are merged to obtain the uncertain component of the entity set. , The knowledge sequence With the input sequence The input sequence of fused knowledge is obtained by splicing them together. and the input sequence of the fused knowledge The predicted label sequence Lp is embedded into a dense vector and added to it before being input into the knowledge fusion model to obtain the entity recognition result.
2. The Chinese named entity recognition and retrieval enhancement method based on uncertain components according to claim 1, characterized in that: in, The named entity recognition model processes the input sequence. Prediction is performed to obtain the predicted label sequence Lp and the candidate sequence Lc. The uncertain components of the entity set are obtained by parsing the predicted label sequence Lp and the candidate sequence Lc. .
3. The method for enhancing Chinese named entity recognition and retrieval based on uncertain components according to claim 1, characterized in that: in, When the entity set has uncertain components If the value is empty, the predicted label sequence Lp is taken as the entity recognition result. When the entity set has uncertain components If not empty, the uncertain components of the entity set will be... The retrieved content is then analyzed using keywords to obtain the knowledge sequence. ].
4. The Chinese named entity recognition and retrieval enhancement method based on uncertain components according to claim 1, characterized in that: in, The knowledge fusion model is a BERT model based on Transformer.
5. The Chinese named entity recognition and retrieval enhancement method based on uncertain components according to claim 1, characterized in that: in, The named entity recognition model is based on the NER model.