Named entity recognition method and device based on context, and electronic equipment
By using a context-based named entity recognition method, multiple preset models and modules are used to process text, generate context related to entities, and calculate attention weights and relevance. This solves the problem of low accuracy in traditional methods and achieves higher recognition accuracy.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD
- Filing Date
- 2026-01-22
- Publication Date
- 2026-05-05
AI Technical Summary
Traditional named entity recognition methods are prone to producing incorrect recognition results when dealing with polysemous words or unclear contexts, resulting in low accuracy.
A context-based named entity recognition method is adopted. By acquiring the original text, text prompts are generated and processed using multiple preset models, including BiLSTM-CRF, BERT and sequence labeling models. Combined with dynamic context generation and context evaluation modules, attention weights and relevance are calculated to obtain entity labels.
It improves the accuracy of named entity recognition, reduces ambiguity, enhances semantic understanding, and improves the model's recognition performance.
Smart Images
Figure CN121981120A_ABST
Abstract
Description
Technical Field
[0001] This application relates to artificial intelligence, and to, but is not limited to, a context-based named entity recognition method, apparatus, and electronic device. Background Technology
[0002] Named Entity Recognition (NER) is an important task in the field of Natural Language Processing (NLP). Its goal is to identify entities with specific meanings in text and classify them into predefined categories, such as names of people, places, organizations, time expressions, quantity expressions, currency values, etc.
[0003] Traditional methods use large models to directly generate recognition results, which may produce incorrect recognition results when dealing with polysemous words or unclear contexts, and also lead to low accuracy. Summary of the Invention
[0004] In view of this, embodiments of this application provide a context-based named entity recognition method, apparatus, and electronic device.
[0005] The technical solution of this application embodiment is implemented as follows: This application provides a context-based named entity recognition method, the method comprising: acquiring original text; obtaining text prompts based on the original text; processing the text prompts using a first preset model to obtain context text; processing the context text using a second preset model to obtain attention weights; performing relevance calculation based on the attention weights to obtain related text; processing the related text using a third preset model to obtain entity labels; and performing named entity recognition based on the entity labels.
[0006] Optionally, obtaining text prompt words based on the original text includes: performing similarity matching on the original text based on RAG retrieval to obtain relevant examples; and constructing prompt words based on the relevant examples to obtain the text prompt words.
[0007] Optionally, before processing the text prompt words using the first preset model to obtain the context text, the process includes: acquiring historical log data; obtaining an initial dataset based on the historical log data; performing data annotation on the initial dataset based on the BiLSTM-CRF model to obtain an entity training dataset; and learning from the entity training dataset using a large language model to obtain the first preset model.
[0008] Optionally, the context text is processed by a second preset model to obtain attention weights, including: performing word segmentation on the context text to obtain text in a preset format; and processing the text in the preset format by the second preset model to obtain the attention weights; wherein the second preset model is the BERT model.
[0009] Optionally, processing the preset format text using a second preset model to obtain the attention weight further includes: performing data conversion on the preset format text to obtain time-series data; and obtaining the attention weight based on the time-series data.
[0010] Optionally, relevance calculation based on the attention weights to obtain relevant text includes: performing relevance calculation based on the attention weights to obtain a similarity score; determining whether the similarity score is greater than a first threshold to obtain a first determination result; and if the first determination result is yes, obtaining relevant text.
[0011] Optionally, the preset format text is converted to obtain time-series data, including: the formula is: ; ; ; ; ; in K, V These are q, k, and v obtained through linear mapping, respectively. Returns the K lowest frequency components with the largest amplitudes, where K is a predefined hyperparameter. , , ( The numbers () represent the k lowest-frequency Fourier components with the largest amplitudes extracted after FFT transformation and decomposition of the input features. This represents the attention weight values before performing the inverse fast Fourier transform, and then... Fill to zero Y This represents the final attention weight value calculated by converting from the frequency domain back to the time domain using IFFT.
[0012] Optionally, the relevant text is processed by a third preset model to obtain entity labels, including: extracting features from the relevant text to obtain relevant features; and predicting the relevant features by the third preset model to obtain entity labels; wherein the third preset model is a sequence labeling model.
[0013] A recognition device, comprising: an acquisition unit, an analysis unit, and a processing unit; the acquisition unit being used to acquire original text; the analysis unit being used to obtain text prompt words based on the original text; process the text prompt words using a first preset model to obtain context text; process the context text using a second preset model to obtain attention weights; and perform relevance calculation based on the attention weights to obtain related text; the processing unit being used to process the related text using a third preset model to obtain entity labels, and perform named entity recognition based on the entity labels.
[0014] An electronic device includes: a memory for storing at least one set of instructions; a processor for acquiring raw text; obtaining text prompts based on the raw text; processing the text prompts using a first preset model to obtain context text; processing the context text using a second preset model to obtain attention weights; performing relevance calculation based on the attention weights to obtain related text; processing the related text using a third preset model to obtain entity labels; and performing named entity recognition based on the entity labels.
[0015] This invention provides a context-based named entity recognition method, apparatus, and electronic device. First, it acquires the original text. Then, based on the original text, it obtains text prompts. Next, it processes the text prompts using a first preset model to obtain context text. Then, it processes the context text using a second preset model to obtain attention weights. Based on these attention weights, it calculates relevance to obtain related text. Finally, it processes the related text using a third preset model to obtain entity labels. Based on these entity labels, it performs named entity recognition. In other words, this solution, after acquiring the context text, obtains attention weights based on the second preset model and the context text, and then obtains entity labels based on the third preset model and the attention weights, thereby achieving named entity recognition and avoiding the problem of low accuracy. Attached Figure Description
[0016] Figure 1 A flowchart illustrating the context-based named entity recognition method provided in this embodiment of the invention; Figure 2 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 3 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 4 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 5Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 6 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 7 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 8 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 9 Another flowchart of the context-based named entity recognition method provided in this embodiment of the invention; Figure 10 A schematic diagram of the identification device provided in an embodiment of the present invention; Figure 11 This is a schematic diagram of the structural composition of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Please refer to Figure 1 ,in, Figure 1 A flowchart illustrating an implementation of the context-based named entity recognition method provided in this application embodiment may include: Step S101: Obtain the original text; Step S102: Obtain text prompts based on the original text; Step S103: Process the text prompt words using the first preset model to obtain the context text; Step S104: Process the context text using the second preset model to obtain attention weights; Step S105: Calculate relevance based on attention weights to obtain relevant text; Step S106: Process the relevant text using the third preset model to obtain entity labels, and then perform named entity recognition based on the entity labels.
[0019] The original text can be Chinese text or foreign language text; the first preset model can be a model generated based on a large language model; the second preset model can be the BERT model; and the third preset model can be a sequence labeling model.
[0020] Specifically, the entire architecture mainly consists of two parts: a semantic enhancement preprocessing mechanism and an entity recognition mechanism. First, the test Chinese text is input into the semantic enhancement preprocessing mechanism. A dynamic context generation module generates multiple contexts related to the current entity (including entity descriptions, attributes, etc.). These generated contexts are then fed into a context evaluation module, where a BERT model evaluates and selects the most relevant contexts. The extracted context embedding representation features are then input into the entity recognition mechanism, where a sequence labeling model predicts the entity labels for each word in the text. Finally, the extracted specific entities are obtained, and the results are evaluated. The recognition results are used as feedback to guide the large language model to dynamically generate more accurate and relevant contexts through iterative optimization.
[0021] The semantic enhancement preprocessing mechanism provides additional semantic cues by automatically and dynamically generating entity-related context, reducing ambiguity in specific entity recognition, enhancing semantic understanding of entity recognition, and thus improving the model's recognition accuracy. It mainly comprises two core modules: dynamic context generation and context evaluation.
[0022] Please refer to Figure 2 The method in this embodiment may include: obtaining text prompt words based on the original text, including: Step S201: Perform similarity matching on the original text based on RAG retrieval to obtain relevant examples; Step S202: Construct prompt words based on relevant examples to obtain text prompt words.
[0023] Please refer to Figure 3 The method in this embodiment may include: processing the text prompt words through a first preset model to obtain the context text, including: Step S301: Obtain historical log data; Step S302: Obtain the initial dataset based on historical log data; Step S303: Perform data annotation on the initial dataset based on the BiLSTM-CRF model to obtain the entity training dataset; Step S304: Learn from the entity training dataset based on the large language model to obtain the first preset model.
[0024] Specifically, the various modules of semantic enhancement preprocessing are described below: (1) Dynamic context generation; The main implementation process of the dynamic context generation module is as follows: Figure 4As shown, before recognizing the test text, initial datasets are first collected from operator customer service historical logs. Then, the initial dataset is labeled using a deep learning model BiLSTM-CRF and manual correction to obtain a training dataset for named entities of customer service historical content. All text in the training set is preprocessed and saved to a locally constructed knowledge base. A large language model is then trained on the labeled training set. After the preparatory work is completed, the user's test text content is input and vectorized. Then, RAG retrieval enhancement technology is used to perform similarity matching retrieval on the text content in the local knowledge base, obtaining the Top K highly relevant sample examples. Prompt words are constructed based on the retrieval demonstration examples and used as input to the large language model. Finally, the large model generates multiple contextual contents related to the current entity and inputs them into the context evaluation module for evaluation and filtering.
[0025] Traditional methods of generating results using large models rely on manual design of prompts, making it difficult to cover all relevant contexts. Furthermore, the quality of prompts directly affects the accuracy of recognition, which presents significant limitations. In contrast, the dynamic context generation module in this proposal can automatically and dynamically generate contexts related to the current entity, reducing the need and time for manual prompt design. The generated context can improve the model's recognition accuracy.
[0026] Please refer to Figure 5 The method in this embodiment may include: processing the context text using a second preset model to obtain attention weights, including: Step S401: Perform word segmentation on the context text to obtain text in a preset format; Step S402: Process the preset format text using the second preset model to obtain attention weights; The second preset model is the BERT model.
[0027] Specifically, (2) context evaluation; The main implementation process of the context evaluation module is as follows: Figure 6 As shown. The context evaluation module obtains multiple context contents generated by the dynamic context generation module and uses the BERT model to evaluate the relevance between the input text and the context. The specific steps of context relevance evaluation are as follows: First, the input text and generated context are concatenated and then segmented into tokens, converting them into a format that the BERT model can understand (such as token IDs and attention masks). The processed input is then fed into the BERT model to obtain the embedding representation of each token (containing rich contextual information). Attention weights are generated using the attention layer in the BERT model; these weights indicate which parts of the input sequence the model focuses on when processing each token. Analyzing these attention weights reveals the model's focus when processing the input text and generated context. It allows us to see how much attention each token in the generated context receives in the original text. Attention weights for the generated context content are selected for analysis. Next, the average attention weight of each token in the input text to the generated context, and the average attention weight of each token in the generated context to the input text, are calculated. The attention cosine similarity between the two weight vectors is calculated and used as the relevance score between the input text and the generated context. A threshold is set, and the calculated similarity score is judged accordingly; a score higher than the threshold indicates that the generated context is relevant to the entities in the current input text. By using relevance assessment to filter out contexts that are strongly related to entities, the assessment results can be used as feedback to guide subsequent context generation or model optimization.
[0028] The context evaluation module of this proposal can capture complex semantic relationships in the context and resolve semantic ambiguity. By using the BERT model to filter out contextual features that are more relevant to the entity, it can provide rich semantic information for the sequence labeling model and help improve the model's accuracy in recognizing specific professional entities.
[0029] Please refer to Figure 7 The method in this embodiment may include: processing preset format text through a second preset model to obtain attention weights, including: Step S501: Perform data conversion on the preset format text to obtain time series data; Step S502: Obtain attention weights based on time-series data.
[0030] Specifically, (3) frequency domain signal enhancement attention mechanism; In the traditional attention mechanism of the BERT model, each query needs to be multiplied by all keys, the result is normalized using softmax, and finally multiplied by the values to obtain the attention weights. The definition of the attention mechanism is: , in These are represented as queries, keys, and values, respectively, which are obtained by linearly projecting the corresponding source inputs.
[0031] Traditional attention mechanisms are computationally expensive and inefficient when processing long texts, and struggle to capture long-distance dependencies or important but not obvious patterns in the text. They are also susceptible to noisy data and may allocate too much attention to unimportant information.
[0032] This proposal modifies the traditional attention mechanism from a frequency domain perspective, proposing a frequency domain signal enhancement attention mechanism. Through frequency domain enhancement, key features in the text can be highlighted more effectively, improving recognition accuracy. Simultaneously, it can reduce noise interference and improve computational efficiency. The main implementation principle of the frequency domain signal enhancement attention mechanism is as follows: The vectorized text data is converted into time-series data. Leveraging the greater sparsity of general time-series signals in the frequency domain compared to traditional time-series data, the main features of the text are extracted more effectively from the frequency domain perspective. In the frequency domain, text data features can be represented as signals of different frequencies. High-frequency noise signals are filtered out, and the low-frequency signals with the largest amplitudes (the main features) are retained to reconstruct the signal. This proposal decomposes queries, keys, and values using FFT to extract the k lowest-frequency components with the largest amplitudes from each, and performs similar attention calculations in the frequency domain. The structure of the frequency domain signal enhancement attention mechanism is as follows:
[0033]
[0034]
[0035]
[0036]
[0037] in K, V These are q, k, and v obtained through linear mapping, respectively. Returns the K lowest frequency components with the largest amplitude, where K is a predefined hyperparameter. , , ( ) represent the k low-frequency Fourier components with the largest amplitude extracted after FFT transformation decomposition of the input features. This represents the attention weight values before performing the inverse fast Fourier transform, and then... Fill to zero Y This represents the final attention weight value calculated by converting from the frequency domain back to the time domain using IFFT.
[0038] Please refer to Figure 8 The method in this embodiment may include: calculating relevance based on attention weights to obtain relevant text, including: Step S601: Calculate the relevance based on the attention weight to obtain the similarity score; Step S602: Determine whether the similarity score is greater than the first threshold, and obtain the first judgment result; Step S603: If the first judgment result is yes, obtain the relevant text.
[0039] Specifically, the attention cosine similarity between two weight vectors is calculated and used as a relevance score between the input text and the generated context. A threshold is set, and the calculated similarity score is judged accordingly. A score higher than the threshold indicates that the generated context is relevant to the entities in the current input text. Contexts with strong entity relevance are selected through relevance evaluation, and the evaluation results can be used as feedback to guide subsequent context generation or model optimization.
[0040] The method in this embodiment may include: performing data conversion on preset format text to obtain time-series data, including: The formula is: ; ; ; ; ; in K, V These are q, k, and v obtained through linear mapping, respectively. Returns the K lowest frequency components with the largest amplitudes, where K is a predefined hyperparameter. , , ( The numbers () represent the k lowest-frequency Fourier components with the largest amplitudes extracted after FFT transformation and decomposition of the input features. This represents the attention weight values before performing the inverse fast Fourier transform, and then... Fill to zero Y This represents the final attention weight value calculated by converting from the frequency domain back to the time domain using IFFT.
[0041] Please refer to Figure 9The method in this embodiment may include: processing the relevant text using a third preset model to obtain entity tags, including: Step S701: Extract features from the relevant text to obtain relevant features; Step S702: Predict relevant features using a third preset model to obtain entity labels; wherein, the third preset model is a sequence labeling model.
[0042] Please refer to Figure 10 The apparatus of this embodiment may include the following structure: Acquisition unit 801 is used to acquire the original text; Analysis unit 802 is used to obtain text prompt words based on the original text; process the text prompt words through a first preset model to obtain context text; process the context text through a second preset model to obtain attention weights; and perform relevance calculation based on the attention weights to obtain related text. The processing unit 803 is used to process the relevant text through a third preset model to obtain entity labels, and then perform named entity recognition based on the entity labels.
[0043] Please refer to Figure 11 This embodiment of the present application also discloses an electronic device, which includes at least one processor 901, and at least one memory 902 and a bus 903 connected to the processor 901; wherein the processor 901 and the memory 902 communicate with each other through the bus 903; the processor 901 is used to call program instructions in the memory 902 to execute the above-described context-based named entity recognition method.
[0044] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A context-based named entity recognition method, characterized in that, The method includes: Get the original text; Based on the original text, obtain the text prompt words; The text prompt words are processed using a first preset model to obtain the context text; The context text is processed using a second preset model to obtain attention weights; Relevance calculation is performed based on the attention weights to obtain relevant text; The relevant text is processed by a third preset model to obtain entity tags, and named entity recognition is performed based on the entity tags.
2. The method according to claim 1, characterized in that, Based on the original text, text prompt words are obtained, including: The original text was matched for similarity based on RAG retrieval to obtain relevant examples; Based on the relevant examples, prompt words are constructed to obtain the text prompt words.
3. The method according to claim 2, characterized in that, Before processing the text prompt words using the first preset model to obtain the context text, the process includes: Retrieve historical log data; Based on the historical log data, an initial dataset is obtained; The initial dataset was labeled using the BiLSTM-CRF model to obtain the entity training dataset. The first preset model is obtained by learning from the entity training dataset based on a large language model.
4. The method according to claim 3, characterized in that, The context text is processed using a second preset model to obtain attention weights, including: The context text is segmented to obtain text in a preset format; The attention weights are obtained by processing the preset format text using a second preset model. The second preset model is the BERT model.
5. The method according to claim 4, characterized in that, The process of processing the preset format text using a second preset model to obtain the attention weights further includes: The preset format text is converted to obtain time-series data; The attention weights are obtained based on the time-series data.
6. The method according to claim 5, characterized in that, Based on the attention weights, relevance calculations are performed to obtain relevant text, including: Based on the attention weights, a relevance calculation is performed to obtain a similarity score; Determine whether the similarity score is greater than a first threshold to obtain a first determination result; If the first judgment result is yes, obtain the relevant text.
7. The method according to claim 5, characterized in that, The preset format text is converted to obtain time-series data, including: The formula is: ； ； ； ； ； in K, V These are q, k, and v obtained through linear mapping, respectively. Returns the K lowest frequency components with the largest amplitudes, where K is a predefined hyperparameter. , , ( The numbers () represent the k lowest-frequency Fourier components with the largest amplitudes extracted after FFT transformation and decomposition of the input features. This represents the attention weight values before performing the inverse fast Fourier transform, and then... Fill to zero Y This represents the final attention weight value calculated by converting from the frequency domain back to the time domain using IFFT.
8. The method according to claim 1, characterized in that, The relevant text is processed using a third preset model to obtain entity tags, including: Feature extraction is performed on the relevant text to obtain relevant features; Entity labels are obtained by predicting the relevant features using the third preset model. The third preset model is a sequence labeling model.
9. An identification device, characterized in that, The device includes: an acquisition unit, an analysis unit, and a processing unit. The acquisition unit is used to acquire the original text; The analysis unit is configured to obtain text prompt words based on the original text; process the text prompt words using a first preset model to obtain context text; process the context text using a second preset model to obtain attention weights; and perform relevance calculation based on the attention weights to obtain related text. The processing unit is used to process the relevant text through a third preset model to obtain entity tags, and then perform named entity recognition based on the entity tags.
10. An electronic device, characterized in that, include: Memory, used to store at least one set of instructions; A processor used to acquire raw text; Based on the original text, obtain the text prompt words; The text prompt words are processed using a first preset model to obtain the context text; The context text is processed using a second preset model to obtain attention weights; Relevance calculation is performed based on the attention weights to obtain relevant text; The relevant text is processed by a third preset model to obtain entity tags, and named entity recognition is performed based on the entity tags.