Neural network model-based few-sample named entity identification method and system

Abstract

The invention discloses a few-sample named entity recognition method and system based on a neural network model, and the method comprises the following steps: obtaining a to-be-recognized original input text, and retrieving from an external knowledge base to obtain a group of candidate knowledge statements; a dynamic similarity threshold algorithm is adopted to filter the candidate knowledge statements, and enhanced knowledge is screened out; inputting into a first encoder, and generating a context text representation matrix of deep fusion knowledge; inputting the matrix into a sequence labeling classification layer, classifying each lexical element, and identifying a boundary span of a potential entity in a text; and inputting a text corresponding to the identified entity boundary span into a second encoder to generate an entity embedding representation, and determining a final category of the entity by calculating a distance between the entity embedding representation and a predefined category prototype to complete named entity identification. According to the method, the problems caused by data sparseness and semantic fuzziness in a few-sample scene are solved, and the accuracy of named entity recognition is improved.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically to a method and system for few-shot named entity recognition based on a neural network model. Background Technology

[0002] Named Entity Recognition (NER) is a fundamental task in information extraction and natural language processing, aiming to identify and classify entities in text. It is crucial for downstream applications such as building knowledge graphs and question-answering systems. Traditional deep learning models typically require large-scale labeled data to achieve good results, but labeled data is scarce in many real-world scenarios. This has spurred research into Few-Shot Named Entity Recognition (FS-NER).

[0003] In the field of few-shot named entity recognition, metric learning-based methods, especially Prototypical Networks, have become a mainstream approach. These methods generate a prototype for each entity category using a small number of labeled samples (i.e., the support set), and then classify the entity by calculating the distance between the query sample and each type of prototype in the embedding space.

[0004] Despite the progress made by prototype networks in few-shot learning, several inherent challenges remain. First, the model's knowledge acquisition relies entirely on the limited support set provided for each task. When the support set is sparse or biased, the generated prototype representation is insufficient, limiting the model's ability to generalize to new entities and making it difficult to identify entities with significant semantic differences from the support set. Second, while some research has attempted to introduce external knowledge bases to enhance model performance, most of these methods retrieve and fuse knowledge in real-time for each new task during the model's inference phase. This "online" processing mechanism inevitably introduces a significant computational burden, leading to a substantial increase in inference latency, which contradicts the principles of low overhead and rapid adaptation required by few-shot learning, greatly limiting its practical application value. Furthermore, directly introducing unfiltered external knowledge may introduce noise or redundant information, interfering with the model's effective learning.

[0005] Therefore, how to efficiently integrate high-quality external knowledge to enhance the model's generalization ability in few-sample named entity recognition tasks, while avoiding the introduction of high computational costs in the inference stage, is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for few-shot named entity recognition based on a neural network model. By integrating knowledge in advance to the pre-training stage and designing an efficient knowledge filtering strategy, the accuracy and robustness of the model in sparse data scenarios are significantly improved without sacrificing reasoning efficiency.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a few-shot named entity recognition method based on a neural network model, comprising the following steps: S1. Obtain the original input text to be identified, and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text; S2. A dynamic similarity threshold algorithm is used to filter the candidate knowledge statements and select the enhanced knowledge that semantically matches the original input text. S3. The original input text and the enhanced knowledge are concatenated to form a knowledge-enhanced input sequence, which is then input into the first encoder to generate a contextual text representation matrix of deep fusion knowledge. S4. Input the context text representation matrix into the sequence labeling classification layer, classify each word in the input sequence, and identify the boundary span of one or more potential entities in the text. S5. Input the text corresponding to the identified entity boundary span into the second encoder to generate an entity embedding representation, and determine the final category of the entity by calculating the distance between the entity embedding representation and the predefined category prototype, thus completing named entity recognition.

[0008] Furthermore, the external knowledge base is a large-scale structured or semi-structured knowledge source, including Wikipedia, WordNet, and ConceptNet.

[0009] Furthermore, step S2 specifically includes: S21. Calculate the semantic similarity score between each candidate knowledge statement and the original input text; S22. Determine the length type based on the sequence length of the original input text; the length types include: short sentences, medium-length sentences, and long sentences; each length type corresponds to a different similarity score filtering range; S23. Retain candidate knowledge statements whose semantic similarity scores fall within the filtering range corresponding to the length type of the original input text, as enhanced knowledge.

[0010] Furthermore, step S21 specifically includes: The original input text and a candidate knowledge statement are treated as two text sequences, and semantic similarity analysis is performed. Calculate the cosine similarity of the word embeddings in two text sequences to obtain precision, recall, and semantic similarity scores; expressed by the formula:

[0011]

[0012]

[0013] in, This represents the i-th embedding vector of the original input text. This represents the j-th embedding vector of the candidate knowledge statement; m represents the sequence length of the original input text, and n represents the sequence length of the candidate knowledge statement; R BERT Indicates recall rate, P BERT Indicates accuracy. F BERT This represents the semantic similarity score.

[0014] Furthermore, step S3 specifically includes: S31. The original input text is fused with the enhanced knowledge to obtain a knowledge-enhanced input sequence. The formula is expressed as follows:

[0015] Where ⊕ represents the feature concatenation operation, S i This represents the entity span in the original input text. This indicates that the content retrieved from an external knowledge base is related to... S i Related augmented knowledge: [SEP] represents a separator; S32. Input the input sequence into the first encoder, use a multi-head self-attention mechanism to perform context modeling on all lexical units in the sequence, and integrate the semantic information of the augmented knowledge into the representation of each lexical unit to obtain the context text representation.

[0016] Furthermore, step S4 specifically includes: S41. Input the context text representation matrix into the sequence labeling classification layer, and process the matrix... The embedded vectors in the sequence are used for sequence labeling, and the probability that each word belongs to an entity is calculated, expressed by the formula:

[0017] in, and These represent the weight matrix and bias vector of the linear layer, respectively. Representation matrix The embedding vector of the i-th dimension; S42. Using a binary classification span detection model, span detection is performed on each word corresponding to the probability; the boundary span of the potential entity in the text is identified.

[0018] Furthermore, the binary span detection model optimizes its parameters by minimizing the cross-entropy loss between the predicted probability distribution and the true label; The cross-entropy loss is expressed by the formula:

[0019] Where L is the total length of the sequence. It is the actual label of the i-th word element.

[0020] Furthermore, in step S5, the second encoder is trained using supervised contrastive learning, specifically including: Construct a dataset for model training; for each entity, concatenate its span text and semantic category name in two different orders to create a pair of semantically equivalent but sequentially different positive samples, and add them to the dataset; The positive sample pairs are input into the second encoder to generate entity embedding representations, which can be expressed by the following formula:

[0021]

[0022] Where ⊕ represents the concatenation operator, and This represents a pair of constructed positive samples. Indicates the second encoder, Represents the i-th entity word element. r i This represents the i-th correct entity label. This is a conversion function.

[0023] Furthermore, in step S5, the second encoder is optimized using a supervised contrastive learning loss function, expressed by the formula:

[0024]

[0025]

[0026] in, This represents the contrastive loss, where I represents the total number of samples in the training batch. rz represents the set of all positive samples belonging to the same class as sample i; rz represents the true class label of the sample with index z; and ri represents the true class label of sample i. and These represent samples constructed according to "entity-label" and "label-entity" respectively; sim (.) represents cosine similarity; τ represents temperature hyperparameter.

[0027] Secondly, the present invention provides a few-shot named entity recognition system based on a neural network model, comprising the following modules: The knowledge acquisition module is used to acquire the original input text to be identified and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text. The knowledge filtering module is used to filter the candidate knowledge statements using a dynamic similarity threshold algorithm, and select the enhanced knowledge that semantically matches the original input text. The text representation module is used to concatenate the original input text with the enhanced knowledge to form a knowledge-enhanced input sequence, and input it into the first encoder to generate a context text representation matrix of deep fusion knowledge; The span detection module is used to input the context text representation matrix into the sequence labeling classification layer, classify each word in the input sequence, and identify the boundary span of one or more potential entities in the text; The type classification module is used to input the text corresponding to the boundary span of the identified entity into the second encoder to generate an entity embedding representation, and to determine the final category of the entity by calculating the distance between the entity embedding representation and the predefined category prototype, thus completing named entity recognition.

[0028] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method and system for few-shot named entity recognition based on a neural network model, which has the following beneficial effects: This invention completely separates the computationally intensive knowledge retrieval and fusion process from the inference stage, moving it to a dedicated pre-training stage. This allows the model to perform lightweight prototype computations without real-time interaction with external knowledge bases during inference, thus significantly improving inference speed and meeting the requirements of rapid adaptation in few-shot learning.

[0029] Furthermore, the dynamic similarity threshold algorithm proposed in this invention can effectively filter out noise and redundant information in the retrieved knowledge, ensuring that only the most relevant knowledge with the highest information entropy is used to enhance the model. This enables the encoder to learn more robust and discriminative semantic representations, thereby significantly improving the accuracy of named entity recognition, especially in extreme data sparsity scenarios with only a single sample, where the performance advantage is particularly prominent.

[0030] By deeply integrating high-quality external knowledge during the pre-training phase, this invention enables the model to transcend its dependence on the limited context of the support set. The model is able to build a reliable understanding of new domains, effectively address the problem of support set sample bias, and exhibit strong generalization ability. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0032] Figure 1 The flowchart illustrates the few-shot named entity recognition method based on a neural network model provided in this embodiment of the invention.

[0033] Figure 2 This is a schematic diagram of the overall framework topology of the neural network model provided in an embodiment of the present invention.

[0034] Figure 3 A block diagram of a few-shot named entity recognition system based on a neural network model provided in an embodiment of the present invention. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Example 1 This invention discloses a few-shot named entity recognition method based on a neural network model, referring to... Figure 1 As shown, it includes the following steps: S1. Obtain the original input text to be identified, and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text; S2. A dynamic similarity threshold algorithm is used to filter the candidate knowledge statements and select the enhanced knowledge that semantically matches the original input text. S3. The original input text and the enhanced knowledge are concatenated to form a knowledge-enhanced input sequence, which is then input into the first encoder to generate a contextual text representation matrix of deep fusion knowledge. S4. Input the context text representation matrix into the sequence labeling classification layer, classify each word in the input sequence, and identify the boundary span of one or more potential entities in the text. S5. Input the text corresponding to the identified entity boundary span into the second encoder to generate an entity embedding representation, and determine the final category of the entity by calculating the distance between the entity embedding representation and the predefined category prototype, thus completing named entity recognition.

[0037] This embodiment applies to few-sample named entity recognition in the financial field, used for high-end client activity management and risk monitoring in financial institutions. Large banks or wealth management institutions can utilize this invention to automatically process text information obtained from channels such as news, internal announcements, and client manager reports, accurately identifying key entities related to high-end client services and business opportunities.

[0038] Reference Figure 2 As shown, the implementation steps of this embodiment are described in detail below: Step S1: Context generation.

[0039] Obtain the original input text to be identified, and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text.

[0040] The original input text in this example is: "The International Finance Association's conference is being held for two days at the Shangri-La Hotel in Singapore, and an opening dinner will be organized for the association's representatives."

[0041] A large-scale, comprehensive, and high-quality text corpus is selected as the external knowledge base to ensure that the retrieved knowledge is authoritative and abundant. In this embodiment, Wikipedia is preferably used as the external knowledge source.

[0042] To achieve efficient and real-time knowledge retrieval, it is necessary to process and index the massive knowledge base beforehand. This embodiment utilizes the Wikipedia2Vec toolkit to convert offline Wikipedia data from its raw XML format into an easily processed structured JSON file format. This step organizes unstructured text content into structured data entries, laying the foundation for subsequent indexing.

[0043] We use Elasticsearch, a search engine technology, to build an index on the collection of JSON files generated in the previous step. By creating an index, we establish a retrieval system that supports fast and scalable queries, avoiding time-consuming scanning of the entire knowledge base in every recognition task.

[0044] In performing a specific named entity recognition task, this embodiment uses the original input text to be recognized as the query statement and inputs it into the pre-constructed retrieval system. The retrieval system then returns the Top-k knowledge statements that are semantically most relevant to the query statement, based on its internal relevance ranking algorithm. In the specific experiment of this embodiment, the preferred value for Top-k is 10.

[0045] The k retrieved knowledge statements are used to construct a candidate knowledge set, which contains unfiltered background information related to the original input text. This candidate knowledge set will serve as input for the next stage, the S2 knowledge filtering module, for further refinement and optimization.

[0046] The retrieval system in this embodiment returns a set of candidate knowledge statements based on the above text. These include: Knowledge 1: The Institute of International Finance is a global financial services industry association headquartered in Washington, D.C., USA.

[0047] Knowledge 2: The Shangri-La Hotel, Singapore is a five-star luxury hotel under the Shangri-La Hotels and Resorts group, located in the Orchard Road business district.

[0048] Knowledge 3: A two-day conference of the Institute of International Finance was held at the Shangri-La Hotel in Singapore, and a dinner was prepared for the delegates.

[0049] Knowledge 4: The most basic function of financial markets is to effectively allocate capital resources and reflect asset value through price discovery mechanisms.

[0050] Knowledge 5: Singapore is a tropical city-state, renowned for its clean streets, Merlion Park, and multicultural cuisine.

[0051] Step S2: Conduct knowledge screening.

[0052] A dynamic similarity threshold algorithm is used to filter candidate knowledge statements to remove statements that are not very relevant to the original input text or are redundant, thereby selecting high-quality augmented knowledge.

[0053] This embodiment uses a dynamic similarity threshold algorithm to filter candidate knowledge statements, thereby removing statements that are not very relevant to the original input text or are redundant, thus selecting high-quality augmented knowledge.

[0054] Two key issues may exist in the candidate knowledge set: first, it contains noisy knowledge with low relevance to the current context, potentially introducing interfering signals; second, it contains redundant information that highly overlaps with the semantics of the original text, failing to provide effective incremental information. Noisy knowledge can mislead the model, while redundant knowledge leads to diminishing returns on information. To accurately balance the relevance and novelty of knowledge, this embodiment employs a dynamic similarity threshold algorithm. Its core is to automatically prune these two extremes to retain the most informative context. The specific implementation of this algorithm is as follows: In this embodiment, BERTScore is preferably used to calculate the similarity score between each candidate knowledge statement and the original input text. This approach can better understand synonyms, paraphrases, and complex syntactic structures, thereby providing a more reliable measure of semantic relevance.

[0055] in, The calculation process includes accuracy P BERT Recall rate R BERT and the final semantic similarity score F BERT The calculation formula is as follows:

[0056]

[0057]

[0058] in, This represents the i-th embedding vector of the original input text. represents the j-th embedding vector of the candidate knowledge statement; m represents the sequence length of the original input text, and n represents the sequence length of the candidate knowledge statement. By Each tag in the middle is relative to The recall rate is calculated by aggregating the maximum similarity scores. Then the accuracy is calculated in the opposite direction. The final calculated BERTS score indicates that the two sentences are more semantically similar.

[0059] This embodiment does not use a fixed single threshold, but rather depends on the length of the original input text. Dynamically adjust a filtering interval. Set a baseline similarity interval. And adjust accordingly based on text length: For short sentences ( Short texts contain limited semantic information, and high similarity scores are likely due to accidental word overlap rather than genuine semantic association. To avoid introducing noisy knowledge, a narrower and stricter filtering interval [0.65, 0.80] is adopted.

[0060] For sentences of medium length ( The text length is moderate, and the semantic information is relatively stable, so the baseline interval can be used directly. .

[0061] For long sentences ( Long texts are semantically rich, and even if the similarity score is slightly low, they may contain valuable supplementary information; while very high similarity scores constitute strong and valid evidence. Therefore, a wider screening range [0.55, 0.90] is adopted.

[0062] Only candidate knowledge statements whose BERTS score falls within a dynamically determined filtering range based on text length are considered high-quality augmented knowledge and retained for subsequent S3 knowledge fusion steps. Scores below the lower limit of the range are considered noisy knowledge, while scores above the upper limit are considered redundant knowledge and are discarded. Through this refined filtering mechanism, this invention ensures that the most valuable external information is input into the model.

[0063] The original input text length was analyzed and determined to be a "medium-long sentence". Based on the above settings, the corresponding basic filtering range [0.60, 0.85] was applied; the BERTS score for each candidate knowledge was calculated as follows: Knowledge 1: 0.65 → Reserved Knowledge 2: 0.66 → Retain Knowledge 3: 0.88 → Discarded (basically a restatement of the original text, without providing any new information) Knowledge 4: 0.57 → Discard (Focus on finance, irrelevant to the context) Knowledge 5: 0.56 → Discard (Focus on Singapore, irrelevant to the context) Output: A high-quality set of augmented knowledge {knowledge1, knowledge2}.

[0064] Step S3: Perform knowledge fusion and representation.

[0065] The original input text is concatenated with the enhanced knowledge to form a knowledge-enhanced input sequence, which is then fed into the first encoder to generate a contextual text representation of the deeply fused knowledge.

[0066] This embodiment employs sequence concatenation for fusion. The original input text sequence is connected to one or more enhanced knowledge sequences output by S2 using a predefined special delimiter [SEP]. The [SEP] marker helps the model distinguish different parts of the sequence during internal processing. In this embodiment, to achieve optimal performance, the most informative enhanced knowledge statement from the S2 filtering results is selected for concatenation. The resulting knowledge-enhanced input sequence... X i It can be represented as:

[0067] Where ⊕ represents the feature concatenation operation, S i This represents the entity span in the original input text. This indicates that the content retrieved from an external knowledge base is related to... S i The relevant augmented knowledge description is then used; the two are then concatenated using a special separator [SEP] to form a knowledge-enhanced input sequence. X i .

[0068] This embodiment will use the enhanced input sequence. X i The input is fed into a first encoder that serves as the backbone network. In this embodiment, the first encoder is BERT, a deep bidirectional pre-trained language model based on the Transformer architecture. We denote this encoder as... ,in It is the model weights that contain a large number of pre-trained parameters.

[0069] encoder Receive complete knowledge-enhanced input sequence X i It utilizes its internal multi-head self-attention mechanism to perform deep context modeling on all lexical units in the sequence, thereby deeply integrating the semantic information of external knowledge into the representation of each lexical unit.

[0070] The encoder output is a hidden state vector matrix of the same length as the input sequence. , can be represented as:

[0071] Where N is the input sequence Total number of lexical elements, It is the hidden state matrix. This represents the d-dimensional context representation vector corresponding to the i-th word. Each vector in this output matrix contains rich contextual information with deep fusion knowledge. It serves as the direct input feature for the subsequent S4 knowledge-aware span detection module, providing a solid foundation for accurate entity boundary judgment. Based on the aforementioned text content, this embodiment outputs a context representation matrix rich in financial domain knowledge. .

[0072] Step S4: Perform a knowledge fusion span detection.

[0073] The contextual text representation is input into the sequence labeling classification layer, which classifies each word in the input sequence to identify and output the boundary span of one or more potential entities in the text.

[0074] The goal of this step is to accurately identify the location boundaries of all potential entities in the text based on the contextual text representation matrix output by S3, which deeply integrates external knowledge. To this end, this embodiment constructs the task as a sequence labeling problem.

[0075] The context text representation matrix generated in the previous stage The input is fed into a dedicated classification layer for sequence labeling. This classification layer employs a standard design, consisting of a Dropout layer and a linear layer. The Dropout layer is a regularization technique used during training to randomly "deactivate" a portion of neuron connections at a certain ratio, enhancing the model's generalization ability and effectively preventing overfitting to the training data. The subsequent linear layer acts as a feature projection layer, mapping the d-dimensional high-dimensional feature vector output from the first encoder to a predefined label space dimension.

[0076] This embodiment uses a Softmax activation function to transform the raw output scores of the linear layer into a valid probability distribution. This distribution clearly represents the probability that each word in the input sequence belongs to a label in the predefined label set C. Its calculation formula can be expressed as:

[0077] in, and These are the weight matrix and bias vector of the linear layer, respectively. For the binary classification task of span detection, the predefined label set C contains only two categories: Inside (indicating that the word is inside an entity) and Outside (indicating that the word is outside an entity).

[0078] During the model training phase, the parameters of the entire span detection module are optimized by minimizing the cross-entropy loss between the predicted probability distribution and the true label. The loss function is denoted as... The calculation formula is as follows:

[0079] Where L is the total length of the sequence. It is the true label of the i-th word (1 for Inside and 0 for Outside).

[0080] Through this step, the model is able to accurately distinguish between the lexical sequences that constitute entities and background lexical sequences that are not entity parts. The final output of this step is one or more identified entity boundary spans (i.e., consecutive lexical sequences labeled "Inside"), which will serve as input to the S5 knowledge-guided type classification module.

[0081] This embodiment uses an IO system to predict a sequence label for each word. Based on the label sequence, consecutive entity labels are merged, and entity boundaries are output. In this embodiment, the International Finance Association is identified as a span (organization, organization, organization); the Shangri-La Hotel, Singapore is identified as (location-hotel, location-hotel, location-hotel).

[0082] Step S5: Perform knowledge-based type classification.

[0083] The text corresponding to the entity boundary span identified in S4 is input into a second encoder trained with supervised contrastive learning to generate an entity embedding representation. The final category of the entity is determined by calculating the distance between the entity embedding representation and the predefined category prototype, thereby completing named entity recognition.

[0084] To enable the model to fully utilize the rich semantic information inherent in the category labels themselves, this embodiment introduces a category semanticization preprocessing step. This is achieved by defining a transformation function. This approach transforms the symbolic labels like PER and ORG used in traditional NER tasks, which lack inherent meaning, into natural language descriptions with clear semantics, such as "person" and "organization." These semantically derived category names will become a key component in constructing category prototypes.

[0085] This embodiment employs a separate, dedicated second encoder for type classification. To enable this encoder to deeply learn the inherent semantic relationships between entities and their category names, this embodiment designs an innovative positive sample pair construction and data augmentation strategy. Specifically, for each entity span text and its corresponding semantically coded category name in the training set, a pair of semantically equivalent but sequentially distinct positive sample pairs is constructed by concatenating the sequences in two different orders:

[0086]

[0087] Where ⊕ represents the concatenation operator, and This represents a pair of constructed positive samples. Indicates the second encoder, Represents the i-th entity word element. r i This represents the i-th correct entity label. This is a transformation function that converts symbolic labels into more descriptive natural language names.

[0088] Preferably, the present invention employs a supervised contrastive learning framework for the second encoder. Specialized pre-training is performed. Its core objective is to utilize a specially designed type-aware contrastive loss function. This loss function optimizes encoder parameters. Regardless of the original text or concatenation order, it brings all embeddings generated from entities of the same class closer together in the feature space, while pushing away embeddings from entities of different classes. The formula for this loss function is as follows:

[0089]

[0090]

[0091] in, This represents the contrastive loss, where I represents the total number of samples in the training batch. This represents the set of all positive samples that belong to the same class as sample i; r z This represents the true class label of the sample with index z. r i This represents the true category label of sample i. and These represent samples constructed according to "entity-label" and "label-entity" respectively; similarity function. sim (.) represents cosine similarity; τ represents temperature hyperparameter.

[0092] In this embodiment, after completing the above pre-training, the second encoding... An entity embedding representation is generated for this category. Then, the distance between this representation and the prototype of each entity category in the support set is calculated by averaging the embedding representations of all support set samples for that category. The entity to be classified is finally assigned the label of the closest prototype of the category.

[0093] Through this series of steps, the present invention finally achieves high-precision classification of all potential entities and outputs complete named entity recognition results.

[0094] This embodiment calculates the distance between each entity embedding and all category prototypes. Entities are assigned to the nearest category. Specifically, "Institute of International Finance" is closest to the "Organization" prototype and is classified as: Organization; "Shangri-La Hotel, Singapore" is closest to the "Location-Hotel" prototype and is classified as: Location-Hotel. The output is a structured named entity recognition result: (Institute of International Finance, Organization) (Shangri-La Hotel, Singapore, Location-Hotel).

[0095] This embodiment also verifies the effectiveness of the knowledge-enhanced neural network model proposed in this invention on the task of few-shot named entity recognition by conducting comparative experiments on a series of publicly available benchmark tests.

[0096] All experiments in this embodiment were conducted on Few-NERD, a large-scale, widely used public benchmark dataset for few-shot named entity recognition (FS-NER). This dataset contains 8 coarse-grained entity types and 66 fine-grained entity types, and provides two highly challenging evaluation settings: INTRA setting: The coarse-grained categories in the training, validation, and test sets do not overlap, and are used to evaluate the model's ability to generalize to new domain knowledge.

[0097] INTER setting: The training set, validation set, and test set may share the same coarse-grained categories, but the fine-grained categories under them do not overlap. It is used to evaluate the model's ability to identify new knowledge in existing domains with fine-grained characteristics.

[0098] To ensure the fairness and reproducibility of the experiment, this embodiment uses the following uniform hyperparameter configuration in the NVIDIA RTX 4070 GPU hardware environment: training epoch is set to 100; batch size is set to 32; learning rate is set to 2e-5; dropout ratio is set to 0.2; contrast learning temperature is set to 0.01; and learning rate warmup is set to True.

[0099] To comprehensively evaluate the performance of the model of this invention, several representative existing technology models were selected as baselines for comparison, including: ProtoBERT, a classic baseline model combining prototype networks and BERT; NNShot, a classification method based on the nearest neighbor idea; StructShot, a model that introduces a Viterbi decoder for sequence optimization on the basis of NNShot; FSLS, a model that learns label semantic information by constructing "label-entity" pairs as input; ESD, a method that optimizes the feature space by using contrastive learning between entity prototypes and non-entity prototypes; DecomposedMetaNER, a meta-learning model that decomposes the named entity recognition task into two sub-tasks: boundary detection and type classification; CONTAINER, an advanced model that enhances entity representation through contrastive learning; and TadNER, a method that uses template and prompting techniques to guide a pre-trained language model for entity recognition.

[0100] The performance evaluation metric for all models was the standard F1 score (%) based on entity span. Key results are shown in Tables 1 and 2 below: Table 1. Comparative experimental results set by Intra

[0101] Table 2. Results of the comparative experiment set by Inter.

[0102] Tables 1 and 2 present the experimental results of different models on the Few-NERD dataset under two different settings. As can be seen from the experimental data in the tables, the proposed method significantly outperforms all baseline models in F1 scores across all evaluation settings. Particularly noteworthy is the most challenging INTRA5-way1-shot scenario, which simulates the extreme case where the model needs to generalize to a completely new domain relying solely on one sample from each new category. Under this condition, the proposed model achieved an F1 score of 63.36%, a significant improvement of 2.58% compared to the then-best baseline model, TadNER. This is because models relying solely on a limited support set face severe semantic inadequacy and overfitting risks. The proposed method, by efficiently injecting high-quality external knowledge during the pre-training stage, greatly compensates for the lack of support set information, enabling the model to build a more reliable understanding of new categories.

[0103] In other scenarios where data is relatively sparse, such as 5-shot or INTER, the model of this invention also maintains a stable performance advantage. Although the performance lead is reduced, the model of this invention still achieves the best performance, which proves that the knowledge fusion mechanism of this invention is robust and can provide effective performance gains without compromising the existing information in the model.

[0104] The experimental results on the impact of different knowledge sources on model performance are shown in Tables 3 and 4: Table 3. Experimental results of different knowledge sources under the Inter10way1shot settings.

[0105] Table 4. Experimental results of different knowledge sources under Intra10way1shot settings.

[0106] As shown in Tables 3 and 4, the performance under different knowledge sources exhibits a clear hierarchical relationship. When using Wikipedia as the knowledge source, the model achieved the best performance across all evaluation configurations. This highlights the crucial role of rich and accurate prior knowledge in improving entity recognition capabilities during few-shot learning. WordNet, which provides a strict hierarchical classification relationship, offers greater performance gains than ConceptNet, which focuses on broader common-sense associations. This indicates that for tasks requiring precise classification, such as named entity recognition, hierarchical knowledge that can be directly mapped to entity types provides more effective and direct support than loose common-sense associations. The gains from ConceptNet are relatively limited, which may be attributed to the sparsity and potential noise issues inherent in its knowledge, hindering the model from efficiently and accurately utilizing it.

[0107] The experimental results ultimately confirmed that the performance of this invention is closely related to the quality and structural strength of the knowledge base. Selecting a high-quality knowledge source that is highly relevant to the target task is key to maximizing model performance.

[0108] The experimental results on the impact of different encoders on model performance are shown in Tables 5 and 6: Table 5. Experimental results of different encoders under Inter5way1shot settings

[0109] Table 6 Experimental results of different encoders under Intra5way1shot settings

[0110] The experimental results in Tables 5 and 6 show that different encoders significantly impact model performance: the ERT-based model outperforms its more technically advanced successors, such as RoBERTa and DeBERTa. This phenomenon may be attributed to the specific nature of few-shot learning environments. While more complex encoders (such as RoBERTa) perform better in scenarios with ample data, they may be more prone to overfitting to the limited support set in extremely sparse scenarios, thus impairing their generalization ability. In contrast, BERT-based models achieve a better balance between model complexity and representational power, making them more suitable for few-shot scenarios. The performance of lightweight models varies depending on the task type. On the INTRA task, ALBERT, which employs a parameter-sharing mechanism, performs better, indicating that its ability to preserve core language patterns helps in generalizing to new, unseen categories within the same domain. On the INTER cross-domain task, DistilBERT, which retains the high-level semantic discriminative power of the teacher model (BERT) through knowledge distillation, performs better, demonstrating its ability to better address cross-domain challenges with more significant semantic gaps. While SciBERT performs exceptionally well in the scientific literature domain, it performs poorly on the general Few-NERD dataset. This suggests that the knowledge it gains through pre-training in a specific domain may actually become a constraint in general-domain scenarios, hindering its generalization ability.

[0111] The results of this experiment ultimately confirm that when selecting a backbone encoder for the method of this invention, a more complex or specialized model is not necessarily better. In scenarios with few samples, models like BERT-base, which have a balanced representational capability and complexity, are the preferred option for achieving optimal performance.

[0112] The experimental results on the impact of different numbers of knowledge sentences on model performance are shown in Table 7: Table 7. Experimental results for different numbers of knowledge sentences under Intra5way1shot settings.

[0113] As shown in Table 7, the experimental results indicate that increasing the amount of external knowledge actually led to a decrease in model performance. The model achieved the best performance when using single-sentence knowledge for enhancement, with an F1 score of 63.36%. When the number of knowledge sentences increased to two and three, the F1 score continued to decline, dropping to 61.04% and 57.96%, respectively.

[0114] Descriptive statements retrieved from knowledge bases, while containing core definitions, often contain a significant amount of noise information irrelevant to the target entity type. For example, when describing an "organization" entity, a knowledge statement might also mention the organization's "awards" or its "geographical location." When multiple such statements are merged, this irrelevant noise accumulates, potentially blurring or even distorting the entity's core semantic representation within the current context. Forcibly compressing information from multiple descriptive statements into a single knowledge representation dilutes semantic focus. Key attributes defining the entity become less prominent due to the mixing of too much other information. This can cause the final generated category prototype vector to deviate from its optimal position in the feature space, thus affecting classification accuracy.

[0115] The results of this experiment ultimately confirm that, for the knowledge enhancement mechanism of this invention, the quality of knowledge is far more important than the quantity. Selecting and integrating single, high-quality knowledge statements that are most relevant to the context is the preferred strategy for achieving optimal performance.

[0116] Example 2 This invention discloses a few-shot named entity recognition system based on a neural network model, referring to... Figure 3 As shown, it includes the following modules: The knowledge acquisition module is used to acquire the original input text to be identified and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text; The knowledge filtering module is used to filter candidate knowledge statements using a dynamic similarity threshold algorithm, and select the enhanced knowledge that semantically matches the original input text. The text representation module is used to concatenate the original input text with the enhanced knowledge to form a knowledge-enhanced input sequence, which is then input into the first encoder to generate a contextual text representation matrix of deep fusion knowledge. The span detection module is used to input the context text representation matrix into the sequence labeling classification layer, classify each word in the input sequence, and identify the boundary span of one or more potential entities in the text; The type classification module is used to input the text corresponding to the boundary span of the identified entities into the second encoder to generate entity embedding representations, and to determine the final category of the entity by calculating the distance between the entity embedding representation and the predefined category prototype, thus completing named entity recognition.

[0117] This embodiment applies to few-sample named entity recognition in the healthcare field. Within a hospital information system, it automatically identifies key medical entities from doctors' clinical records (such as outpatient medical records and discharge summaries). This is crucial for assisting diagnosis, constructing patient knowledge graphs, and clinical research.

[0118] Specifically, this embodiment is applied to a medical consultation platform system. The few-shot named entity recognition system based on a neural network model in this embodiment is used to identify entities such as diseases, symptoms, and drugs from user-input text. The system comprises: a knowledge acquisition module to acquire user-input text and retrieve a set of candidate knowledge statements; a knowledge filtering module to filter candidate knowledge statements and select enhanced knowledge that semantically matches the user-input text; a text representation module to concatenate the user-input text with the enhanced knowledge to form a knowledge-enhanced input sequence, which is then input into a first encoder to generate a deep-fusion knowledge context text representation matrix; a span detection module to input the context text representation matrix into a sequence labeling classification layer to classify each lexical unit in the input sequence and identify the boundary span of one or more potential entities in the text; and a type classification module to input the text corresponding to the identified entity boundary spans into a second encoder to generate entity embedding representations, and to determine the final category of the entity by calculating the distance between the entity embedding representation and a predefined category prototype, thus completing named entity recognition.

[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0120] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A few-shot named entity recognition method based on a neural network model, characterized in that, Includes the following steps: S1. Obtain the original input text to be identified, and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text; S2. A dynamic similarity threshold algorithm is used to filter the candidate knowledge statements and select the enhanced knowledge that semantically matches the original input text. S3. The original input text and the enhanced knowledge are concatenated to form a knowledge-enhanced input sequence, which is then input into the first encoder to generate a contextual text representation matrix of deep fusion knowledge. S4. Input the context text representation matrix into the sequence labeling classification layer, classify each word in the input sequence, and identify the boundary span of one or more potential entities in the text. S5. Input the text corresponding to the identified entity boundary span into the second encoder to generate an entity embedding representation, and determine the final category of the entity by calculating the distance between the entity embedding representation and the predefined category prototype, thus completing named entity recognition.

2. The method for few-shot named entity recognition based on a neural network model as described in claim 1, characterized in that, Step S2 specifically includes: S21. Calculate the semantic similarity score between each candidate knowledge statement and the original input text; S22. Determine the length type based on the sequence length of the original input text; the length types include: short sentences, medium-length sentences, and long sentences; each length type corresponds to a different similarity score filtering range; S23. Retain candidate knowledge statements whose semantic similarity scores fall within the filtering range corresponding to the length type of the original input text, as enhanced knowledge.

3. The few-shot named entity recognition method based on a neural network model as described in claim 2, characterized in that, Step S21 specifically includes: The original input text and a candidate knowledge statement are treated as two text sequences, and semantic similarity analysis is performed. Calculate the cosine similarity of the word embeddings in two text sequences to obtain precision, recall, and semantic similarity scores; expressed by the formula: ； ； ； in, This represents the i-th embedding vector of the original input text. This represents the j-th embedding vector of the candidate knowledge statement; m represents the sequence length of the original input text, and n represents the sequence length of the candidate knowledge statement; Indicates recall rate, Indicates accuracy. This represents the semantic similarity score.

4. The method for few-shot named entity recognition based on a neural network model as described in claim 1, characterized in that, Step S3 specifically includes: S31. The original input text is fused with the enhanced knowledge to obtain a knowledge-enhanced input sequence. The formula is expressed as follows: ； Where ⊕ represents the feature concatenation operation, This represents the entity span in the original input text. This indicates that the content retrieved from an external knowledge base is related to... Related augmented knowledge: [SEP] represents a separator; S32. Input the input sequence into the first encoder, use a multi-head self-attention mechanism to perform context modeling on all lexical units in the sequence, and integrate the semantic information of the augmented knowledge into the representation of each lexical unit to obtain the context text representation.

5. The method for few-shot named entity recognition based on a neural network model as described in claim 1, characterized in that, Step S4 specifically includes: S41. Input the context text representation matrix into the sequence labeling classification layer, and process the matrix... The embedded vectors in the sequence are used for sequence labeling, and the probability that each word belongs to an entity is calculated, expressed by the formula: ； in, and These represent the weight matrix and bias vector of the linear layer, respectively. Representation matrix The embedding vector of the i-th dimension; S42. Using a binary classification span detection model, span detection is performed on each word corresponding to the probability; the boundary span of the potential entity in the text is identified.

6. The few-shot named entity recognition method based on a neural network model as described in claim 5, characterized in that, The binary span detection model optimizes its parameters by minimizing the cross-entropy loss between the predicted probability distribution and the true label. The cross-entropy loss is expressed by the formula: ； Where L is the total length of the sequence. It is the actual label of the i-th word element.

7. The method for few-shot named entity recognition based on a neural network model as described in claim 1, characterized in that, In step S5, the second encoder is trained using supervised contrastive learning, specifically including: Construct a dataset for model training; for each entity, concatenate its span text and semantic category name in two different orders to create a pair of semantically equivalent but sequentially different positive samples, and add them to the dataset; The positive sample pairs are input into the second encoder to generate entity embedding representations, which can be expressed by the following formula: ； ； Where ⊕ represents the concatenation operator, and This represents a pair of constructed positive samples. Indicates the second encoder. Represents the i-th entity word element. This represents the i-th correct entity label. This is a conversion function.

8. The few-shot named entity recognition method based on a neural network model as described in claim 7, characterized in that, In step S5, the second encoder is optimized using a supervised contrastive learning loss function, expressed by the formula: ； ； ； in, This represents the contrastive loss, where I represents the total number of samples in the training batch. r represents the set of all positive samples belonging to the same class as sample i; z r represents the true class label of the sample with index z. i This represents the true category label of sample i. and These represent samples constructed according to "entity-label" and "label-entity" respectively; τ represents the cosine similarity; τ represents the temperature hyperparameter.

9. The method for few-shot named entity recognition based on a neural network model as described in claim 1, characterized in that, The external knowledge base is a large-scale structured or semi-structured knowledge source, including Wikipedia, WordNet, and ConceptNet.

10. A few-shot named entity recognition system based on a neural network model, characterized in that, The few-shot named entity recognition method based on a neural network model as described in any one of claims 1-9 includes the following modules: The knowledge acquisition module is used to acquire the original input text to be identified, and retrieve a set of candidate knowledge statements from an external knowledge base based on the original input text; The knowledge filtering module is used to filter the candidate knowledge statements using a dynamic similarity threshold algorithm, and select the enhanced knowledge that semantically matches the original input text. The text representation module is used to concatenate the original input text with the enhanced knowledge to form a knowledge-enhanced input sequence, and input it into the first encoder to generate a context text representation matrix of deep fusion knowledge; The span detection module is used to input the context text representation matrix into the sequence labeling classification layer, classify each word in the input sequence, and identify the boundary span of one or more potential entities in the text; The type classification module is used to input the text corresponding to the boundary span of the identified entity into the second encoder to generate an entity embedding representation, and to determine the final category of the entity by calculating the distance between the entity embedding representation and the predefined category prototype, thus completing named entity recognition.

Images