Low-resource vertical domain named entity recognition method and system

Abstract

The application relates to the technical field of natural language processing, and provides a low-resource vertical field named entity recognition method and system.The method comprises the following steps: a shared encoder is used to extract deep context representation of a text; the deep context representation is decoded through a cascaded two-stage decoding architecture, the cascaded two-stage decoding architecture comprises a coarse-grained stage and a fine-grained stage, and the coarse-grained stage and the fine-grained stage share the deep context representation; the coarse-grained stage predicts a core interval of an entity through a pointer network, and outputs a coarse-grained entity span set; the fine-grained stage extracts fine-grained entities based on the coarse-grained entity span set and in relation to unoccupied text regions; through cooperation of the coarse-grained stage and the fine-grained stage, the named entity recognition is completed based on the deep context representation, including entity span prediction and category labeling.The application can realize low-resource vertical field named entity recognition with light weight and high precision without relying on external resources.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and system for low-resource vertical domain named entity recognition. Background Technology

[0002] Named Entity Recognition (NER) is one of the core foundational tasks in the field of Natural Language Processing. Its goal is to automatically identify and classify entities with specific semantic meanings from unstructured text. NER technology is a core prerequisite for advanced tasks such as Natural Language Understanding, Machine Translation, Intelligent Question Answering, and Knowledge Graph Construction, and has been widely applied in vertical fields such as finance, healthcare, government affairs, and the internet.

[0003] From a technological development perspective, NER has evolved from rule-based and statistical learning-based approaches to deep learning-based approaches. Current research focuses on low-resource scenarios, vertical domain adaptation, privacy protection, and lightweight deployment. Among these, low-resource vertical domain NER has become a research hotspot and a technical challenge due to issues such as scarce labeled data, complex entity hierarchies, strong domain specificity, and limited external auxiliary resources.

[0004] Currently, for low-resource scenarios, existing research mainly focuses on three technical approaches: methods based on external dictionary enhancement (such as FLAT and LEBERT), methods based on multi-source feature fusion (such as MECT), and methods based on transfer learning (such as PCBERT). However, all three methods still rely on external resources, and their specific technical solutions and limitations are as follows: The FLAT (Flat-Lattice Transformer) model transforms a complex dynamic lattice structure composed of characters and latent words matched from a dictionary into a flat span structure containing head and tail position indices. This preserves the lattice structure information while adapting to the parallel computing characteristics of the Transformer. It also incorporates customized relative position encodings based on four relative distances: head-to-head, head-to-tail, tail-to-head, and tail-to-tail. These encodings are integrated into the Transformer's self-attention computation, allowing self-attention to capture both token semantics and span positional relationships. Through the Transformer's fully connected self-attention, it enables direct interaction between characters and matched words and long-distance dependency modeling. The model uses a single-layer Transformer encoder to encode the flat lattice structure, extracting only the character-specific encoding representations and inputting them into the CRF layer for entity labeling and recognition. This eliminates the need to modify the core model structure to adapt to lattice inputs; instead, it characterizes the lattice structure solely through positional encoding. This solves the problems of poor parallelism and difficulty in modeling long-distance dependencies inherent in traditional lattice models. Furthermore, the model structure is simple and compatible with pre-trained models such as BERT.

[0005] LEBERT Model: Addressing the issue in existing Chinese sequence labeling methods that only fuse dictionary features through shallow, randomly initialized layers without integrating them into the underlying BERT model, we propose the dictionary-enhanced LEBERT model. Its core functionality involves using a dictionary adapter (Lexicon). The Adapter integrates external dictionary knowledge directly into the Transformer layers of BERT, achieving deep fusion of dictionary features and BERT representations. First, the Chinese sentence matching dictionary is transformed into a sequence of character-word pairs. Each character is assigned a potential matching word. Then, a dictionary adapter is designed. First, a non-linear transformation is applied to the word embeddings to align the representation dimensions of characters and words. Then, a character-word bilinear attention mechanism is used to select the most relevant matching words for each character and weighted fused word features. The fused dictionary features are injected into the BERT layer output of the corresponding character. Simultaneously, this adapter is deployed between shallow Transformer layers of BERT. During training, the original BERT parameters and adapter parameters are fine-tuned simultaneously, rather than fixing the BERT layer. Finally, the character representation of the final BERT layer is input into the CRF layer to complete sequence labeling. This scheme achieves deep fusion of dictionary features at the BERT level, rather than the traditional shallow fusion at the model level. It achieves state-of-the-art (SOTA) performance on ten Chinese benchmark datasets for three major tasks: named entity recognition, word segmentation, and part-of-speech tagging. It also demonstrates stronger robustness in long sentence processing, validating the improvement effect of low-level feature fusion on entity boundary detection and type determination.

[0006] MECT Model: This paper addresses the issue of existing vocabulary augmentation methods neglecting the structural information of Chinese characters in Chinese NER tasks. Building upon the FLAT model, it proposes the MECT cross-Transformer model based on multivariate data embedding. It employs a dual-stream architecture to fuse character, vocabulary, and radical structural information to improve recognition performance. First, Chinese characters are decomposed into structural components as radical features. Radical-level embedding features are extracted using a CNN. Simultaneously, the FLAT method is used to construct lexical lattice embeddings to preserve lexical semantics and boundary information. Then, a Cross-Transformer module is designed as the core of the dual-stream approach, with lattice embeddings and radical-level embeddings serving as inputs to the two streams respectively. Through linear transformation... The model obtains the Q, K, and V values ​​of each stream and introduces FLAT relative position encoding to calculate attention scores. Simultaneously, a randomly initialized attention matrix is ​​added as an attention bias to optimize cross-stream attention calculation, achieving cross-interaction and complementarity of features from the two streams. The dual-stream output features are then concatenated and fused through a fully connected layer. Finally, the lexical features are masked, and the fused features are input into a CRF layer for entity labeling. This model can also be combined with BERT to further improve performance, achieving superior recognition results on four Chinese NER benchmark datasets: Weibo, Resume, MSRA, and Ontonotes 4.0, with only a slight reduction in inference speed, maintaining good computational efficiency.

[0007] PCBERT Model: This paper addresses the issues of poor transfer learning performance and implicit character boundaries caused by semantic distribution differences in Chinese few-shot NER. It proposes a cue-based parent-child BERT model, PCBERT, which includes two components, P-BERT and C-BERT, to fuse dictionary features and implicit label features. First, a high-resource dataset is constructed to train the labeled model. Then, a low-resource target dataset is labeled to obtain a labeled extended dataset. Finally, P-BERT is fine-tuned on this dataset using cue-based techniques (the input is transformed into "[Index] is..." through a template function). [Z]” format prompt input, mask label slots and train recovery), freeze P-BERT after training to provide label extension features; C-BERT is an improvement on LEBERT, which fuses the label extension features of P-BERT with its own word embedding as input, adds a label dictionary adapter (LLA) after the first encoder layer, calculates the correlation between the dictionary and multi-label extension features through label attention and weighted fusion, and then injects the fused dictionary features into BERT through word attention; finally, the C-BERT output is enhanced with temporal information by BiLSTM, and further fused with the prompt features of P-BERT through interactive attention, and finally input into the CRF layer to complete entity annotation. This scheme achieves cross-domain label transfer through label extension strategy, effectively improving the recognition performance of Chinese NER in few-shot scenarios.

[0008] In summary, the traditional approach has the following objective drawbacks: Dependence on external resources makes it unsuitable for privacy-sensitive scenarios: FLAT and LEBERT both rely on external dictionaries or word embeddings to fuse lexical information, MECT requires the introduction of external radical structure datasets and dictionaries, and PCBERT not only relies on external dictionaries but also requires the construction of large-scale, high-resource datasets to achieve label expansion. The introduction of these external resources limits its deployment in vertical fields where privacy is restricted or resources are difficult to obtain.

[0009] Insufficient adaptability to low resources: The performance of FLAT, LEBERT, and MECT depends on sufficient training data. Their performance degrades in extremely low-resource scenarios with fewer than 1300 training samples. Although PCBERT is designed for scenarios with few samples, it still requires high-resource datasets to assist in label transfer. When vertical domain data is extremely scarce and there is no high-resource transfer foundation, the effect is difficult to guarantee.

[0010] The above solutions fail to address the issue of entity hierarchy consistency: their core design focuses on word fusion, radical features, or label transfer, but none of them specifically address the pain point of "dense entity hierarchy structure" in low-resource vertical domains, which can easily lead to label conflicts in complex hierarchical entity recognition.

[0011] The models are highly complex: FLAT requires the construction of a flat grid structure and the design of complex positional encoding; MECT adopts a dual-stream Transformer architecture and introduces CNN to extract radical features; PCBERT includes dual BERT components and a multi-attention fusion mechanism. All three have a large number of parameters. Insufficient optimization of entity boundary localization: FLAT relies on span position encoding, LEBERT fuses lexical features through adapters, and MECT improves semantic capture by leveraging cross-stream attention, but none of them adopt a dedicated boundary optimization loss function, making the problem of blurred entity boundaries more prominent in low-resource scenarios. Summary of the Invention

[0012] The purpose of this invention is to solve at least one technical problem in the background art and to provide a low-resource vertical domain named entity recognition method and system.

[0013] To achieve the above objectives, the present invention provides a low-resource vertical domain named entity recognition method, comprising: A shared encoder is used to extract deep contextual representations of the text; The deep context representation is decoded using a cascaded two-stage decoding architecture, which includes a coarse-grained stage and a fine-grained stage, both of which share the deep context representation. Specifically, the coarse-grained stage predicts the core regions of entities using a pointer network and outputs a coarse-grained entity span set. The fine-grained stage then determines unoccupied regions in the deep context representation based on this coarse-grained entity span set and further extracts fine-grained entities within these regions. Through the collaboration of the coarse-grained and fine-grained stages, named entity recognition is completed based on the deep contextual representation, including entity span prediction and category labeling.

[0014] According to one aspect of the invention, the shared encoder is a BERT encoder.

[0015] According to one aspect of the invention, it further includes: Construct a hard mutual exclusion mask, and based on the coarse-grained entity span set output by the coarse-grained stage, mark the text positions in the coarse-grained entity span set as prohibited areas and the unoccupied text positions as allowed areas; During the fine-grained decoding stage, a hard mask constraint is applied to the original logits output by the decoder to set the logits of the prohibited region to a minimum value, so as to prevent the same token from being labeled by multiple levels of entities at the same time, and to force the mutual exclusion of hierarchical labels.

[0016] According to one aspect of the invention, it further includes: Based on the boundary probability distribution output by the decoder in the fine-grained stage, the discrete token position prediction is transformed into the expected starting and ending coordinates of the prediction boundary through probability weighted averaging; at the same time, the discrete boundary labels of the real entity span are mapped to the corresponding continuous starting and ending coordinates. The prediction interval is formed by the starting and ending expected coordinates of the predicted boundary, and the real interval is formed by the starting and ending continuous coordinates of the real entity span. A 1D Distance-IoU regression loss is introduced to calculate the overlap rate, center distance, and minimum enclosing interval length between the predicted and real intervals to measure the boundary positioning error. A classification loss for optimizing category labeling is introduced, and the classification loss is combined with the 1D Distance-IoU regression loss to construct a joint optimization objective of the classification loss and the 1D Distance-IoU regression loss, thereby optimizing both the boundary positioning accuracy and the category labeling accuracy.

[0017] According to one aspect of the invention, it further includes: An adaptive gating fusion mechanism is designed, which dynamically balances the deep context representation output by the shared encoder with the boundary probability features output by the coarse-grained stage pointer network through learnable gating coefficients to generate fused features for the fine-grained stage decoding.

[0018] According to one aspect of the invention, it further includes: In the coarse-grained stage, a double affine mechanism is introduced. The deep context representation of each token is mapped to start and end features by a multilayer perceptron. The semantic interaction between the start and end positions is explicitly modeled by the double affine transformation, and the entity span score is calculated to suppress invalid spans where the start position is greater than the end position.

[0019] To achieve the above objectives, the present invention also provides a low-resource vertical domain named entity recognition system, comprising: The deep context representation extraction module uses a shared encoder to extract the deep context representation of the text. The cascaded two-stage decoding architecture decoding module decodes the deep context representation using a cascaded two-stage decoding architecture, which includes a coarse-grained stage and a fine-grained stage. The coarse-grained stage and the fine-grained stage share the deep context representation. Specifically, the coarse-grained stage predicts the core interval of entities through a pointer network and outputs a coarse-grained entity span set. The fine-grained stage determines the unoccupied regions in the deep context representation based on the coarse-grained entity span set and further extracts fine-grained entities within these regions. The named entity recognition module, through the collaboration of the coarse-grained and fine-grained stages, completes named entity recognition based on the deep context representation, including entity span prediction and category labeling.

[0020] To achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the low-resource vertical domain named entity recognition method as described above.

[0021] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the low-resource vertical domain named entity recognition method as described above.

[0022] According to the solution of the present invention, compared with the conventional solution, the present invention can achieve the following beneficial effects: (i) No external resource dependency, suitable for privacy-sensitive and low-resource scenarios Traditional solutions (such as FLAT, LEBERT, and MECT) rely on external dictionaries, while the traditional PCBERT solution depends on pre-trained annotators on high-resource datasets, limiting deployment in vertical domains with privacy concerns or scarce resources. This invention, however, can complete training using only a limited amount of labeled data in the target domain (less than 1300 samples), without introducing any external resources, thus avoiding privacy risks from the outset and maintaining stable performance even in extremely low-resource scenarios.

[0023] (ii) Enforce mutual exclusion of hierarchical labels to improve recognition consistency Traditional solutions lack a dedicated mechanism for handling hierarchical entities, relying solely on feature fusion to indirectly mitigate conflicts. This can easily lead to the same token being simultaneously labeled by entities at multiple levels. This invention uses a hard mutual exclusion mask to enforce mutual exclusion of hierarchical tags from the decoding logic, ensuring that tokens occupied by coarse-grained entities are not re-labeled at the fine-grained stage. This guarantees the consistency of the entity hierarchy and effectively reduces tag conflict errors.

[0024] (iii) Improved boundary positioning accuracy and reduced boundary offset Traditional methods rely on indirect approaches such as location encoding and dictionary features to optimize boundaries, but they do not design a dedicated loss function for boundary localization, resulting in the neglect of spatial continuity. This invention introduces 1D Distance-IoU regression loss into entity boundary localization, transforming discrete token prediction into geometric optimization of continuous intervals. Simultaneously, it constrains the overlap rate and center alignment between the predicted and ground truth intervals, reducing entity boundary offset and improving the accuracy of matching.

[0025] (iv) Alleviate semantic deprivation and reduce error propagation Traditional cascaded models (such as PCBERT) use hard masks to remove features from non-entity regions, leading to semantic loss in downstream stages and amplification of errors at each stage. This invention uses an adaptive gating fusion mechanism to dynamically balance global semantics with the positional priors of the coarse-grained stages, generating fused features for decoding in the fine-grained stages. This preserves complete contextual information and uses the recognition results from preceding stages to guide the decoding direction, effectively suppressing error propagation.

[0026] (v) Lightweight architecture, balancing performance and efficiency Traditional solutions involve a large number of parameters (all exceeding 10M), resulting in high inference latency and limited deployment flexibility. This invention achieves a lightweight design through a shared encoder and simplified decoding structure, with a parameter range of 0.01M to 110M and an inference latency as low as 3.2ms, meeting the requirements of industrial-grade low-latency deployment while maintaining leading performance.

[0027] (vi) It is robust to complex scenarios and adaptable to diverse text types. Traditional solutions lack robustness for complex scenarios such as short texts, high noise levels, and nested entities. This invention combines dual affine interaction with an adaptive fusion mechanism to enhance the semantic association of entity boundaries and the ability to preserve global context. On complex datasets such as Weibo, the recognition performance of named entities (NE) and nominal entities (NM) is superior to existing models, and the invention has a wider range of applications. Attached Figure Description

[0028] Figure 1 The flowchart illustrates a low-resource vertical domain named entity recognition method according to one embodiment of the present invention. Detailed Implementation

[0029] The invention will now be discussed with reference to exemplary embodiments. It should be understood that the described embodiments are merely intended to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0030] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment".

[0031] Figure 1 The flowchart illustrates a low-resource vertical domain named entity recognition method according to one embodiment of the present invention. Figure 1 As shown, in this embodiment, the low-resource vertical domain named entity recognition method includes: A shared encoder is used to extract deep contextual representations of the text; The deep context representation is decoded using a cascaded dual-stage decoding architecture, which includes a coarse-grained stage and a fine-grained stage. The coarse-grained stage and the fine-grained stage share the deep context representation to avoid redundant encoding. Specifically, the coarse-grained stage predicts the core regions of entities using a pointer head network and outputs a coarse-grained entity span set to clarify the approximate boundaries and top-level categories of entities. The fine-grained stage determines the unoccupied regions in the deep context representation based on the coarse-grained entity span set and further extracts fine-grained entities within these regions, achieving complementary recognition of hierarchical entities. Through the collaboration of coarse-grained and fine-grained stages, named entity recognition is completed based on deep contextual representation, including entity span prediction and category labeling.

[0032] In this implementation, the coarse-grained stage and the fine-grained stage share the deep context representation, avoiding redundant encoding. The number of parameters is 0.01M-110M, which is much lower than that of existing resource-dependent models.

[0033] In this embodiment, a lightweight, efficient, and high-precision named entity recognition can be achieved in low-resource vertical domains through the collaborative design of a shared encoder and a cascaded two-stage decoding architecture. Specifically, the coarse-grained and fine-grained stages share the deep contextual representation output by the same encoder, avoiding computational redundancy caused by repeated encoding and significantly reducing the number of model parameters and inference latency. The coarse-grained stage quickly locates the core region of the entity and outputs the top-level category through a pointer network, while the fine-grained stage further mines fine-grained information in unoccupied text regions, forming a collaborative mechanism of coarse-to-fine and complementary recognition, effectively solving the problem of complete coverage of complex-level entities. This method does not rely on external dictionaries, pre-trained annotators, or additional corpora; it can be trained using only the limited labeled data of the target domain itself. It can maintain stable recognition performance even in privacy-sensitive scenarios with scarce data. At the same time, the progressive localization strategy from coarse to fine optimizes the entity boundary accuracy, taking into account both the efficiency requirements of industrial deployment and the adaptation needs of vertical domains.

[0034] Furthermore, according to one embodiment of the present invention, the shared encoder is a BERT encoder.

[0035] Furthermore, according to one embodiment of the present invention, the low-resource vertical domain named entity recognition method further includes: Construct a Hard Mutual-Exclusion Mask (ME-Mask). Based on the coarse-grained entity span set output from the coarse-grained stage, mark occupied text positions (text positions in the coarse-grained entity span set) as prohibited regions and unoccupied text positions as allowed regions. During the fine-grained decoding stage, a hard mask constraint is applied to the original logits output by the decoder to set the logits of the prohibited region to a minimum value, so as to prevent the same token from being labeled by multiple levels of entities at the same time (strictly ensuring that the same token will not belong to both coarse and fine level entities at the same time), thus forcibly implementing mutual exclusion of hierarchical tags.

[0036] In this embodiment, the hard mutual exclusion masking mechanism mathematically forces the same token to belong to only a single-level entity during the decoding stage by rigidly dividing prohibited and allowed regions, thus completely solving the multi-label conflict problem caused by insufficient feature fusion in traditional models. Specifically, this method constructs a binary mask based on the entity span already identified in the coarse-grained stage, marking occupied text positions as prohibited regions. During the fine-grained stage decoding, a hard mask (set to a minimum value) is directly applied to the original logits, making the softmax output probability of the prohibited positions approach zero, thereby strictly ensuring that the fine-grained stage does not overlap with the coarse-grained stage. This mechanism not only achieves deterministic mutual exclusion of hierarchical labels, eliminating recognition ambiguity caused by label conflicts, but also avoids the additional computational overhead caused by complex post-processing to resolve conflicts. At the same time, it provides a clear decoding boundary for the fine-grained stage, improving the recognition consistency and reliability of the model in dense hierarchical entity scenarios.

[0037] Furthermore, according to one embodiment of the present invention, the low-resource vertical domain named entity recognition method further includes: Based on the boundary probability distribution output by the decoder in the fine-grained stage, discrete token position predictions are transformed into the expected starting and ending coordinates of the prediction boundary through probability weighted averaging; at the same time, discrete boundary labels of the real entity span are mapped to the corresponding continuous starting and ending coordinates. The prediction interval is formed by the starting and ending expected coordinates of the predicted boundary, and the real interval is formed by the starting and ending continuous coordinates of the actual entity span. A 1D Distance-IoU regression loss is introduced to calculate the overlap rate, center distance, and minimum enclosing interval length between the prediction interval and the real interval to measure the boundary positioning error. A classification loss for optimizing category labeling is introduced, and the classification loss is combined with the 1D Distance-IoU regression loss to construct a joint optimization objective of the classification loss and the 1D Distance-IoU regression loss, thereby optimizing both the boundary positioning accuracy and the category labeling accuracy.

[0038] In this implementation, by transforming discrete token boundary predictions into a continuous coordinate space and introducing 1D Distance-IoU (DIoU) regression loss and Focal classification loss for joint optimization, a synergistic improvement in entity boundary localization accuracy and category labeling accuracy is achieved. Specifically, firstly, the boundary probability distribution output by the fine-grained stage decoder is used to transform discrete token position predictions into continuous start and end expected coordinates through probability-weighted averaging. Simultaneously, the discrete boundary labels of the real entity span are mapped to corresponding continuous coordinates, thereby transforming the boundary localization problem from a discrete classification space to a continuous geometric space. Based on this, 1D Distance-IoU regression loss is introduced. By calculating the overlap rate, center distance, and minimum enclosing interval length between the predicted and real intervals, the geometric quality of the boundary is directly constrained, solving the problems of "ignoring spatial continuity" in traditional cross-entropy loss and "gradient vanishing in non-overlapping regions" in IoU loss. At the same time, the joint classification loss is used to optimize entity category labeling, constructing a joint optimization objective. This approach enables the model to simultaneously address entity boundary localization and category labeling during training, effectively reducing boundary offset errors and improving the overall accuracy of entity recognition. It is particularly suitable for vertical fields with blurred entity boundaries and high labeling noise in low-resource scenarios.

[0039] In this embodiment, the formula for the 1D Distance-IoU regression loss is: ; In the formula, Here, I is the intersection area of ​​the predicted bounding box and the ground truth bounding box, U is the union area of ​​the predicted bounding box and the ground truth bounding box, and I / U is the IoU (Intersection over Union); ρ is the Euclidean distance function, which calculates the straight-line distance between two points; C is the coordinate of the center point of the predicted bounding box; C gt The coordinates of the center point of the true bounding box; Let c be the squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box; c is the diagonal length of the smallest bounding rectangle (CMB) that can simultaneously contain both the predicted and ground truth bounding boxes; c 2 It is the square of the diagonal length of the smallest bounding rectangle, used to normalize the center distance.

[0040] Furthermore, according to one embodiment of the present invention, the low-resource vertical domain named entity recognition method further includes: To address the semantic deprivation problem in traditional cascaded architectures, an Adaptive Gated Fusion mechanism is designed, which uses learnable gating coefficients. Deep context representation of the output of a dynamically balanced shared encoder H bert Boundary probability feature P of the coarse-grained stage pointer network output stageGenerate fusion features for fine-grained stage decoding; The activation function is H; Linear is a linear transformation layer that transforms H... bert and P stage The two feature vectors are concatenated and then input into a linear layer for transformation.

[0041] In this embodiment, the fusion formula based on learnable gating coefficients is: This approach preserves global semantics while incorporating entity location priors from the coarse-grained stage, mitigating error propagation in cascaded models. Specifically, Proj(P stage ) indicates that for P stage Features are projected.

[0042] In this implementation, the adaptive gating fusion mechanism effectively solves the error propagation problem caused by "semantic deprivation" in traditional cascaded architectures by dynamically balancing global semantics and location priors through learnable gating coefficients. This mechanism dynamically weights and fuses the deep contextual representation (containing complete textual semantics) output by the shared encoder with the boundary probability features (carrying entity location priors) output by the coarse-grained stage pointer network, generating fused features rich in dual information for decoding in the fine-grained stage. Compared to the traditional approach of directly deleting non-entity region features in cascaded models, this mechanism preserves the complete global semantic integrity while incorporating the recognition priors from the coarse-grained stage. This allows the fine-grained stage to both rely on the global context for semantic understanding and utilize the localization results from preceding stages to guide the decoding direction, thereby effectively suppressing the problem of error amplification at each stage in cascaded architectures and improving the model's robustness and overall performance in complex scenarios.

[0043] Furthermore, according to one embodiment of the present invention, the low-resource vertical domain named entity recognition method further includes: To address the issue of complex entity boundary dependencies in short texts, a dual affine mechanism is introduced in the coarse-grained stage. A multilayer perceptron (MLP) is used to map the deep contextual representation of each token to the initial feature h. s (i) and the ending feature h e (j) The semantic interaction between the starting and ending positions is explicitly modeled through double affine transformation, and the entity span score is calculated. In the formula, U is the weight matrix (usually a low-dimensional dense matrix, such as d×d dimension, where d is the dimension of the feature vector), which is used to weight the initial feature h. s (i) and the ending feature h e(j) Perform a bilinear transformation to capture the interaction between the two (such as the semantic matching degree between entities and text). This is commonly used in bilinear attention and entity-text matching models and is a core parameter for modeling feature interactions. W is a weight vector (or weight matrix, depending on the model design). If it is a vector, its dimension is the same as that of the feature vector. It is used to transform the initial feature h. s (i) and the ending feature h e (j) concatenation, element-wise summation and linear weighting of combined features capture the global importance of features; W[h s (i);h e [j] usually represents a linear transformation of the vector after concatenating two features ([;] is the concatenation operation), or it can represent summation after element-wise multiplication; b is a bias scalar (single value) used to add a global bias to the scoring function, to compensate for the score offset after the linear transformation, and to improve the model's fitting ability.

[0044] This method aims to suppress invalid spans where the starting position is greater than the ending position, thereby improving boundary positioning accuracy.

[0045] In this embodiment, the dual affine mechanism effectively addresses the complex entity boundary dependencies in short texts by explicitly modeling the semantic interaction between the start and end positions at the coarse-grained stage, significantly improving boundary localization accuracy. The mechanism first uses a multilayer perceptron to map the deep contextual representation of each token into start and end features, decoupling the entity boundary information into two independent but related feature spaces. Then, it calculates the interaction score between the start and end features through dual affine transformation, enabling the model to directly learn the complete span dependency relationship of "where the entity starts and ends." Based on this, it applies low-score suppression to invalid spans where the start position is greater than the end position, geometrically constraining the legitimacy of entity boundaries. Compared to traditional methods that implicitly learn boundaries through sequence labeling, this mechanism explicitly models the internal structure of the span, strengthening the model's ability to recognize scenarios with closely spaced entity boundaries and complex dependencies in short texts, effectively reducing boundary offsets and invalid predictions, and improving the overall robustness of entity recognition.

[0046] According to the above-described solution of the present invention, compared with the conventional solution, the present invention can achieve the following beneficial effects: (i) No external resource dependency, suitable for privacy-sensitive and low-resource scenarios Traditional solutions (such as FLAT, LEBERT, and MECT) rely on external dictionaries, while the traditional PCBERT solution depends on pre-trained annotators on high-resource datasets, limiting deployment in vertical domains with privacy concerns or scarce resources. This invention, however, can complete training using only a limited amount of labeled data in the target domain (less than 1300 samples), without introducing any external resources, thus avoiding privacy risks from the outset and maintaining stable performance even in extremely low-resource scenarios.

[0047] (ii) Enforce mutual exclusion of hierarchical labels to improve recognition consistency Traditional solutions lack a dedicated mechanism for handling hierarchical entities, relying solely on feature fusion to indirectly mitigate conflicts. This can easily lead to the same token being simultaneously labeled by entities at multiple levels. This invention uses a hard mutual exclusion mask to enforce mutual exclusion of hierarchical tags from the decoding logic, ensuring that tokens occupied by coarse-grained entities are not re-labeled at the fine-grained stage. This guarantees the consistency of the entity hierarchy and effectively reduces tag conflict errors.

[0048] (iii) Improved boundary positioning accuracy and reduced boundary offset Traditional methods rely on indirect approaches such as location encoding and dictionary features to optimize boundaries, but they do not design a dedicated loss function for boundary localization, resulting in the neglect of spatial continuity. This invention introduces 1D Distance-IoU regression loss into entity boundary localization, transforming discrete token prediction into geometric optimization of continuous intervals. Simultaneously, it constrains the overlap rate and center alignment between the predicted and ground truth intervals, reducing entity boundary offset and improving the accuracy of matching.

[0049] (iv) Alleviate semantic deprivation and reduce error propagation Traditional cascaded models (such as PCBERT) use hard masks to remove features from non-entity regions, leading to semantic loss in downstream stages and amplification of errors at each stage. This invention uses an adaptive gating fusion mechanism to dynamically balance global semantics with the positional priors of the coarse-grained stages, generating fused features for decoding in the fine-grained stages. This preserves complete contextual information and uses the recognition results from preceding stages to guide the decoding direction, effectively suppressing error propagation.

[0050] (v) Lightweight architecture, balancing performance and efficiency Traditional solutions involve a large number of parameters (all exceeding 10M), resulting in high inference latency and limited deployment flexibility. This invention achieves a lightweight design through a shared encoder and simplified decoding structure, with a parameter range of 0.01M to 110M and an inference latency as low as 3.2ms, meeting the requirements of industrial-grade low-latency deployment while maintaining leading performance.

[0051] (vi) It is robust to complex scenarios and adaptable to diverse text types. Traditional solutions lack robustness for complex scenarios such as short texts, high noise levels, and nested entities. This invention combines dual affine interaction with an adaptive fusion mechanism to enhance the semantic association of entity boundaries and the ability to preserve global context. On complex datasets such as Weibo, the recognition performance of named entities (NE) and nominal entities (NM) is superior to existing models, and the invention has a wider range of applications.

[0052] Furthermore, to achieve the above objectives, the present invention also provides a low-resource vertical domain named entity recognition system, comprising: The deep context representation extraction module uses a shared encoder to extract the deep context representation of the text. The cascaded two-stage decoding architecture decodes the deep context representation. This architecture includes a coarse-grained stage and a fine-grained stage, which share the deep context representation. Specifically, the coarse-grained stage predicts the core regions of entities using a pointer network and outputs a coarse-grained entity span set. The fine-grained stage, based on this coarse-grained entity span set, determines unoccupied regions in the deep context representation and further extracts fine-grained entities within these regions. The named entity recognition module completes named entity recognition based on deep contextual representation through the collaboration of coarse-grained and fine-grained stages, including entity span prediction and category labeling.

[0053] The low-resource vertical domain named entity recognition system according to the present invention can realize the low-resource vertical domain named entity recognition method. The specific process steps are as described above and will not be repeated here.

[0054] Furthermore, to achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the low-resource vertical domain named entity recognition method as described above.

[0055] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the low-resource vertical domain named entity recognition method as described above.

[0056] Those skilled in the art will recognize that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0057] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and equipment can be referred to the corresponding process in the foregoing method implementation, and will not be repeated here.

[0058] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0059] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs.

[0060] In addition, the functional modules in the embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0061] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the sending / receiving methods of various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0062] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

[0063] It should be understood that the sequence number of each step in the invention and its embodiments does not absolutely imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. A method for naming entities in low-resource vertical domains, characterized in that, include: A shared encoder is used to extract deep contextual representations of the text; The deep context representation is decoded using a cascaded two-stage decoding architecture, which includes a coarse-grained stage and a fine-grained stage, both of which share the deep context representation. Specifically, the coarse-grained stage predicts the core regions of entities using a pointer network and outputs a coarse-grained entity span set. The fine-grained stage then determines unoccupied regions in the deep context representation based on this coarse-grained entity span set and further extracts fine-grained entities within these regions. Through the collaboration of the coarse-grained and fine-grained stages, named entity recognition is completed based on the deep contextual representation, including entity span prediction and category labeling.

2. The low-resource vertical domain named entity recognition method according to claim 1, characterized in that, The shared encoder is a BERT encoder.

3. The low-resource vertical domain named entity recognition method according to claim 1, characterized in that, Also includes: Construct a hard mutual exclusion mask, and based on the coarse-grained entity span set output by the coarse-grained stage, mark the text positions in the coarse-grained entity span set as prohibited areas and the unoccupied text positions as allowed areas; During the fine-grained decoding stage, a hard mask constraint is applied to the original logits output by the decoder to set the logits of the prohibited region to a minimum value, so as to prevent the same token from being labeled by multiple levels of entities at the same time, and to force the mutual exclusion of hierarchical labels.

4. The low-resource vertical domain named entity recognition method according to claim 1, characterized in that, Also includes: Based on the boundary probability distribution output by the decoder in the fine-grained stage, the discrete token position prediction is transformed into the expected starting and ending coordinates of the prediction boundary through probability weighted averaging; at the same time, the discrete boundary labels of the real entity span are mapped to the corresponding continuous starting and ending coordinates. The prediction interval is formed by the starting and ending expected coordinates of the predicted boundary, and the real interval is formed by the starting and ending continuous coordinates of the real entity span. A 1D Distance-IoU regression loss is introduced to calculate the overlap rate, center distance, and minimum enclosing interval length between the predicted and real intervals to measure the boundary positioning error. A classification loss for optimizing category labeling is introduced, and the classification loss is combined with the 1D Distance-IoU regression loss to construct a joint optimization objective of the classification loss and the 1D Distance-IoU regression loss, thereby optimizing both the boundary positioning accuracy and the category labeling accuracy.

5. The low-resource vertical domain named entity recognition method according to claim 1, characterized in that, Also includes: An adaptive gating fusion mechanism is designed, which dynamically balances the deep context representation output by the shared encoder with the boundary probability features output by the coarse-grained stage pointer network through learnable gating coefficients to generate fused features for the fine-grained stage decoding.

6. The low-resource vertical domain named entity recognition method according to claim 1, characterized in that, Also includes: In the coarse-grained stage, a double affine mechanism is introduced. The deep context representation of each token is mapped to start and end features by a multilayer perceptron. The semantic interaction between the start and end positions is explicitly modeled by the double affine transformation, and the entity span score is calculated to suppress invalid spans where the start position is greater than the end position.

7. A low-resource vertical domain named entity recognition system, characterized in that, include: The deep context representation extraction module uses a shared encoder to extract the deep context representation of the text. The cascaded two-stage decoding architecture decoding module decodes the deep context representation using a cascaded two-stage decoding architecture, which includes a coarse-grained stage and a fine-grained stage. The coarse-grained stage and the fine-grained stage share the deep context representation. Specifically, the coarse-grained stage predicts the core interval of entities through a pointer network and outputs a coarse-grained entity span set. The fine-grained stage determines the unoccupied regions in the deep context representation based on the coarse-grained entity span set and further extracts fine-grained entities within these regions. The named entity recognition module, through the collaboration of the coarse-grained and fine-grained stages, completes named entity recognition based on the deep context representation, including entity span prediction and category labeling.

8. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the low-resource vertical domain named entity recognition method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the low-resource vertical domain named entity recognition method as described in any one of claims 1-6.

Images