A multi-model collaborative entity recognition method based on hierarchical checksum and dynamic gating

Abstract

The application provides a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating, comprising: obtaining original text to be recognized and inputting the original text into a lightweight candidate generator to generate an entity candidate pool containing a plurality of candidate entities and confidence thereof; extracting features of the original text and the entity candidate pool and jointly inputting the features into a strategy controller to adaptively predict an optimal filtering threshold corresponding to the original text; filtering all candidate entities in the entity candidate pool according to the confidence and the optimal filtering threshold to obtain a plurality of high-value candidate entities; jointly inputting the high-value candidate entities and the original text into a hierarchical verifier for hierarchical decoding, utilizing a hierarchical attention mask mechanism to sequentially recognize corresponding entity categories from outside to inside to obtain a structured entity recognition result containing hierarchical relationships; and the application filters entities based on a dynamic gating mechanism and recognizes entities based on hierarchical verification, and can efficiently and accurately recognize entities of thousands of categories and nested entities.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating. Background Technology

[0002] In today's era of rapid information technology development, automatically and accurately extracting valuable key information from massive amounts of unstructured text data (such as financial research reports, legal documents, medical records, and industrial safety logs)—named entity recognition (NER)—has become a core foundational technology driving the deep application of artificial intelligence across various industries. However, modern industrial applications present NER technology with three unprecedented challenges, primarily manifested in: 1) Ultra-large scale categories (thousand-category scenarios): In professional fields such as financial risk control and intelligent manufacturing, the number of entity categories that need to be identified can reach thousands, which poses a severe test to the model's category differentiation and generalization capabilities.

[0003] 2) Complex entity structure: Entity nesting and discontinuous entities are common in texts, which requires the model to have a deep level of structured semantic understanding ability.

[0004] 3) Stringent real-time requirements: In scenarios such as real-time public opinion monitoring and online recommendation systems, models are required to have millisecond-level processing speeds. While existing high-precision large language models (LLMs) have excellent performance, their high inference latency and computational cost make it difficult to deploy them on a large scale and at low cost in the industry.

[0005] Therefore, there is an urgent need for an innovative entity recognition technology that can simultaneously address the three major challenges of large-scale categories, complex entity structures, and high real-time requirements, in order to overcome the bottlenecks of existing technologies. Currently, existing NER technologies have the following shortcomings: 1) The "Impossible Triangle" of Accuracy, Efficiency, and Scalability: High-precision models (such as large Transformers) are typically complex in structure and computationally intensive, resulting in slow inference speeds; while lightweight models, although fast, suffer a significant drop in accuracy when handling thousands of categories and complex nested entities. A key challenge is achieving the low latency and high throughput required for industrial-grade deployments while maintaining industry-leading accuracy.

[0006] 2) The challenge of disambiguation for nested and overlapping entities: When dealing with nested structures, existing models are prone to confusing the precise boundaries of inner and outer entities, or making incorrect choices among multiple overlapping candidate entities. Traditional sequence labeling methods (such as CRF) are mathematically difficult to handle non-parallel nested relationships, while brute-force decoding using a single model lacks effective structured constraints, which can easily lead to the accumulation and propagation of errors.

[0007] 3) Mismatch between static strategies and dynamic data: Traditional models typically use fixed processing procedures and static thresholds to handle all input text, failing to dynamically adjust their computational strategies based on the complexity of the text itself (such as entity density, sentence length, nesting depth, etc.). This leads to a waste of valuable computational resources when processing simple text, while sacrificing recall or precision when processing complex text due to its "one-size-fits-all" strategy. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies in effectively balancing the three key dimensions of inference efficiency, nested processing, and dynamic optimization, this invention provides a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating. By employing a collaborative intelligent division of labor between a lightweight candidate generator (small model) and a high-parameter hierarchical verifier (large model), and simultaneously introducing a policy controller (adaptive gating mechanism) for dynamic decision-making, this method efficiently and accurately solves the recognition challenges in scenarios with thousands of categories and nested entities.

[0009] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating includes the following steps: S1: Obtain the original text to be recognized and input it into the preset lightweight candidate generator to generate an entity candidate pool containing several candidate entities and their confidence levels. S2: Extract the features of the original text and the entity candidate pool, and input them into the preset policy controller to adaptively predict the optimal screening threshold corresponding to the original text; S3: Based on the confidence level and the optimal screening threshold, filter all candidate entities in the entity candidate pool to obtain several high-value candidate entities; S4: Input the high-value candidate entities and the original text into a preset hierarchical verifier for hierarchical decoding. Use the hierarchical attention masking mechanism to identify the corresponding entity categories from the outside to the inside, and obtain the structured entity recognition result containing hierarchical relationships.

[0010] Preferably, in step S1, the preset lightweight candidate generator is specifically a trained first model; In step S2, the preset policy controller is specifically the trained second model; In step S4, the preset hierarchical verifier is specifically the trained third model. The number of parameters in the first model is less than the number of parameters in the third model.

[0011] Preferably, the structure of the first model includes, in sequence, a hybrid text encoding layer, a feature concatenation layer, and a span scoring layer; The hybrid text encoding layer includes a Char-CNN module and a Subword-BERT module arranged in parallel. The Char-CNN module is used to scan the original text using a preset set of convolutional kernels to extract the morphological features of several candidate entities. The Subword-BERT module is used to segment the original text using the Subword algorithm and to process the segmentation results using multi-layer BERT to generate semantic features. The feature splicing layer is used to splice the morphological features and semantic features to generate hybrid features; In the span scoring layer, a preset fully connected network is used to calculate the span score of the hybrid feature. The span score is then processed by the LeakyReLU activation function and the Sigmoid function to generate the confidence score corresponding to each candidate entity.

[0012] Preferably, the second model is any one of the following: LightGBM model, HistGradientBoosting model, CatBoost model, XGBoost model, MART model, and NGBoost model.

[0013] Preferably, the structure of the third model includes, in sequence, a candidate entity embedding layer, a decoder, and a classifier; The decoder includes several Transformer decoding modules connected in sequence; each Transformer decoding module has the same structure, including the following connected in sequence: an improved multi-head attention layer, a first normalization layer, a feedforward network layer, and a second normalization layer; the input of the improved multi-head attention layer and the input of the first normalization layer form a residual summation connection; the input of the feedforward network layer and the input of the second normalization layer form a residual summation connection. The improved multi-head attention layer incorporates a hierarchical attention masking mechanism. The classifier comprises at least one linear layer and a Softmax function connected in sequence to predict the final entity category.

[0014] Preferably, in the improved multi-head attention layer, the hierarchical attention masking mechanism is represented as follows:

[0015] in, Two high-value candidate entities and Attention mask scores between levels; For high-value candidate entities The query matrix; For high-value candidate entities The key matrix; It is a value matrix; Indicates matrix transpose; for The dimension; Indicates two high-value candidate entities and The mask between; The The expression is:

[0016] in, Represents a set of parent entities of a single identified type; This represents the set of all identified entities from level 1 to level (k-1).

[0017] Preferably, the trained first model and third model are obtained according to the following steps: Obtain the first training set; the first training set includes several training texts and their corresponding entity lists; each entity in the entity list is marked with a hierarchical relationship to identify the parent entity of each entity; Initialize the first model, input the first training set into the initialized first model for iterative training, and use the binary cross-entropy loss function to supervise the training process until the first model converges, and obtain the trained first model. The third model is initialized by inputting the first training set into the initialized third model for iterative training, and the training process is supervised by the cross-entropy loss function until the third model converges, thus obtaining the trained third model.

[0018] Preferably, the trained second model is obtained according to the following steps: Obtain a second training set, which includes several training texts; The second training set is input into the trained first model to generate an entity candidate pool corresponding to the second training set. For each training text in the second training set, several different filtering thresholds are set. Each filtering threshold is traversed, and the entity candidate pool corresponding to each training text is filtered. The filtered entities are input into the trained third model. The scores corresponding to each filtering threshold are calculated using a preset utility function. The filtering threshold with the highest score is used as the real label of the current training text, and the labeled second training set is obtained. The second model is initialized, and the labeled second training set is input into the initialized second model for iterative training. The mean squared error loss function is used to supervise the training process until the second model converges, and the trained second model is obtained.

[0019] Preferably, the preset utility function is specifically:

[0020] in, The current filtering threshold Corresponding rating; The current filtering threshold The corresponding F1 score; The current filtering threshold The corresponding computational delay; and These are the first and second weight hyperparameters, respectively.

[0021] Preferably, in step S2, the features include at least: the text length of the original text, the number of punctuation marks, the total number of candidate entities, the mean, variance, and information entropy of the confidence scores of the candidate entities.

[0022] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention provides a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating. First, the original text to be recognized is acquired and input into a preset lightweight candidate generator to generate an entity candidate pool containing several candidate entities and their confidence levels. Next, features are extracted from the original text and the entity candidate pool, and input together into a preset policy controller to adaptively predict the optimal screening threshold corresponding to the original text. Then, all candidate entities in the entity candidate pool are screened based on the confidence level and the optimal screening threshold to obtain several high-value candidate entities. Finally, the high-value candidate entities and the original text are input together into a preset hierarchical verifier for hierarchical decoding. A hierarchical attention masking mechanism is used to sequentially identify the corresponding entity categories from the outside in, obtaining a structured entity recognition result containing hierarchical relationships. This invention has the following advantages: 1) A breakthrough solution to the core contradiction between accuracy and efficiency: Through an innovative multi-model collaborative architecture, over 99% of the initial screening computation is handled by lightweight small models, while the computationally expensive large models only need to process less than 1% of the high-value candidate entities selected by the policy controller. This allows the invention to maintain a recognition accuracy comparable to pure large model schemes (over 90%), while reducing end-to-end inference time to less than 1 / 5 of that of pure large model schemes, achieving a perfect combination of state-of-the-art (SOTA) accuracy and industrial-grade efficiency.

[0023] 2) It fundamentally overcomes the problem of identifying "nested entities": The invention's unique nested entity hierarchical attention masking mechanism introduces powerful structured prior knowledge into the model's decoding process; by dynamically shielding the interference of outer entities when decoding inner entities, it ensures that the model can clearly, unambiguously, and layer by layer resolve complex nested relationships, improving the F1 score of nested entity recognition by about 15% compared to the baseline model.

[0024] 3) This invention achieves a paradigm shift from "static strategy" to "dynamic intelligence": Based on offline optimization and online prediction, the strategy controller endows the overall model with unprecedented adaptive decision-making capabilities. This invention can perceive the complexity of the input text in real time and dynamically predict the optimal processing strategy, adaptively selecting high-value candidate entities. This achieves Pareto optimality of computational resources and recognition performance across various input types, avoiding waste of computing power. Attached Figure Description

[0025] Figure 1 This is a flowchart of a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating provided in Example 1.

[0026] Figure 2 This is a diagram illustrating the specific implementation process of a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating provided in Example 2.

[0027] Figure 3 This is a structural diagram of the lightweight candidate generator provided in Example 2.

[0028] Figure 4 This is a structural diagram of the hierarchical verifier provided in Example 2.

[0029] Figure 5 This is a schematic diagram of the structure of a single Transformer decoding module provided in Example 2.

[0030] Figure 6 This is a flowchart of the process of dynamically generating screening thresholds using a policy controller, as provided in Example 2.

[0031] Figure 7 This is the model training flowchart provided in Example 2. Detailed Implementation

[0032] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application. To better illustrate this embodiment, some parts in the accompanying drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions; It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0033] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0034] Example 1 like Figure 1 As shown, this embodiment provides a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating, including the following steps: S1: Obtain the original text to be recognized and input it into the preset lightweight candidate generator to generate an entity candidate pool containing several candidate entities and their confidence levels. S2: Extract the features of the original text and the entity candidate pool, and input them into the preset policy controller to adaptively predict the optimal screening threshold corresponding to the original text; S3: Based on the confidence level and the optimal screening threshold, filter all candidate entities in the entity candidate pool to obtain several high-value candidate entities; S4: Input the high-value candidate entities and the original text into a preset hierarchical verifier for hierarchical decoding. Use the hierarchical attention masking mechanism to identify the corresponding entity categories from the outside to the inside, and obtain the structured entity recognition result containing hierarchical relationships.

[0035] In the specific implementation process, the method of this embodiment mainly relies on a small model (lightweight candidate generator) for quickly generating candidate entities and their initial confidence, a large model (hierarchical validator) for fine-grained category verification of candidate entities, and a policy controller. The policy controller is a pre-trained prediction model that receives a set of text-level features describing the complexity of the input text and predicts an optimal screening threshold based on these features. This threshold is used to select a high-quality subset of candidate entities from the entity candidate pool generated by the small model and finally send it to the large model for verification, outputting the final entity classification result. In the process of entity recognition in large models, this method adopts a hierarchical attention masking mechanism. Specifically, when recognizing a potential inner entity, it first determines all its potential parent entities (i.e., the outer entities that have been identified and contain the inner entity). Then, in the attention calculation of the model, the text positions occupied by the parent entities are masked, so that the model's attention focuses on the interior and context of the entity to be recognized, thereby achieving unambiguous structured entity parsing from the outside to the inside. This method is based on a dynamic gating mechanism of a policy controller, which can dynamically predict and apply a processing hyperparameter (a confidence threshold for filtering candidate entities) based on the real-time features of the input text (e.g., the entropy of the confidence distribution of candidate entities generated by the small model, and the text length), so as to adaptively balance computational cost and recognition accuracy on input texts of different complexities.

[0036] This method, through multi-model collaboration, ultimately solves the recognition challenge in scenarios with thousands of categories and nested entities efficiently and accurately.

[0037] Example 2 like Figure 2 As shown, this embodiment provides a multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating, including the following steps: S1: Obtain the original text to be recognized and input it into the preset lightweight candidate generator to generate an entity candidate pool containing several candidate entities and their confidence levels. S2: Extract the features of the original text and the entity candidate pool, and input them into the preset policy controller to adaptively predict the optimal screening threshold corresponding to the original text; S3: Based on the confidence level and the optimal screening threshold, filter all candidate entities in the entity candidate pool to obtain several high-value candidate entities; S4: Input the high-value candidate entities and the original text into a preset hierarchical verifier for hierarchical decoding, and use the hierarchical attention masking mechanism to identify the corresponding entity categories from the outside to the inside in order to obtain the structured entity recognition result containing hierarchical relationships; In step S1, the preset lightweight candidate generator is specifically the trained first model; In step S2, the preset policy controller is specifically the trained second model; In step S4, the preset hierarchical verifier is specifically the trained third model. The number of parameters in the first model is less than the number of parameters in the third model; The structure of the first model includes, in sequence: a hybrid text encoding layer, a feature concatenation layer, and a span scoring layer; The hybrid text encoding layer includes a Char-CNN module and a Subword-BERT module arranged in parallel. The Char-CNN module is used to scan the original text using a preset set of convolutional kernels to extract the morphological features of several candidate entities. The Subword-BERT module is used to segment the original text using the Subword algorithm and to process the segmentation results using multi-layer BERT to generate semantic features. The feature splicing layer is used to splice the morphological features and semantic features to generate hybrid features; In the span scoring layer, a preset fully connected network is used to calculate the span score of the hybrid feature. The span score is then processed by the LeakyReLU activation function and the Sigmoid function to generate the confidence score corresponding to each candidate entity. The second model is any one of the following: LightGBM model, HistGradientBoosting model, CatBoost model, XGBoost model, MART model, and NGBoost model; The structure of the third model includes, in sequence, a candidate entity embedding layer, a decoder, and a classifier; The decoder includes several Transformer decoding modules connected in sequence; each Transformer decoding module has the same structure, including the following connected in sequence: an improved multi-head attention layer, a first normalization layer, a feedforward network layer, and a second normalization layer; the input of the improved multi-head attention layer and the input of the first normalization layer form a residual summation connection; the input of the feedforward network layer and the input of the second normalization layer form a residual summation connection. The improved multi-head attention layer incorporates a hierarchical attention masking mechanism. The classifier comprises at least one linear layer and a Softmax function connected in sequence to predict the final entity category; In the improved multi-head attention layer, the hierarchical attention masking mechanism is represented as follows:

[0038] in, Two high-value candidate entities and Attention mask scores between levels; For high-value candidate entities The query matrix; For high-value candidate entities The key matrix; It is a value matrix; Indicates matrix transpose; for The dimension; Indicates two high-value candidate entities and The mask between; The The expression is:

[0039] in, Represents a set of parent entities of a single identified type; This represents the set of all identified entities from level 1 to level (k-1). The trained first and third models are obtained according to the following steps: Obtain the first training set; the first training set includes several training texts and their corresponding entity lists; each entity in the entity list is marked with a hierarchical relationship to identify the parent entity of each entity; Initialize the first model, input the first training set into the initialized first model for iterative training, and use the binary cross-entropy loss function to supervise the training process until the first model converges, and obtain the trained first model. The third model is initialized by inputting the first training set into the initialized third model for iterative training, and the training process is supervised by the cross-entropy loss function until the third model converges, thereby obtaining the trained third model. The trained second model is obtained by following these steps: Obtain a second training set, which includes several training texts; The second training set is input into the trained first model to generate an entity candidate pool corresponding to the second training set. For each training text in the second training set, several different filtering thresholds are set. Each filtering threshold is traversed, and the entity candidate pool corresponding to each training text is filtered. The filtered entities are input into the trained third model. The scores corresponding to each filtering threshold are calculated using a preset utility function. The filtering threshold with the highest score is used as the real label of the current training text, and the labeled second training set is obtained. The second model is initialized, and the labeled second training set is input into the initialized second model for iterative training. The mean squared error loss function is used to supervise the training process until the second model converges, and the trained second model is obtained. The preset utility function is specifically as follows:

[0040] in, The current filtering threshold Corresponding rating; The current filtering threshold The corresponding F1 score; The current filtering threshold The corresponding computational delay; and These are the first and second weighted hyperparameters, respectively. In step S2, the features include at least: the text length of the original text, the number of punctuation marks, the total number of candidate entities, the mean, variance, and information entropy of the confidence scores of the candidate entities.

[0041] In the specific implementation process, first, a lightweight candidate generator (Dual-Encoder SpanExtractor) is constructed to quickly scan the full text within milliseconds, generate a candidate pool of all possible entity segments (Spans), and assign a preliminary confidence level to them. As Figure 3 shown, it is the structural diagram of the candidate generator, which includes a hybrid text encoding layer, a feature concatenation layer, and a span scoring layer connected in sequence; among them, the hybrid text encoding layer includes a Char-CNN module and a Subword-BERT module arranged in parallel; The Char-CNN module is used to scan the original text using a preset convolution kernel group to extract the morphological features of several candidate entities. In this embodiment, a three-dimensional convolution kernel group (3 / 4 / 5-gram) is used to scan the character sequence in parallel to capture morphological features; then, a max-pooling layer is used to retain the most distinguishable local patterns in each channel; The Subword-BERT module is used to tokenize the original text using the Subword algorithm and use 3 layers of BERT to process the tokenization results to generate semantic features (context-related vectors); for example, for the character "city" in the nested entity "Haidian District, Beijing", it can be associated with the context of "Beijing" and "Haidian" through self-attention; The feature concatenation layer is used to concatenate the morphological features and semantic features to generate hybrid features; in this embodiment, the feature concatenation layer concatenates the morphological features of CNN (768 dimensions) and the semantic features of BERT (768 dimensions) into a 1536-dimensional hybrid vector. By feature concatenation rather than simple addition, the independence of the two modes can be retained; In the span scoring layer, a preset fully connected network is used to calculate the span scores of the hybrid features, and the span scores are sequentially processed through the LeakyReLU activation function and the Sigmoid function to generate the confidence levels corresponding to each candidate entity; in this embodiment, a three-layer fully connected network (512-256-1) is used to calculate the span scores, and the LeakyReLU activation function is used to prevent gradient disappearance, and finally, the Sigmoid-normalized confidence level is output; in this embodiment, the span scoring layer enumerates all possible in the text and calculates its possibility as an "arbitrary entity" through a feed-forward network; the input features of the scoring function are:

[0042] Among them, is the hybrid vector representation of the span start and end positions (reflecting boundary information), is the position embedding of the span length, used to distinguish short entities (such as "city") from long entities (such as "Chinese Academy of Sciences"), This captures the core semantic signals within the span by extracting the highest activation value of the CNN within the span through a sliding window (capturing the core semantic signals). The lightweight candidate generator designed in this embodiment can achieve efficient parallel processing. The parallel architecture of CNN and BERT can achieve a speed 10 times faster than pure BERT. Secondly, it can perform fine-grained feature capture. The 3-5 gram convolutional kernel covers the length of common Chinese words, improving the ability to capture long entities. In addition, this design achieves complementary advantages through hybrid encoding. CNN ensures strong generalization ability for out-of-vocabulary words, while mini BERT maintains basic semantic understanding. Finally, it achieves a recall rate close to that of a large model with a small model capacity.

[0043] This embodiment also constructs a nested entity transformer, which acts as an "expert" for the entire system, performing refined and authoritative category judgment and nested structure parsing on a small number of high-quality candidates selected by the policy controller; like Figure 4 The diagram shows the structure of the hierarchical validator. This embodiment's hierarchical validator (large model) is based on a standard Transformer decoder architecture (such as GPT-style). Its core innovation lies in injecting a dynamically generated "hierarchical attention mask" into the standard self-attention layer. The candidate span embedding layer receives a small number of candidate entities selected by the policy controller. For each candidate span(i,j), it aggregates the embeddings of its internal tokens (such as start and end token embeddings and pooling representations within the span) and position embeddings to form an initial vector representation h_span. This is then input into the Transformer-based decoder, which is the core of the large model and consists of multiple stacked identical decoding modules. Each module contains an improved multi-head self-attention layer and a feedforward neural network. Finally, a classifier receives the output vector from the top layer of the Transformer decoder and, through one or more linear layers and a softmax function, predicts the final entity category and confidence for each input candidate span. like Figure 5 The diagram shows the structure of the improved multi-head self-attention layer. In the traditional Transformer self-attention mechanism, the updated representation of a token (or span in this embodiment) is the weighted sum of its representations with all other tokens. The weights (i.e., attention scores) are calculated by the dot product of the query vector and the key vector, using the following formula:

[0044] Where Q, K, and V are the query, key, and value matrices, respectively, and d K This is the dimension of the key vector; the softmax function converts scores into weights. In this embodiment, the traditional self-attention mechanism is replaced by a hierarchical attention mask (H-Mask) mechanism, as shown in the formula:

[0045] in, Two high-value candidate entities and Attention mask scores between levels; For high-value candidate entities The query matrix; For high-value candidate entities The key matrix; It is a value matrix; Indicates matrix transpose; for The dimension; Indicates two high-value candidate entities and The mask between; Assuming we are currently decoding the first... The entity in the layer, at this point, already possesses the first to... The set of all confirmed entities in the layer When the model needs to calculate two high-value candidate entities and When attention scores are between, The expression is:

[0046] in, Represents a set of parent entities of a single identified type; This represents the set of all identified entities from level 1 to level (k-1). when When this is the case, it indicates that attention is allowed; that is, if a certain previous mask exists. And both A and B belong to If A and B are not in the same mask entity, then interaction between them is allowed; otherwise, if A and B are not in the same mask entity, interaction is prohibited (the probability is 0 after softmax). The intuitive meaning of this rule is "people from different places should not inquire about each other". If two candidate entities do not belong to the same known outermost entity "family", then they should not have direct attention communication to prevent "misattribution". During the decoding process, the large model employs an outside-in hierarchical decoding strategy. Layer 1 decoding identifies the outermost entity among all high-value candidate entities. Layer k decoding (k>1) activates a hierarchical attention mask mechanism when identifying entities in layer k (i.e., inner entities contained within entities in layer k-1). This hierarchical attention mask is a dynamically generated attention mask. When calculating the class probability of candidate entities in layer k, the model's self-attention mechanism is forcibly masked from paying attention to text tokens completely covered by outer entities (layers 1 to k-1). This effectively "shiels" the interference of outer entities during computation, allowing the model to focus all computational resources on the effective text region of the current layer, thus fundamentally solving the disambiguation problem of nested entities. An example of the large model's decoding process is shown below: The input text is "Tsinghua University located in Haidian District, Beijing". The entities to be identified (Ground Truth) include: E_outer_1 = “Haidian District, Beijing” (address); E_inner_1 = “Beijing” (city) <-- nested inside E_outer_1; E_outer_2 = “Tsinghua University” (organizational structure); First, during decoding at layer 1 (k=1), the outermost entity is searched. The entire attention mask is fully open, allowing the model to freely focus on any word in the sentence. At this point, Mask_1 is: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] (0 represents no masking, visible to all); the large model (layered validator) analyzes all incoming candidate entities without any restrictions; the first layer decoder outputs the two outermost entities successfully identified by the model: "Beijing Haidian District" and "Tsinghua University", and then performs a state update. At this point, the set of identified entities becomes... = {"Haidian District, Beijing", "Tsinghua University"}; During the second layer (k=2) decoding, it is necessary to further search for nested entities within the already identified external entities. This step is the most critical, and its goal is to find nested entities within... The internal entities in the code are as follows: Candidate entity: Beijing Municipality; Text: Tsinghua University, located in Haidian District, Beijing; Attention mask: [1, 1, 1] 1, 1, 1, 1 {0, 0, 0, 0, 0} (1 represents attentionable, 0 represents blocked); At this point, the large model analyzes the candidate "Beijing" in a "purified" and "focused" context. Since the interference of "Tsinghua University" has been eliminated, the model can more purely use the strongly related context such as "located in" and "Haidian District" to easily and confidently determine that "Beijing" is a "city" entity. The final output result (layer 2) is that the model has successfully identified the inner entity "Beijing".

[0047] The large model designed in this embodiment enhances local dependency, forcing the model to rely only on the context information within the same masked segment when predicting the masked segment, rather than global information. This prevents information leakage and avoids different masked segments from "seeing" each other, thereby improving the model's ability to model local continuous structures.

[0048] In this embodiment, the number of parameters of the large model is greater than that of the small model. For example, the large model can be the qwen2.5-7b model, or a large model with a higher number of parameters can be configured according to its own hardware resources. There are no restrictions here.

[0049] Furthermore, this embodiment also designs a policy controller as an adaptive gating module, serving as the "intelligent scheduling hub" of the entire system. It connects the small model and the large model, and is responsible for making dynamic decisions to achieve optimal global performance. It is an independently trained, lightweight prediction model, such as gradient boosting trees, small multilayer perceptrons, HistGradientBoosting, CatBoost, XGBoost, MART (Multiple Additive Regression Trees), NGBoost (Natural Gradient Boosting), etc. This embodiment uses the LightGBM model as an example for illustration. The policy controller receives a set of feature vectors that describe the complexity of the current text, such as: text-level features: sentence length, number of punctuation marks, etc.; candidate pool statistical features: the total number of candidate spans generated by the small model, the mean and variance of candidate confidence, and information entropy (the higher the entropy, the greater the uncertainty and the more complex the text); and finally predicts an optimal screening threshold τ, which is "tailor-made" for the current input text and is used to select high-value candidates from the candidate pool to be sent to the large model. like Figure 6 The diagram shows the structure of the policy controller. The input layer receives real-time text features from the small model, including text length, confidence distribution entropy, and candidate entity nesting levels. The feature engineering layer processes the original features through standardization (Z-Score) and discretization, and handles text length and position sensitivity through an embedded position encoder. It also generates feature interaction terms (such as entropy value × nesting level) to enhance the model's representation ability. Then, the LightGBM framework is used to implement gradient boosting decision trees. Each subtree focuses on different feature combinations. For example, shallow trees handle numerical features (length / entropy), and deep trees handle combined features (nested correlation). Adaptive pruning is achieved through dynamic feature importance evaluation. Finally, the outputs of each decision tree are weighted and aggregated. The Sigmoid activation function is applied to constrain the threshold range, and Gaussian noise is added to enhance prediction robustness. In this embodiment, the policy controller outputs a threshold τ that is negatively correlated with text complexity. High-complexity text generates a low threshold (relaxed screening), while low-complexity text generates a high threshold (strict screening).

[0050] After constructing the small model, large model, and policy controller, they need to be trained separately. The overall training process is as follows: Figure 7 As shown, the training process is decoupled into three stages to ensure optimal performance of each module and a stable and efficient training process: a) The first stage is basic pre-training, where small and large models are trained separately; the small model performs binary classification, and the large model performs multi-class classification. b) The second stage is to generate strategy labels offline. Here, it is necessary to simulate different thresholds, calculate the utility function, and then generate the optimal τ label. c) The third stage is the training of the policy controller, with input features including text features and candidate statistical features, and output dynamic thresholds; Specifically, to enable the hierarchical validator (large model) to accurately learn complex nested structures, this embodiment designs a training data format containing explicit hierarchical relationships. The training data adopts JSON format, and each sample contains text content and a list of entities. Crucially, each entity object contains a `parent_entity_id` field to explicitly indicate its parent entity, thereby constructing a clear entity tree structure. This explicit hierarchical labeling is the cornerstone of the training hierarchy attention masking mechanism. Real data examples are as follows: { "request_id": "req-xyz-12345", "input_text": "CC Company, headquartered in State B of Country A, announced the acquisition of startup DDDDDDDD." "entities": [ { "entity_id": "T1", "text_span": "Country A, State B", "label": "Geographic Location", "start_char": 4, "end_char": 10, "confidence": 0.985, "parent_entity_id": null }, { "entity_id": "T2", "text_span": "Country A", "label": "Country", "start_char": 4, "end_char": 6, "confidence": 0.991, "parent_entity_id": "T1" }, { "entity_id": "T3", "text_span": "CC Company", "label": "Company", "start_char": 11, "end_char": 15, "confidence": 0.995, "parent_entity_id": null }, { "entity_id": "T4", "text_span": "DDDDDDDD", "label": "Company", "start_char": 22, "end_char": 30, "confidence": 0.978, "parent_entity_id": null } ] } Data Explanation: request_id: A unique identifier for each request, used for log tracking and troubleshooting; input_text: The original input text for easy comparison; entities: A list containing all identified entity objects; entity_id: A unique ID assigned to each entity within the current document; this is crucial for establishing hierarchical relationships. text_span: The text segment in the original text that corresponds to the entity; label: The entity category determined by the final validation of the large model (one of the thousands of categories); start_char, end_char: The precise character-level start and end positions of the entity in the original text; confidence: The confidence score of the large model for this identification result (including boundaries and categories), which can be used to set thresholds in downstream applications; parent_entity_id: The core nested relationship field. If the value is null, it indicates that this is the outermost entity (such as T1, T3, T4); if the value is the entity_id of another entity (such as the parent ID of T2 being "T1"), it explicitly indicates that this is an inner entity and specifies its parent entity.

[0051] In the first stage, a lightweight small model is trained into an efficient entity boundary detector. At this stage, the goal is not precise entity classification, but rather high recall, ensuring that all real-world entity segments can be identified by the model with a high initial confidence level, recognizing as many entities as possible. To train the small model, this embodiment converts the original labeled data in the above format into the following binary classification training samples: Positive Samples (Label: 1): All entity spans that appear in the labeled data; for example, input: text: "Headquarters located in Country A, State B...", span(4, 8) -> Target: 1; Negative Samples (Label: 0): Samples spans from text that do not constitute entities, especially "hard negative samples" that are easily misclassified; for example, input: text: "Headquarters located in Country A, State B...", span(0,2) ("Headquarters") -> Target output: 0; The loss function used for training the small model is the standard binary cross-entropy loss (BCE Loss):

[0052] Where y is the ground truth label, y=1 for positive samples and y=0 for negative samples; Predicted Probability is the floating-point number between 0 and 1 that the small model outputs for the input span, representing the probability that the span "is an entity".

[0053] The goal of large model training is to enable the hierarchical validator (large model) to quickly master the ability to accurately classify thousands of entity categories and to initially learn to handle nested structures. This stage is essentially a multi-class classification task. In this embodiment, a multi-class classification training sample is created for each known entity at this stage. For example, "Input: text: "Headquarters located in Country A, State B...", span(4, 8) ("Country A, State B") -> Target output (Target): Geographic location (index in 1001 categories); Input: text: "Headquarters located in Country A, State B...", span(0, 2) ("Headquarters") -> Target output (Target): [O] (Non-entity category index)"; The loss function used is the multi-class cross-entropy loss, which consists of two parts: First, the Softmax function is used to convert the model's original output (logits) into a probability distribution, and then the cross-entropy is used to calculate the difference between this distribution and the true label.

[0054] in The original score of category c in the Logits vector z; C is the total number of categories. In this example, this is thousands of entity categories plus one non-entity category, for example, C=1001. To predict probabilities, this is The probability value after Softmax normalization represents the confidence level of the model in predicting that the span belongs to class c. The sum of is 1; After obtaining the probability distribution p, calculate the loss using the cross-entropy formula:

[0055] in, This represents the loss value for a single sample. One-hot encoding of the true label; this is a vector of length C, where the c-th element of the vector is 1 and all other elements are 0 if the true class of the sample is c. is the probability that the model predicts the input to belong to class c, obtained from the Softmax function; log is the natural logarithm. To minimize the cross-entropy loss, the model must adjust its parameters (in this embodiment, the MoRA adapter is used to adjust the parameters) to make the probability of predicting the true class as close to 1 as possible.

[0056] In the second stage, policy labels are generated offline (constructing data), a crucial step in the entire training process, used to generate "teacher" data for the policy controller. This stage uses a deterministic offline optimization method to find a theoretically optimal screening threshold τ for each text sample in the training / development set, and uses it as the label (ground truth) for supervised learning. The specific process is as follows: using a development set, offline simulation optimization is performed for each text in it. For a given text, a series of different screening thresholds are tried. (For example, from 0.1 to 0.9, with a step size of 0.05), for each τ value, the entire "small model screening - large model validation" process is run once, and the final F1 score and computational latency are recorded. Then, the score for each τ is calculated based on a predefined utility function.

[0057] in, This represents the overall utility score at a given threshold τ; To identify the F1 score of the entity under τ; The end-to-end inference latency under this τ is mainly determined by the number of candidates fed into the large model; α and γ are weight hyperparameters used to balance the importance of accuracy and efficiency. For this text, select the value of τ that maximizes U(τ), and denote it as the theoretically optimal threshold τ. At the same time, pair the feature vector of this text with τ to form a training sample for the policy controller, namely (feature, label τ) data pair.

[0058] In the third stage, using the large number of (feature, label τ) data pairs generated in the second stage, the policy controller is trained through standard supervised learning methods. During both training processes, Mean Squared Error (MSE) is used as the loss function to measure the model's prediction accuracy τ. pred The difference between the model and the true label τ; after training, the policy controller model is saved for subsequent online inference.

[0059] For online inference, the original text to be recognized is first obtained, and then three pre-trained models are used for entity recognition. The specific process is as follows: 1) Rapid Candidate Generation: This step is achieved through a lightweight small model (candidate generator), aiming to quickly and comprehensively identify all text fragments that could potentially represent entities, pursuing high recall. The original text string is input into the lightweight small model, with an input example: "CC Company, headquartered in State B of Country A, announced the acquisition of startup DDDDDDDDD." The small model then scores all possible n-grams (n ranging from 1 to a preset maximum length) in the input text in parallel at extremely high speeds (typically in milliseconds), finally outputting a large pool of entity candidates. Each element is a tuple (span_text, start_char, end_char, confidence_score), as shown in the example below: [ ("Country A, State B", 4, 8, 0.85) ("CC Company", 9, 13, 0.91) ("Country A", 4, 6, 0.78), ("State B", 6, 8, 0.65) ("CC", 9, 11, 0.72) ("DDDDDDDD", 22, 30, 0.95) ("Startups", 18, 22, 0.55) ... (hundreds of other low-confidence candidates) ... ].

[0060] 2) Dynamic threshold prediction: This step extracts features (length, number of candidates, confidence entropy, etc.) from the text and the candidate pool, and feeds them into a pre-trained policy controller, which immediately predicts a screening threshold τ tailored to the text. For text feature extraction, the input is the original text and the candidate pool generated in the previous step. By running a predefined feature extraction function (script), this function can calculate features that reflect the current text complexity in real time, such as text length, etc. Size, The mean, variance, and information entropy of the confidence scores are calculated; the final output is a standardized feature vector. ; For dynamic threshold prediction, the input is the feature vector generated in the previous step. It makes predictions by calling the deployed policy controller (adaptive gating mechanism), such as the loaded LightGBM model, and finally outputs a floating-point number. This refers to the optimal filtering threshold predicted by the controller for the current text. For example, it might predict 0.85 for simple text and 0.55 for complex text.

[0061] 3) High-Value Candidate Selection: This step is achieved through a simple filtering mechanism to significantly reduce the number of candidates that need to be fed into a large model for validation. This is a key breakthrough in achieving high-performance recognition. The complete candidate pool and dynamic threshold obtained in the previous step are then... Input filter, iterate through the candidate pool once, and only retain those with confidence scores. > The system selects candidate entities and outputs a new candidate list that is very small in number but extremely valuable; an example is shown below: Sample input: The complete candidate pool, = 0.8; Sample output: [ ("Country A, State B", 4, 8, 0.85) ("CC Company", 9, 13, 0.91) ("DDDDDDDD", 22, 30, 0.95) ] It is worth noting that after filtering, countries such as A (0.78), B (0.65), and CC (0.72) were all filtered out.

[0062] 4) Authoritative Hierarchical Verification: This step uses a powerful large model (hierarchical validator) to accurately classify the selected high-value candidates into thousands of categories and correctly resolve the nesting relationships between them. First, the selected candidate list and the original text are input into the hierarchical validator. The large model starts its hierarchical decoding process and activates the "hierarchical attention masking mechanism". Specifically, the first round of decoding first identifies the outermost entity. The large model first classifies all candidates ["Country A, State B", "Company CC", "DDDDDDDD"] in the context of the complete text, assuming that it identifies them as valid outermost entities. Next, a second round of decoding is prepared. If the candidate "Country A, State B" is found to contain inner entities, a hierarchical attention masking mechanism is applied. When searching for entities within "Country A, State B", the attention mechanism is modified so that when the model calculates the representation of the inner entity (such as "Country A"), its attention score is highly focused on the token of "Country A, State B", ignoring interference from other external entities such as "Company CC". The second round of decoding is used to accurately identify the inner entities. In the "focused" context, the large model classifies "Country A" and successfully identifies it as a "country". Finally, a series of finally confirmed entities and their categories and hierarchical relationships are output. Sample output: {"Country A, State B": "Geographical Location"}, {"Company CC": "Company"}, {"DDDDDDDD": "Company"}, {"Country A": "Country", "parent": "Country A, State B"}.

[0063] 5) Result Aggregation and Output: This step is an optional post-processing step used to integrate the scattered verification results into a structured, user-friendly final output. It constructs an entity tree based on the entity list and their relationships confirmed in the previous step, and assembles all information (text, boundaries, category, ID, parent ID) into a final JSON object, outputting the final entity recognition result containing complete information. A sample output (JSON format) is shown below: { "text": "...", "entities": [ { "entity_id": "E1", "text_span": "Country A, State B", "label": "Geographical Location", "start_char": 4, "end_char": 8, "parent_entity_id": null}, { "entity_id": "E3", "text_span": "CC Company", "label": "Company", "start_char": 9, "end_char": 13, "parent_entity_id": null}, { "entity_id": "E4", "text_span": "DDDDDDDD ", "label": "company", "start_char": 22, "end_char": 30, "parent_entity_id": null}, { "entity_id": "E2", "text_span": "Country A", "label": "Country", "start_char": 4, "end_char": 6, "parent_entity_id": "E1"} ] } This method achieves efficient and accurate identification of thousands of categories and nested entities by employing a collaborative intelligent division of labor between a lightweight candidate generator (small model) and a hierarchical validator with large parameters (large model), while introducing a policy controller (adaptive gating mechanism) for dynamic decision-making.

[0064] The same or similar labels correspond to the same or similar parts; The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this application. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating, characterized in that, Includes the following steps: S1: Obtain the original text to be recognized and input it into the preset lightweight candidate generator to generate an entity candidate pool containing several candidate entities and their confidence levels. S2: Extract the features of the original text and the entity candidate pool, and input them into the preset policy controller to adaptively predict the optimal screening threshold corresponding to the original text; S3: Based on the confidence level and the optimal screening threshold, filter all candidate entities in the entity candidate pool to obtain several high-value candidate entities; S4: Input the high-value candidate entities and the original text into a preset hierarchical verifier for hierarchical decoding, and use the hierarchical attention masking mechanism to identify the corresponding entity categories from the outside to the inside in order to obtain the structured entity recognition result containing hierarchical relationships; In step S1, the preset lightweight candidate generator is specifically the trained first model; In step S2, the preset policy controller is specifically the trained second model; In step S4, the preset hierarchical verifier is specifically the trained third model. The number of parameters in the first model is less than the number of parameters in the third model; The trained first and third models are obtained according to the following steps: Obtain the first training set; the first training set includes several training texts and their corresponding entity lists; each entity in the entity list is marked with a hierarchical relationship to identify the parent entity of each entity; Initialize the first model, input the first training set into the initialized first model for iterative training, and use the binary cross-entropy loss function to supervise the training process until the first model converges, and obtain the trained first model. The third model is initialized by inputting the first training set into the initialized third model for iterative training, and the training process is supervised by the cross-entropy loss function until the third model converges, thereby obtaining the trained third model. The trained second model is obtained by following these steps: Obtain a second training set, which includes several training texts; The second training set is input into the trained first model to generate an entity candidate pool corresponding to the second training set. For each training text in the second training set, several different filtering thresholds are set. Each filtering threshold is traversed, and the entity candidate pool corresponding to each training text is filtered. The filtered entities are input into the trained third model. The scores corresponding to each filtering threshold are calculated using a preset utility function. The filtering threshold with the highest score is used as the real label of the current training text, and the labeled second training set is obtained. The second model is initialized, and the labeled second training set is input into the initialized second model for iterative training. The mean squared error loss function is used to supervise the training process until the second model converges, and the trained second model is obtained.

2. The multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating according to claim 1, characterized in that, The structure of the first model includes, in sequence: a hybrid text encoding layer, a feature concatenation layer, and a span scoring layer; The hybrid text encoding layer includes a Char-CNN module and a Subword-BERT module arranged in parallel. The Char-CNN module is used to scan the original text using a preset set of convolutional kernels to extract the morphological features of several candidate entities. The Subword-BERT module is used to segment the original text using the Subword algorithm and to process the segmentation results using multi-layer BERT to generate semantic features. The feature splicing layer is used to splice the morphological features and semantic features to generate hybrid features; In the span scoring layer, a preset fully connected network is used to calculate the span score of the hybrid feature. The span score is then processed by the LeakyReLU activation function and the Sigmoid function to generate the confidence score corresponding to each candidate entity.

3. The multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating according to claim 1, characterized in that, The second model is any one of the following: LightGBM model, HistGradientBoosting model, CatBoost model, XGBoost model, MART model, and NGBoost model.

4. The multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating according to claim 1, characterized in that, The structure of the third model includes, in sequence, a candidate entity embedding layer, a decoder, and a classifier; The decoder includes several Transformer decoding modules connected in sequence; each Transformer decoding module has the same structure, including the following connected in sequence: an improved multi-head attention layer, a first normalization layer, a feedforward network layer, and a second normalization layer; the input of the improved multi-head attention layer and the input of the first normalization layer form a residual summation connection; the input of the feedforward network layer and the input of the second normalization layer form a residual summation connection. The improved multi-head attention layer incorporates a hierarchical attention masking mechanism. The classifier comprises at least one linear layer and a Softmax function connected in sequence to predict the final entity category.

5. The multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating according to claim 4, characterized in that, In the improved multi-head attention layer, the hierarchical attention masking mechanism is represented as follows: in, Two high-value candidate entities and Attention mask scores between levels; For high-value candidate entities The query matrix; For high-value candidate entities The key matrix; It is a value matrix; Indicates matrix transpose; for The dimension; Indicates two high-value candidate entities and The mask between; The The expression is: in, Represents a set of parent entities of a single identified type; This represents the set of all identified entities from level 1 to level (k-1).

6. The multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating according to claim 1, characterized in that, The preset utility function is specifically as follows: in, The current filtering threshold Corresponding rating; The current filtering threshold The corresponding F1 score; The current filtering threshold The corresponding computational delay; and These are the first and second weighted hyperparameters, respectively.

7. A multi-model collaborative entity recognition method based on hierarchical verification and dynamic gating according to any one of claims 1 to 6, characterized in that, In step S2, the features include at least: the text length of the original text, the number of punctuation marks, the total number of candidate entities, the mean, variance, and information entropy of the confidence scores of the candidate entities.

Images