An entity recognition method, device, apparatus and storage medium
Patent Information
- Application Number
- CN202611216962.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-12
- Publication Date
- 2026-10-30
AI Technical Summary
然而,这种方法往往面临“竞争稀释”与旧类特征目标漂移的问题
[0013]本申请实施例提供的计算机可读存储介质,其上存储有计算机程序,该计算机程序被处理器执行时实现本申请实施例提供的所述的方法。
Smart Images

Figure CN123013923A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of natural language processing technology, and in particular to an entity recognition method, apparatus, device and storage medium. Background Technology
[0002] With the continuous development of information technology and the explosive growth of text data, many intelligent application scenarios require the rapid and accurate extraction of entity information from massive amounts of text data. Named entity recognition, as a key foundational technology in the field of natural language processing, aims to identify entities with specific meanings in text. However, in practical applications, entity categories often increase dynamically over time, which requires models to be able to perform incremental category learning, that is, to continuously learn the ability to recognize new entities while only encountering new category data.
[0003] Current methods typically perform incremental recognition by expanding the class head weight matrix with each new category. However, this approach often faces problems such as "competitive dilution" and feature drift of older classes. As new categories are added, the normalized denominator increases, leading to a systematic suppression of the predicted probabilities of older classes. Simultaneously, older classes lack sufficient supervision during incremental training, and their features are prone to geometric drift under the influence of new class data, resulting in forgetting and recognition errors, thus reducing the accuracy of entity recognition.
[0004] Therefore, how to effectively alleviate catastrophic forgetting and competitive dilution in the incremental learning process and improve the accuracy of entity recognition is an urgent problem to be solved. Summary of the Invention
[0005] In view of this, the entity recognition method, apparatus, device, and storage medium provided in this application can effectively alleviate catastrophic forgetting and competitive dilution during incremental learning, and improve the accuracy of entity recognition. The entity recognition method, apparatus, device, and storage medium provided in this application are implemented as follows: This application provides an entity recognition method, including: Geometric construction and parameter locking are performed on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule, and freezing control parameters to obtain a preset isoangular prototype matrix. The preset isoangular prototype matrix includes multiple fixed prototype vectors with equal angles between each pair of vectors. Deep semantic feature extraction, residual bottleneck projection mapping and modulus normalization are performed on the text sequence to be processed to obtain the corresponding text word unit feature vector. The identifiers of all entity categories that have been opened in the current incremental task phase are truncated by category index to obtain the current active prototype matrix composed of the fixed prototype vectors corresponding to the currently opened categories. The text word unit feature vector, each fixed prototype vector in the current activated prototype matrix, and the preset temperature scaling factor are subjected to vector inner product and numerical amplification processing to obtain the classification score of each open entity category. Based on the constructed normalized transition probability matrix, the classification score is processed by Viterbi dynamic programming search to obtain the target entity recognition result.
[0006] In some embodiments, the geometric construction and parameter locking process performed on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule, and freeze control parameters to obtain a preset isoangular prototype matrix includes: Gaussian random sampling and orthogonal decomposition are performed on the preset maximum number of categories and feature dimensions to obtain the orthogonal basis matrix; The orthogonal basis matrix is subjected to row-wise mean calculation and row-wise mean subtraction to obtain a zero-mean centered matrix; The centered matrix is normalized by modulus and scaled by ratio to obtain a preset isoangular prototype matrix in which the cosine similarity between any two vectors is a preset negative constant. The gradient update state of the preset isoangular prototype matrix is subjected to backpropagation truncation and parameter locking to obtain a preset isoangular prototype matrix that remains unchanged during both model training and inference phases.
[0007] In some embodiments, the process of performing deep semantic feature extraction, residual bottleneck projection mapping, and modulus normalization on the text sequence to be processed to obtain the corresponding text lexical unit feature vector includes: The text sequence to be processed is encoded by a pre-trained language model to obtain deep semantic feature vectors; The deep semantic feature vector is subjected to linear dimensionality reduction, nonlinear activation, linear dimensionality increase and residual connection fusion to obtain the projection mapping feature vector; The projection mapping feature vector is normalized to its L2 norm to obtain the corresponding text word unit feature vector.
[0008] In some embodiments, the step of performing class-indexed truncation processing on all entity category identifiers already opened in the current incremental task phase and the preset isoangular prototype matrix to obtain a currently active prototype matrix composed of fixed prototype vectors corresponding to the currently opened categories includes: The set of open category indexes is obtained by summarizing all historical old entity categories and current new entity categories included in the current incremental task phase. Based on the set of open category indexes, the corresponding column vectors in the preset isogonal prototype matrix are subjected to mask filtering and slice extraction to obtain submatrices arranged in category order; The submatrix is then subjected to activation state marking and multi-stage isolation processing to obtain the current activation prototype matrix, which is composed of the fixed prototype vectors corresponding to the currently open categories.
[0009] In some embodiments, the step of performing vector dot product and numerical amplification processing on the text lexical unit feature vector, each fixed prototype vector in the current activated prototype matrix, and a preset temperature scaling factor to obtain the classification score for each open entity category includes: The cosine similarity matrix is obtained by performing matrix multiplication and dot product calculations on the text word unit feature vector and each fixed prototype vector in the current activated prototype matrix. The cosine similarity numerical matrix is multiplied and scaled by a preset temperature scaling factor to obtain the original scoring matrix with unnormalized probability. The numerical values corresponding to each word position in the original scoring matrix are mapped and associated with each open entity category to obtain the classification score of each open entity category.
[0010] In some embodiments, the Viterbi dynamic programming search process performed on the classification score based on the constructed canonical transition probability matrix to obtain the target entity recognition result includes: The label transition legality rule is mapped to a binary mask and a numerical penalty matrix to obtain the normalized transition probability matrix. The classification score is used as the emission probability value of each word position, and Viterbi dynamic programming is used to solve the state transition process with the normalized transition probability matrix to obtain the optimal label path sequence. The optimal label path sequence is subjected to entity boundary parsing and category mapping to obtain the target entity recognition result.
[0011] This application provides an entity recognition device, comprising: The processing module is used to perform geometric construction and parameter locking processing on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule and freeze control parameters to obtain a preset isoangular prototype matrix, which includes multiple fixed prototype vectors with equal included angles in pairs. The processing module is also used to perform deep semantic feature extraction, residual bottleneck projection mapping and modulus normalization on the text sequence to be processed, so as to obtain the corresponding text word unit feature vector. The processing module is also used to perform class index truncation processing on all entity category identifiers that have been opened in the current incremental task stage and the preset isogonal prototype matrix to obtain the current active prototype matrix composed of fixed prototype vectors corresponding to the currently opened categories. The acquisition module is used to perform vector inner product and numerical amplification processing on the text word unit feature vector, each fixed prototype vector in the current activated prototype matrix and the preset temperature scaling factor to obtain the classification score of each open entity category. The module is used to perform Viterbi dynamic programming search on the classification score based on the constructed normalized transition probability matrix to obtain the target entity recognition result.
[0012] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.
[0013] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method described in this application embodiment.
[0014] The entity recognition method, apparatus, device, and storage medium provided in this application embodiment obtain a preset isogonal prototype matrix by performing geometric construction and parameter locking processing on a pre-generated orthogonal basis matrix, centering rules, modulus scaling rules, and freezing control parameters; perform deep semantic feature extraction, residual bottleneck projection mapping, and modulus normalization processing on the text sequence to be processed to obtain text word unit feature vectors; index and truncate the entity category identifiers that have been opened in the current incremental stage and the preset isogonal prototype matrix to obtain the current activation prototype matrix; perform vector inner product and numerical amplification processing on the text word unit feature vectors, the current activation prototype matrix, and the temperature scaling factor to obtain a classification score; and perform Viterbi dynamic programming search on the classification score based on the normalized transition probability matrix to obtain the target entity recognition result. This effectively alleviates catastrophic forgetting and competitive dilution in the incremental learning process, improves the accuracy of entity recognition, and solves the technical problems proposed in the background art. Attached Figure Description
[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 A schematic diagram illustrating the implementation process of an entity recognition method provided in this application embodiment; Figure 2 A schematic diagram illustrating the implementation process of obtaining a preset isogonal prototype matrix provided in an embodiment of this application; Figure 3This is a schematic diagram of the structure of an entity recognition device provided in an embodiment of this application. Detailed Implementation
[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0018] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.
[0019] Figure 1 This is a schematic diagram illustrating the implementation flow of an entity recognition method provided in an embodiment of this application, including steps 101 to 105. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for an entity recognition method. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.
[0020] Step 101: Perform geometric construction and parameter locking on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule, and freeze control parameters to obtain the preset isogonal prototype matrix.
[0021] In this embodiment, before all incremental training tasks are officially launched, the geometric construction and parameter locking of the isoangular prototype matrix are completed offline to generate a preset isoangular prototype matrix. The maximum number of prototypes, the prototype feature dimension, and the freeze control parameters are pre-configured to constrain the prototype feature dimension value to be greater than or equal to the maximum number of prototypes, ensuring that the subsequent orthogonal decomposition can generate a sufficient number of compliant orthogonal vectors. The original numerical matrix with the corresponding row and column size is randomly generated, and the QR decomposition operation is performed on the original matrix to extract the matrix in which the columns are pairwise orthogonal and the magnitude of each column vector is 1, which is used as the pre-generated orthogonal basis matrix.
[0022] Construct identity matrices and all-1 column vectors of the corresponding order according to the preset centering rules, derive the centering projection transformation matrix, multiply the orthogonal basis matrix with the projection matrix, perform a global mean-removing centering transformation on all orthogonal vectors, widen the angles between vectors, and form the basic shape of the isoangular configuration; then, according to the modulus scaling rules, use a fixed scaling factor to perform a scaling operation on the matrix after centering, compensate for the vector modulus attenuation caused by the centering operation, and restore all vectors in the matrix to unit length.
[0023] After the entire geometric transformation operation is completed, the generated matrix is marked as a non-trainable fixed tensor based on the freeze control parameters. In all subsequent incremental training and inference processes, gradient backpropagation is blocked, and the values within the matrix do not undergo any parameter updates throughout the process. Finally, a preset isoangular prototype matrix is obtained. Within this matrix, the angles between each pair of prototype vectors are completely equal, and the cosine similarity between any two prototype vectors is always a fixed negative value. The whole matrix forms an isoangular tight-frame geometric configuration, as shown in the following equation:
[0024] in, The number of vectors, For a matrix, For the input matrix, This is a transpose.
[0025] Step 102 involves performing deep semantic feature extraction, residual bottleneck projection mapping, and modulus normalization on the text sequence to be processed to obtain the corresponding text word unit feature vector.
[0026] In this embodiment, a text sequence to be processed is received, and deep semantic feature extraction, residual bottleneck projection mapping, and modulus normalization are performed sequentially to obtain text lexical unit feature vectors. A BERT pre-trained encoder is selected as the deep semantic feature extraction module. WordPiece word segmentation is performed on the text to be processed, and a first-word alignment strategy is adopted. Only the encoder hidden state corresponding to the first word of each original word is extracted as the original lexical feature, and the feature vectors corresponding to the remaining redundant words are discarded, thus obtaining the original semantic vector whose dimension matches the dimension of the hidden layer of the BERT model. The original semantic vectors are fed into the residual bottleneck projection module for projection mapping. This module consists of two parts: a direct shortcut branch and a nonlinear bottleneck backbone branch. The shortcut branch directly maps the original semantic vectors to the feature dimension corresponding to the prototype through a single-layer linear transformation combined with the LeakyReLU activation function. The backbone branch first compresses the features to an intermediate low-dimensional space, then performs two layers of linear transformation, layer normalization, and GELU nonlinear activation before mapping them back to the prototype dimension. The feature vectors output by the two branches are added element-wise to complete the dimensional alignment from the high-dimensional semantic space to the low-dimensional geometric space of the prototype. The aligned feature vectors undergo L2 modulus normalization, dividing the vector by its own L2 norm while introducing a minimal constant to avoid division by zero errors. Finally, a unit-length text word unit feature vector is generated, making the inner product of the vector and the prototype vector equivalent to their cosine similarity.
[0027] Step 103: Perform class-indexed truncation on all entity category identifiers and preset isogonal prototype matrices that have been opened in the current incremental task phase to obtain the current active prototype matrix composed of fixed prototype vectors corresponding to the currently opened categories.
[0028] In this embodiment, based on the entity category identifiers already available in the current incremental task stage, a class-indexed truncation process is performed on the preset isometric prototype matrix to generate the currently activated prototype matrix. This application fully adheres to the ExtendNER label mask incremental protocol. In each incremental stage, only some entity-related BIO labels are available; unavailable labels are uniformly masked as non-entity identifiers in the training data. The number of categories corresponding to all available labels in the current incremental stage is read, and column vectors corresponding to the preceding number are truncated according to the preset label index order, combining them to form the currently activated prototype matrix. The truncation operation only reads the matrix column indices and does not modify the values of any prototype vectors within the preset isometric prototype matrix. Prototype vectors already activated in previous incremental stages retain their original values throughout the process; only prototype vectors corresponding to newly available entity labels are added to participate in subsequent calculations, completing the logical processing of prototype class-by-class activation.
[0029] Step 104: Perform vector inner product and numerical amplification on the text word unit feature vector, each fixed prototype vector in the current activated prototype matrix, and the preset temperature scaling factor to obtain the classification score of each open entity category.
[0030] In this embodiment, the text lexical unit feature vector, the current activated prototype matrix, and a preset temperature scaling factor are used to perform vector dot product and numerical amplification processing to obtain the classification score for each open entity category. The dot product of a single lexical unit feature vector and each column of the fixed prototype vector in the current activated prototype matrix is calculated to obtain the basic cosine similarity between the feature and each type of prototype. A temperature scaling factor greater than 1 is introduced to perform numerical amplification. This operation is equivalent to sharpening the similarity distribution using a temperature parameter less than 1, multiplying the basic similarity by the scaling factor to widen the numerical difference between different categories of similarity. This prevents the difference in probability values between categories after normalization from being too small, resulting in weak gradient forces that cannot drive features to align with their corresponding prototypes. The amplified values are the classification scores for the corresponding open entity categories. All category scores are calculated independently; adding a new entity category only increases the scoring dimension and does not change the historical scores of older categories.
[0031] Step 105: Based on the constructed normalized transition probability matrix, perform Viterbi dynamic programming search on the classification score to obtain the target entity recognition result.
[0032] In this embodiment, a BIO tag standardized transition probability matrix is pre-constructed. Global path search processing is performed on the classification score using Viterbi dynamic programming, outputting the target entity recognition result. A fixed transition probability matrix is constructed according to the legal transition rules of BIO tags. It stipulates that non-entity tags can transition to themselves or any entity starting tag; entity starting tags can transition to non-entity tags, tags within the same type of entity, or any other entity starting tag; entity internal tags can only transition to non-entity tags, tags within the same type of entity, or any other entity starting tag; and the probability of all other transition paths that do not conform to the rules is uniformly set to zero. The logarithm of the transition matrix is taken to generate a logarithmic transition cost tensor, which is registered as a fixed parameter that does not participate in gradient updates. The classification score corresponding to each word position is used as the emission score. A Viterbi dynamic programming forward recursion operation is performed, accumulating the total score of each legal label path position by position. Illegal transition paths are automatically eliminated during the recursion process due to the negative infinite logarithmic cost. After the forward calculation is completed for all word positions in the text, the label corresponding to the largest accumulated score at the end of the sequence is selected as the sequence endpoint. The optimal predecessor label for each position is then backtracked to obtain the complete BIO label sequence that is legal and has the best score globally. Based on this label sequence, continuous entity fragments are extracted, entity boundaries and entity categories are distinguished, and the target entity recognition result is finally output. This step is only performed during the model inference and prediction stage; the transition matrix is not introduced into the loss calculation during the model training stage, and model parameter updates and optimizations are completed solely based on the classification score.
[0033] This application addresses two core shortcomings of incremental named entity recognition (NER) at the feature geometry level. First, it eliminates the competitive dilution problem caused by traditional scalable classification heads, ensuring that adding new entity categories does not alter the original classification scores of older categories, thus preventing the systematic suppression of old category prediction probabilities. Second, it applies global geometric constraints to the feature space by fixing isometric prototypes, constraining the distribution of new and old category features within isometric tight-frame configurations, suppressing old category feature drift during incremental training, and mitigating catastrophic forgetting. The entire process is adapted to the ExtendNER incremental NER label masking protocol, providing a complete set of adaptation logic for tasks with a high proportion of BIO and O labels, including projection normalization, prototype-based activation, and Viterbi compliant decoding. Unlike the common ETF incremental solution in computer vision, this approach is naturally suited to text entity recognition scenarios. The inference phase only adds vector inner product and Viterbi path search; prototype construction is completed offline in a single step, resulting in minimal additional computational overhead during training and inference. It eliminates the need to store massive amounts of historical text data, reducing the computational, storage, and data compliance costs of incremental iterations.
[0034] In the above Figure 1 Based on the above, this application embodiment also provides a schematic diagram of the implementation process for obtaining a preset isogonal prototype matrix. For example... Figure 2As shown, steps 201 to 204 are included: Step 201: Perform Gaussian random sampling and orthogonal decomposition on the preset maximum number of categories and feature dimensions to obtain the orthogonal basis matrix.
[0035] In this embodiment, an orthogonal basis matrix generation operation is first performed. An original random matrix is generated based on a pre-defined maximum number of categories and prototype feature dimension. Gaussian random sampling is used to fill all values within the matrix. The number of rows in this matrix corresponds to the prototype feature dimension, and the number of columns corresponds to the maximum number of categories. Simultaneously, it is strictly guaranteed that the value of the prototype feature dimension is greater than or equal to the maximum number of categories, satisfying the preconditions for orthogonal decomposition. A QR orthogonal decomposition operation is then performed on the generated Gaussian random matrix. The column orthogonal matrix in the decomposition result is extracted as the orthogonal basis matrix. Each column vector in this orthogonal basis matrix is of unit length, and any two different column vectors are orthogonal to each other. The vector dot product is zero, as shown in the following equation:
[0036] Step 202: Perform row-wise mean calculation and row-wise mean subtraction on the orthogonal basis matrix to obtain a centered matrix with zero mean.
[0037] In this embodiment, the orthogonal basis matrix is then centered. The orthogonal basis matrix is subjected to row-wise mean calculation and row-wise element-wise mean subtraction according to the preset centering rules. This is equivalent to performing matrix multiplication on the orthogonal basis matrix using the centered projection matrix obtained by combining the identity matrix and the all-one matrix. This completes the global mean removal process for all prototype vectors, eliminates the overall vector offset, widens the angle between different vectors, and after the operation, a centered matrix with a mean of zero for all vectors is obtained.
[0038] Step 203: Perform modulus normalization and scaling on the centered matrix to obtain a preset isoangular prototype matrix in which the cosine similarity between any two vectors is a preset negative constant.
[0039] In this embodiment, the centralized matrix is then subjected to modulus normalization and scaling. First, the L2 norm of each column vector in the centralized matrix is calculated, and the norm value is used to normalize the corresponding column vector to eliminate the vector length change caused by the centralization operation. Then, the normalized matrix is scaled as a whole using a fixed scaling factor. The scaling factor is the square root of the ratio of the maximum number of categories to the maximum number of categories minus one. After the scaling operation, all column vectors in the matrix are restored to unit length, and the inner product between any two different column vectors is fixed as a negative constant. The value of this negative constant is equal to negative one divided by the maximum number of categories minus one, thereby forming a preset isoangular prototype matrix that satisfies the isoangular tight frame standard and the included angles between any two prototype vectors are completely equal.
[0040] Step 204: Perform backpropagation truncation and parameter locking on the gradient update state of the preset isoangular prototype matrix to obtain the preset isoangular prototype matrix that remains unchanged during both model training and inference phases.
[0041] In this embodiment, gradient truncation and parameter locking are performed on the generated preset isoangular prototype matrix. Based on the backpropagation logic of the frozen control parameter configuration, the gradient data stream pointing to the preset isoangular prototype matrix is truncated in the model gradient backpropagation link, blocking the gradient from updating the values in the matrix. The matrix is registered as a non-trainable fixed tensor, and parameter update permissions are not enabled in all subsequent incremental training stages and online inference prediction stages. The values of all prototype vectors inside the matrix remain in their original state after construction and will not change with incremental training of new classes or model fine-tuning. Finally, a preset isoangular prototype matrix that remains unchanged throughout the process is obtained.
[0042] This application's embodiments utilize Gaussian random sampling and QR orthogonal decomposition to generate orthogonal bases. Combined with centering, normalization, scaling, and closed-form transformations, this stably generates prototype matrices that satisfy the mathematical properties of equiangular tight boxes. The cosine similarity between any two prototype classes is uniformly set to a fixed negative value, maximizing the theoretical inter-class margin in the feature space. The geometric constraint effect is stable and reproducible. By centering and removing the mean to widen the vector angle, the deficiency of insufficient inter-class margin in simple orthogonal basis vectors is addressed. This ensures that prototype vectors corresponding to different entity labels are evenly distributed on a unit sphere, significantly improving the distinguishability of new and old entity category features. Gradient backpropagation truncation and parameter permanent locking mechanisms guarantee that the prototype vector values remain completely unchanged throughout all incremental stages. Old class prototypes are not disturbed by the training gradients of new categories, eliminating the forgetting of old classes caused by the weight drift of traditional trainable classifier heads. The decision boundary of old classes remains stable throughout the entire process.
[0043] In some embodiments, deep semantic feature extraction, residual bottleneck projection mapping, and modulus normalization are performed on the text sequence to be processed to obtain the corresponding text lexical unit feature vector, including: encoding the text sequence to be processed using a pre-trained language model to obtain the deep semantic feature vector.
[0044] Specifically, the text sequence to be processed is encoded using a pre-trained language model, outputting a deep semantic feature vector. A BERT-base-uncased pre-trained language model is selected as the encoder. First, WordPiece word segmentation is performed on the input text sequence. To address the mismatch between word segmentation and the granularity of the original vocabulary, a first-word alignment strategy is adopted. Only the model's hidden state corresponding to the first word of each original word is extracted, discarding features corresponding to other words of the same word. Invalid features from sequence-filling placeholders are filtered out. Finally, a deep semantic feature vector with dimensions matching the hidden layer dimensions of the BERT model is obtained. This vector carries the complete contextual semantic information of the corresponding word.
[0045] Furthermore, the deep semantic feature vectors are subjected to linear dimensionality reduction, nonlinear activation, linear dimensionality increase, and residual connection fusion to obtain the projection mapping feature vectors.
[0046] Specifically, linear dimensionality reduction, nonlinear activation, linear dimensionality increase, and residual connection fusion are performed on deep semantic feature vectors to generate projected mapping feature vectors. This application employs a residual bottleneck projection structure, with two parallel computation paths: a shortcut branch and a bottleneck trunk branch. The shortcut branch directly maps the deep semantic feature vectors to the prototype target dimension through a single-layer linear transformation, and completes a basic nonlinear transformation using the LeakyReLU activation function, serving as a residual direct path to ensure stable gradient backpropagation. The bottleneck trunk branch first performs linear dimensionality reduction through a first-layer linear transformation, compressing the features to an intermediate dimension lower than the prototype dimension. After dimensionality reduction, layer normalization and GELU nonlinear activation operations are performed sequentially, followed by a second-layer linear transformation to complete linear dimensionality increase, restoring the features to the prototype target dimension. The output features of the bottleneck trunk branch and the shortcut branch are added element-wise to complete residual connection fusion, resulting in a projected mapping feature vector with the same dimension as the prototype vector. This achieves alignment and adaptation from the high-dimensional semantic space to the low-dimensional isoangular prototype geometric space, as shown in the following equation:
[0047] in, For the complete projection mapping function, The original word deep semantic feature vector is represented by `<vector>`, `Dropout` is a common random deactivation operation in deep learning, and `LeakyReLU` is a modified linear activation function. The weight matrix is a linear transformation matrix. It is a one-dimensional bias vector. To fix the hidden feature dimension, For feature dimensions.
[0048] Furthermore, the projection mapping feature vector is normalized to the second norm modulus to obtain the corresponding text word unit feature vector.
[0049] Specifically, the L2 norm normalization process is performed on the projection mapping feature vectors to obtain the text word unit feature vectors. The L2 norm of a single projection mapping feature vector is calculated. To avoid division by zero errors caused by a zero norm, a minimal constant of 10 to the power of -8 is superimposed on the norm value. The norm with the added minimal constant is then used to perform division operations on each element of the projection mapping feature vector, completing the overall norm normalization. After normalization, all word feature vectors are unit-length vectors. The result of the inner product operation between this unit feature vector and the fixed prototype vector is directly equivalent to their cosine similarity, ensuring that the geometric constraint calculation based on the vector angle has accurate physical meaning. The final output is the text word unit feature vector, as shown in the following formula:
[0050] in, It is a very small constant.
[0051] This application employs a BERT pre-trained model to extract deep contextual semantics. A first-word alignment strategy addresses the mismatch between word segmentation and entity word granularity, fully preserving the contextual distinguishing features of textual lexical units and ensuring basic semantic representation capabilities for entity recognition. The residual bottleneck projection structure integrates linear dimensionality reduction, nonlinear activation, linear dimensionality increase, and residual connections. While aligning high-dimensional BERT features to the prototype's low-dimensional space, it avoids deep projection gradient vanishing by relying on residual pathways, balancing semantic information preservation and dimensionality matching requirements. L2 norm normalization constrains all lexical features to unit vectors, making the inner product of features and prototype vectors directly equivalent to cosine similarity, unifying the geometric calculation scale, eliminating scoring bias caused by feature magnitude drift at different incremental stages, and ensuring consistency in cross-stage feature comparisons.
[0052] In some embodiments, the identifiers of all entity categories that have been opened in the current incremental task phase and the preset isogonal prototype matrix are subjected to class index truncation processing to obtain the current active prototype matrix composed of the fixed prototype vectors corresponding to the currently opened categories. This includes: performing set aggregation processing on all historical old entity categories and current new entity categories included in the current incremental task phase to obtain the open category index set.
[0053] Specifically, the process begins by summarizing the open category index set. For the current incremental task phase, all historical entity categories that have been gradually opened from the first incremental phase to the current phase, as well as the newly added entity categories in this phase, are collected. All BIO tag numbers corresponding to these two types of entities are integrated and summarized to form a complete open category index set. This index set is ordered sequentially according to the order in which entities were opened. Each index number within the set corresponds to a fixed prototype vector in a pre-defined isometric prototype matrix. Indexes corresponding to unopened entity tags are not included in this set and do not participate in subsequent calculations during the current incremental phase.
[0054] Furthermore, based on the open category index set, the corresponding column vectors in the preset isogonal prototype matrix are subjected to mask filtering and slice extraction to obtain submatrices arranged in category order.
[0055] Specifically, based on the set of open category indexes, a masking and slicing extraction process is performed on the preset isometric prototype matrix. The preset isometric prototype matrix uses column vectors as prototype vectors corresponding to individual entity labels. Following the ordered numbering within the index set, mask identifiers are set for the matrix column dimensions, retaining only the columns included in the index set and masking the column vectors corresponding to other unopened categories. Based on the mask, a matrix slicing extraction operation is performed, extracting all index-matched column vectors from the complete preset isometric prototype matrix to generate sub-matrices neatly arranged according to the category openness order. The slicing process only reads the original matrix values; there is no modification or recalculation of the prototype vector values.
[0056] Furthermore, the submatrix is marked with activation status and subjected to multi-stage isolation processing to obtain the current activation prototype matrix, which is composed of the fixed prototype vectors corresponding to the currently open categories.
[0057] Specifically, the extracted submatrix is then subjected to activation state labeling and multi-stage isolation processing to generate the current activation prototype matrix. An activation state label is added to each prototype vector in the submatrix, indicating that the prototype vector can participate in similarity and scoring-related calculations in the current incremental stage. Simultaneously, multi-stage isolation logic is configured to distinguish between prototypes already activated in previous stages and newly added prototypes in the current stage. Prototype vectors from previous stages retain their original values without any change, while newly added prototypes only participate in calculations as supplementary branches. There is no parameter coupling or gradient interference between the old and new prototypes, and the activation ranges of different incremental stages are isolated from each other, preventing the problem of old prototypes being modified by the new class training process. The submatrix after labeling and isolation processing is the current activation prototype matrix.
[0058] This application's embodiments generate a unified index set by summarizing all historical old classes and the current new classes, fully covering all identifiable entity labels at the current stage. This strictly adheres to the ExtendNER cumulative evaluation rules, ensuring that all learned entity categories can participate in the calculation during incremental evaluation. Masking and slicing extraction based on the index set only read the original prototype matrix data, without modifying any prototype vector values. Activated historical prototypes remain unchanged throughout the process, preventing the old class prototypes from being overwritten or tampered with due to the addition of new categories. Activation status marking and a multi-stage isolation mechanism distinguish the computational branches of the old and new prototypes, completely isolating their gradients. The gradient generated by the new class training only acts on the text features and projection layer, without coupling or interfering with the feature alignment relationship corresponding to the old class prototypes, further suppressing feature drift of the old class.
[0059] In some embodiments, the text lexical unit feature vector, each fixed prototype vector in the current activated prototype matrix, and the preset temperature scaling factor are subjected to vector inner product and numerical amplification processing to obtain the classification score of each open entity category, including: performing matrix multiplication and dot product calculation processing on the text lexical unit feature vector and each fixed prototype vector in the current activated prototype matrix to obtain the cosine similarity numerical matrix.
[0060] Specifically, matrix multiplication and dot product are performed on the text word unit feature vector and all fixed prototype vectors in the current activated prototype matrix, outputting a cosine similarity matrix. The text word unit feature vector is a unit-length vector, and each column of the prototype vector in the current activated prototype matrix is also a unit-length vector. Performing matrix multiplication on both is equivalent to performing vector dot products word-by-word and category-by-category. The result directly corresponds to the cosine similarity between the word feature and the corresponding prototype vector. The similarity values of all words and all open categories are integrated into a two-dimensional cosine similarity matrix. The matrix rows correspond to the positions of each word in the text sequence, and the matrix columns correspond to the open entity labels of each category in the current stage, as shown in the following formula:
[0061] in, This is the currently active prototype matrix.
[0062] Furthermore, the cosine similarity numerical matrix is multiplied and scaled by a preset temperature scaling factor to obtain the original scoring matrix with unnormalized probability.
[0063] Specifically, the cosine similarity matrix is multiplied by a preset temperature scaling factor to amplify the values, generating the original scoring matrix without probability normalization. The preset temperature scaling factor is greater than 1, which is equivalent to sharpening the similarity distribution using a temperature parameter less than 1. Each value in the cosine similarity matrix is uniformly multiplied by the scaling factor to widen the numerical difference in similarity between different entity categories, avoiding a narrow range of original similarity values that would result in a weak gradient in the subsequent cross-entropy loss and an inability to effectively drive word features to align with their corresponding prototype vectors. All values after uniform scaling constitute the original scoring matrix. All values in the matrix are only linearly amplified without cross-category normalization operations such as softmax, and the scores for each category are independent.
[0064] Furthermore, the numerical values corresponding to each word position in the original scoring matrix are mapped and associated with each open entity category to obtain the classification score of each open entity category.
[0065] Specifically, a mapping relationship is established between the values at each position in the original scoring matrix and each open entity category, resulting in a classification score for each open entity category. The column dimensions of the original scoring matrix correspond one-to-one with the column dimensions of the currently activated prototype matrix, with each column fixedly matching an open entity BIO tag category. All values within the matrix are traversed, and category binding is completed according to column indices. The scaled values corresponding to all word positions in the same column are defined as the classification scores for the corresponding entity category. The scores for different entity categories are independent of each other. Adding a new entity category only adds the corresponding column's score value to the matrix, without changing any existing scores for historical entity categories. Finally, the complete classification scores for all open entity categories are output.
[0066] This application's embodiments directly output cosine similarity through matrix multiplication of unit features and a fixed prototype matrix. The calculation logic is simple, naturally relying on the geometric constraints of isogonal prototypes to quantify the matching degree between words and various entities, and the scoring has clear geometric interpretability. A temperature scaling factor amplifies the similarity value range, widening the score gap between different categories, solving the problems of narrow original cosine similarity value ranges and weak gradients, enhancing the gradient force of auxiliary geometric constraints, and accelerating the convergence of word features to their corresponding prototypes. The original scores do not perform cross-category normalization; scores for each category are independent. Adding a new entity only adds a scoring dimension and does not change the original score values corresponding to previous categories.
[0067] In some embodiments, the classification score is processed by Viterbi dynamic programming search based on the constructed normalized transition probability matrix to obtain the target entity recognition result, including: performing binary mask and numerical penalty matrix mapping processing on the label transition legality rule to obtain the normalized transition probability matrix.
[0068] Specifically, based on the BIO tag transfer legality rules, a binary mask and a numerical penalty matrix are mapped to generate a normalized transfer probability matrix. The legal transfer constraints for all open BIO tags are pre-defined: non-entity O tags can transfer to themselves or any entity-starting BX tags; any BX entity-starting tag can transfer to O tags, tags within the same type of IX entities, or tags starting with any other BY entities; tags within any IX entity can only transfer to O tags, tags within the same type of IX entities, or tags starting with any other BY entities; all other cross-tag transfers are considered illegal. A binary mask matrix is constructed based on these rules, with legal transfer positions marked with a value of 1 and illegal transfer positions marked with a value of 0. The binary mask matrix is then mapped to a logarithmic form numerical penalty matrix, where legal transfer positions retain a finite logarithmic probability value, and illegal transfer positions are filled with negative infinity as penalty terms. This penalty matrix is the normalized transfer probability matrix, which is registered as a fixed tensor throughout the process and does not participate in gradient updates.
[0069] Furthermore, the classification score is used as the emission probability value of each word position, and Viterbi dynamic programming is used to solve the state transition process with the normalized transition probability matrix to obtain the optimal label path sequence.
[0070] Specifically, the classification score corresponding to each word position is used as the emission probability value. A Viterbi dynamic programming state transition solution is then performed using the canonical transition probability matrix to output the globally optimal label path sequence. The classification score obtained by scaling each word position in the text sequence is used as the emission score for that position, corresponding to the emission probability of the Viterbi algorithm. The total path score is accumulated word by word position according to the Viterbi forward recursion formula. During the recursion process, the logarithmic transition cost within the canonical transition probability matrix is simultaneously added. Illegal transition paths are automatically excluded due to the negative infinity penalty value, and only legal paths conforming to the BIO rules are retained for optimal value calculation. After accumulating the forward scores of all text words, the label with the highest accumulated score at the end of the text is selected as the path endpoint. The optimal preceding label recorded at each position is traced back from the endpoint to completely reconstruct the entire globally optimal legal label path sequence. This sequence does not contain any illegal label jumps that violate the BIO format, as shown in the following formula:
[0071] in, Let be the position of the t-th word in the text sequence. For the candidate BIO tag corresponding to the current t-th word, For the candidate BIO tags of the previous word element, For Viterbi's cumulative best score, Assign a classification score to the t-th word and its corresponding tag y.
[0072] Furthermore, entity boundary parsing and category mapping are performed on the optimal label path sequence to obtain the target entity recognition result.
[0073] Specifically, entity boundary parsing and entity category mapping are performed on the optimal label path sequence to output the final target entity recognition result. The tags within the optimal label path are traversed word by word, distinguishing between B-class starting tags, I-class internal tags, and O-class non-entity tags. A BX tag is used as the starting boundary of an entity segment, and subsequent consecutive IX tags of the same type are classified as belonging to the same entity segment until an O tag or other type B tag appears, at which point the entity segment is considered to have ended. Consecutive BX and IX word segments are integrated into complete named entities, and the entity type identifier carried within the tags is bound simultaneously. The text segment corresponding to the entity, the entity's start position, the entity's end position, and the entity's category are separated. Invalid tag information corresponding to sequence-filling placeholders is filtered out, and the target entity recognition result is obtained after integrating all valid entity information.
[0074] This application's embodiments construct a binary mask plus a negative infinity penalty transition matrix based on BIO label rules. This shields illegal label jumps at the path level, avoiding erroneous sequences of Class I labels without valid starts generated by single-point independent argmax decoding. This reduces entity boundary and entity type recognition errors, improving recognition accuracy and recall. The transition matrix is only introduced into path search during the inference phase and is not included in loss calculation during the training phase. This does not compromise the core characteristic of independent classification scoring of this invention and does not interfere with the geometric constraint effect of the isoangular prototype on the feature space. Viterbi dynamic programming globally searches for the optimal valid path, correcting local scoring errors at individual word positions, preventing single-point misjudgments from propagating continuously along the text sequence, and significantly reducing the accumulation of sequence errors in long text entity recognition.
[0075] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed sequentially according to this embodiment or the accompanying drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).
[0076] like Figure 3 As shown in the illustration, this application also provides an entity recognition device 300. The device includes: The processing module 301 is used to perform geometric construction and parameter locking processing on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule and freeze control parameters to obtain a preset isogonal prototype matrix. The preset isogonal prototype matrix includes multiple fixed prototype vectors with equal included angles in pairs.
[0077] The processing module 301 is also used to perform deep semantic feature extraction, residual bottleneck projection mapping and modulus normalization on the text sequence to be processed, so as to obtain the corresponding text word unit feature vector.
[0078] The processing module 301 is also used to perform class-indexed truncation processing on all entity category identifiers and preset isogonal prototype matrices that have been opened in the current incremental task stage, so as to obtain the current active prototype matrix composed of fixed prototype vectors corresponding to the currently opened categories.
[0079] The acquisition module 302 is used to perform vector inner product and numerical amplification processing on the text word unit feature vector, each fixed prototype vector in the current activated prototype matrix and the preset temperature scaling factor to obtain the classification score of each open entity category.
[0080] Module 303 is used to perform Viterbi dynamic programming search on the classification score based on the constructed normalized transition probability matrix to obtain the target entity recognition result.
[0081] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0082] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.
[0083] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.
[0084] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.
[0085] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application embodiment to be implemented.
[0086] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.
[0087] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.
[0088] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.
[0089] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.
[0090] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.
Claims
1. An entity recognition method, characterized in that, include: Geometric construction and parameter locking are performed on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule, and freezing control parameters to obtain a preset isoangular prototype matrix. The preset isoangular prototype matrix includes multiple fixed prototype vectors with equal angles between each pair of vectors. Deep semantic feature extraction, residual bottleneck projection mapping and modulus normalization are performed on the text sequence to be processed to obtain the corresponding text word unit feature vector. The identifiers of all entity categories that have been opened in the current incremental task phase are truncated by category index to obtain the current active prototype matrix composed of the fixed prototype vectors corresponding to the currently opened categories. The text word unit feature vector, each fixed prototype vector in the current activated prototype matrix, and the preset temperature scaling factor are subjected to vector inner product and numerical amplification processing to obtain the classification score of each open entity category. Based on the constructed normalized transition probability matrix, the classification score is processed by Viterbi dynamic programming search to obtain the target entity recognition result.
2. The method according to claim 1, characterized in that, The process of geometrically constructing and locking parameters on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule, and freeze control parameters to obtain a preset isogonal prototype matrix includes: Gaussian random sampling and orthogonal decomposition are performed on the preset maximum number of categories and feature dimensions to obtain the orthogonal basis matrix; The orthogonal basis matrix is subjected to row-wise mean calculation and row-wise mean subtraction to obtain a zero-mean centered matrix; The centered matrix is normalized by modulus and scaled by ratio to obtain a preset isoangular prototype matrix in which the cosine similarity between any two vectors is a preset negative constant. The gradient update state of the preset isoangular prototype matrix is subjected to backpropagation truncation and parameter locking to obtain a preset isoangular prototype matrix that remains unchanged during both model training and inference phases.
3. The method according to claim 1, characterized in that, The text sequence to be processed undergoes deep semantic feature extraction, residual bottleneck projection mapping, and modulus normalization to obtain the corresponding text word unit feature vector, including: The text sequence to be processed is encoded by a pre-trained language model to obtain deep semantic feature vectors; The deep semantic feature vector is subjected to linear dimensionality reduction, nonlinear activation, linear dimensionality increase and residual connection fusion to obtain the projection mapping feature vector; The projection mapping feature vector is normalized to its L2 norm to obtain the corresponding text word unit feature vector.
4. The method according to claim 1, characterized in that, The step of performing class-indexed truncation processing on all entity category identifiers already opened in the current incremental task phase and the preset isoangular prototype matrix to obtain the current active prototype matrix composed of fixed prototype vectors corresponding to the currently opened categories includes: The set of open category indexes is obtained by summarizing all historical old entity categories and current new entity categories included in the current incremental task phase. Based on the set of open category indexes, the corresponding column vectors in the preset isogonal prototype matrix are subjected to mask filtering and slice extraction to obtain submatrices arranged in category order; The submatrix is then subjected to activation state marking and multi-stage isolation processing to obtain the current activation prototype matrix, which is composed of the fixed prototype vectors corresponding to the currently open categories.
5. The method according to claim 1, characterized in that, The process of performing vector dot product and numerical amplification on the text lexical unit feature vector, each fixed prototype vector in the current activated prototype matrix, and a preset temperature scaling factor to obtain the classification score for each open entity category includes: The cosine similarity matrix is obtained by performing matrix multiplication and dot product calculations on the text word unit feature vector and each fixed prototype vector in the current activated prototype matrix. The cosine similarity numerical matrix is multiplied and scaled by a preset temperature scaling factor to obtain the original scoring matrix with unnormalized probability. The numerical values corresponding to each word position in the original scoring matrix are mapped and associated with each open entity category to obtain the classification score of each open entity category.
6. The method according to claim 1, characterized in that, The classification score is processed using Viterbi dynamic programming based on the constructed canonical transition probability matrix to obtain the target entity recognition result, including: The label transition legality rule is mapped to a binary mask and a numerical penalty matrix to obtain the normalized transition probability matrix. The classification score is used as the emission probability value of each word position, and Viterbi dynamic programming is used to solve the state transition process with the normalized transition probability matrix to obtain the optimal label path sequence. The optimal label path sequence is subjected to entity boundary parsing and category mapping to obtain the target entity recognition result.
7. An entity recognition device, characterized in that, include: The processing module is used to perform geometric construction and parameter locking processing on the pre-generated orthogonal basis matrix, centering rule, modulus scaling rule and freeze control parameters to obtain a preset isoangular prototype matrix, which includes multiple fixed prototype vectors with equal included angles in pairs. The processing module is also used to perform deep semantic feature extraction, residual bottleneck projection mapping and modulus normalization on the text sequence to be processed, so as to obtain the corresponding text word unit feature vector. The processing module is also used to perform class index truncation processing on all entity category identifiers that have been opened in the current incremental task stage and the preset isogonal prototype matrix to obtain the current active prototype matrix composed of fixed prototype vectors corresponding to the currently opened categories. The acquisition module is used to perform vector inner product and numerical amplification processing on the text word unit feature vector, each fixed prototype vector in the current activated prototype matrix and the preset temperature scaling factor to obtain the classification score of each open entity category. The module is used to perform Viterbi dynamic programming search on the classification score based on the constructed normalized transition probability matrix to obtain the target entity recognition result.
8. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.
