Named entity recognition method based on maximum pooling extraction fragment boundary and adjacent fragment gap features
By combining pre-trained language models and multi-layer hollow convolutional neural networks and other methods, using maximum pooling to extract fragment boundaries and adjacent fragment gap features, the unified processing problem of multi-type naming entity recognition in the existing technology is solved, and a high accuracy and robust naming entity recognition is achieved.
Patent Information
- Application Number
- CN202510439211.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-08-01
AI Technical Summary
The prior art is difficult to effectively and uniformly deal with flat named entities, discontinuous named entities, overlapped named entities and long named entities, and there are problems of error propagation between tasks and local error propagation in pipeline tasks.
The combination methods of pre-trained language model, BiLSTM neural network, multi-layer hollow convolutional neural network are used to extract fragment boundaries and adjacent fragment gap characteristics through maximum pooling, and the joint learning method is used to identify the named entity type boundaries to avoid error propagation.
It improves the accuracy and robustness of named entity recognition, can uniformly handle multi-type named entities, reduces local error propagation, improves boundary recognition and feature expression capabilities, and optimizes the performance of experimental data verification.
Smart Images
Figure CN120409478A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of natural language processing, and particularly to a named entity recognition method based on maximum pooling to extract segment boundary and adjacent segment gap features. Background Art
[0002] Named entity recognition is a fundamental component of natural language processing. It is an upstream task for many NLP tasks, such as relation extraction, entity linking, knowledge graph generation, etc. In recent years, great progress has been made in the field of uniformly recognizing flat named entities, discontinuous named entities, overlapping named entities, and long named entities. However, it is still a challenging task to process the above four different types of named entities using a unified method.
[0003] Early named entity recognition adopted the BIO tag-based approach. Although this method is simple and effective, it cannot handle discontinuous or overlapping named entities.
[0004] In recent years, methods based on named entity segment recognition have achieved great success in the field of unified named entity recognition. For example, the paper "Packed Levitated Marker for Entity and Relation Extraction" published by Deming Ye et al. in ACL 2022 proposed a method for recognizing named entity segments based on floating tags and stitching different named entity segments into named entities. However, this type of method introduces an additional named entity segment extraction network, resulting in complex problems and increased time complexity, and there is error propagation between the segment extraction task and the segment classification task.
[0005] The paper "Unified named entity recognition as word-word relation classification." published by Jingye Li et al. in AAAI 2022 starts from the perspective of character pair relations, structures the text into a two-dimensional matrix of character pairs, and uniformly processes flat named entities, discontinuous named entities, overlapping named entities, and long named entities by recognizing the next character relation and entity type boundary relation in the entity.
[0006] Although the method based on named entity adjacent character relations simply and effectively unifies the processing of flat named entities, discontinuous named entities, overlapping named entities, and long named entities, in this method, if any adjacent character relation is not recognized, it will cause the entire entity recognition to be incorrect, that is, there is local error propagation. Summary of the Invention
[0007] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a named entity recognition method for extracting fragment boundaries and adjacent fragment gap features based on max-pooling. This method is applicable to flat named entities, discontinuous named entities, overlapping named entities, and long named entities, and uses a joint learning method to avoid the error propagation between tasks commonly found in pipeline tasks.
[0008] The purpose of the present invention is achieved through the following technical solutions:
[0009] A named entity recognition method for extracting fragment boundaries and adjacent fragment gap features based on max-pooling, including a pre-trained language model PLM, a BiLSTM neural network, a named entity fragment boundary and adjacent fragment gap feature recognition neural network, a named entity type boundary recognition network, a multi-layer dilated convolutional neural network, and a named entity label feature recognition neural network:
[0010] S1. Use the pre-trained language model to encode the text to obtain the feature representation of each character;
[0011] S2. Use the BiLSTM neural network to enhance the context awareness ability of the character feature representation; a continuous segment of characters in a named entity is called a named entity fragment, and the feature representations of the first character and the last character of the named entity fragment are used as the named entity fragment feature representation; in a named entity composed of two or more than two named entity fragments, the gap between two adjacent named entity fragments forms an adjacent fragment gap, and the feature representations of the last character of the preceding named entity fragment and the first character of the subsequent named entity fragment are used as the named entity adjacent fragment gap feature representation; through the named entity fragment boundary and adjacent fragment gap feature recognition neural network, use the max-pooling operator to activate the significant features in the two character feature representations to distinguish and obtain the named entity fragment feature representation, the named entity adjacent fragment gap feature representation, the non-named entity fragment boundary and adjacent fragment gap feature;
[0012] S3. The named entity type boundary recognition network uses the pre-trained language model to encode the text to obtain the feature representation of each character, performs linear transformations on the feature representations of the last character and the first character of the named entity respectively, maps them to the named entity type boundary feature extraction space, and uses the max-pooling operator to activate the significant features of the last character and the first character of the named entity in the named entity type boundary feature extraction space; the named entity type boundary recognition network superimposes the feature representation of the relationship between the last character and the first character in the named entity type boundary feature extraction space on the named entity fragment feature representation, the named entity adjacent fragment gap feature representation, and the non-named entity fragment boundary and adjacent fragment gap feature obtained in S2 to obtain the named entity type boundary feature representation;
[0013] S4. Use a multi-layer dilated convolutional neural network to map the named entity segment feature representation, the named entity adjacent segment gap feature representation, and the named entity type boundary feature representation to a unified character relationship feature space;
[0014] S5. Use three types of relationships, namely the named entity segment boundary relationship, the named entity adjacent segment gap relationship, and the named entity type boundary relationship, as named entity labels; use a pre-trained language model to encode the named entity labels to obtain a named entity label feature representation; use average pooling to reduce the dimension of the named entity label feature representation and align it with the unified character relationship feature space; adopt a unified method based on similarity matching to achieve the recognition of flat named entities, discontinuous named entities, overlapping named entities, and long named entities in the text.
[0015] Further, step S1 includes the following steps:
[0016] S101. Encode the input text using a pre-trained language model PLM. The pre-trained language model PLM uses a BERT neural network to obtain the feature representation of each character:
[0017] X = {x1, x2, …, x n};
[0018] h i = PLM(x i );
[0019] where x i represents the i-th character in the text, n represents the number of characters in the text, h i represents the feature representation of the i-th character, and the feature representation sequence of the characters in the text is:
[0020]
[0021] where d h represents the dimension of the character feature representation output by the pre-trained language model PLM; R represents the set of real numbers.
[0022] Further, step S2 includes the following steps:
[0023] S201. Use a BiLSTM neural network to process the character feature representation in the text:
[0024] h i ' = BiLSTM(h i );
[0025] where h u ' represents the feature representation of the character output by the BiLSTM neural network; the character feature representation sequence H' in the processed text is:
[0026]
[0027] where d h′ represents the output dimension of the BiLSTM neural network;
[0028] S202. After performing a linear transformation on H′, use the max pooling operator to obtain the feature representation g of the relationship between two characters ij :
[0029] h i "" = h i ′ × W;
[0030] g ij = max(h i ″, h j ″);
[0031] where h i ″ and h j ″ respectively represent the feature representations of the two characters in the character relationship; W represents the linear transformation parameter matrix; the character relationship feature representation matrix G is:
[0032]
[0033] where d g represents the output dimension of the character relationship feature representation matrix, and i and j represent the i-th character and the j-th character.
[0034] Furthermore, step S3 includes the following steps:
[0035] S301. In the lower triangular matrix of the character relationship feature representation matrix, for identifying the type boundary feature representations of the end character and the start character of the named entity, perform linear transformations on the feature representations of the end character and the start character using different weights respectively, and extract the relationship feature representations of the end character and the start character:
[0036]
[0037] where h i , h j respectively represent the feature representations of the i-th and j-th characters; represents the character feature representation with the i-th character as the start character of the named entity, represents the character feature representation with the j-th character as the end character of the named entity; W head represents the start character linear transformation parameter matrix; W tail represents the start character linear transformation parameter matrix;
[0038] S302. Use max pooling to synthesize the feature representations of the end character and the start character into the relationship feature representation of these two characters
[0039]
[0040] Lower triangular matrix G composed of the relationship characteristics of the tail character and the head character tril It is expressed as:
[0041]
[0042] d g Represents the output dimension of the character relationship feature representation matrix
[0043] Furthermore, step S4 includes the following steps:
[0044] S401. Add the character relationship feature representation matrix G in step S2 and the lower triangular matrix G of the character relationship feature representation matrix in step S3 tril to obtain a unified character relationship feature representation matrix:
[0045]
[0046] where g ij represents the feature representation of the relationship between two characters represents the relationship feature representation after the synthesis of the tail character and the head character represents the feature representation between the i-th character and the j-th character. If i < j, that is, located in the upper triangular matrix, the relationship between the i-th character and the j-th character belongs to the named entity segment boundary relationship, or the named entity boundary gap relationship, or no relationship; if i ≤ j, that is, located in the lower triangular matrix, the relationship between the i-th character and the j-th character belongs to the named entity type boundary relationship, or no relationship
[0047] Finally, the feature representation matrix G of the relationship between two characters final is:
[0048]
[0049] S402. Use the multi-layer dilated convolutional neural network DConv(·) to extract features from the feature representation matrix of the relationship between two characters to obtain the convolutional layer feature representation matrix C of all character relationships:
[0050] C = δ(DConv(G final )) = {c 00 , …, c ik, …, c ji , …c nn};
[0051] where δ(·) is the activation function, the number of layers and the dilation size of the multi-layer dilated convolutional neural network DConv(·) are hyperparameters, the common number of dilated convolutional layers is 3, and the dilation sizes are [1, 2, 3]; c ij is the convolutional layer feature representation of the relationship between the i-th character and the j-th character.
[0052] Further, step S5 includes the following steps:
[0053] S501. Encode the named entity segment boundary relationship label, the named entity adjacent segment gap relationship label, the named entity type boundary relationship label, and the no-relationship label, and input the relationship label r i = r1, r2, …, r , , to obtain the feature identification of the relationship label
[0054]
[0055] where PLM refers to the pre-trained language model, mean(·) is the average pooling used to reduce the dimension of the feature representation of the character relationship label to the dimension of the feature representation of the character relationship; represents the feature representation of the i-th character relationship label;
[0056] S502. Calculate the distance between each character relationship feature representation in the two-character relationship Convolutional layer feature representation matrix and the feature representation of the character relationship label:
[0057]
[0058] where dis ijk represents the L2 distance between the feature representation of the relationship between character i and character j and the feature representation of the k-th character relationship label;
[0059] The predicted output y ijk of the character relationship feature representation matrix is:
[0060] y ijk = softmax(-dis ijk );
[0061] S503. The loss function uses the cross-entropy loss L, which is defined as:
[0062]
[0063] where, is the supervision signal; m is the number of character relationship labels.
[0064] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the named entity recognition method for extracting fragment boundaries and adjacent fragment gap features based on max-pooling are implemented.
[0065] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the named entity recognition method for extracting fragment boundaries and adjacent fragment gap features based on max-pooling are implemented.
[0066] Compared with the prior art, the beneficial effects brought by the technical solution of the present invention are as follows:
[0067] 1. Improved boundary recognition ability: The max-pooling operator is used to activate the features of character relationships. The two-character relationship feature representation generated through linear transformation can accurately extract the boundaries of named entity fragments and the features of adjacent fragment gaps, thereby greatly improving the accuracy of boundary recognition.
[0068] 2. Unified processing of multiple types of named entities: This method defines character relationships as named entity fragment boundary relationships, named entity fragment gap boundary relationships, named entity type boundary relationships, and no relationships, and uses a joint learning method to achieve unified recognition of flat named entities, discontinuous named entities, overlapping named entities, and long named entities, effectively avoiding the overall recognition errors caused by local error propagation in pipeline tasks.
[0069] 3. Reasonable model structure and strong feature expression ability: Through the organic combination of modules such as pre-trained language models, BiLSTM neural networks, and multi-layer dilated convolutional neural networks, the boundaries and gap features of each level of named entities are mapped to a unified character relationship feature space, enhancing the model's feature capture and expression ability in complex text scenarios.
[0070] 4. Experimental data verifies performance optimization: Experimental results show that this method reaches F1 values of 94.09, 73.34, 80.30, and 95.38 on the four datasets of CoNLL2003, CADEC, GENIA, and Resume Chinese respectively, and its recognition performance exceeds or reaches the level of the latest research results, fully demonstrating the high accuracy and robustness of this method in various application scenarios.
[0071] 5. Broad application prospects: The present invention not only improves the boundary accuracy of named entity recognition, but also provides strong technical support for upstream natural language processing tasks such as subsequent relation extraction, entity linking, and knowledge graph construction, having good promotion application prospects and practical application value. Description of the Drawings
[0072] Figure 1 It is a framework diagram of the named entity recognition method according to an embodiment of the present invention.
[0073] Figure 2 It is an example diagram of entity segment relationship and entity segment gap relationship.
[0074] Figure 3 It is an example diagram of the character relationship matrix label. Detailed implementation manners
[0075] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0076] The present invention identifies flat named entities, discontinuous named entities, overlapping named entities, and long named entities in a unified manner. That is, the pairwise relationships between characters in a text are defined as entity boundary relationships, entity segment boundary relationships, segment gap boundary relationships, and no relationships, and a two-dimensional matrix is used to represent the above relationships. Among them, the upper triangular matrix is used to represent the entity segment boundary relationship and the segment gap boundary relationship, and the lower triangular matrix is used to represent the entity boundary relationship. The character relationship representation is generated through a maximization operator, and a convolutional neural network is used to process the two-dimensional character relationship matrix to obtain the character relationship label. The method of the present invention solves the problem of error propagation between tasks in the previous unified named entity recognition model based on the pipeline method and the problem of local error propagation in the method based on adjacent character relationships.
[0077] Specifically, as Figures 1 - 3 shown, the named entity recognition method based on max-pooling to extract segment boundary and adjacent segment gap features provided in this embodiment specifically includes the following steps:
[0078] S1: Preprocess the data set to obtain the data and annotation matrix required for pre-training the language model. The label introduction is as Figure 2 , Figure 3 shown. In this embodiment, the pre-trained language model PLM uses a BERT neural network.
[0079] The relationship between two characters is divided into: named entity segment boundary relationship, named entity segment gap boundary relationship, named entity type boundary relationship, and no relationship.
[0080] Among them, the named entity segment boundary relationship refers to the relationship between the first character and the last character of all mutually isolated segments, or segments that are segmented due to being covered by other named entities in a named entity.
[0081] The named entity segment gap boundary relationship refers to the gap relationship between two adjacent segments in the same named entity, which is composed of the last character of the previous segment and the first character of the next segment.
[0082] The boundary relationship of named entity types refers to the relationship between the last character and the first character of the entire named entity, and its label is determined by the named entity type.
[0083] No relationship means that the character pair does not have the three relationships mentioned above.
[0084] Generate an annotation matrix according to the character pair relationship, where the upper triangular matrix (excluding the diagonal) is used to represent the boundary relationship of named entity segments and the boundary relationship of named entity segment gaps; the lower triangular matrix (including the diagonal) is used to represent the boundary relationship of named entity types.
[0085] S2: Construct a feature representation matrix of the relationship between two characters. Generate the feature representation of the character pair relationship through linear transformation and max pooling, replacing the traditional bi-affine or conditional layer normalization method. For the lower triangular matrix, add additional linear transformation and max pooling feature activations of the last character and the first character to enhance the ability to identify the boundary relationship of entity types. Then perform multi-layer dilated convolution feature extraction on the entire matrix.
[0086] S201: Use a BiLSTM neural network to process the character feature representation in the text, then perform a linear transformation on the character feature representation, and finally use max pooling to generate a feature representation matrix of the relationship between two characters. Specifically:
[0087] h i ′ = BiLSTM(h i );
[0088] where h i ′ represents the feature representation of the character output by the BiLSTM neural network. The sequence of character feature representations in the processed text is:
[0089]
[0090] where d h′ represents the output dimension of the BiLSTM neural network.
[0091] After performing a linear transformation on H′, use the max pooling operator to obtain the feature representation of the relationship between two characters:
[0092] h i "" = h i ′ × W;
[0093] g ij = max(h i ″, h j ″);
[0094] where, h i ″ and h j″respectively represent the feature representations of two characters in the character relationship; W represents the linear transformation parameter matrix; the character relationship feature representation matrix is:
[0095]
[0096] where d g represents the output dimension of the character relationship feature representation matrix, and i and j represent the i-th character and the j-th character.
[0097] S202: In the lower triangular matrix of the two-character relationship feature representation, add head and tail linear transformations and max pooling. Then add the lower triangular matrix and the original matrix to update the two-character relationship feature representation matrix. Specifically:
[0098] In the lower triangular matrix of the character relationship feature representation matrix, to identify the type boundary feature representations of the last character and the first character of the named entity, linear transformations are performed on the feature representations of the last character and the first character using different weights respectively, and the relationship feature representations of the last character and the first character are extracted:
[0099]
[0100] where h i 、h j respectively represent the feature representations of the i-th and j-th characters; represents taking the i-th character as the first character of the named entity, represents taking the j-th character as the last character of the named entity; W head represents the first character linear transformation parameter matrix; W tail represents the first character linear transformation parameter matrix;
[0101] Use max pooling to synthesize the feature representations of the last character and the first character into the relationship feature representation of these two characters
[0102]
[0103] The lower triangular matrix G composed of the relationship feature representations of the last character and the first character tril is expressed as:
[0104]
[0105] d g represents the output dimension of the character relationship feature representation matrix.
[0106] S203: Use the multi-layer dilated convolutional neural network DConv(·) to extract features from the two-character relationship feature representation matrix to obtain the convolutional layer feature representation matrix C of all character relationships:
[0107] C = δ(DConv(G final )) = {c 00 , …, c ij, …, c ji , … c nn};
[0108] Where δ(·) is the activation function, the number of layers and the dilation size of the multi-layer dilated convolutional neural network DConv(·) are hyperparameters. The commonly used number of dilated convolutional layers is 3, and the dilation sizes are [1, 2, 3]; c ij, is the convolutional layer feature representation of the relationship between the i-th character and the j-th character.
[0109] S3: Encode the named entity segment boundary relationship label, the named entity adjacent segment gap relationship label, the named entity type boundary relationship label, and the no-relationship label to obtain the feature representation of the character pair relationship label. This feature representation is equivalent to the prototype of the character relationship label in the input context.
[0110] h i = PLM(x i );
[0111]
[0112] Calculate the distance between the feature representation in the two character relationship feature representation matrices and the feature representation of the character relationship label, and select the label with the smallest distance for the two character relationship feature representation matrices.
[0113]
[0114] Where i, j are the matrix dimensions, k is the number of labels, and dis ijk represents the L2 distance between the relationship feature representation between character i and character j and the k-th character relationship label.
[0115] S4: Calculate the loss of the distance between the two character relationship feature representation matrices and the feature representation of the character relationship label. The loss function uses the cross-entropy function.
[0116] y ijk = softmax(-dis ijk );
[0117]
[0118] Where, is the supervision signal; m is the number of character relationship labels.
[0119] Preferably, an embodiment of the present application further provides a specific implementation manner of an electronic device that can implement all steps in the named entity recognition method based on max - pooling for extracting fragment boundaries and adjacent fragment gap features in the above - mentioned embodiment. The electronic device specifically includes the following:
[0120] A processor, a memory, a communications interface, and a bus;
[0121] Among them, the processor, the memory, and the communications interface complete mutual communication through the bus; the communications interface is used to implement information transmission between related devices such as server - side devices, metering devices, and user - side devices.
[0122] The processor is used to call a computer program in the memory. When the processor executes the computer program, it implements all steps in the named entity recognition method based on max - pooling for extracting fragment boundaries and adjacent fragment gap features in the above - mentioned embodiment.
[0123] An embodiment of the present application also provides a computer - readable storage medium that can implement all steps in the named entity recognition method based on max - pooling for extracting fragment boundaries and adjacent fragment gap features in the above - mentioned embodiment. A computer program is stored on the computer - readable storage medium, and when the computer program is executed by the processor, it implements all steps of the named entity recognition method based on max - pooling for extracting fragment boundaries and adjacent fragment gap features in the above - mentioned embodiment.
[0124] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program - type embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0125] The above - mentioned specific embodiments of this specification have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0126] Although the present application provides method operation steps such as in the embodiments or flowcharts, more or fewer operation steps may be included based on routine or non-creative labor. The step sequences listed in the embodiments are only one way among numerous step execution sequences and do not represent the only execution sequence. When the actual device or client product is executed, it may be executed in the method sequence shown in the embodiments or the drawings or in parallel (e.g., in an environment of parallel processors or multi-threaded processing).
[0127] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0128] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks specified in the function.
[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks specified in the function.
[0130] The present invention is not limited to the above-described embodiments. The above description of the specific embodiments is intended to describe and illustrate the technical solutions of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the spirit of the present invention and the scope protected by the claims, those of ordinary skill in the art can also make many specific transformations in various forms under the inspiration of the present invention, and these all fall within the protection scope of the present invention.
Claims
1. A named entity recognition method based on maximum pooling for extracting fragment boundary and adjacent fragment gap features, characterized in that Including a pre-trained language model PLM, a BiLSTM neural network, a named entity segment boundary and adjacent segment gap feature recognition neural network, a named entity type boundary recognition network, a multi-layer dilated convolutional neural network, and a named entity label feature recognition neural network: S1. Use the pre-trained language model to encode the text to obtain the feature representation of each character; S2. Use the BiLSTM neural network to enhance the context awareness ability of the character feature representation; A continuous segment of characters in a named entity is called a named entity segment, and the feature representations of the first character and the last character of the named entity segment are used as the named entity segment feature representation; In a named entity composed of two or more than two named entity segments, an adjacent segment gap is formed between two adjacent named entity segments, and the feature representations of the last character of the preceding named entity segment and the first character of the subsequent named entity segment are used as the named entity adjacent segment gap feature representation; Through the named entity segment boundary and adjacent segment gap feature recognition neural network, use the max pooling operator to activate the significant features in the two character feature representations to distinguish and obtain the named entity segment feature representation, the named entity adjacent segment gap feature representation, the non-named entity segment boundary and adjacent segment gap feature; S3. The named entity type boundary recognition network uses the pre-trained language model to encode the text to obtain the feature representation of each character, and performs linear transformation on the feature representations of the last character and the first character of the named entity respectively, maps them to the named entity type boundary feature extraction space, and uses the max pooling operator to activate the significant features of the last character and the first character of the named entity in the named entity type boundary feature extraction space; The named entity type boundary recognition network superimposes the feature representation of the relationship between the last character and the first character in the named entity type boundary feature extraction space on the named entity segment feature representation, the named entity adjacent segment gap feature representation, and the non-named entity segment boundary and adjacent segment gap feature obtained in S2 to obtain the named entity type boundary feature representation; S4. Use the multi-layer dilated convolutional neural network to map the named entity segment feature representation, the named entity adjacent segment gap feature representation, and the named entity type boundary feature representation to a unified character relationship feature space; S5. Use three types of relationships, namely the named entity segment boundary relationship, the named entity adjacent segment gap relationship, and the named entity type boundary relationship, as the named entity label; Use the pre-trained language model to encode the named entity label to obtain the named entity label feature representation; Use average pooling to reduce the dimension of the named entity label feature representation and align the named entity label feature representation with the unified character relationship feature space; Adopt a unified method based on similarity matching to realize the recognition of flat named entities, discontinuous named entities, overlapping named entities, and long named entities in the text.
2. The named entity recognition method based on extracting segment boundaries and adjacent segment gap features by max pooling according to claim 1, wherein Step S1 includes the following steps: S101. Encode the input text using the pre-trained language model PLM. The pre-trained language model PLM uses the BERT neural network to obtain the feature representation of each character: X = {x1, x2, …, x n}; h i = PLM(x i ); where x i represents the i-th character in the text, n represents the number of characters in the text, h i represents the feature representation of the i-th character, and the sequence of feature representations of the characters in the text is: Among them, d h represents the dimensionality of the character feature representation output by the pre-trained language model PLM; R represents the set of real numbers.
3. The named entity recognition method based on extracting segment boundaries and adjacent segment gap features by maximum pooling according to claim 1, characterized in that Step S2 includes the following steps: S201. Process the character feature representation in the text using a BiLSTM neural network: h i ' = BiLSTM(h i ); where h i ′ represents the feature representation of the character output by the BiLSTM neural network; the sequence H′ of character feature representations in the processed text is: where d hi represents the output dimension of the BiLSTM neural network; S202. After linearly transforming H′, use the max pooling operator to obtain the feature representation g of the relationship between two characters ij : h i ″ = h i ′ × W; g ij = max(h i ″, h j ″); where h i ″ and h j ″ respectively represent the feature representations of two characters in the character relationship; W represents the linear transformation parameter matrix; the character relationship feature representation matrix G is: where d g represents the output dimension of the character relationship feature representation matrix, and i and j represent the i-th character and the j-th character.
4. The named entity recognition method based on maximum pooling for extracting segment boundaries and adjacent segment gap features according to claim 3, wherein Step S3 includes the following steps: S301. In the lower triangular matrix of the character relationship feature representation matrix, to identify the type boundary feature representation of the end character and the start character of the named entity, perform linear transformations on the feature representations of the end character and the start character using different weights respectively, and extract the relationship feature representation of the end character and the start character: where h i and h j represent the feature representations of the i-th and j-th characters respectively; represents that the i-th character is the first character of the named entity, represents the character feature representation with the j-th character as the last character of the named entity; W head represents the first character linear transformation parameter matrix; W tail represents the first character linear transformation parameter matrix; S302. Use max pooling to synthesize the feature representations of the tail character and the head character into the relationship feature representation of these two characters. The lower triangular matrix G formed by the relationship characteristics of the tail character and the head character tril It is expressed as: d g Indicates the output dimension of the character relationship feature representation matrix.
5. The named entity recognition method based on extracting segment boundaries and adjacent segment gap features by max pooling according to claim 4, characterized in that, Step S4 includes the following steps: S401. Add the character relationship feature representation matrix G in step S2 and the lower triangular matrix G of the character relationship feature representation matrix in step S3 to obtain a unified character relationship feature representation matrix: tril where g ij is a feature representation of the relationship between two characters, is a combined relationship feature representation of the feature representations of the tail character and the head character; is a feature representation between the i-th character and the j-th character. If i < j, that is, in the upper triangular matrix, the relationship between the i-th character and the j-th character belongs to the named entity segment boundary relationship, or the named entity boundary gap relationship, or no relationship; if i ≤ j, that is, in the lower triangular matrix, the relationship between the i-th character and the j-th character belongs to the named entity type boundary relationship, or no relationship; Feature representation matrix G of the final two-character relationship final is as follows: S402. Use a multi-layer dilated convolutional neural network DConv(·) to extract features from the two character relationship feature representation matrices, and obtain the convolutional layer feature representation matrix C of all character relationships: C = δ(DConv(G final )) = {c 00 , …, c ij, …, c ji , …c nn}; where δ(·) is the activation function, the number of layers and the dilation sizes of the multi-layer dilated convolutional neural network DConv(·) are hyperparameters. The commonly used number of dilated convolutional layers is 3, and the dilation sizes are [1, 2, 3]; c ij is the convolutional layer feature representation of the relationship between the i-th character and the j-th character.
6. The named entity recognition method based on maximum pooling for extracting fragment boundaries and adjacent fragment gap features according to claim 1, characterized in that, Step S5 includes the following steps: S501. Encode the named entity segment boundary relation label, the named entity adjacent segment gap relation label, the named entity type boundary relation label, and the no-relation label, and input the relation label r i = r1, r2, …, r m , to obtain the feature identifier of the relation label Among them, PLM refers to the pre-trained language model, and mean(·) is the average pooling, which is used to reduce the dimension of the feature representation of the character relation label to the dimension of the feature representation of the character relation; represents the feature representation of the i-th character relation label; S502. Calculate the relationship between two characters Convolutional layer Calculate the distance between the feature representation of each character relationship in the feature representation matrix and the feature representation of the character relationship label where dis ijk represents the L2 distance between the relationship feature representation between character i and character j and the feature representation of the k-character relationship label; The predicted output y of the character relationship feature representation matrix ijk is as follows: y ijk = softmax(-dis ijk ); S503. The loss function uses cross-entropy loss L, which is defined as: Among them, is a supervision signal; m is the number of character relationship tags.
7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the named entity recognition method for extracting fragment boundaries and adjacent fragment gap features based on max pooling according to any one of claims 1 to 6.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the named entity recognition method for extracting fragment boundaries and adjacent fragment gap features based on max pooling according to any one of claims 1 to 6.