A Method and System for Semantic Enhancement of Nested Named Entities Based on Edge Gradient

Abstract

The present invention belongs to the field of natural language processing, and specifically relates to a method and system for enhancing the semantics of nested named entities based on edge gradients. Step 1: Preprocess the text dataset, that is, process the original data to be suitable for processing by the entity model; Step 2: Input the sentences preprocessed in Step 1 into the model to obtain context information features; Step 3: Two-dimensionalize the sentences with the context information features in Step 2; Step 4: Combine the two-dimensionalized sentences in Step 3 with the method of per-channel convolution and edge gradients to enhance the semantics, and use pointwise convolution to perform spatial connection on the sentence features after enhancing the semantics to obtain high-level features; Step 5: The high-order features obtained in Step 4 enter the multi-layer perceptron and are encoded with the shallow information formed by bi-affine, and Softmax and Argmax are used to predict the classification and return the index value. It is used to solve the problems in the prior art that semantic information will penetrate into the representations of adjacent nested entities; there is less supervision in boundary detection, resulting in generally poor performance when determining the boundaries of named entities; and there will be noise interference when identifying nested named entities.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing, and particularly relates to a nested named entity semantic enhancement method and system based on edge gradient. Background Art

[0002] Finding useful information manually from a large amount of text is a time-consuming and laborious task. In this situation, the research on information extraction has emerged as the times require. Named entity recognition is a basic task in the field of information extraction, and its main purpose is to identify entities with practical meanings such as person names, place names, and organization names in the text. With the improvement of computer computing power and recognition requirements, the methods for completing the entity recognition task have shifted from traditional methods based on dictionaries and statistics to deep learning-based methods that combine attention mechanisms and transfer learning.

[0003] Natural language has a complex semantic structure, and there is a special type of named entity - nested named entity in named entities. For example, "Nanjing Yangtze River Bridge" is a complete named entity, but in this entity, "Nanjing City", "Yangtze River", and "Yangtze River Bridge" can all be classified as entity types. Entity nesting is a common phenomenon in NLP. The nested named entity structure is complex and variable, and the nesting granularity and nesting level lack regularity. Since the semantic information between entities at each layer is shared, the existence of nested named entities spreads the semantic information to adjacent entities. When identifying nested named entities, most of the spread semantic information represents the noise range, and the lack of specific semantics will lead to false positive errors of entities. Therefore, studying the solution to this problem is of great significance for the application of named entity recognition.

[0004] The research work of traditional named entity recognition usually regards it as a sequence labeling task (entity start (B), entity middle and end (I), and non-entity (O)), and assigns a single entity label to each word in the text to achieve the extraction of named entities. For example, "Nanjing Yangtze River Bridge" is assigned the label "B-LOC, I-LOC, I-LOC, I-LOC, I-LOC, I-LOC, I-LOC" and classified as a place name. However, the sequence annotation method can only assign one label to the same entry and cannot solve the multi-label problem.

[0005] From the perspective of multi-labeling, the current research attempts to transform the multi-labeling problem into a single-labeling problem by assigning multiple labels to the same lexical entry. Ju et al. proposed a hierarchical BiLSTM-CRF model that dynamically stacks flat NER layers to identify nested named entities and uses sufficient internal named entity encoding information to identify external named entities. There is error propagation between layers in this idea. When identifying external entities first, internal entities cannot be detected. Compared with the multi-angle method, the span idea does not have the problem of error propagation. Eberts proposed a local detection method that directly classifies all subsequences of a sentence by encoding each subsequence into a fixed-size representation. The key idea is to enumerate all possible potential entities mentioned and classify them using a deep neural network. However, most of the extracted fragments are non-entities and lack precise boundary information. But these methods all have the following problems: 1. Semantic information penetrates into the representations of adjacent nested entities; 2. Less supervision in boundary detection leads to generally poor performance in determining the boundaries of named entities; 3. There is inevitable noise interference when identifying nested named entities. Summary of the Invention

[0006] The present invention provides a method for enhancing the semantics of nested named entities based on edge gradients, which makes up for the shortcomings of traditional span models and avoids the noise problems generated by traditional methods, thereby improving the performance of nested named entity recognition.

[0007] The present invention provides a system for enhancing the semantics of nested named entities based on edge gradients to solve the problems in the prior art that semantic information will penetrate into the representations of adjacent nested entities; less supervision in boundary detection leads to generally poor performance in determining the boundaries of named entities; there is noise interference when identifying nested named entities.

[0008] The present invention is realized through the following technical solutions:

[0009] A method for enhancing the semantics of nested named entities based on edge gradients, the method comprising the following steps:

[0010] Step 1: Preprocess the text data set, that is, process the original data to be suitable for processing by the entity model;

[0011] Step 2: Input the sentences preprocessed in Step 1 into the model to obtain context information features;

[0012] Step 3: Two-dimensionalize the sentences with the context information features in Step 2;

[0013] Step 4: Combine the two-dimensionalized sentences in Step 3 with the method of per-channel convolution and edge gradients to enhance the semantics; use pointwise convolution on the sentence features with enhanced semantics to obtain high-order features;

[0014] Step 5: The high-order features obtained in Step 4 enter the multi-layer perceptron and are encoded with the shallow information formed by the bi-affine. Softmax and Argmax are used to predict the classification and return the index value, completing the screening of candidate entities.

[0015] A method for enhancing the semantics of nested named entities based on edge gradients, where Step 1 is specifically as follows:

[0016] Mark the positions of entities in the sentence, from the start to the end position, and record the entity types with type. The sentence structure is obtained through pre-training.

[0017] A method for enhancing the semantics of nested named entities based on edge gradients, where Step 2 is specifically as follows:

[0018] Vectorize the text as the input to the network; use a pre-trained model to vector-encode the semantics of each token in the sentence, and use BiLSTM to extract the context information features of the text, obtaining a one-dimensional sentence word vector L = [L1, L2, ···, L N

[0019] A method for enhancing the semantics of nested named entities based on edge gradients, where Step 3 is specifically as follows:

[0020] Obtain the two-dimensional sentence matrix L m , the two-dimensional sentence position matrix D m , the two-dimensional sentence triangular matrix T m from the one-dimensional sentence word vector L through cross mapping. Connect these matrices to fuse information and use linear projection to form a sequential representation M = Linear([L m ; D m ; T m )

[0021] A method for enhancing the semantics of nested named entities based on edge gradients, where Step 4 is specifically as follows:

[0022] Using the method of edge gradients, after enhancing the semantics in the per-channel convolutional network using the edge detection operator as the kernel, and using pointwise convolution for spatial connection to model the correlation of information in the same space for different channels, it is defined as:

[0023] Per-channel convolution and gradient operator:

[0024] Pointwise convolution: f (p) (M 1:n ; x p ) = x p * f (d) ​

[0025] x - direction: f (x) (M 1:n ; G x , x p ) = f (p) (f (d) (M i ; G x ) ; x p )

[0026] y - direction: f (y) (M 1:n ; G y , x p ) = f (p) (f (d) (M i ; G y ) ; x p )

[0027]

[0028] Gradient calculation: G ≈ |f (x) | + |f (y) |

[0029] where Gx and Gy are the first - order operators of the edge detection operator in two directions. In the second - order operator, there is no direction distinction. Since the Laplace operator already has a gradient, the operator convolution can be directly used.

[0030] A method for enhancing the semantics of nested named entities based on edge gradients, specifically, step 5 is as follows:

[0031] The shallow information obtained by the bi - affine classifier and the detailed information of the high - level feature reconstruction after passing through the multi - layer perceptron are used to calculate the entity label distribution of each span through the Softmax layer and the Argmax layer to complete the screening of candidate entities.

[0032] A system for enhancing the semantics of nested named entities based on edge gradients, the system includes a pre - processing unit, a sentence two - dimensionalization unit, an edge gradient extraction and high - order feature unit, and an entity screening unit;

[0033] The pre - processing unit: pre - processes the text data set, that is, extracts entities, their attributes, and sentences from the text data set;

[0034] The sentence two - dimensionalization unit: based on the extracted entities, their attributes, and sentences, obtains context information features; and performs two - dimensionalization processing on the sentences;

[0035] The edge gradient extraction and high - order feature unit: combines the two - dimensionalized sentences through per - channel convolution and the method of edge gradients to enhance semantics; and uses point - wise convolution on the enhanced semantic sentence features to obtain high - order features;

[0036] The entity screening unit: After the high-order features enter the multi-layer perceptron, they are encoded with the shallow information formed by the bi-affine. The Softmax and Argmax are used to predict the classification and return the index value to complete the screening of candidate entities.

[0037] An electronic device includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete the communication with each other through the communication bus;

[0038] The memory is used to store computer programs;

[0039] The processor is used to implement the above method steps when executing the program stored on the memory.

[0040] A computer-readable storage medium stores a computer program therein, and when the computer program is executed by a processor, the above method steps are implemented.

[0041] The beneficial effects of the present invention are:

[0042] Adopting the technical solution of the present invention, on the basis of the convolutional neural network, the method of combining the per-channel neural network and the gradient detection edge is used. While giving full play to the characteristics of the neural network to automatically extract high-dimensional abstract features, the traditional edge detection operator is integrated into the popular convolutional operation in the modern CNN to enhance the performance of the entity semantic task.

[0043] Experiments show that compared with enhancing entity boundaries in the past, the method proposed by the present invention effectively utilizes the neural network integrated gradient detection edge. The main reason for the improvement of the model performance is the enhancement of the gradient operator on the boundary where the entity is located. In addition, the gradient operator can be easily integrated into any 2D CNN model. The method proposed by the present invention integrates the traditional edge detection operator into the convolutional operation in the CNN, breaking through the optimization of the CNN kernel starting from random initialization, and the lack of explicit encoding of gradient information makes it difficult to focus on edge-related features. Excellent results have been achieved in entity recognition and good application effects have been obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic flowchart of the method of the present invention.

[0045] Figure 2 It is a model diagram of the present invention.

[0046] Figure 3 It is a detailed diagram of the gradient algorithm integrated with the CNN of the present invention.

[0047] Figure 4 It is a schematic diagram of entity recognition of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0049] Fully utilize the entity boundaries with finer granularity, adopt a gradient-enhanced semantic strategy, introduce edge detection operators, combine the characteristics of automatically extracting high-dimensional abstract features by neural network layering, use a semantic convergence operator of a deep neural network and local semantic features to extract high-level semantic features, and use a bi-affine classifier to extract shallow information, and divide the span into corresponding semantic labels to corresponding two-dimensional table positions.

[0050] A method for enhancing the semantics of nested named entities based on edge gradients, the method comprising the following steps:

[0051] Step 1: Preprocess the text dataset (the text dataset is GENIA and RESUME), that is, process the original data to be suitable for processing by the entity model;

[0052] Step 2: Input the sentences preprocessed in Step 1 into the model to obtain context information features;

[0053] Step 3: Two-dimensionalize the sentences of the context information features in Step 2;

[0054] Use a matrix encoder to learn the representation of the two-dimensional sentence matrix M, where the vector in the i-th row and j-th column corresponds to the span of the i-th to j-th words of the input sentence. Through cross mapping, the one-dimensional sentence word vector L is cross-mapped to obtain the two-dimensional sentence matrix L m . To effectively distinguish the positions of words in the table and the positions in different regions of the triangle, one-dimensional position embeddings and triangular position embeddings are obtained through cross mapping to obtain the two-dimensional sentence position matrix D m and the triangular table T m . We concatenate these matrices to fuse information and use linear projection to form a sequential representation M = Linear([L m ; D m ; T m ).

[0055] Step 4: Combine the two-dimensionalized sentences in Step 3 with edge gradients through per-channel convolution to enhance semantics; use pointwise convolution on the sentence features with enhanced semantics to obtain high-order features;

[0056] Step 5: The high-order features obtained in Step 4 enter a multi-layer perceptron and are encoded with the shallow information formed by the bi-affine. Softmax and Argmax are used to predict the classification and return the index value to complete the screening of candidate entities.

[0057] A method for enhancing the semantics of nested named entities based on edge gradients. In Step 1, the text dataset is preprocessed, that is, the original data is processed to be suitable for processing by the entity model. Specifically:

[0058] Mark the positions of the entities in the sentence, from the start to the end position, and use type to record the entity type. The sentence structure can be obtained through pre-training.

[0059] Taking the GENIA dataset as an example, the form is as follows: [{"sentence": ["GATA-1", "and", "ER", "bind", "to", "each", "other", "in", "vitro", "in", "the", "absence", "of", "DNA."], "ner": [{"index": [2], "type": "protein"}, {"index": [0], "type": "protein"}]}]. This method enables the neural network to obtain the entity matrix and the position matrix (from the start to the end of the entity), and then obtain the sentence structure information with the entity as the feature in the sentence.

[0060] A method for enhancing the semantics of nested named entities based on edge gradients. In Step 2, based on the entities extracted in Step 1 and their entity attributes and sentences, context information features are obtained. Specifically:

[0061] Vectorize the text as the input of the network; use a pre-trained model to perform vector encoding on the semantics of each token in the sentence, and use BiLSTM to extract the context information features of the text to obtain the one-dimensional sentence word vector L = [L1, L2, ···, L N

[0062] A method for enhancing the semantics of nested named entities based on edge gradients. In Step 3, the sentence with entity attributes in Step 1 or the sentence with context information features in Step 2 is two-dimensionalized. Specifically:

[0063] The one-dimensional sentence word vector L is cross-mapped to obtain the two-dimensional sentence matrix L m , the two-dimensional sentence position matrix D m , the two-dimensional sentence triangular matrix T m , connect these matrices to fuse the information, and use linear projection to form a sequential representation M = Linear([L m ; D m ; T​m )

[0064] A nested named entity semantic enhancement method based on edge gradient. In step 4, the two-dimensionalized sentence is enhanced in semantics by combining per-channel convolution and edge gradient; specifically, the sentence features with enhanced semantics are used to obtain high-order features using pointwise convolution as follows:

[0065] CNN can effectively capture the semantic features of edges through the deep stacking of convolutions. However, the CNN kernel is optimized from random initialization and it is difficult to focus on edge-related features. Using the edge gradient method, the edge detection operator is used as the kernel to enhance semantics in the per-channel convolution network, and then pointwise convolution is used for spatial connection to model the correlation of information in the same space from different channels, which is defined as:

[0066] Per-channel convolution and gradient operator:

[0067] Pointwise convolution: f (p) (M 1:n ; x p ) = x p * f (d)

[0068] x direction: f (x) (M 1:n ; G x , x p ) = f (p) (f (d) (M i ; G x ) ; x p )

[0069] y direction: f (y) (M 1:n ; G y , x p ) = f (p) (f (d) (M i ; G y ) ; x p )

[0070] where M i ;

[0071] Gradient calculation: G ≈ |f (x) | + |f (y) |

[0072] where G represents an operator, Gx and Gy are the first-order operators of the edge detection operator in two directions. In the second-order operator, there is no direction distinction. Since the Laplace operator already has a gradient, two convolutions can be directly used.

[0073] A method for enhancing the semantics of nested named entities based on edge gradients. After the high-order features in step 5 enter the multi-layer perceptron, they are encoded with the shallow information formed by the bi-affine. Softmax and Argmax are used to predict the classification and return the index value to complete the screening of candidate entities. Specifically:

[0074] The shallow information obtained by the bi-affine classifier and the reconstructed detailed information of the high-order features after passing through the multi-layer perception. Under the constraint of the two, a linear transformation is performed on the unit representation to calculate the entity label distribution of each span. Usually, we predict the entity label of each span by selecting the category with the highest probability, that is, by calculating the entity label distribution of each span through the Softmax layer and the Argmax layer to complete the screening of candidate entities.

[0075] The method of the present invention mainly studies the edge gradient phenomenon in the two-dimensional representation of sentences. The method proposes a gradient operator for traditional CNNs and enhances the semantic representation of spans. The method uses the information of adjacent elements for weighting, which can not only calculate the semantic gradient but also eliminate noise. Its advantage is that it enhances the semantic information of the neighborhood, thereby enhancing the semantic span representation in the two-dimensional sentence representation.

[0076] The present invention uses the GENIA and RESUME-ZH standard datasets to verify the effectiveness of the above method. Through the preprocessed dataset, the obtained dataset samples are vectorized to obtain L. Through the constructed edge gradient network, a semantic convergence operator is used in the two-dimensional sentence matrix to replace the convolution weights in the deep convolution to strengthen the span edges, realizing an effective enhancement of the semantics of nested multi-layer named entities. The performance of the bi-affine is significantly better than that of the directly connected LSTM output pair. We use the bi-affine to score all possible sentence fragments after the BiLSTM. Through the shallow information obtained by the bi-affine and the reconstructed detailed information of the high-order features, feature fusion is achieved to improve the classification accuracy of the target area. To verify the effectiveness of the method, the evaluation criteria are accuracy (P), recall (R), and F1 value, where F = 2 * P * R / (P + R).

[0077] A system for enhancing the semantics of nested named entities based on edge gradients, the system includes a preprocessing unit, a sentence two-dimensionalization unit, an edge gradient extraction high-order feature unit, and an entity screening unit;

[0078] The preprocessing unit: preprocesses the text dataset (the text dataset is GENIA and RESUME), that is, extracts entities, their attributes, and sentences from the text dataset;

[0079] The sentence two-dimensionalization unit: based on the extracted entities, their attributes, and sentences, obtains context information features; and performs two-dimensionalization processing on the sentences;

[0080] Learn the representation of the two-dimensional sentence matrix M using a matrix encoder, where the vector in the i-th row and j-th column corresponds to the span of the i-th to j-th words of the input sentence. Through cross-mapping, the one-dimensional sentence word vector L is obtained as the two-dimensional sentence matrix L through cross-mapping. m . To effectively distinguish the positions of words in the table and the positions in different regions of the triangle, a one-dimensional position embedding and a triangular position embedding are obtained through cross-mapping to obtain the two-dimensional sentence position matrix D. m and the triangular table Tm. We concatenate these matrices to fuse information and use linear projection to form the sequential representation M = Linear([L m ; D m ; T m ).

[0081] The edge gradient extraction high-order feature unit: The two-dimensionalized sentence is enhanced in semantics by combining per-channel convolution with the edge gradient; the sentence features after enhanced semantics are used to obtain high-order features using pointwise convolution;

[0082] The entity screening unit: After the high-order features enter the multi-layer perceptron, they are encoded with the shallow information formed by the bi-affine, and the Softmax and Argmax are used to predict the classification and return the index value to complete the screening of candidate entities.

[0083] An electronic device includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0084] The memory is used to store computer programs;

[0085] The processor is used to implement the above method steps when executing the programs stored on the memory.

[0086] A computer-readable storage medium stores a computer program therein, and the computer program implements the above method steps when executed by a processor.

[0087] The following is further described in conjunction with embodiments:

[0088] To implement the method of the present invention, first perform step 1 for data preprocessing. Process it into [{"sentence": ["GATA-1", "and", "ER", "bind", "to", "each", "other", "in", "vitro", "in", "the", "absence", "of", "DNA."], "ner": [{"index": [2], "type": "protein"}, {"index": [0], "type": "protein"}]}] through the location of the entity and the entity classification type. This method enables the neural network to obtain the entity matrix and the position matrix (from the start to the end of the entity), and then obtain the sentence structure information featuring the entity in the sentence;

[0089] After data preprocessing, perform step 2 for text vectorization.

[0090] BERT uses Word Piece to split words. Since the table cells are annotated at the word level, both the training and prediction processes need to represent it at the word level. Input the sentence into the BERT encoder and apply scalar mixing to the last four layers. We use a pre-trained model to encode the semantics of each token in the sentence. Specifically, consider inputting the token into BERT to obtain the token embedding sequence H = {h1, …, h n}. The last four layers are concatenated together to get B. To obtain the word embedding L with context, B is input into a bidirectional LSTM to obtain the sequence of context representations L = {L1, …, L n}.

[0091] Perform step 3 on the obtained one-dimensional vector to realize the two-dimensionalization of the sentence.

[0092] Obtain the entity representation from the matrix obtained after mapping the sentence, and construct the entity span from the matrix. Use the matrix encoder to learn the two-dimensional sentence matrix M, where the vector in the i-th row and j-th column corresponds to the span of the i-th to j-th words of the input sentence. Through cross-mapping, the obtained one-dimensional word vector L is promoted to the two-dimensional sentence matrix L m . To effectively distinguish the position of the word in the table and the positions in different regions of the triangle, the one-dimensional position embedding and the triangular position embedding are cross-mapped to obtain the two-dimensional sentence position matrix D m and the triangular matrix T m . Connect the three matrices to fuse the information and use linear projection to form M.

[0093] Secondly, perform step 4.

[0094] Convolutional neural networks (CNNs) can effectively capture the semantic features of edges through deep stacking of convolutions. However, the CNN kernel functions are optimized from random initializations and it is difficult to focus on edge-related features. On the other hand, operators such as Roberts, Sobel, and Dynamic use differential information to characterize the mutations and detailed features of edge contexts. However, these manual traditional operators are often limited by their shallow representation capabilities. Combining the properties of operators and CNNs, the edge convergence operator is used as the convolution kernel in CNNs to play the role of capturing local semantic overlaps.

[0095] To enhance the semantic of span edges, in the invention, the standard convolutional layer is replaced with a depthwise separable convolutional layer, as Figure 3 shown. Using the method of edge gradients, after enhancing the semantics in the per-channel convolutional network using the edge detection operator as the kernel, and using pointwise convolution for spatial connection, the information of different channels in the same space is modeled for correlation, defined as:

[0096] Per-channel convolution and gradient operator:

[0097] Pointwise convolution: f (p) (M 1:n ; x p ) = x p * f (d)

[0098] x direction: f (x) (M 1:n ; G x , x p ) = f (p) (f (d) (M i ; G x ) ; x p )

[0099] y direction: f (y) (M 1:n ; G x , x p ) = f (p) (f (d) (M i ; G y ) ; x p )

[0100]

[0101] Gradient calculation: G ≈ |f (x) | + |f (y) |

[0102] Among them, Gx and Gy are the first-order operators of the edge detection operator in two directions. In the second-order operator, there is no direction distinction. Since the Laplace operator already has a gradient, two convolutions can be directly used.

[0103] Finally, perform step 5 to implement nested named entity recognition.

[0104] The effective span connection in the multi-dimensional vector encodes the context information and internal structure of the span. This step can use a bilinear or bi-affine classifier to increase the dimension. The performance of the bi-affine is significantly better than that of the directly connected LSTM output pair. Use the bi-affine to score all possible sentence fragments after BiLSTM.

[0105] The shallow information obtained by the bi-affine and the high-level feature reconstruction details information are used to achieve feature fusion and improve the classification accuracy of the target area. Under the constraints of both, a linear transformation is performed on the matrix unit representation to calculate the entity label distribution of each span. Usually, the category with the highest probability is selected to predict the entity label of each span. Provide a representation of the predefined type c for each span in the mask matrix C.

[0106] The text corpus of this embodiment is sourced from the GENIA and RUSUME datasets. The GENIA dataset has been annotated according to different levels of language information and semantic information. There are 36 fine-grained entity categories in this corpus. The collection includes 32 entity categories, 92,681 mentions, 9,533 phrases, and 2,000 abstracts. It contains five entity types, including DNA, RNA, protein, cell lineage, and cell type categories. Approximately 17% of the sentences contain nested named entities. The RESUME dataset is the aggregated data of resumes, generated after filtering and manual annotation. This dataset contains 1,027 resume abstracts, and the entity annotations are divided into 8 categories, such as name, nationality, origin, ethnicity, occupation, degree, institution, and title.

[0107] The dataset is divided into a training set, a test set, and a validation set in the ratio of 8:1:1 and input into the model to obtain the experimental results. In addition, a comparison is made with the traditional model, and the data is shown in Table 1:

[0108] Table 1 Comparison of entity recognition performance under the traditional model

[0109]

[0110] The present invention has achieved state-of-the-art performance in the nested entity dataset GENIA and the flat dataset. The present invention first uses the gradient operator for NER, effectively improving the results. Compared with the previous methods for enhancing entity boundaries, the performance of this method is optimal on all datasets.

[0111] Table 2 verifies the different effects of different gradient operators on this method. Compared with other first-order operators, the Roberts operator uses a local difference method to find the entity boundary and is more accurate in determining the position of the entity boundary; for most Sobel kernel variants, the dilated operator does not bring better results, indicating that the dilated kernel function does not bring better results. The Laplace 3x3 V1 not only focuses on information in different directions but also can focus on the position of the convergent entity in a smaller local area. Comparing the manual operator and the dynamic operator, it can be seen from the results that the performance of the dynamic operator is not as good as that of the manual operator because of the universality problem of the dynamic operator.

[0112] In view of all the results of the present invention, the operator based on the second derivative has better detection effects in terms of accuracy and positioning than the operator based on the first derivative. This is because the higher the order of differentiation, the stronger the feature extraction ability. Since the convergence of the entity boundary requires attention to local features, which causes the dilated operator to miss more features, therefore, no matter whether it is a first-order operator or a second-order operator, the performance cannot be improved by expanding the kernel. In summary, the nested named entity semantic enhancement method based on edge gradient proposed by the present invention has excellent performance.

[0113] Table 2 Performance of various nested recognition models

[0114]

Claims

1. A semantic enhancement method for nested named entities based on edge gradients, characterized in that, The method includes the following steps: Step 1: Preprocess the text dataset, that is, process the original data into sentences suitable for parallel processing by the entity model; Step 2: Input the sentences preprocessed in Step 1 into the model to obtain context information features; Step 3: Two-dimensionalize the sentences with context information features in Step 2; Step 4: Combine the two-dimensionalized sentences in Step 3 with the method of per-channel convolution and edge gradient to enhance semantics; use pointwise convolution to perform spatial connection on the sentence features with enhanced semantics to obtain high-order features; Step 5: The high-order features obtained in Step 4 enter the multi-layer perceptron and are encoded with the shallow information formed by the bi-affine. Use Softmax and Argmax to predict the classification and return the index value to complete the screening of candidate entities; Specifically, Step 3 is as follows: The one-dimensional sentence word vector L is cross-mapped to obtain the two-dimensional sentence matrix L m , the two-dimensional sentence position matrix D m , the two-dimensional sentence triangular matrix T m . These matrices are concatenated to fuse information and a linear projection is used to form the sequential representation M = Linear([L m ; D m ; T m ); Specifically, Step 4 is as follows: Using the method of edge gradient, using the edge detection operator as the kernel to enhance semantics in the per-channel convolution network, and then using pointwise convolution for spatial connection to model the correlation of information in the same space of different channels, which is defined as: Channel-wise Convolution and Gradient Operator: Pointwise Convolution: f (p) (M 1:n ; x p ) = x p * f (d) x - direction: f (x) (M 1:n ; G x , x p ) = f (p) (f (d) (M i ; G x ); x p ) y - direction: f (y) (M 1:n ; G y , x p ) = f (p) (f (d) (M i ; G y ); x p ) Where M i ; Gradient calculation: G ≈ |f (x) | + |f (y) | Among them, G represents an operator, Gx and Gy are the first-order operators of the edge detection operator in two directions. In the second-order operator, there is no direction distinction. Since the Laplace operator already has a gradient, the operator convolution can be directly used.

2. The method for enhancing the semantics of nested named entities based on edge gradients according to claim 1, wherein Specifically, Step 1 is as follows: Mark the positions of entities in the sentence, from the start to the end position, and mark the entity types with type. Obtain the sentence structure through pre-training.

3. The method for enhancing the semantics of nested named entities based on edge gradients according to claim 1, wherein Specifically, Step 2 is as follows: Vectorize the text as the input to the network; use a pre-trained model to perform vector encoding on the semantics of each token in the sentence, and use BiLSTM to extract the context information features of the text to obtain a one-dimensional sentence word vector L = [L1, L2, ···, L N .

4. The method for enhancing the semantics of nested named entities based on edge gradients according to claim 1, wherein Specifically, Step 5 is as follows: The shallow information obtained by the bi-affine classifier and the high-order feature reconstruction detail information passing through the multi-layer perceptron calculate the entity label distribution of each span through the Softmax layer and the Argmax layer to complete the screening of candidate entities.

5. A nested named entity semantic enhancement system based on edge gradient, characterized in that, The system uses the nested named entity semantic enhancement method based on edge gradient as described in Claim 1. The system includes a preprocessing unit, a sentence two-dimensionalization unit, an edge gradient extraction high-order feature unit, and an entity screening unit; The preprocessing unit: Preprocess the text dataset, that is, extract entities, entity attributes, and sentences from the text dataset; The sentence two-dimensionalization unit: Based on the extracted entities, entity attributes, and sentences, obtain context information features; And perform two-dimensionalization processing on the sentences; The edge gradient extraction high-order feature unit: Combine the two-dimensionalized sentences with the method of per-channel convolution and edge gradient to enhance semantics; use pointwise convolution to perform spatial connection on the sentence features with enhanced semantics to obtain high-order features; The entity screening unit: The high-order features enter the multi-layer perceptron and are encoded with the shallow information formed by the bi-affine. Use Softmax and Argmax to predict the classification and return the index value to complete the screening of candidate entities.

6. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus; The memory is used to store computer programs; The processor is used to implement the method steps described in any one of Claims 1-4 when executing the programs stored on the memory.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1-4 are implemented.

Images