A sequence labeling optimization method, system, computer device and medium

Abstract

The application provides a sequence labeling optimization method and system, computer equipment and medium, and belongs to the technical field of computer application and natural language processing. The method comprises the following steps: S1, preprocessing a text data set to construct a preprocessed data set; S2, obtaining a word vector in the preprocessed data set; S3, inputting the obtained word vector into a label classifier and two different offset classifiers at the same time to obtain a label sequence and an offset sequence respectively; S4, extracting an entity span set based on the obtained label sequence; S5, based on the obtained offset sequence, performing smoothing processing, smoothing noise in the offset sequence, and extracting a candidate span set; S6, filtering out low-quality candidate spans through an intersection-over-union strategy to obtain filtered candidate spans; and S7, updating corresponding entity spans in the label sequence based on the filtered candidate spans. The application can improve the accuracy of named entity recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer application and natural language processing, and particularly relates to a sequence labeling optimization method and system, a computer device and a medium. BACKGROUND

[0002] Named entity recognition (NER) is an important basic task in the field of natural language processing (NLP). The purpose of named entity recognition is to identify the type and boundary of an entity in a given text. The entity type mainly includes time, name, place name, organization name and the like. The performance of named entity recognition directly affects the performance and efficiency of downstream tasks such as knowledge graph, search engine and question and answer system.

[0003] The sequence labeling model can output the most probable labeling path and capture the global structure information of a sentence, and performs well in the task of named entity recognition. However, the sequence labeling model only relies on the original input sentence, and does not introduce any additional information about the entity boundary when identifying the entity type and boundary. The related research 'Sequence labeling optimization method combined with entity boundary offset' proposes a sequence labeling optimization method combined with entity boundary offset, aiming to solve the problem that the entity boundary identified by the sequence labeling model has a positional deviation from the real entity boundary. By introducing the offset of the entity boundary, the semantic dependency relationship between the entity boundary and its context is constructed, so that the entity boundary closer to the real boundary is obtained, the entity recognition result output by the sequence labeling model is optimized, and the performance of entity recognition is improved. However, in the process of generating the offset sequence, it may be disturbed by noise, resulting in deviation of the fluctuation of the offset sequence data.

[0004] To solve this problem, the present application is based on the fact that the positional offset of adjacent labels is regular and symmetric, which implies the spatial shape of the named entity, which can be used to construct the entity position dependency relationship in the sentence. Inspired by the noise elimination in image smoothing, a sliding window is designed to obtain the value in it in a specified calculation manner, which is updated to the current value. By capturing the offset information around the entity boundary, the data with excessive deviation is smoothed to reduce the influence of noise and improve the accuracy of the entity boundary. SUMMARY

[0005] The purpose of the present application is to provide a sequence labeling optimization method, system, computer device and medium to solve the problem of position deviation between the sequence labeling model and the real entity boundary when identifying the entity boundary. To solve this problem, the present application introduces the concept of offset to quantify this boundary deviation and associate the position relationship between each word and the entity boundary, thereby providing more rich entity boundary information. What is particularly key is that the present application adopts a smoothing processing technology. This technology effectively reduces the interference of noise and improves the data quality by smoothing the sequence of entity boundary offsets. On this basis, candidate spans are generated according to the smoothed entity boundary offsets, and these candidate spans are further used to adjust the entity boundary in the label sequence, thereby significantly improving the accuracy of entity recognition.

[0006] To achieve the above purpose, the present application provides a sequence labeling optimization method, comprising the following steps:

[0007] S1, preprocessing the text data set to construct a preprocessed data set;

[0008] S2, obtaining the word vector in the preprocessed data set;

[0009] S3, inputting the obtained word vector into the label classifier and two different offset classifiers at the same time, and obtaining the label sequence and the offset sequence respectively;

[0010] S4, extracting the entity span set based on the obtained label sequence;

[0011] S5, based on the obtained offset sequence, smoothing processing is performed to smooth the noise in the offset sequence, and a candidate span set is extracted;

[0012] S6, filtering out low-quality candidate spans through the intersection-over-union strategy to obtain filtered candidate spans;

[0013] S7, updating the corresponding entity span in the label sequence based on the filtered candidate spans.

[0014] Preferably, in step S2, the word vector in the preprocessed data is obtained, and the specific operation is as follows:

[0015] S21, for the sentence in the preprocessed data set, given the input sequence X, using BERT to obtain the deep semantic features in the input sequence X to obtain the word embedding representation of each word wherein, represents the word vector of the i-th word;

[0016] S22, using BiLSTM to further process the output of BERT to capture the bidirectional semantic dependency in the sentence to obtain the context representation H={h1, h2,..., hn}, wherein h i ∈ R d represents a word vector, and d represents the dimension of the word vector.

[0017] The formal representation of steps S21 and S22 is as follows:

[0018]

[0019] wherein, respectively represent the BiLSTM forward and backward hidden states; and [; ] represents a splicing operation.

[0020] Preferably, in step S3, the obtained word vector is input into a label classifier to obtain a label sequence, and the specific operation is as follows:

[0021] The word vector h i is input into the label classifier, the word vector h i is converted from a high dimension to a dimension corresponding to the number of labels, a probability matrix of each word vector h i is obtained by using a softmax layer, and the score of the label sequence is decoded, and each word vector h i is finally mapped into a label set:

[0022]

[0023] wherein the label sequence is y, and is represented as: represents the label of the i-th word xi; MLP1 represents a multi-layer perception machine, i.e., a label classifier, which is composed of two layers of Linear and GELU activation functions.

[0024] Preferably, in step S3, the obtained word vector is input into two different offset classifiers to obtain an offset sequence, and the specific operation is as follows:

[0025] The word vector h i is input into two different offset classifiers to generate an offset of the start position of the entity and an offset of the end position of the entity, respectively, and a softmax layer is used to obtain the probability of each word vector h i relative to the offset of the entity boundary position, and finally, the highest score of the offset sequence is decoded:

[0026]

[0027] wherein, and respectively represent the offset of the start position of an entity and the offset of the end position of an entity; MLP2 and MLP3 both represent a multi-layer perception, i.e., an offset classifier, and have the same structure as MLP1.

[0028] Preferably, in step S4, the entity span set is extracted based on the obtained label sequence, and the specific operation is as follows:

[0029] The entity span set S = {s0, s1,..., sm} is extracted from the label sequence y. m} where m represents the number of spans, s q = (yb q , ye q ) represents the qth entity span, yb q and ye q represent the start position and the end position of the entity span, respectively.

[0030] Preferably, in step S5, the obtained offset sequence is smoothed to smooth the noise in the offset sequence, and the candidate span set is extracted, and the specific operation is as follows:

[0031] S51, the entity boundary offset sequence and are regarded as one-dimensional digital signals for smoothing, and the entity boundary offset sequence is smoothed by using the sliding window method;

[0032] S52, the smoothing process is regarded as a sliding window operation, and a smoothing check window (left2, left1, mid) is used to determine whether the mid in the current window needs to be smoothed, and when the smoothing check window finds abnormal data fluctuation, a smoothing calculation window (left2, left1, mid, right1, right2) is used to calculate the new value of the mid at the abnormal point;

[0033] wherein mid is the smoothing value of the window, left1 is the previous value of the smoothing value, left2 is the previous value of left1, right1 is the next value of the smoothing value, and right2 is the next value of right1;

[0034] S521, edge values are added to the beginning and end of the entity boundary offset sequence to prevent window out-of-bounds, the beginning edge value is the first value of the sequence, and the end edge value is the last value of the sequence;

[0035] S522, in the smoothing check window sliding, the window is smoothed from left to right, starting from the data at the index position 1 of the entity boundary offset sequence, and in this process, it is assumed that the data that have passed through the window are consistent with the true offset, and two variables v1 = |left1-mid| and v2 = |left2-mid| are used to determine whether the data fluctuate.

[0036] S523, if the value range of v1 is not in [0, 1] or the value of v2 is not 0 or 2, it is judged that the numerical value is abnormal, and a new smoothing value is calculated at the abnormal window using a smoothing calculation window;

[0037] It is judged whether left1±1==right1±1 is established, if the equation is established, the smoothing value is the value that makes the equation established;

[0038] If the equation is not established, the smoothing value is:

[0039]

[0040] Entity boundary offset sequence O b and O e After smoothing, a new sequence of entity boundary offsets O b′ and O e′ , O b′ and O e w extracts the entity start position and entity end position information and combines them into an offset span;

[0041] S53, set two sets B={b1, b2,..., b c} and E={e1, e2,..., e r}, which are the position index sets of offsets in O b′ and O e′ , respectively, where 0≤c≤n, 0≤r≤n;

[0042] S54, by enumerating all candidate start positions and candidate end positions in sets B and E, an offset candidate span set D={d1, d2,..., d k} of entity boundary is formed, where d w =(b w ,e j ) represents the wth enumerated candidate span, 1≤w≤c, 1≤j≤r, b w represents the wth candidate start position in the candidate span set, e j represents the jth candidate end position in the candidate span set, and k=c*r represents the number of candidate spans.

[0043] Preferably, in step S6, low-quality candidate spans are filtered out by an intersection-union ratio strategy to obtain filtered candidate spans, and the specific operation is as follows:

[0044] Calculate the intersection-union ratio:

[0045]

[0046] where min(ej min(b q ,yb w ) represents the minimum value of the end position in the candidate span d q and the entity span s w ; max(b q ,yb w ) represents the maximum value of the start position in the candidate span d q and the entity span s j ; min(b q ,yb w ) represents the maximum value of the end position in the candidate span d q and the entity span s w ; max(b q ,yb w ) represents the minimum value of the start position in the candidate span d q and the entity span s

[0047] Based on the calculated intersection over union value, the candidate span with an intersection over union value greater than 0.8 is selected as the final result of the entity span.

[0048] The application also provides a sequence labeling optimization system, comprising:

[0049] A data preprocessing module is configured to preprocess a text data set and construct a preprocessed data set.

[0050] A word vector acquisition module is configured to acquire word vectors in the preprocessed data set.

[0051] A sequence decoding and offset classification module is configured to input the word vectors into a sequence decoding module to acquire a label sequence and input the word vectors into two offset classifiers to acquire offset sequences.

[0052] An entity span and candidate span extraction module is configured to extract an entity span set based on the label sequence, perform smoothing processing on the acquired offset sequences to remove noise in the offset sequences, and then extract a candidate span set.

[0053] A candidate span filtering module is configured to filter out low-quality candidate spans through an intersection over union strategy to obtain filtered candidate spans.

[0054] An entity span updating module is configured to update corresponding entity spans in the label sequence based on the filtered candidate spans.

[0055] The application also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the sequence labeling optimization method when executing the computer program.

[0056] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is characterized in that the computer program is executed by a processor to realize the steps of the sequence labeling optimization method.

[0057] Therefore, the sequence labeling optimization method, system, computer device and medium have the beneficial technical effects that: the concept of entity boundary offset is introduced to quantify the deviation of the recognized entity boundary and the real entity boundary of the sequence labeling model in view of the context information and semantic ambiguity of the entity boundary in the text; the semantic dependency relationship between the entity boundary and its context is constructed by using the boundary offset as a connection to capture more boundary information and enhance the recognition ability of the sequence labeling model for the entity boundary; the F1 values of the method on the data sets CLUENER2020, Resume-zh and MSRA reach 80.61, 96.53 and 94.96, which verifies the universality and effectiveness of the method. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 The flowchart of the sequence labeling optimization method of the application is shown in the following.

[0059] Figure 2 The entity recognition model diagram is shown in the following.

[0060] Figure 3 The adjustment example diagram is shown in the following. DETAILED DESCRIPTION

[0061] The technical solutions of the application are further described below with reference to the drawings and examples.

[0062] Unless otherwise defined, the technical terms or scientific terms used in the application should be understood as the usual meanings understood by those skilled in the art to which the application belongs.

[0063] Example 1

[0064] As shown in the following, the flowchart of the sequence labeling optimization method of the application includes the following steps: Figure 1

[0065] S1, pre-process the text data set to construct a pre-processed data set; the pre-processing includes word segmentation and named entity annotation.

[0066] S2, obtain the word vector in the pre-processed data set, and the specific operation is as follows:

[0067] S21, for the sentence in the pre-processed data set, given an input sequence X, use BERT to obtain the deep semantic features in the input sequence X to obtain the word embedding representation of each word wherein,​ Represents the word vector of the i-th word;

[0068] S22. Use BiLSTM to further process the output of BERT and capture the bidirectional semantic dependencies in the sentence to obtain the context representation H = {h1,h2,...,h n}, where h i ∈R d Represents the word vector, d represents the dimension of the word vector;

[0069] The formal expression of step S21 and step S22 is:

[0070]

[0071] in, Represent the forward and backward hidden states of BiLSTM respectively, and [;] represents the concatenation operation.

[0072] S3. Input the obtained word vector into the label classifier and two different offset classifiers at the same time to obtain the label sequence and offset sequence respectively;

[0073] Get the tag sequence. The specific operations are as follows:

[0074] The word vector h i In the input label classifier, the word vector h i Convert from high dimension to the dimension corresponding to the number of labels, and use the softmax layer to obtain the word vector h for each word i The probability matrix decodes the score of the label sequence and converts the word vector h of each word i Finally mapped to a label set:

[0075]

[0076] Among them, the label sequence is y, which is expressed as: Represents the i-th word x i label; MLP1 represents a multi-layer perceptron, i.e., a label classifier, which consists of two layers of Linear and GELU activation functions.

[0077] Get the offset sequence. The specific operations are as follows:

[0078] The word vector h i Input into two different offset classifiers to generate the offset of the entity start position and the offset of the entity end position respectively, and use the softmax layer to obtain the word vector h of each word i The probability of the offset relative to the entity boundary position, and finally, the highest scoring offset sequence is decoded:

[0079]

[0080] in, and They represent the offset of the entity's starting position and the offset of the entity's ending position respectively; MLP2 and MLP3 both represent multi-layer perceptrons, i.e., offset classifiers, and their structures are consistent with MLP1.

[0081] S4. Extract the entity span set based on the obtained tag sequence. The specific operations are as follows:

[0082] Extract entity span set S = {s0,s1,...,s m}, where m represents the number of spans, s q =(yb q ,ye q ) represents the span of the qth entity, yb q and ye q Respectively represent the starting and ending positions of the entity span.

[0083] S5. Based on the obtained offset sequence, smoothing processing is performed to smooth the noise in the offset sequence and extract the candidate span set. The specific operations are as follows:

[0084] S51, the entity boundary offset sequence and It is regarded as a one-dimensional digital signal for smoothing, and the entity boundary offset sequence is smoothed using the sliding window method;

[0085] S52, the smoothing process is regarded as a sliding window operation, and the smoothing check window (left2, left1, mid) is used to determine whether mid in the current window needs to be smoothed. When the smoothing check window finds abnormal data fluctuations, the smoothing calculation window (left2, left1, mid, right1, right2) is used to calculate the new value of mid at the abnormal point;

[0086] Where mid is the smoothed value of the window, left1 is the previous value of the smoothed value, left2 is the previous value of left1, right1 is the next value of the smoothed value, and right2 is the next value of right1.

[0087] S521. Add edge values ​​to the beginning and end of the entity boundary offset sequence to prevent the window from crossing the boundary. The beginning edge value is the first value of the sequence, and the ending edge value is the last value of the sequence.

[0088] S522, in the smooth check window sliding, the window is smooth from left to right, starting from the data of the entity boundary offset sequence index position 1, in the process, it is assumed that the data that has passed through the window are consistent with the true offset, two variables v1 = |left1-mid| and v2 = |left2-mid| are used to determine whether the data fluctuates;

[0089] S523, when the value range of v1 is not in [0, 1] or the value of v2 is not 0 or 2, it is judged as a numerical anomaly, and a smooth calculation window is used to calculate the new smooth value at the abnormal window;

[0090] It is judged whether left1±1==right1±1 is established, if the equation is established, the smooth value is the value that makes the equation established;

[0091] If the equation is not established, the smooth value is:

[0092]

[0093] Entity boundary offset sequence O b And O e After smoothing, a new sequence of entity boundary offsets O b′ And O e′ , O b′ And O e′ Extract the entity start position and entity end position information, and combine them into offset span;

[0094] S53, let two sets B = {b1, b2,..., b c} and E = {e1, e2,..., e r} be the position index set of the offset in O b′ And O e′ , respectively, where 0≤c≤n, 0≤r≤n;

[0095] S54, by enumerating all candidate start positions and candidate end positions in sets B and E, the offset candidate span set D = {d1, d2,..., d k} of the entity boundary is formed, where d w =(b w ,e j ) represents the wth enumerated candidate span, 1≤w≤c, 1≤j≤r, b w represents the wth candidate start position in the candidate span set, e j represents the jth candidate end position in the candidate span set, and k = c*r represents the number of candidate spans.

[0096] S6, filter out low-quality candidate spans through intersection-over-union strategy to obtain filtered candidate spans, and the specific operation is as follows:

[0097] Calculate intersection-over-union:

[0098]

[0099] Wherein, min(e j ,ye q ) represents the minimum value of the end position of the candidate span d w and the entity span s q ; max(b w ,yb q ) represents the maximum value of the start position of the candidate span d w and the entity span s q ; max(e j ,ye q ) represents the maximum value of the end position of the candidate span d w and the entity span s q ; min(b w ,yb q ) represents the minimum value of the start position of the candidate span d w and the entity span s q ;

[0100] Based on the calculated intersection-over-union value, the candidate span with an intersection-over-union value greater than 0.8 is selected as the final result of the entity span.

[0101] S7, based on the filtered candidate span, update the corresponding entity span in the label sequence, and the specific operation is as follows:

[0102] S71, create an empty new label sequence with the same length as the label sequence. For each candidate span, find its corresponding position in the new label sequence. Update the label at this position to the entity category label corresponding to the candidate span, and update the rest of the positions to non-entity "O".

[0103] S72, finally, output an updated label sequence.

[0104] The application will be further described below through specific examples.

[0105] As Figure 2The entity recognition model graph is shown. First, the abstract semantic representation of each word in the sentence is obtained by pre-training the language model, and the context semantic information of the sentence is obtained by using the context encoder to obtain the word vector. Then, the word vector is used to generate a label sequence and an offset sequence. Then, the boundary information in the offset sequence is combined after smoothing to generate candidate spans, and low-quality candidate spans are filtered out by the method of intersection over union, and finally the candidate spans that are most likely to represent the entity boundary are left. Finally, the candidate spans are used to update the corresponding entity span in the label sequence.

[0106] As Figure 3 The adjustment example of the invention is shown. The input sequence "Nanjing City Yangtze River Bridge" contains two entities "Nanjing City" and "Yangtze River Bridge", and the true label should be {B, I, E, B, I, I, E}. However, the boundaries of the two entities are ambiguous in the semantic level, and the end boundary "city" of the entity "Nanjing City" and the start boundary "long" of the entity "Yangtze River Bridge" can be combined to form a new semantic "city long". According to the context information in the sentence, the word "Nanjing" can also be combined with "city long", further complicating the determination of the boundary. When predicting the entity boundary, the sequence labeling model is affected by the semantic ambiguity relationship and the complex context information, and the model incorrectly labels "Nanjing City" and "Yangtze River Bridge" as entities {B, I, I, E, B, I, E}, resulting in a deviation Δ in the division of the entity boundary. The invention introduces the concept of entity boundary offset to quantify this boundary deviation. The start boundary or end boundary of the entity is regarded as the starting reference point of the boundary offset, and the index offset of the word distance from the starting point is calculated as the quantitative expression of the position characteristics of each word and the entity boundary, thereby establishing the semantic dependence relationship between the entity boundary and its context. "Nan" and "long" are the start boundaries of the entities "Nanjing City" and "Yangtze River Bridge", and their offsets are the starting points B. The offset of the non-boundary word is the minimum value of the absolute index difference between the entity boundary "Nan" or "long", so the offset sequence {B, B-1, B+1, B, B-1, B-2, B-3} is obtained, where the index difference between the non-boundary word "city" and the entity boundary "Nan" and "long" is -2 and 1 respectively, and the minimum value of the absolute index difference is 1, so the boundary offset representing its position is B+1. Similarly, taking the entity end word "city" and "bridge" as the starting point, the entity end offset sequence {E+2, E+1, E, E-1, E+2, E+1, E} can be obtained. The offset sequence can provide more entity boundary features for the model, and clearly define the word boundary to avoid ambiguous boundaries caused by semantic ambiguity.

[0107] The embodiment is experimented on three Chinese data sets of CLUENER2020, Resume-zh and MSRA. The CLUENER2020 data set is a Chinese data set screened based on an open source text classification data set THUCNEWS.

[0108] The Resume-zh is a Chinese entity recognition data set for resume.

[0109] The MSRA is a Chinese named entity recognition data set for news field.

[0110] Since the data sets CLUENER2020 and MSRA are not divided into verification sets, 10% of the training set is randomly selected to form a verification set to adjust the model parameters.

[0111] The evaluation indexes used in the experiment include accuracy P, recall R and measure value F1. Among them, P represents the ratio of the correctly recognized entities to all recognized entities, R represents the ratio of the correctly recognized entities to the entities that should be recognized, and F1 is a comprehensive evaluation index combining P and R. In order to verify the effectiveness of the model, the model proposed in the application is compared with other models on different data sets, and the comparison results are shown in Table 1.

[0112] Table 1 Comparison of experimental results of each model on different data sets Unit: %

[0113]

[0114] The application is innovated on the basis of the paper “Sequence labeling optimization method combined with entity boundary offset”. The smoothing module is added on the basis of the original text. The generated offset sequence may be disturbed by noise, resulting in data fluctuation and deviation. Because the position offsets of adjacent labels are regular and symmetrical, they imply the spatial shape of the named entity, which can be used to construct the entity position dependency relationship in the sentence. Therefore, an effective boundary offset updating algorithm and smoothing algorithm are designed with respect to the true offset, so as to reduce the influence of noise and improve the regression accuracy of the entity boundary.

[0115] Embodiment two

[0116] A sequence labeling optimization system comprises:

[0117] A data preprocessing module is configured to preprocess a text data set and construct a preprocessed data set.

[0118] A word vector acquisition module is configured to acquire word vectors in the preprocessed data set.

[0119] A sequence decoding and offset classification module is configured to input the word vectors into a sequence decoding module to acquire a label sequence, and input the word vectors into two offset classifiers to acquire offset sequences.

[0120] The entity span and candidate span extraction module extracts an entity span set based on the label sequence, and performs smoothing processing on the obtained offset sequence, filters out noise in the offset sequence, and then extracts a candidate span set;

[0121] The candidate span filtering module filters out low-quality candidate spans through an intersection-over-union strategy to obtain filtered candidate spans.

[0122] The entity span updating module updates the corresponding entity span in the label sequence based on the filtered candidate spans.

[0123] If the above functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered a list of executable instructions for implementing logic functions, which can be specifically embodied in any computer-readable medium for use by or in conjunction with an instruction execution system, device or apparatus, such as a computer-based system, a system including a processor or other system that can fetch and execute instructions from an instruction execution system, device or apparatus. For the purpose of this specification, the "computer-readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by or in conjunction with an instruction execution system, device or apparatus, or in conjunction with these instruction execution systems, devices or apparatus.

[0125] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for instance via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then suitable for use by the computer. Therefore, the computer-readable medium can be paper or another suitable medium that can act to transfer a program for implementation by a computer.

[0126] It is worth noting that the content not elaborated in the present application is all prior art, which is well known to those skilled in the art.

[0127] Therefore, the present application adopts the above-mentioned sequence labeling optimization method, system, computer device and medium, which can improve the accuracy of named entity recognition.

[0128] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application but not to limit it, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the present application, and these modifications or equivalent replacements also cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. A sequence labeling optimization method, characterized in that, The method comprises the following steps: S1, preprocessing the text data set to construct a preprocessed data set; S2, obtaining word vectors in the preprocessed data set; S3, inputting the obtained word vectors into a label classifier and two different offset classifiers to obtain a label sequence and an offset sequence, respectively; S4, extracting an entity span set based on the obtained label sequence; S5, based on the obtained offset sequence, performing smoothing processing to smooth the noise in the offset sequence, and extracting a candidate span set; S6, filtering out low-quality candidate spans through an intersection-over-union strategy to obtain filtered candidate spans; S7, updating the corresponding entity span in the label sequence based on the filtered candidate spans. Based on the obtained offset sequence, the noise in the offset sequence is smoothed, and a candidate span set is extracted, and the specific operations are as follows: S51, the entity boundary offset sequence is smoothed and The entity boundary offset sequence is smoothed by a method of a sliding window, as a one-dimensional digital signal S52, consider the smoothing process as a sliding window operation, use a smoothing check window to decide whether to smooth in the current window or not, when the smoothing check window finds abnormal data fluctuation, use a smoothing calculation window to calculate the new value at the abnormal point ; wherein, is a smoothed value of the window, is a previous value of the smoothed value, is a previous value of is a previous value of is a next value of the smoothed value, is a next value of is a next value of S521, an edge value is added to the beginning and end of the entity boundary offset sequence to prevent window out-of-bounds, the beginning edge value is the first value of the sequence, and the end edge value is the last value of the sequence; S522, in the smooth check window sliding, the window is smooth from left to right, starting from the data at the entity boundary offset sequence index position 1, in the process, it is assumed that the data that has passed through the window are consistent with the true offset, and two variables are used to determine whether the data fluctuates and two variables to determine whether the data fluctuates; S523、When the value range is not in [0, 1] or the value is not 0 or 2, it is judged as a numerical anomaly, and a new smooth value is calculated at the abnormal window using a smoothing calculation window. determining whether the equation holds, and if the equation holds, the smoothed value is the value that makes the equation hold; If the equation is not established, the smoothing value is: ； Entity boundary offset sequence and After smoothing, a new entity boundary offset sequence is obtained and From and Extract entity start position and entity end position information, and combine them into offset spans; S53, set two sets and respectively and the position index set with offset 0 in and, wherein , ; S54, enumerate the set with all candidate start positions and candidate end positions in the entity boundary, a set of offset candidates wherein represents the th enumerated candidate offset, , , represents the th candidate start position in the set of candidate offsets, represents the th candidate end position in the set of candidate offsets, represents the number of candidate offsets.

2. The sequence labeling optimization method of claim 1, wherein, In step S2, the word vectors in the preprocessed data are obtained, and the specific operations are as follows: S21, for sentences in the preprocessed dataset, given an input sequence , using BERT to obtain the semantic features of the input sequence deep, get the word embedding representation of each word , wherein, represents the word vector of the first word S22, further processing the output of BERT using BiLSTM to capture the bidirectional semantic dependencies in the sentence to obtain context representation wherein, denotes a word vector, denotes the dimension of the word vector; The formal representation of steps S21 and S22 is as follows: ； ； ； ； wherein, , denote BiLSTM forward and backward hidden states, respectively; [;] denotes a concatenation operation, denotes a BiLSTM model.

3. The sequence labeling optimization method of claim 2, wherein, In step S3, the obtained word vectors are input into the label classifier to obtain the label sequence, and the specific operations are as follows: Convert the word vectors In the input tag classifier, the word vectors Convert from high dimension to dimension corresponding to the number of tags, using The layer obtains the word vector of each word The probability matrix of the word vector of each word is decoded to score the label sequence, and the word vector of each word Finally, map to a label set: ； Wherein, the label sequence is , is expressed as: , indicates the label of the th word ; indicates a multi-layer perception, that is, a label classifier, which is composed of two layers of Linear and GELU activation function.

4. The sequence labeling optimization method of claim 3, wherein, In step S3, the obtained word vectors are input into two different offset classifiers to obtain the offset sequence, and the specific operations are as follows: word vectors into two different offset classifiers, one for the start position of the entity and one for the end position of the entity, and use layer to obtain word vectors for each word the probabilities of the offsets relative to the entity boundary positions, and finally, decode the highest-scoring offsets sequence: ； ； wherein, and respectively represent an offset of an entity start position and an offset of an entity end position; and both represent a multi-layer perceptron, i.e., an offset classifier, whose structure is identical to both.

5. The sequence labeling optimization method of claim 4, wherein, In step S4, the entity span set is extracted based on the obtained label sequence, and the specific operations are as follows: From the tag sequence Extract entity span sets from ,in, Indicates the number of spans, Indicates the entity span, and Respectively represent the starting and ending positions of the entity span.

6. The sequence labeling optimization method of claim 5, wherein, In step S6, low-quality candidate spans are filtered out through an intersection-over-union strategy to obtain filtered candidate spans, and the specific operations are as follows: Calculate the intersection-over-union ratio: ； wherein, denotes the minimum of the end positions in the candidate spans and the entity span ; denotes the maximum of the start positions in the candidate spans and the entity span ; denotes the maximum of the end positions in the candidate spans and the entity span ; denotes the minimum of the start positions in the candidate spans and the entity span ; Based on the calculated intersection-over-union ratio, select the candidate span with an intersection-over-union ratio greater than 0.8 as the final result of the entity span.

7. A sequence labeling optimization system, comprising: For implementing the sequence labeling optimization method according to any one of claims 1-6, comprising: a data preprocessing module for preprocessing the text data set to construct a preprocessed data set; a word vector obtaining module for obtaining word vectors in the preprocessed data set; a sequence decoding and offset classification module for inputting the word vectors into a sequence decoding module to obtain a label sequence, and inputting the word vectors into two offset classifiers to obtain an offset sequence; an entity span and candidate span extraction module for extracting an entity span set based on the label sequence, and performing smoothing processing based on the obtained offset sequence to smooth the noise in the offset sequence, and then extracting a candidate span set; a candidate span filtering module for filtering out low-quality candidate spans through an intersection-over-union strategy to obtain filtered candidate spans; an entity span updating module for updating the corresponding entity span in the label sequence based on the filtered candidate spans.

8. A computer device comprising: a memory and a processor; The memory stores a computer program, and the processor executes the computer program to implement the steps of the sequence labeling optimization method according to any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the sequence labeling optimization method according to any one of claims 1-6.

Images