Named entity recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a named entity recognition method and device, electronic equipment and a storage medium. The named entity recognition method comprises: inputting a to-be-recognized text into a pre-trained named entity recognition model; and performing the following operations in the named entity recognition model: for each entity type, obtaining a relative position vector, a query vector and a key vector of a segmented word in the to-be-recognized text under the entity type, and obtaining global pointer information of the to-be-recognized text under the entity type based on the relative position vector, the query vector and the key vector, the global pointer information indicating a probability that the segmented word belongs to the entity type and a probability that the segmented word belongs to an entity start-end position pair; and identifying an entity type and a start-end position pair of a named entity in the to-be-recognized text based on the global pointer information. The embodiments of the present application can improve the accuracy of named entity recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a named entity recognition method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of information processing technology and the Internet, the amount of data people process has surged. How to quickly and efficiently identify named entity information from text in these open domains has become an important problem that urgently needs to be solved.

[0003] Named entity recognition is an important fundamental task in natural language processing. Essentially, named entity recognition is the identification of desired entities from unstructured text, and it is widely used in automatic question answering, information extraction, knowledge graphs, and other fields.

[0004] In existing technologies, named entity recognition typically uses models to first identify the location of entities in the text, and then identify the entity type at that location. However, in practical applications, texts often contain a large number of long entities, and existing models have low sensitivity to entity length, resulting in low recognition accuracy. Summary of the Invention

[0005] In view of the above problems, embodiments of this application propose a named entity recognition method, apparatus, electronic device and storage medium to improve the accuracy of named entity recognition.

[0006] According to one aspect of an embodiment of this application, a named entity recognition method is provided, the method comprising:

[0007] Input the text to be recognized into a pre-trained named entity recognition model;

[0008] The following operations are performed in the named entity recognition model:

[0009] For each preset entity type, the relative position vector, query vector, and key vector of the word segment in the text to be identified under the entity type are obtained. Based on the relative position vector, the query vector, and the key vector, the global pointer information of the text to be identified under the entity type is obtained. The global pointer information indicates the probability that the word segment belongs to the entity type and the probability that it belongs to the entity start and end position pair.

[0010] Based on the global pointer information, the entity type and start / end position pairs of the named entities in the text to be identified are determined.

[0011] Optionally, obtaining the query vector and key vector of the word segmentation in the text to be identified under the entity type includes: obtaining the target semantic feature vector of the word segmentation; using the first fully connected layer corresponding to the entity type to extract features from the target semantic feature vector of the word segmentation to obtain the query vector of the word segmentation under the entity type; using the second fully connected layer corresponding to the entity type to extract features from the target semantic feature vector of the word segmentation to obtain the key vector of the word segmentation under the entity type.

[0012] Optionally, obtaining the target semantic feature vector of the word segmentation includes: using a converter-based bidirectional encoder to represent the BERT model to extract text semantic features from the word segmentation, thereby obtaining the semantic encoding feature vector of the word segmentation; and using a bidirectional long short memory (BiLSTM) model to extract text context-dependent features from the semantic encoding feature vector of the word segmentation, thereby obtaining the target semantic feature vector of the word segmentation.

[0013] Optionally, obtaining the relative position vector of the word segment in the text to be identified under the entity type includes: performing rotational position encoding on the word segment corresponding to the entity type to obtain the relative position vector of the word segment under the entity type.

[0014] Optionally, based on the relative position vector, the query vector, and the key vector, obtaining global pointer information of the text to be identified under the entity type includes: obtaining entity type information of the text to be identified under the entity type based on the query vector, the key vector, and the relative position vector, wherein the entity type information indicates the probability that the word segment belongs to the entity type; obtaining entity position information of the text to be identified under the entity type based on the query vector and the key vector, wherein the entity position information indicates the probability that the word segment belongs to an entity start-end position pair; and fusing the entity type information and the entity position information to obtain the global pointer information.

[0015] Optionally, obtaining entity type information of the text to be identified under the entity type based on the query vector, the key vector, and the relative position vector includes: performing an inner product calculation based on the query vector, the key vector, and the relative position vector to obtain entity type information of the text to be identified under the entity type.

[0016] Optionally, based on the query vector and the key vector, obtaining the entity location information of the text to be identified under the entity type includes: concatenating the query vector and the key vector to obtain a concatenated vector; and using the third fully connected layer corresponding to the entity type to extract features from the concatenated vector to obtain the entity location information of the text to be identified under the entity type.

[0017] According to another aspect of the embodiments of this application, a named entity recognition device is provided, the device comprising:

[0018] The input module is used to input the text to be recognized into the pre-trained named entity recognition model;

[0019] The processing module is used in the named entity recognition model to obtain, for each preset entity type, the relative position vector, query vector and key vector of the word segment in the text to be recognized under the entity type, and to obtain the global pointer information of the text to be recognized under the entity type based on the relative position vector, the query vector and the key vector, wherein the global pointer information indicates the probability that the word segment belongs to the entity type and the probability that it belongs to the entity start and end position pair;

[0020] The recognition module is used to identify the entity type and start / end position pairs of named entities in the text to be recognized based on the global pointer information.

[0021] Optionally, the processing module includes: a first acquisition unit, used to acquire the target semantic feature vector of the word segmentation; a first extraction unit, used to extract features from the target semantic feature vector of the word segmentation using a first fully connected layer corresponding to the entity type, to obtain a query vector of the word segmentation under the entity type; and a second extraction unit, used to extract features from the target semantic feature vector of the word segmentation using a second fully connected layer corresponding to the entity type, to obtain a key vector of the word segmentation under the entity type.

[0022] Optionally, the first acquisition unit is specifically used to extract text semantic features from the word segmentation using a converter-based bidirectional encoder BERT model to obtain the semantic encoding feature vector of the word segmentation; and to extract text context-dependent features from the semantic encoding feature vector of the word segmentation using a bidirectional long short memory (BiLSTM) model to obtain the target semantic feature vector of the word segmentation.

[0023] Optionally, the processing module includes: an encoding unit, used to perform rotational position encoding on the word segmentation corresponding to the entity type, to obtain the relative position vector of the word segmentation under the entity type.

[0024] Optionally, the processing module includes: a second acquisition unit, configured to acquire entity type information of the text to be identified under the entity type based on the query vector, the key vector, and the relative position vector, wherein the entity type information indicates the probability that the word segment belongs to the entity type; a third acquisition unit, configured to acquire entity position information of the text to be identified under the entity type based on the query vector and the key vector, wherein the entity position information indicates the probability that the word segment belongs to an entity start-end position pair; and a fusion unit, configured to fuse the entity type information and the entity position information to obtain the global pointer information.

[0025] Optionally, the second acquisition unit is specifically used to perform inner product calculation based on the query vector, the key vector and the relative position vector to obtain the entity type information of the text to be identified under the entity type.

[0026] Optionally, the third acquisition unit is specifically used to concatenate the query vector and the key vector to obtain a concatenated vector; and to extract features from the concatenated vector using the third fully connected layer corresponding to the entity type to obtain the entity location information of the text to be identified under the entity type.

[0027] According to another aspect of the embodiments of this application, an electronic device is provided, comprising: one or more processors; and one or more computer-readable storage media having instructions stored thereon; wherein, when the instructions are executed by the one or more processors, the processors perform the named entity recognition method as described in any of the preceding claims.

[0028] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the named entity recognition method as described in any of the preceding claims.

[0029] In this embodiment, on the one hand, the named entity recognition model integrates the relative position vectors of word segmentation during analysis. By integrating the relative position vectors, the model can increase its sensitivity to the length and span of entities, thereby enabling it to more accurately identify long entities. On the other hand, the named entity recognition model recognizes the start and end positions of entities as a whole, which can improve the accuracy of entity position recognition. Furthermore, by simultaneously recognizing the start and end position pairs of entities and the entity type, the model can decode the corresponding entities more accurately from a global perspective, effectively improving the recognition accuracy of the model. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some drawings of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating the steps of a named entity recognition method according to an embodiment of this application.

[0032] Figure 2 This is a schematic diagram of the structure of a named entity recognition model according to an embodiment of this application.

[0033] Figure 3 This is a flowchart illustrating the steps of another named entity recognition method according to an embodiment of this application.

[0034] Figure 4 This is a structural block diagram of a named entity recognition device according to an embodiment of this application.

[0035] Figure 5 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0036] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0037] Reference Figure 1 The diagram shows a flowchart of the steps of a named entity recognition method according to an embodiment of this application.

[0038] like Figure 1 As shown, the named entity recognition method may include the following steps:

[0039] Step 101: Input the text to be recognized into the pre-trained named entity recognition model.

[0040] The text to be recognized refers to a segment of text composed of multiple characters. These characters can include letters, numbers, symbols, and so on.

[0041] Named entity recognition models can include vector acquisition networks, pointer acquisition networks, and entity recognition networks. The text to be recognized is sequentially analyzed and processed by these networks to obtain the types and locations of named entities in the text. The types of named entities can include, but are not limited to: personal names, place names, organization names, proper nouns from different fields, etc.

[0042] Step 102: In the named entity recognition model, for each preset entity type, the relative position vector, query vector, and key vector of the word segmentation in the text to be recognized under the entity type are obtained, and based on the relative position vector, the query vector, and the key vector, the global pointer information of the text to be recognized under the entity type is obtained.

[0043] The text to be recognized is first input into a vector acquisition network. In this network, for each entity type, the relative position vector, query vector, and key vector of the word segments in the text to be recognized within that entity type are obtained. The vector acquisition network then outputs these vector vectors for each entity type.

[0044] The output of the vector acquisition network serves as the input of the pointer acquisition network. In the pointer acquisition network, for each entity type, based on the relative position vector of the word segment in the text to be identified under that entity type, the query vector, and the key vector, global pointer information of the text to be identified under that entity type is obtained. The global pointer information of the text to be identified under that entity type indicates the probability that the word segment in the text belongs to that entity type and the probability that it belongs to an entity start-end position pair.

[0045] Step 103: Based on the global pointer information, identify the entity type and start / end position pairs of the named entities in the text to be identified.

[0046] The output of the pointer acquisition network is used as the input of the entity recognition network. In the entity recognition network, for each entity type, based on the global pointer information of the text to be recognized under the entity type, it can be identified whether there is a named entity of that entity type in the text to be recognized. When there is a named entity of that entity type, the start and end position pairs of the named entity can also be identified at the same time, so as to finally identify the named entity in the text to be recognized.

[0047] For example, for any entity type, if the probability of a word segment belonging to that entity type exceeds the type probability threshold corresponding to that entity type, then the word segment can be determined to belong to that entity type; if the probability of a word segment belonging to an entity start-end position pair exceeds the position pair probability threshold corresponding to that entity type, then the word segment can be determined to belong to an entity start-end position pair under that entity type.

[0048] In this embodiment, on the one hand, the named entity recognition model integrates the relative position vectors of word segmentation during analysis. By integrating the relative position vectors, the model can increase its sensitivity to the length and span of entities, thereby enabling it to more accurately identify long entities. On the other hand, the named entity recognition model recognizes the start and end positions of entities as a whole, which can improve the accuracy of entity position recognition. Furthermore, by simultaneously recognizing the start and end position pairs of entities and the entity type, the model can decode the corresponding entities more accurately from a global perspective, effectively improving the recognition accuracy of the model.

[0049] Reference Figure 2 The diagram shows a structural schematic of a named entity recognition model according to an embodiment of this application.

[0050] like Figure 2 As shown, the input to the named entity recognition model is text; the text is segmented into multiple words T1, T2, T3, T4, ..., Tn; the words T1, T2, T3, T4, ..., Tn are input to the Bidirectional Encoder Representations from Transformers (BERT) model; the output of the BERT model is used as the input to the Bidirectional Long Short-Term Memory (BiLSTM) model; the output of the BiLSTM model is used as the input to the first fully connected layer (first Dense layer) and the second fully connected layer (second Dense layer); the query vector output by the first Dense layer, the key vector output by the second Dense layer, and the relative position vector obtained by the rotational positional encoding part are calculated by the inner product (Dot); the query vector output by the first Dense layer and the key vector output by the second Dense layer are concatenated, and the concatenation result is input to the third fully connected layer (third Dense layer); the result of the Dot calculation and the output of the third Dense layer are used together as the score-matrix fusion network. The input is the scoring matrix fusion network; based on the global pointer information about entity type and entity start and end position pairs output by the scoring matrix fusion network, the entity type and start and end position pairs of named entities in the text are identified, and finally the identified named entities are output.

[0051] Among them, the BERT model, BiLSTM model, first Dense layer, second Dense layer and rotational position encoding part belong to vector acquisition network; Dot calculation, concatenate, third Dense layer and score-matrix fusion network belong to pointer acquisition network; the following part belongs to entity recognition network.

[0052] It should be noted that, for each entity type, a first Dense layer, a second Dense layer, a rotational position encoding part, a dot calculation, concatenation, a third Dense layer, and a score-matrix fusion network can be set separately; alternatively, sub-parts for each entity type can be set within the first Dense layer, the second Dense layer, the rotational position encoding part, the dot calculation, concatenation, the third Dense layer, and the score-matrix fusion network.

[0053] Below, in conjunction with Figure 2 The named entity recognition model shown provides a detailed explanation of the named entity recognition method.

[0054] Reference Figure 3 The diagram illustrates a flowchart of another named entity recognition method according to an embodiment of this application.

[0055] like Figure 3 As shown, the named entity recognition method may include the following steps:

[0056] Step 301: Input the text to be recognized into the pre-trained named entity recognition model.

[0057] The named entity recognition model is trained in advance.

[0058] First, obtain the sample text and its annotation information. The annotation information of the sample text indicates the entity type and start / end position pairs of the actual named entities in the sample text.

[0059] Then, an initial named entity recognition model is constructed. The structure of the initial named entity recognition model is the same as described above. Figure 2 The structures shown are the same.

[0060] The parameters in the initial named entity recognition model are initial parameters, which are iteratively optimized during model training. For example, the parameters that need to be iteratively optimized in the initial named entity recognition model may include, but are not limited to: the parameters corresponding to the first Dense layer, the parameters corresponding to the second Dense layer, the parameters corresponding to the rotational positional encoding part, the parameters corresponding to the third Dense layer, and so on.

[0061] Next, during the model training process, the sample text is used as the input to the initial named entity recognition model, and the annotation information of the sample text is used as the output target of the initial named entity recognition model. The initial named entity recognition model is trained, and the trained model is used as the named entity recognition model in this embodiment of the application.

[0062] Specifically, the sample text is used as the input to the initial named entity recognition model. Relevant analysis and processing are performed inside the initial named entity recognition model (the specific analysis and processing process is basically the same as the analysis and processing process of the text to be recognized in subsequent steps 302 to 306, which will be described in detail below) to obtain the output of the initial named entity recognition model. The output is the entity type and start and end position pairs of the predicted named entities in the sample text.

[0063] Based on the output of the initial named entity recognition model and the annotation information of the sample text, the model loss function can be calculated. Training is considered complete when the model loss function meets a preset condition (e.g., less than a certain threshold); if the model loss function does not meet the preset condition, the parameters in the initial named entity recognition model are optimized, and training continues. For example, the aforementioned model loss function may include, but is not limited to: cross-entropy loss function, exponential loss function, squared loss function, absolute value loss function, etc.

[0064] Step 302: Use the BERT model to extract the semantic features of the word segments in the text to be identified, and obtain the semantic encoding feature vector of the word segments.

[0065] The BERT model utilizes the encoder part of the Transformer to extract bidirectional contextual information. It relies on the self-attention mechanism to learn unlabeled text in an unsupervised manner. By using the masking method, it transforms "extraction of information about a word" into "extraction of contextual information about the word", reducing the dependence on external information and making it better at capturing the internal correlation of words or features, thus effectively solving the ambiguity problem.

[0066] Before inputting the text to be recognized into the BERT model, the text is preprocessed, including word segmentation and other processes. For example... Figure 2As shown, the text to be identified is divided into multiple segments T1, T2, T3, T4, ..., Tn.

[0067] The preprocessed text to be recognized is input into the BERT model, and the BERT model is used to extract the semantic features of the word segments in the text to be recognized, thereby obtaining the semantic encoding feature vector of the word segments in the text to be recognized.

[0068] The processing of the BERT model may include the following steps A1 to A2:

[0069] Step A1: The input text to be recognized is a text sentence. When this text is input into the BERT model, it needs to undergo Word Piece embedding to obtain the token word embedding representation; it also needs to undergo Segment Embedding to obtain the segment embedding representation of the sentence containing the token; and it also needs to undergo Position Embedding to obtain the position embedding representation of the token. Based on the task in this embodiment, the maximum input to the BERT model can be set to 300 tokens.

[0070] Step A2: For the three encoding representations corresponding to the word segmentation in the text to be identified: Token word embedding representation, segment embedding representation and position embedding representation, add special symbols [CLS] and [SEP] to the beginning and end of each sentence respectively. After training, the overall semantic information of the sentence will be retained in CLS. After the BERT model extracts the text features, the encoding result of the CLS position is finally output. This encoding result summarizes the semantic encoding information of the entire sentence.

[0071] Step 303: Use the BiLSTM model to extract text context-dependent features from the semantic encoding feature vector of the segmented words to obtain the target semantic feature vector of the segmented words.

[0072] The BiLSTM model is composed of a forward LSTM network and a backward LSTM network. Based on an RNN (Recurrent Neural Network), the BiLSTM model adds input gates, forget gates, unit states, and output gates. BiLSTM introduces a gating mechanism to capture long-range sequence information, combining the forward and backward LSTM networks to capture bidirectional semantic dependencies and further obtain better semantic encoding information.

[0073] The semantic encoding feature vectors of word segmentation in the text to be identified, output by the BERT model, are input into the BiLSTM model. The BiLSTM model is then used to extract the context-dependent features of the text to obtain the target semantic feature vectors of word segmentation in the text to be identified.

[0074] The processing of a BiLSTM model may include the following steps B1 to B2:

[0075] Step B1: Input the semantic encoding feature vector of word segmentation in the text to be identified output by the BERT model into the feedforward LSTM network to obtain the feedforward semantic dependency feature vector of word segmentation.

[0076] Step B2: Input the semantic encoding feature vector of word segmentation in the text to be identified output by the BERT model into the backward LSTM network to obtain the backward semantic dependency feature vector of word segmentation.

[0077] Step B3: Concatenate the forward semantic dependency feature vector and the backward semantic dependency feature vector of the word segmentation to obtain the contextual semantic dependency encoding information of the word segmentation, which is the target semantic feature vector of the word segmentation.

[0078] Step 304: For each preset entity type, obtain the relative position vector, query vector, and key vector of the word segmentation in the text to be identified under the entity type.

[0079] For each entity type, a first fully connected layer corresponding to that entity type is set up. For a query vector of a word segment under any entity type, the first fully connected layer corresponding to that entity type can be used to extract features from the target semantic feature vector of the word segment to obtain the query vector of the word segment under that entity type.

[0080] Specifically, for any entity type, the target semantic feature vector of word segmentation in the text to be identified output by the BiLSTM model is input into the first fully connected layer corresponding to that entity type. After feature extraction processing in the first fully connected layer, the first fully connected layer outputs the query vector query under that entity type.

[0081] For each entity type, a second fully connected layer corresponding to that entity type is set up. For the key vector of the word segment under any entity type, the target semantic feature vector of the word segment can be extracted using the second fully connected layer corresponding to that entity type to obtain the key vector of the word segment under that entity type.

[0082] Specifically, for any entity type, the target semantic feature vector of word segmentation in the text to be identified output by the BiLSTM model is input into the second fully connected layer corresponding to that entity type. After feature extraction processing in the second fully connected layer, the second fully connected layer outputs the key vector of word segmentation under that entity type.

[0083] For each entity type, a rotational position encoding method corresponding to that entity type is set. For the relative position vector of a word under any entity type, the word can be rotated and encoded according to the entity type to obtain the relative position vector of the word under that entity type.

[0084] Rotary Position Embedding (RoPE) is a design that, in conjunction with a self-attention mechanism, achieves relative position encoding in the same way as absolute position encoding. The dimension of the relative position vector can be set according to the actual situation, matching the dimensions of the query vector and key vector. The addition of the relative position vector in RoPE increases the model's sensitivity to the length and span of entities, thus enabling more accurate identification of long entities.

[0085] For example, if the text to be identified is a sentence of length n, after processing by the BERT model and the BiLSTM model, the target semantic feature vector of each word in the text to be identified is obtained as a matrix [h1, h2, ..., h]. n ].

[0086] The query vector q of the i-th word in the text to be recognized under entity type α αi It is expressed as follows:

[0087] q αi =ω qα ·h i +b qα

[0088] Among them, h i ω represents the target semantic feature vector of the i-th word in the text to be identified. qα b represents the weight parameters of the first fully connected layer corresponding to entity type α. qα ω represents the bias parameter of the first fully connected layer corresponding to entity type α. qα and b qα As trainable parameters in model training.

[0089] The key vector k of the i-th word in the text to be recognized under entity type α αi It is expressed as follows:

[0090] k αi =ω kα·h i +b kα

[0091] Among them, h i ω represents the target semantic feature vector of the i-th word in the text to be identified. kα b represents the weight parameters of the second fully connected layer corresponding to entity type α. kα ω represents the bias parameter of the second fully connected layer corresponding to entity type α. kα and b kα As trainable parameters in model training.

[0092] In the two-dimensional case (other dimensions are handled similarly), the relative position vector u of the i-th word in the text to be identified under entity type α is... αi It is expressed as follows:

[0093]

[0094] Where, α i This represents the parameter of the i-th word segment corresponding to the rotational positional encoding of entity type α. i As trainable parameters in model training.

[0095] Step 305: Based on the relative position vector, the query vector, and the key vector, obtain the global pointer information of the text to be identified under the entity type.

[0096] The global pointer information of the text to be identified under any entity type indicates the probability that the word segment in the text to be identified belongs to the entity type, and the probability that it belongs to the start and end position pair of the entity under that entity type.

[0097] For any entity type, the process of obtaining the global pointer information of the text to be identified under the entity type based on the relative position vector, the query vector, and the key vector may include the following steps C1 to C3:

[0098] Step C1: Based on the query vector, the key vector, and the relative position vector, obtain the entity type information of the text to be identified under the entity type.

[0099] For the query vector output by the first fully connected layer corresponding to this entity type and the key vector output by the second fully connected layer corresponding to this entity type, the relative position vector obtained by the rotational positional encoding corresponding to this entity type is fused, and a Dot inner product is calculated to obtain the entity type information of the text to be recognized under this entity type. The entity type information of the text to be recognized under this entity type indicates the probability that the word segmentation in the text to be recognized belongs to this entity type. The entity type information can be in the form of an entity type scoring matrix.

[0100] Corresponding to the example above, the entity type scoring matrix of the text to be identified under entity type α It is expressed as follows:

[0101]

[0102] Where, q αi This represents the query vector of the i-th word in the text to be identified under entity type α. q αi transpose, u αi This represents the relative position vector of the i-th word in the text to be identified under entity type α. Indicate u αi transpose, u αj Let k represent the relative position vector of the j-th word in the text to be recognized under entity type α. αj This represents the key vector of the j-th word in the text to be identified under entity type α.

[0103] Step C2: Based on the query vector and the key vector, obtain the entity location information of the text to be identified under the entity type.

[0104] The query vector output from the first fully connected layer corresponding to this entity type and the key vector output from the second fully connected layer corresponding to this entity type are concatenated to obtain the concatenated vector corresponding to this entity type. Using the third fully connected layer corresponding to this entity type, feature extraction is performed on the concatenated vector to obtain the entity position information of the text to be identified under this entity type. The entity position information of the text to be identified under this entity type indicates the probability that a word segment in the text belongs to an entity start-end position pair under this entity type. The entity position information can be in the form of an entity start-end position pair scoring matrix.

[0105] Corresponding to the example above, the scoring matrix is ​​used to determine the start and end positions of entities in the text to be identified under entity type α. It is expressed as follows:

[0106]

[0107] Where, q αi Let k represent the query vector of the i-th word in the text to be recognized under entity type α. αi Let q represent the key vector of the i-th word in the text to be recognized under entity type α. αj Let k represent the query vector of the j-th word in the text to be recognized under entity type α. αj ω represents the key vector of the j-th word in the text to be recognized under entity type α. α b represents the weight parameters of the third fully connected layer corresponding to entity type α.α ω represents the bias parameter of the third fully connected layer corresponding to entity type α. α and b α As trainable parameters in model training.

[0108] Step C3: The entity type information and the entity location information are fused to obtain the global pointer information.

[0109] For the entity type scoring matrix and the entity start-end position pair scoring matrix corresponding to this entity type, both are input into the score-matrix fusion network. The score-matrix fusion network adds and fuses the entity type scoring matrix and the entity start-end position pair scoring matrix to obtain the global pointer information corresponding to this entity type. The global pointer information can be in the form of a global pointer scoring matrix.

[0110] Corresponding to the example above, the global pointer scoring matrix of the text to be identified under entity type α It is expressed as follows:

[0111]

[0112] in, This represents the entity type scoring matrix of the text to be identified under entity type α. This represents the scoring matrix for the start and end positions of entities in the text to be identified under entity type α.

[0113] Step 306: Based on the global pointer information, identify the entity type and start / end position pairs of the named entities in the text to be identified.

[0114] In one optional implementation, for any entity type, if the probability of a word belonging to that entity type exceeds the type probability threshold corresponding to that entity type, then the word can be determined to belong to that entity type. In this case, the value of the entity type at the corresponding position of the word in the global pointer scoring matrix can be set to 1; otherwise, it can be set to 0. Correspondingly, if the probability of the word belonging to the start-end position pair of an entity exceeds the position pair probability threshold corresponding to that entity type, then the word can be determined to belong to the start-end position pair of an entity under that entity type. In this case, the value of the start-end position pair of an entity at the corresponding position of the word in the global pointer scoring matrix can be set to 1; otherwise, it can be set to 0.

[0115] In this method, it is determined whether the value of the entity type and the value of the entity start and end position pair in the global pointer scoring matrix are both greater than 0 (i.e., both are 1), so as to simultaneously identify the entity start and end position pair and the entity type. If so, the segment covered by the entity start and end position pair is determined to be the named entity, and the type of the named entity is the current entity type.

[0116] In another optional implementation, for any entity type, if the probability of a word belonging to that entity type exceeds the type probability threshold corresponding to that entity type, and the probability of the word belonging to the start and end position pair of the entity exceeds the position pair probability threshold corresponding to that entity type, then it is determined that the word belongs to that entity type and belongs to the start and end position pair of the entity under that entity type. At this time, the comprehensive value of the position corresponding to that word in the global pointer scoring matrix can be set to the value 1, otherwise it can be set to the value 0.

[0117] In this method, it checks whether the comprehensive value in the global pointer scoring matrix is ​​greater than 0 (i.e., 1), thereby simultaneously identifying the entity's start and end position pairs and entity type. If so, the segment covered by the entity's start and end position pairs is determined to be a named entity, and the type of the named entity is the current entity type. For example... Figure 2 As shown, in the global pointer scoring matrix output by the Score-matrix Fusion network for each entity type, the position of the value 1 indicates that the word belongs to that entity type and is a start and end position pair of entities under that entity type. Therefore, Figure 2 The segment between two 1 values ​​is the named entity under this entity type.

[0118] The global pointer network method based on the fusion of entity start and end position pairs and entity type realizes the named entity recognition task that treats the entity start and end positions as a whole, effectively improving the model's generalization ability and recognition accuracy.

[0119] Step 307: The named entity recognition model outputs the named entity.

[0120] Once the named entity recognition model identifies the entity type and start / end position pairs of the named entities in the text to be recognized, it can output the final recognized named entities.

[0121] This application provides a global pointer network that integrates position and type information to achieve named entity recognition. Position information and entity type information are simultaneously injected into the neural network to improve the model's recognition accuracy. This method of integrating position and type information for named entity recognition can be viewed as an end-to-end model. The encoder uses BERT+BiLSTM as the encoder. The BERT model, based on the Transformer module, effectively extracts semantic features from the text, while the BiLSTM network further captures context-dependent semantic features, demonstrating good performance and high accuracy in named entity recognition. The decoder integrates text position and type information to design a novel global pointer network that simultaneously identifies the start and end position pairs of entities and their types, decoding entities of the corresponding types from a global perspective. This enables the indiscriminate recognition of overlapping and long entities, effectively improving the accuracy of named entity recognition.

[0122] Reference Figure 4 The diagram shows a structural block diagram of a named entity recognition device according to an embodiment of this application.

[0123] like Figure 4 As shown, the named entity recognition device may include the following modules:

[0124] Input module 401 is used to input the text to be recognized into a pre-trained named entity recognition model;

[0125] Processing module 402 is used in the named entity recognition model to obtain, for each preset entity type, the relative position vector, query vector and key vector of the word segment in the text to be recognized under the entity type, and based on the relative position vector, the query vector and the key vector, to obtain global pointer information of the text to be recognized under the entity type, wherein the global pointer information indicates the probability that the word segment belongs to the entity type and the probability that it belongs to the entity start and end position pair;

[0126] The recognition module 403 is used to identify the entity type and start / end position pairs of named entities in the text to be recognized based on the global pointer information.

[0127] Optionally, the processing module 402 includes: a first acquisition unit, used to acquire the target semantic feature vector of the word segmentation; a first extraction unit, used to extract features from the target semantic feature vector of the word segmentation using a first fully connected layer corresponding to the entity type, to obtain a query vector of the word segmentation under the entity type; and a second extraction unit, used to extract features from the target semantic feature vector of the word segmentation using a second fully connected layer corresponding to the entity type, to obtain a key vector of the word segmentation under the entity type.

[0128] Optionally, the first acquisition unit is specifically used to extract text semantic features from the word segmentation using a converter-based bidirectional encoder BERT model to obtain the semantic encoding feature vector of the word segmentation; and to extract text context-dependent features from the semantic encoding feature vector of the word segmentation using a bidirectional long short memory (BiLSTM) model to obtain the target semantic feature vector of the word segmentation.

[0129] Optionally, the processing module 402 includes: an encoding unit, used to perform rotational position encoding on the word segmentation corresponding to the entity type, to obtain the relative position vector of the word segmentation under the entity type.

[0130] Optionally, the processing module 402 includes: a second acquisition unit, configured to acquire entity type information of the text to be identified under the entity type based on the query vector, the key vector, and the relative position vector, wherein the entity type information indicates the probability that the word segment belongs to the entity type; a third acquisition unit, configured to acquire entity position information of the text to be identified under the entity type based on the query vector and the key vector, wherein the entity position information indicates the probability that the word segment belongs to an entity start-end position pair; and a fusion unit, configured to fuse the entity type information and the entity position information to obtain the global pointer information.

[0131] Optionally, the second acquisition unit is specifically used to perform inner product calculation based on the query vector, the key vector and the relative position vector to obtain the entity type information of the text to be identified under the entity type.

[0132] Optionally, the third acquisition unit is specifically used to concatenate the query vector and the key vector to obtain a concatenated vector; and to extract features from the concatenated vector using the third fully connected layer corresponding to the entity type to obtain the entity location information of the text to be identified under the entity type.

[0133] In this embodiment, on the one hand, the named entity recognition model integrates the relative position vectors of word segmentation during analysis. By integrating the relative position vectors, the model can increase its sensitivity to the length and span of entities, thereby enabling it to more accurately identify long entities. On the other hand, the named entity recognition model recognizes the start and end positions of entities as a whole, which can improve the accuracy of entity position recognition. Furthermore, by simultaneously recognizing the start and end position pairs of entities and the entity type, the model can decode the corresponding entities more accurately from a global perspective, effectively improving the recognition accuracy of the model.

[0134] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0135] In embodiments of this application, an electronic device is also provided. This electronic device may include one or more processors and one or more computer-readable storage media storing instructions thereon, such as an application program. When the instructions are executed by the one or more processors, the processors cause the processors to perform the named entity recognition method of any of the above embodiments.

[0136] Reference Figure 5 The diagram illustrates a schematic representation of an electronic device structure according to an embodiment of this application. Figure 5 As shown, the electronic device includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, communication interface 502, and memory 503 communicate with each other via the communication bus 504.

[0137] Memory 503 is used to store computer programs.

[0138] When the processor 501 executes the program stored in the memory 503, it implements the named entity recognition method of any of the above embodiments.

[0139] Communication interface 502 is used for communication between the above-mentioned electronic device and other devices.

[0140] The aforementioned communication bus 504 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, it is represented by only one thick line in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0141] The processor 501 mentioned above may include, but is not limited to: a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0142] The aforementioned memory 503 may include, but is not limited to: Read Only Memory (ROM), Random Access Memory (RAM), Compact Disc Read Only Memory (CD-ROM), Electronic Erasable Programmable Read Only Memory (EEPROM), Hard Disk, Floppy Disk, Flash Memory, etc.

[0143] In embodiments of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which can be executed by a processor of an electronic device, and when the computer program is executed by the processor, the processor performs the named entity recognition method as described in any of the above embodiments.

[0144] The various embodiments in this specification are related to each other and are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0145] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0147] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0148] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0149] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0150] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0151] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0152] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0153] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0154] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. A named entity recognition method, characterized in that, The method includes: Input the text to be recognized into a pre-trained named entity recognition model; The following operations are performed in the named entity recognition model: For each preset entity type, the relative position vector, query vector, and key vector of the word segment in the text to be identified under the entity type are obtained. Based on the relative position vector, the query vector, and the key vector, the global pointer information of the text to be identified under the entity type is obtained. The global pointer information indicates the probability that the word segment belongs to the entity type and the probability that it belongs to the entity start and end position pair. Based on the global pointer information, the entity type and start / end position pairs of named entities in the text to be identified are determined; Obtaining the query vector and key vector of the word segmentation in the text to be identified under the entity type includes: Obtain the target semantic feature vector of the segmented words; The first fully connected layer corresponding to the entity type is used to extract features from the target semantic feature vector of the word segmentation to obtain the query vector of the word segmentation under the entity type. The target semantic feature vector of the word segmentation is extracted by using the second fully connected layer corresponding to the entity type to obtain the key vector of the word segmentation under the entity type. Based on the relative position vector, the query vector, and the key vector, global pointer information of the text to be identified under the entity type is obtained, including: Based on the query vector, the key vector, and the relative position vector, the entity type information of the text to be identified under the entity type is obtained, and the entity type information indicates the probability that the word segmentation belongs to the entity type. Based on the query vector and the key vector, the entity position information of the text to be identified under the entity type is obtained, and the entity position information indicates the probability that the word segmentation belongs to the entity start and end position pair; The entity type information and the entity location information are fused to obtain the global pointer information.

2. The method according to claim 1, characterized in that, Obtaining the target semantic feature vector of the word segmentation includes: The semantic features of the segmented words are extracted by using a converter-based bidirectional encoder to represent the BERT model, thereby obtaining the semantic encoding feature vector of the segmented words; The semantic encoding feature vector of the segmented words is extracted using a bidirectional long short-term memory (BiLSTM) model to obtain the target semantic feature vector of the segmented words.

3. The method according to claim 1, characterized in that, Obtaining the relative position vectors of word segments in the text to be identified under the entity type includes: The word segment is subjected to rotational position encoding corresponding to the entity type to obtain the relative position vector of the word segment under the entity type.

4. The method according to claim 1, characterized in that, Based on the query vector, the key vector, and the relative position vector, the entity type information of the text to be identified under the entity type is obtained, including: The entity type information of the text to be identified under the entity type is obtained by calculating the inner product based on the query vector, the key vector and the relative position vector.

5. The method according to claim 1, characterized in that, Based on the query vector and the key vector, obtain the entity location information of the text to be identified under the entity type, including: The query vector and the key vector are concatenated to obtain a concatenated vector. The concatenated vector is used to extract features by the third fully connected layer corresponding to the entity type to obtain the entity location information of the text to be identified under the entity type.

6. A named entity recognition device, characterized in that, The device includes: The input module is used to input the text to be recognized into the pre-trained named entity recognition model; The processing module is used in the named entity recognition model to obtain, for each preset entity type, the relative position vector, query vector and key vector of the word segment in the text to be recognized under the entity type, and to obtain the global pointer information of the text to be recognized under the entity type based on the relative position vector, the query vector and the key vector, wherein the global pointer information indicates the probability that the word segment belongs to the entity type and the probability that it belongs to the entity start and end position pair; The processing module includes: a first acquisition unit, used to acquire the target semantic feature vector of the word segmentation; a first extraction unit, used to extract features from the target semantic feature vector of the word segmentation using a first fully connected layer corresponding to the entity type, to obtain a query vector of the word segmentation under the entity type; and a second extraction unit, used to extract features from the target semantic feature vector of the word segmentation using a second fully connected layer corresponding to the entity type, to obtain a key vector of the word segmentation under the entity type. The recognition module is used to identify the entity type and start / end position pairs of named entities in the text to be recognized based on the global pointer information. The processing module includes: The second acquisition unit is used to acquire entity type information of the text to be identified under the entity type based on the query vector, the key vector and the relative position vector, wherein the entity type information indicates the probability that the word segmentation belongs to the entity type; The third acquisition unit is used to acquire entity location information of the text to be identified under the entity type based on the query vector and the key vector, wherein the entity location information indicates the probability that the word segmentation belongs to the entity start and end position pair; The fusion unit is used to fuse the entity type information and the entity location information to obtain the global pointer information.

7. An electronic device, characterized in that, include: One or more processors; and One or more computer-readable storage media on which instructions are stored; When the instructions are executed by the one or more processors, the processors perform the named entity recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, causes the processor to perform the named entity recognition method as described in any one of claims 1 to 5.

Images