Text entity recognition method and apparatus

Abstract

This application provides a text entity recognition method and apparatus. The method includes: acquiring text to be recognized; performing feature transformation on the text to be recognized using multiple word vector models to obtain multiple text feature vectors of the text to be recognized; learning the multiple text feature vectors by a pre-trained neural network model to obtain entity feature vectors, wherein the entity feature vectors are used to characterize entity features in the text to be recognized; and determining the entity label sequence corresponding to the text to be recognized based on the entity feature vectors using a conditional random field model, thereby identifying the entity labels of each entity in the text to be recognized. The technical solution of this application embodiment can improve the accuracy of text entity recognition.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and natural language processing technology, and more specifically, to a method and apparatus for text entity recognition. Background Technology

[0002] Named entity recognition, as a fundamental task in natural language processing, aims to identify named entities from unstructured text. Essentially, it involves identifying proper nouns or words or phrases with special meanings from a text description. However, the accuracy of existing named entity recognition technologies still needs improvement. Therefore, how to improve the accuracy of text entity recognition is an urgent technical problem to be solved. Summary of the Invention

[0003] The embodiments of this application provide a text entity recognition method, apparatus, computer program product or computer program, computer-readable medium and electronic device, which can at least improve the accuracy of text entity recognition to a certain extent.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a text entity recognition method is provided, the method comprising: acquiring a text to be recognized; performing feature transformation on the text to be recognized through multiple word vector models to obtain multiple text feature vectors of the text to be recognized; learning the multiple text feature vectors by a pre-trained neural network model to obtain entity feature vectors, the entity feature vectors being used to characterize entity features in the text to be recognized; and determining an entity label sequence corresponding to the text to be recognized based on the entity feature vectors through a conditional random field model to identify the entity labels of each entity in the text to be recognized.

[0006] According to one aspect of the embodiments of this application, a text entity recognition device is provided. The device includes: a first acquisition unit, configured to acquire text to be recognized; a conversion unit, configured to perform feature conversion on the text to be recognized using multiple word vector models to obtain multiple text feature vectors of the text to be recognized; a learning unit, configured to learn the multiple text feature vectors by a pre-trained neural network model to obtain entity feature vectors, the entity feature vectors being used to characterize entity features in the text to be recognized; and a determination unit, configured to determine the entity label sequence corresponding to the text to be recognized based on the entity feature vectors using a conditional random field model, so as to identify the entity labels of each entity in the text to be recognized.

[0007] In some embodiments of this application, based on the foregoing scheme, the plurality of text feature vectors include a first text feature vector and a second text feature vector. The conversion unit is configured to: encode the characters in the text to be identified using the plurality of word vector models to obtain a plurality of text word vectors; select any one of the plurality of text word vectors as the first text feature vector; concatenate the plurality of text word vectors to obtain a concatenated vector, and use the concatenated vector as the second text feature vector.

[0008] In some embodiments of this application, based on the foregoing scheme, the learning unit is configured to: input the plurality of text feature vectors into the neural network model respectively, so that the neural network model outputs a plurality of sub-entity feature vectors; and average the plurality of sub-entity feature vectors to obtain the entity feature vector.

[0009] In some embodiments of this application, based on the foregoing scheme, the apparatus further includes: an allocation unit, configured to assign weights to each feature value in the entity feature vector using an attention mechanism model before determining the entity label sequence corresponding to the text to be identified using a conditional random field model based on the entity feature vector, wherein the weights are used to distinguish the importance of each string in the text to be identified in terms of entity features; and an adjustment unit, configured to adjust each feature value in the entity feature vector based on the weights to update the entity feature vector.

[0010] In some embodiments of this application, based on the foregoing scheme, the device further includes a second acquisition unit, used to acquire a training text set before acquiring the text to be recognized, the training text set including multiple training texts, wherein each training text is configured with an entity label sequence; and a training unit, used to perform supervised training on the neural network model and the attention mechanism model in an iterative manner based on the multiple training texts in the training text set, and adjust the neural network model and the attention mechanism model based on the entity label sequence configured in the training texts until the neural network model and the attention mechanism model converge.

[0011] In some embodiments of this application, based on the foregoing scheme, the second acquisition unit is configured to: acquire a first candidate text set, the first candidate text set including multiple first candidate texts; generate a label sequence for the first candidate texts in the first candidate text set using a far-supervised model; select a first candidate text from the first candidate text set as training text, and obtain the training text set.

[0012] In some embodiments of this application, based on the foregoing scheme, the second acquisition unit is configured to: acquire a second candidate text set, the second candidate text set including multiple second candidate texts, wherein each second candidate text is configured with an entity label sequence; train a reinforcement learning model based on the second candidate text set, the reinforcement learning model being used to identify incomplete label sequences or noisy label sequences in the candidate texts; and select a first candidate text with a complete label sequence or a non-noisy label sequence from the first candidate text set as training text through the reinforcement learning model.

[0013] In some embodiments of this application, based on the foregoing scheme, the various word vector models include the ALBERT model and the word2vec model.

[0014] In some embodiments of this application, based on the foregoing scheme, the neural network model includes a bidirectional long short-term memory recurrent neural network model.

[0015] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the text entity recognition method described in the above embodiments.

[0016] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the text entity recognition method as described in the above embodiments.

[0017] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the text entity recognition method as described in the above embodiments.

[0018] In some embodiments of this application, the technical solutions involve performing feature transformation on the text to be recognized using multiple word vector models. This yields multiple text feature vectors for learning by a neural network model, which in turn generate entity feature vectors representing the entity features in the text. Finally, based on these entity feature vectors, a conditional random field model can determine the entity label sequence corresponding to the text. Since multiple text feature vectors increase the feature dimension of characters in the text to be recognized, ensuring the diversity of character feature representations and enhancing character feature representation, the accuracy of entity recognition in the text to be recognized can be improved.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0021] Figure 1 An exemplary system architecture diagram is shown, to which the technical solutions of the embodiments of this application can be applied;

[0022] Figure 2 A flowchart of a text entity recognition method according to an embodiment of this application is shown;

[0023] Figure 3 A detailed flowchart illustrating how, according to an embodiment of this application, feature transformation is performed on the text to be identified using multiple word vector models to obtain multiple text feature vectors of the text to be identified;

[0024] Figure 4 A schematic diagram of the structure of an ALBERT model according to an embodiment of this application is shown;

[0025] Figure 5 A detailed flowchart illustrating how a pre-trained neural network model learns the plurality of text feature vectors to obtain entity feature vectors according to one embodiment of this application is shown.

[0026] Figure 6 A flowchart of a method according to an embodiment of this application is shown before determining the entity label sequence corresponding to the text to be identified by a conditional random field model;

[0027] Figure 7 A flowchart of a method prior to obtaining the text to be recognized, according to an embodiment of this application, is shown;

[0028] Figure 8 A detailed flowchart of obtaining a training text set according to one embodiment of this application is shown;

[0029] Figure 9 A detailed flowchart illustrating the selection of a first candidate text from a first candidate text set as training text is shown according to an embodiment of this application.

[0030] Figure 10A schematic diagram of the overall model for text entity data processing according to an embodiment of this application is shown;

[0031] Figure 11 A block diagram of a text entity recognition device according to an embodiment of this application is shown;

[0032] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0034] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0035] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.

[0038] Figure 1An exemplary system architecture diagram is shown that can be applied to the technical solutions of the embodiments of this application.

[0039] like Figure 1 As shown, the system architecture may include terminal devices (such as...) Figure 1 The device shown includes one or more of a smartphone 101, tablet 102, and portable computer 103 (which could also be a desktop computer, etc.), a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal device and the server 105. The network 104 can include various connection types, such as wired communication links, wireless communication links, etc.

[0040] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, there can be any number of terminal devices, networks, and servers. For example, server 105 could be a server cluster composed of multiple servers.

[0041] In one embodiment of this application, the text entity recognition method proposed in this application can be executed by server 105. Specifically, a user can upload text to be recognized by entity recognition to server 105 using a terminal device. After obtaining the text to be recognized, server 105 performs feature transformation on the text to be recognized using multiple word vector models to obtain multiple text feature vectors of the text to be recognized. The pre-trained neural network model learns the multiple text feature vectors to obtain entity feature vectors. The entity feature vectors are used to characterize the entity features in the text to be recognized. Finally, based on the entity feature vectors, server 105 determines the entity label sequence corresponding to the text to be recognized using a conditional random field model to identify the entity labels of each entity in the text to be recognized.

[0042] It should be noted that the text entity recognition method provided in this application embodiment is generally executed by server 105, and correspondingly, the text entity recognition device is generally located in server 105. However, in other embodiments of this application, the terminal device may also have similar functions to the server, thereby executing the text entity recognition scheme provided in this application embodiment.

[0043] The text entity recognition method provided in this application relates to deep learning in the field of Artificial Intelligence (AI), specifically to Natural Language Processing (NLP) within deep learning technology. Natural Language Processing is an important direction in computer science and artificial intelligence. It studies various theories and methods that enable effective communication between humans and computers using natural language. Natural Language Processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language, that is, the language people use in daily life, and thus it has a close relationship with linguistic research. Natural Language Processing technologies typically include text processing, semantic understanding, machine translation, robot question answering, knowledge graphs, and other technologies.

[0044] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0045] Figure 2 A flowchart of a text entity recognition method according to an embodiment of this application is shown. This text entity recognition method can be executed by a device with computational processing capabilities, such as a... Figure 1 The server 105 shown is used for execution. (Refer to...) Figure 2 As shown, this text entity recognition method includes at least steps 220 to 280, which are described in detail below:

[0046] In step 220, the text to be recognized is obtained.

[0047] In this application, the text to be identified refers to the text that needs to be named entity recognition. Named entity recognition (NER), also known as proper name recognition, aims to identify named entities from unstructured text, that is, to identify entities with specific meanings in the text, mainly including personal names, place names, organization names, proper nouns, etc.

[0048] For example, for the text "Xiaoming, who works in Beijing, has gone on a business trip to Nanjing", it is necessary to identify the named entities of each word, that is, "Xiaoming (person's name), who works in Beijing (place name), has gone on a business trip to Nanjing (place name)".

[0049] In practical applications, named entity recognition is a fundamental task in natural language processing. Through named entity recognition, various technical tasks such as text processing, semantic understanding, machine translation, robot question answering, and knowledge graphs can be completed.

[0050] Continue to refer to Figure 2In step 240, the text to be identified is transformed using multiple word vector models to obtain multiple text feature vectors of the text to be identified.

[0051] In this application, the text feature vector can be a mathematical matrix, which is a numerical feature representation of the text. It is understood that different text content will correspond to different mathematical matrices. Translating the text to be recognized into this mathematical matrix can be accomplished using a word vector model.

[0052] In this application, the plurality of text feature vectors may include a first text feature vector and a second text feature vector.

[0053] In one embodiment of step 240, the text to be identified is feature-transformed using multiple word vector models to obtain multiple text feature vectors of the text to be identified, which can be performed as follows: Figure 3 Perform the steps shown.

[0054] See Figure 3 This document illustrates a detailed flowchart of a method for feature transformation of the text to be identified using multiple word vector models, according to an embodiment of this application, to obtain multiple text feature vectors of the text to be identified. Specifically, it includes steps 241 to 242:

[0055] Step 241: The characters in the text to be identified are encoded using the various word vector models to obtain multiple text word vectors.

[0056] Step 242: Select any one of the plurality of text word vectors as the first text feature vector.

[0057] Step 243: Concatenate the multiple text word vectors to obtain a concatenated vector, and use the concatenated vector as the second text feature vector.

[0058] It should be noted that the text word vector is actually a kind of text feature vector. Here, in order to distinguish the final text feature vector, the text feature vector obtained directly through the word vector model is defined as a text word vector.

[0059] In this application, a word vector model encodes characters in the text to be recognized, resulting in a text word vector. After obtaining multiple text word vectors through multiple word vector models, any one of the multiple text word vectors can be selected as the first text feature vector. Furthermore, the multiple text word vectors are concatenated to obtain a concatenated vector, which is then used as the second text feature vector.

[0060] For example, for the text to be identified, "Xiaoming went to Shanghai", the text word vector A obtained through the A word vector model is:

[0061]

[0062] The text word vector B of the text to be identified is obtained through the B-word vector model:

[0063]

[0064] The concatenated vector after concatenating text word vectors A and B is:

[0065]

[0066] Therefore, in this embodiment, text word vector A or text word vector B can be used as the first text feature vector, and the concatenated vector can be used as the second text feature vector.

[0067] In this application, the text to be identified is transformed using multiple word vector models to obtain multiple text feature vectors of the text to be identified. The advantage of this is that multiple text feature vectors can increase the feature dimension of the characters in the text to be identified, ensure the diversity of the character feature representation, enhance the character feature representation, and thus improve the accuracy of entity recognition in the text to be identified.

[0068] In this application, the various word vector models may include the ALBERT model and the word2vec model.

[0069] Specifically, for the ALBERT model, Figure 4 A schematic diagram of the structure of an ALBERT model according to an embodiment of this application is shown.

[0070] like Figure 4 As shown, the ALBERT model uses a bidirectional Transformer to obtain the vector representation of the text to be recognized. The input layer 401 consists of "c1, c2, ..., c...". n " represents each character in the sequence. After training through multiple bidirectional Transformer encoding layers 402, the output layer 403 is finally obtained, which is the text word vector "e1, e2, ..., e" of the text to be recognized. n The Transformer model has an Encoder-Decoder structure, and the ALBERT model uses its Encoder part, which consists of multiple identical basic layers.

[0071] Specifically, for the word2vec model, the input text to be recognized is considered as a character sequence s = {c1, c2, ..., c...} n}∈V c V c It is a character vocabulary. Each character 'c'... i Represented by a dense vector (embedding):

[0072]

[0073] Among them, e c A lookup table representing character embeddings.

[0074] Continue to refer to Figure 2 In step 260, the pre-trained neural network model learns the multiple text feature vectors to obtain entity feature vectors, which are used to characterize the entity features in the text to be identified.

[0075] In this application, a pre-trained neural network model can identify entity features in the text to be identified through the text feature vector corresponding to the text to be identified. These entity features can be represented by entity feature vectors. In other words, entity features in the text to be identified are hidden in the entity feature vectors.

[0076] In one embodiment of step 260, the multiple text feature vectors are learned by a pre-trained neural network model to obtain entity feature vectors, which can be processed as follows: Figure 5 Perform the steps shown.

[0077] See Figure 5 This document illustrates a detailed flowchart of an embodiment of the present application, showing how a pre-trained neural network model learns multiple text feature vectors to obtain entity feature vectors. Specifically, it includes steps 261 to 262:

[0078] Step 261: Input the multiple text feature vectors into the neural network model respectively, so that the neural network model outputs multiple sub-entity feature vectors.

[0079] Step 262: Average the feature vectors of the multiple sub-entities to obtain the entity feature vector.

[0080] In this application, a text feature vector is input into the neural network model, and the neural network model can output a corresponding sub-entity feature vector. After obtaining multiple sub-entity feature vectors corresponding to multiple text feature vectors, the entity feature vector can be obtained by averaging the multiple sub-entity feature vectors.

[0081] For example, for sub-entity feature vectors A = [BCDE] and a = [bcde], calculate the averaged entity feature vector (A+a) / 2 = [(B+b) / 2 (C+c) / 2 (D+d) / 2 (E+e) / 2].

[0082] In this application, the neural network model may include a bidirectional long short-term memory recurrent neural network model (Bi-directional LSTM RNN, Bi-LSTM).

[0083] Specifically, the bidirectional long short-term memory recurrent neural network model can be defined as:

[0084]

[0085]

[0086] h t =o t ⊙tanh(c t (4)

[0087] Where σ is the sigmoid function of the elements, and ⊙ represents the product of the elements. W and b are trainable parameters. Reverse LSTM is similar to forward LSTM, but models the sequence in the reverse order. The i-th hidden state of forward and reverse LSTM. Forming c i Context-dependent representation.

[0088] In this application, the neural network model may also include a Convolutional Neural Network (CNN) model or a Transformer model, and may also include an architecture model composed of a bidirectional long short-term memory recurrent neural network model, a convolutional neural network model, and a Transformer model.

[0089] Continue to refer to Figure 2 In step 280, based on the entity feature vector, the entity label sequence corresponding to the text to be identified is determined by a conditional random field model, so as to identify the entity labels of each entity in the text to be identified.

[0090] In this application, the Conditional Random Field (CRF) model can identify the entity label sequence of the text to be identified through the entity feature vector of the text to be identified.

[0091] In this application, a conditional random field model can be applied to sequentially infer entity labels from the entire character sequence of the text to be identified, as follows:

[0092]

[0093] Here, y s Let represent all possible sequence labels of s, and Where w y′,y and b y′,y θ represents the trainable parameters corresponding to the label pair (y′, y), while θ represents the model parameters. For label inference, it searches for the label sequence y with the highest conditional probability given the input sequence s. * :

[0094]

[0095] Before step 280, that is, before determining the entity label sequence corresponding to the text to be identified based on the entity feature vector using a conditional random field model to identify the entity labels of each entity in the text to be identified, the following can also be performed: Figure 6 The steps are shown.

[0096] See Figure 6 This document illustrates a flowchart of a method according to an embodiment of the present application before determining the entity label sequence corresponding to the text to be identified using a conditional random field model. Specifically, it includes steps 271 to 272:

[0097] Step 271: Assign weights to each feature value in the entity feature vector using an attention mechanism model. The weights are used to distinguish the importance of each string in the text to be identified in terms of entity features.

[0098] Step 272: Adjust each feature value in the entity feature vector based on the weights to update the entity feature vector.

[0099] In this application, the attention mechanism model is actually a type of model based on the attention mechanism. Specifically, the attention mechanism allows the network to selectively enhance useful features, enabling subsequent processing to fully utilize these features while suppressing useless features. Based on this idea, the attention mechanism model can be used to distinguish the importance of the labels corresponding to each string in multi-label text data to be recognized, thereby improving the accuracy of entity recognition in the text.

[0100] In this application, as Figure 2 Before step 220 shown, that is, before obtaining the text to be recognized, the following can also be performed: Figure 7 The steps are shown.

[0101] See Figure 7This diagram illustrates a method flowchart according to an embodiment of the present application before obtaining the text to be recognized. Specifically, it includes steps 200 to 210:

[0102] Step 200: Obtain a training text set, which includes multiple training texts, each of which is configured with an entity label sequence.

[0103] Step 210: Based on multiple training texts in the training text set, supervised training is performed on the neural network model and the attention mechanism model through iteration, and the neural network model and the attention mechanism model are adjusted based on the entity label sequence configured in the training text until the neural network model and the attention mechanism model converge.

[0104] In one embodiment of step 200, the training text set is obtained, which can be done as follows: Figure 8 Perform the steps shown.

[0105] See Figure 8 This diagram illustrates a detailed flowchart of obtaining a training text set according to an embodiment of this application. Specifically, it includes steps 201 to 203:

[0106] Step 201: Obtain the first candidate text set, which includes multiple first candidate texts.

[0107] Step 202: Generate a label sequence for the first candidate text in the first candidate text set using a distant supervision model.

[0108] Step 203: Select a first candidate text from the first candidate text set as the training text to obtain the training text set.

[0109] In this application, the first candidate text in the first candidate text set does not have an entity label sequence configured. Therefore, an entity label sequence can be configured for the first candidate text in the first candidate text set through a distant supervision method. The advantage is that the distant supervision model can automatically generate labeled data for large-scale texts without manual annotation, which can improve the efficiency of generating training text data.

[0110] In one embodiment of step 203, a first candidate text is selected from the first candidate text set as training text to obtain the training text set, which can be done according to the following... Figure 9 Perform the steps shown.

[0111] See Figure 9 This document illustrates a detailed flowchart of a process for selecting a first candidate text from a first candidate text set as training text, according to an embodiment of this application. Specifically, it includes steps 2031 to 2033:

[0112] Step 2031: Obtain a second candidate text set, which includes multiple second candidate texts, each of which is configured with an entity label sequence.

[0113] Step 2032: Based on the second candidate text set, train a reinforcement learning model, which is used to identify incomplete label sequences or noisy label sequences in the candidate text.

[0114] Step 2033: Using the reinforcement learning model, select a first candidate text with a complete label sequence or a non-noise label sequence from the first candidate text set as the training text.

[0115] In this application, reinforcement learning (RL) models are used to describe and solve the problem of how an agent learns strategies to maximize rewards or achieve specific goals during its interaction with the environment.

[0116] To enable those skilled in the art to better understand this application, the following will be combined with Figure 10 A brief explanation of the relationships between the models involved in this application is provided.

[0117] Reference Figure 10 The diagram illustrates a structural schematic of an overall model for text entity data processing according to an embodiment of this application.

[0118] Firstly, in the first aspect, such as Figure 10 As shown, a first candidate text set 1001 is obtained, and a label sequence is generated for the first candidate text in the first candidate text set 1001 using a distant supervision model, resulting in a first candidate text set 1001 configured with entity label sequences. A second candidate text set 1002 configured with entity label sequences is then obtained, and a reinforcement learning model 1005 (i.e., an agent) is trained using the second candidate text set 1002. During the training of the reinforcement learning model 1005, the learning parameters in the reinforcement learning model 1005 are updated through rewards and suppression to improve the model performance of the reinforcement learning model 1005. Using the reinforcement learning model 1005, a first candidate text with a complete label sequence or a non-noise label sequence is selected from the first candidate text set as training text, resulting in a training text set 1003.

[0119] In the second aspect, feature transformation is performed on the training samples 1006 in the training text set 1003 using the ALBERT model 1008 and the word2vec model 1007 to obtain the entity feature vectors of the training samples. Based on the entity feature vectors, the bidirectional long short-term memory recurrent neural network model 1009, the attention mechanism model 1010, and the conditional random field model 1011 are trained sequentially. During the training process, the entity label sequence of the training text is output from the conditional random field model 1011. By comparing the output entity label sequence with the entity label sequence configured in the training text, the bidirectional long short-term memory recurrent neural network model 1009 and the attention mechanism model 1010 are adjusted, and rewards and inhibitions are provided for the reinforcement learning model 1005.

[0120] It should be noted that for incomplete label sequences in the candidate text, each mismatched character is considered to be labeled as any appropriate entity label. Let each candidate text obtained through the far-supervised model represent a set of entity label sequences Z, whose probability is the sum of the probabilities of each possible label sequence y in Z. Applying softmax to all candidate output label sequences, the probability of a far-supervised instance is calculated as follows:

[0121]

[0122] Then, CRF-PA is used to obtain the loss function:

[0123] loss(Θ,x,z)=-logp(z|x) (8)

[0124] It should also be noted that for the noisy label sequence in the candidate texts, the initial second candidate text set and the first candidate text set are merged into a single candidate text set. An instance bag of random size is collected from the candidate text set. For each far-supervised instance in the instance bag, an action is performed from the set {1,0} to determine whether to select that instance. When all actions are completed, the agent receives a reward. The reward represents the action feedback on the instance bag and will be used to update the agent, whose goal is to perform the correct action to maximize the reward.

[0125] In this application, multiple word vector models are used to perform feature transformation on the text to be identified, resulting in multiple text feature vectors for neural network model learning. These vectors then form entity feature vectors that characterize the entity features within the text. Finally, based on these entity feature vectors, a conditional random field model is used to determine the entity label sequence corresponding to the text. Because multiple text feature vectors increase the feature dimension of characters in the text, ensuring diverse character feature representations and enhancing character feature representation, the accuracy of entity recognition in the text can be improved.

[0126] The following describes an apparatus embodiment of this application, which can be used to execute the text entity recognition method in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the text entity recognition method described above.

[0127] Figure 11 A block diagram of a text entity recognition device according to an embodiment of this application is shown.

[0128] Reference Figure 11 As shown, a text entity recognition device 1100 according to an embodiment of this application includes: a first acquisition unit 1401, a conversion unit 1402, a learning unit 1403, and a determination unit 1404.

[0129] The first acquisition unit 1401 is used to acquire the text to be identified; the conversion unit 1402 is used to perform feature conversion on the text to be identified through multiple word vector models to obtain multiple text feature vectors of the text to be identified; the learning unit 1403 is used to learn the multiple text feature vectors by a pre-trained neural network model to obtain entity feature vectors, which are used to characterize the entity features in the text to be identified; and the determination unit 1404 is used to determine the entity label sequence corresponding to the text to be identified based on the entity feature vectors through a conditional random field model, so as to identify the entity labels of each entity in the text to be identified.

[0130] In some embodiments of this application, based on the foregoing scheme, the plurality of text feature vectors include a first text feature vector and a second text feature vector. The conversion unit 1402 is configured to: encode the characters in the text to be identified through the plurality of word vector models respectively to obtain a plurality of text word vectors; select any one of the plurality of text word vectors as the first text feature vector; concatenate the plurality of text word vectors to obtain a concatenated vector, and use the concatenated vector as the second text feature vector.

[0131] In some embodiments of this application, based on the foregoing scheme, the learning unit 1403 is configured to: input the plurality of text feature vectors into the neural network model respectively, so that the neural network model outputs a plurality of sub-entity feature vectors; and average the plurality of sub-entity feature vectors to obtain the entity feature vector.

[0132] In some embodiments of this application, based on the foregoing scheme, the apparatus further includes: an allocation unit, configured to assign weights to each feature value in the entity feature vector using an attention mechanism model before determining the entity label sequence corresponding to the text to be identified using a conditional random field model based on the entity feature vector, wherein the weights are used to distinguish the importance of each string in the text to be identified in terms of entity features; and an adjustment unit, configured to adjust each feature value in the entity feature vector based on the weights to update the entity feature vector.

[0133] In some embodiments of this application, based on the foregoing scheme, the device further includes a second acquisition unit, used to acquire a training text set before acquiring the text to be recognized, the training text set including multiple training texts, wherein each training text is configured with an entity label sequence; and a training unit, used to perform supervised training on the neural network model and the attention mechanism model in an iterative manner based on the multiple training texts in the training text set, and adjust the neural network model and the attention mechanism model based on the entity label sequence configured in the training texts until the neural network model and the attention mechanism model converge.

[0134] In some embodiments of this application, based on the foregoing scheme, the second acquisition unit is configured to: acquire a first candidate text set, the first candidate text set including multiple first candidate texts; generate a label sequence for the first candidate texts in the first candidate text set using a far-supervised model; select a first candidate text from the first candidate text set as training text, and obtain the training text set.

[0135] In some embodiments of this application, based on the foregoing scheme, the second acquisition unit is configured to: acquire a second candidate text set, the second candidate text set including multiple second candidate texts, wherein each second candidate text is configured with an entity label sequence; train a reinforcement learning model based on the second candidate text set, the reinforcement learning model being used to identify incomplete label sequences or noisy label sequences in the candidate texts; and select a first candidate text with a complete label sequence or a non-noisy label sequence from the first candidate text set as training text through the reinforcement learning model.

[0136] In some embodiments of this application, based on the foregoing scheme, the various word vector models include the ALBERT model and the word2vec model.

[0137] In some embodiments of this application, based on the foregoing scheme, the neural network model includes a bidirectional long short-term memory recurrent neural network model.

[0138] Figure 12 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0139] It should be noted that, Figure 12 The computer system 1200 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 12 As shown, the computer system 1200 includes a Central Processing Unit (CPU) 1201, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1202 or programs loaded from storage portion 1208 into Random Access Memory (RAM) 1203, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1203. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. An Input / Output (I / O) interface 1205 is also connected to bus 1204.

[0141] The following components are connected to I / O interface 1205: an input section 1206 including a keyboard, mouse, etc.; an output section 1207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to I / O interface 1205 as needed. Removable media 1211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1210 as needed so that computer programs read from them can be installed into storage section 1208 as needed.

[0142] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1209, and / or installed from removable medium 1211. When the computer program is executed by central processing unit (CPU) 1201, it performs various functions defined in the system of this application.

[0143] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0146] In another aspect, this application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the text entity recognition method described in the above embodiments.

[0147] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the text entity recognition method described in the above embodiments.

[0148] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0149] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0150] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0151] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A text entity recognition method, characterized in that, The method includes: Obtain a first candidate text set, which includes multiple candidate texts; generate a label sequence for the candidate texts in the first candidate text set using a distant supervision model; Obtain a second candidate text set, which includes multiple candidate texts, each candidate text being configured with an entity label sequence; based on the second candidate text set, train a reinforcement learning model, which is used to identify incomplete label sequences or noisy label sequences in the candidate texts; Through the reinforcement learning model, a first candidate text with a complete label sequence or a non-noise label sequence is selected from the first candidate text set as a training text to obtain a training text set. The training text set includes multiple training texts, each of which is configured with an entity label sequence. Based on multiple training texts in the training text set, the neural network model and the attention mechanism model are trained in a supervised manner through iteration, and the neural network model and the attention mechanism model are adjusted based on the entity label sequence configured in the training text until the neural network model and the attention mechanism model converge. Obtain the text to be recognized; The text to be identified is transformed by using multiple word vector models to obtain multiple text feature vectors of the text to be identified; The neural network model learns the multiple text feature vectors to obtain entity feature vectors, which are used to characterize the entity features in the text to be identified. The attention mechanism model assigns weights to each feature value in the entity feature vector, and the weights are used to distinguish the importance of each string in the text to be identified in terms of entity features; the entity feature vector is updated by adjusting each feature value in the entity feature vector based on the weights. Based on the entity feature vector, the entity label sequence corresponding to the text to be identified is determined by the conditional random field model, so as to identify the entity label of each entity in the text to be identified.

2. The method according to claim 1, characterized in that, The plurality of text feature vectors includes a first text feature vector and a second text feature vector. The process of performing feature transformation on the text to be identified using multiple word vector models to obtain the plurality of text feature vectors of the text to be identified includes: The characters in the text to be identified are encoded using the various word vector models to obtain multiple text word vectors. Select any one of the plurality of text word vectors as the first text feature vector; The multiple text word vectors are concatenated to obtain a concatenated vector, which is then used as the second text feature vector.

3. The method according to claim 1, characterized in that, The process of learning the multiple text feature vectors by the neural network model to obtain entity feature vectors includes: The multiple text feature vectors are respectively input into the neural network model, so that the neural network model outputs multiple sub-entity feature vectors; The entity feature vector is obtained by averaging the feature vectors of the multiple sub-entities.

4. The method according to claim 1, characterized in that, The various word vector models include the ALBERT model and the word2vec model.

5. The method according to claim 1, characterized in that, The neural network model includes a bidirectional long short-term memory recurrent neural network model.

6. A text entity recognition device, characterized in that, The device includes: The second acquisition unit is used to acquire a first candidate text set, which includes multiple first candidate texts; generate label sequences for the first candidate texts in the first candidate text set using a far-supervised model; acquire a second candidate text set, which includes multiple second candidate texts, each of which is configured with an entity label sequence; train a reinforcement learning model based on the second candidate text set, the reinforcement learning model being used to identify incomplete label sequences or noisy label sequences in the candidate texts; and select first candidate texts with complete label sequences or non-noisy label sequences from the first candidate text set using the reinforcement learning model as training texts to obtain a training text set, which includes multiple training texts, each of which is configured with an entity label sequence. The training unit is used to perform supervised training on the neural network model and the attention mechanism model in an iterative manner based on multiple training texts in the training text set, and to adjust the neural network model and the attention mechanism model based on the entity label sequence configured in the training text until the neural network model and the attention mechanism model converge. The first acquisition unit is used to acquire the text to be recognized; The conversion unit is used to perform feature conversion on the text to be identified using multiple word vector models to obtain multiple text feature vectors of the text to be identified; The learning unit is used to learn the multiple text feature vectors by the neural network model to obtain entity feature vectors, which are used to characterize entity features in the text to be identified. The allocation unit is used to assign weights to each feature value in the entity feature vector through the attention mechanism model, and the weights are used to distinguish the importance of each string in the text to be identified in the entity features; the adjustment unit is used to adjust each feature value in the entity feature vector based on the weights to update the entity feature vector. The determining unit is used to determine the entity label sequence corresponding to the text to be identified based on the entity feature vector through a conditional random field model, so as to identify the entity label of each entity in the text to be identified.

Images