Named Entity Recognition Method, Apparatus and Device

Abstract

The present application discloses a named entity recognition method, apparatus and device. Among them, the named entity recognition method includes: determining a first text feature and a first text structure feature of a target word in a text to be processed; determining a second text structure feature of the target word entering a memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; determining a second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; determining the named entity type of the target word at least according to the second text feature of the target word. By adopting this processing method, it is possible to better integrate the two features of text linear feature and text structure feature; therefore, the recognition accuracy can be effectively improved.

Description

Technical Field

[0001] This application relates to the technical field of natural language processing, and specifically to a named entity recognition method, apparatus, and device. Background Art

[0002] In an e-commerce scenario, named entity recognition can be used to analyze the key entity words in a user's search and the entity words in a product name, thereby helping a buyer more accurately locate relevant products for the search. Named entity recognition is to extract named entities from unstructured text and classify the extracted named entities. For example, in the product name "Huawei Honor V30 mobile phone", "Huawei" is extracted as the mobile phone brand, and "Honor V30" is extracted as the mobile phone model.

[0003] Currently, a typical named entity recognition method is to extract text features through a general neural network model (such as a bidirectional long short-term memory network Bi-LSTM), and then input the extracted features into a conditional random field model (CRF) for sequence labeling, thereby obtaining which named entities appear in the text.

[0004] However, in the process of implementing the present invention, the inventor found that the above solution has at least the following problems: 1) The general model does not well integrate text linear features and text structure features, and finally the extracted features are not well represented, which limits the accuracy of named entity recognition; 2) The meaning of the same word is different in different texts affected by context information, and it is difficult for a general neural network model to capture the dependency relationship between words over a long distance. When the text is too long, information will be lost, and the text structure information existing in the sentence cannot be fully captured, so it is difficult to extract the meaning of the target word in the entire text. These reasons also limit the performance of the existing models in the task of named entity recognition, resulting in a low accuracy of named entity recognition. In summary, how to improve the accuracy of named entity recognition and then improve the search accuracy has become an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] This application provides a named entity recognition method to solve the problem of low accuracy of named entity recognition in the prior art. This application also provides a named entity recognition apparatus and device.

[0006] This application provides a named entity recognition method, including:

[0007] Determine the first text feature and the first text structure feature of the target word in the text to be processed;

[0008] Determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state;

[0009] Determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word;

[0010] Determine the named entity type of the target word at least according to the second text feature of the target word.

[0011] Optionally, determine the first text structure feature of the target word through the structure information extraction module included in the named entity recognition model;

[0012] Through the feature memory sub-module included in the text feature determination module of the named entity recognition model, determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state;

[0013] Through the output sub-module included in the text feature determination module, determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word.

[0014] Through the classifier included in the named entity recognition model, determine the named entity type of the target word at least according to the second text feature of the target word.

[0015] Optionally, it further includes:

[0016] Through the feature memory sub-module, determine the third text structure feature of the target word entering the memory state at least according to the second text feature of the previous word of the target word that fuses text linear information and text structure information, the first text structure feature of the target word, and the third text structure feature of the previous word entering the memory state, where the third text feature includes the second text structure feature of entering the memory state;

[0017] Through the output sub-module, determine the second text feature of the target word at least according to the first text feature of the target word, the second text feature of the previous word, the first text structure feature of the target word, and the third text structure feature of the target word entering the memory state.

[0018] Optionally, it further includes:

[0019] The feature memory sub-module determines the second text structure feature of the target word entering the memory state at least according to the second text feature of the previous word of the target word that fuses text linear information and text structure information and the first text structure feature of the target word through the feature memory controller;

[0020] The feature memory sub-module determines the third text structure feature of the target word entering the memory state at least according to the second text structure feature of the target word entering the memory state and the third text structure feature of the previous word entering the memory state through the feature memory.

[0021] Optionally, it further includes:

[0022] The output sub-module determines the fourth text feature through the first output controller, at least according to the first text feature and the first text structure feature of the target word and the second text feature of the previous word;

[0023] The output sub-module determines the second text feature of the target word through the second output controller, at least according to the fourth text feature and the third text feature of the target word entering the memory state.

[0024] Optionally, it further includes:

[0025] Determining the first text structure feature of the target word through the structure information extraction module included in the named entity recognition model includes:

[0026] Determining the word dependency relationship of the text, where the word dependency relationship includes the dependency relationship type;

[0027] According to the word dependency relationship, determining a word dependency relationship matrix;

[0028] According to the matrix, the word vector of the target word, and the dependency relationship type vector, determining the third text structure feature of the target word;

[0029] Through the structure information extraction module based on a multi-layer graph neural network, according to the third text structure feature of the target word, determining the first text structure feature of the target word.

[0030] Optionally, the determining the first text structure feature of the target word through the structure information extraction module based on a multi-layer graph neural network according to the third text structure feature of the target word includes:

[0031] Through the first-layer graph neural network included in the structure information extraction module, according to the third text structure feature, determining the fourth text structure feature of the target word;

[0032] Through the second-layer graph neural network included in the structure information extraction module, according to the fourth text structure feature, determining the first text structure feature of the target word.

[0033] Optionally, it further includes:

[0034] Through the feature forgetting sub-module included in the named entity recognition model, according to the second text feature of the previous word, the first text feature and the first text structure feature of the target word, determining the feature of the previous word losing memory.

[0035] This application also provides a method for constructing a named entity recognition model, including:

[0036] Obtain a set of corresponding relationships between text and named entity annotation data;

[0037] Construct the network structure of a named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of a target word; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module is used to determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word;

[0038] Learn a named entity recognition model according to the set of corresponding relationships.

[0039] This application also provides a named entity recognition device, including:

[0040] A first feature determination unit, configured to determine the first text feature and the first text structure feature of a target word in the text to be processed;

[0041] A text structure feature control unit, configured to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state;

[0042] A second feature determination unit, configured to determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word;

[0043] A classification unit, configured to determine the named entity type of the target word at least according to the second text feature of the target word.

[0044] This application also provides a named entity recognition model construction device, including:

[0045] A training data acquisition unit, configured to obtain a set of corresponding relationships between text and named entity annotation data;

[0046] A network construction unit for constructing the network structure of a named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of a target word; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module is used to determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word;

[0047] A training unit for learning a named entity recognition model according to the corresponding relationship set.

[0048] This application also provides a computer-readable storage medium, in which instructions are stored, and when it runs on a computer, it causes the computer to execute the above various methods.

[0049] This application also provides a computer program product including instructions, and when it runs on a computer, it causes the computer to execute the above various methods.

[0050] Compared with the prior art, this application has the following advantages:

[0051] The named entity recognition method provided by the embodiment of this application determines the first text feature and the first text structure feature of a target word in the text to be processed; determines the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; determines the second text feature of the target word at least according to the first text feature of the target word and the second text structure feature of the target word entering the memory state; determines the named entity type of the target word at least according to the second text feature of the target word; this processing method enables better integration of the two features of text linear features and text structure features; therefore, the recognition accuracy can be effectively improved.

[0052] The method for constructing a named entity recognition model provided by an embodiment of the present application includes obtaining a set of corresponding relationships between texts and named entity annotation data; constructing a network structure of the named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of a target word; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module is used to determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; learning a named entity recognition model according to the set of corresponding relationships; this processing method enables better integration of two features, namely text linear features and text structure features, thereby enhancing the model prediction ability; therefore, the model accuracy can be effectively improved. Description of the Drawings

[0053] Figure 1 Flow chart of an embodiment of a named entity recognition method provided by the present application;

[0054] Figure 2 Schematic diagram of the model structure of an embodiment of a named entity recognition method provided by the present application;

[0055] Figure 3 Schematic diagram of the structure of the text feature determination module of an embodiment of a named entity recognition method provided by the present application;

[0056] Figure 4 Schematic diagram of the word dependency relationship of an embodiment of a named entity recognition method provided by the present application;

[0057] Figure 5 Schematic diagram of the structure information extraction module based on a two-layer graph neural network of an embodiment of a named entity recognition method provided by the present application. Detailed Embodiment

[0058] Many specific details are set forth in the following description in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0059] In the present application, a named entity recognition method and device, a named entity recognition model construction method and device, and an electronic device are provided. Each solution will be described in detail in the following embodiments.

[0060] First Embodiment

[0061] Please refer to Figure 1 , which is a schematic flowchart of an embodiment of the named entity recognition method of the present application. The execution subject of the method includes but is not limited to the server, and can also be any device capable of implementing the method. In this embodiment, the method may include the following steps:

[0062] Step S101: Determine the first text feature and the first text structure feature of the target word in the text to be processed.

[0063] The text to be processed may be a sentence composed of multiple words. Named Entity Recognition (NER), also known as "proper name recognition", refers to identifying entities with specific meanings in the text, mainly including person names, place names, organization names, proper nouns, etc.

[0064] For example, the text to be processed is: "ACM announced that the three creators of deep learning, Yoshua Bengio, Yann LeCun, and Geoffrey Hinton, won the Turing Award in 2019". The task of NER is to extract "organization name: ACM", "person names: Yoshua Bengio, Yann LeCun, Geoffrey Hinton", "time: 2019", and "proper noun: Turing Award" from this sentence.

[0065] For example, in the scenario of e-commerce, when different users search for products on the e-commerce platform through the client, the server performs named entity recognition on the search terms specified by the users through the named entity recognition model, so as to accurately identify entities with specific meanings such as regions, brands, and product names, which can effectively improve the user experience. For another example, the server can also perform entity recognition on brand names, materials, consumer groups, regions, product categories, etc. of the product descriptions in the product library through the named entity recognition model, so as to recommend products that the users are interested in to the users.

[0066] For the text to be processed, the named entity recognition model can be used to predict the named entity types of each word respectively. The named entity recognition model includes a text feature determination module and a classifier. Among them, the text feature determination module is used to determine the features of each word in the text to be processed (i.e., the second text feature); the classifier is used to determine the named entity type of the word according to the second text feature, so as to obtain the named entities existing in the text to be processed.

[0067] The text feature determination module can be a neural network-based model, which obtains a better text information-based word feature vector, that is, the second text feature, by fusing the linear text feature and the structural information feature of the target word.

[0068] The classifier can adopt structures such as conditional random field (CRF). Taking the conditional random field as an example, this layer mainly uses the conditional random field to capture the relationship between each word and its corresponding label. Through the conditional random field, the optimal label can be found overall based on the relationship between labels, that is, considering the mutual influence between labels, finding the optimal sequence label group, and finally determining the named entity type of each word, so as to obtain the named entities existing in the text to be processed.

[0069] The input data of the named entity recognition model includes: the first text features of each word, such as Figure 2 X in t-1 、X t 、X t+1 and X t+2 etc., and the first text structure features etc. Suppose the target word is the t-th word in the text to be processed, then the first text feature of the target word is X t , and the first text structure feature of the target word is

[0070] The first text features include, but are not limited to: word vectors, character vectors, dependency relation vectors, and part-of-speech vectors. The first text structure features include, but are not limited to: word vectors, character vectors, and dependency relation vectors. Among them, the word vector can adopt 100-dimensional Glove, the character vector can be obtained by a bidirectional LSTM, and the dependency relation vector and part-of-speech vector are randomly obtained.

[0071] Such as Figure 2 shown, in this embodiment, the named entity recognition model may further include a structure information extraction module for determining the first text structure feature of the target word The structure information extraction module can adopt the structure of a graph neural network or other network structures, such as a recurrent neural network, etc.

[0072] In this embodiment, through the structure information extraction module based on the graph neural network, according to the word dependency relationship in the text to be processed, the first text structure feature of the target word is determined Since the graph neural network will consider the adjacent relationship between word i and word j on the graph during operation, it will not weaken the relationship between long-distance words due to the length of the distance between word i and word j in a sentence. Thus, it can be seen that since the graph neural network utilizes the dependency relationship structure, the output of the graph neural network has the structure information in the text, and the graph neural network based on the dependency relationship graph can better capture the relationship between long-distance words.

[0073] A graph neural network means that given a graph with n nodes, the structure of this graph can be represented as an n*n matrix A. For an l-layer graph neural network, its expression can be defined as:

[0074]

[0075] Among them, i represents the i-th node, A(i, j) represents that node i and node j are connected, and h_j^(l - 1) and h_i^(l) respectively represent the input and output vectors of node i at layer l. W and b are learnable parameters.

[0076] In an example, determining the first text structure feature of the target word by the structure information extraction module included in the named entity recognition model may include the following sub-steps:

[0077] 1) Determine the word dependency relationship of the text, and the word dependency relationship includes the dependency relationship type.

[0078] The word dependency relationship of the text refers to the dependency relationship between words in the text. The word dependency relationship can be determined by the prior art, so it will not be elaborated here.

[0079] Such as Figure 4 shown, the text to be processed is "Precision Castparts Corp.,Portlan,will begintrading with the symbol PCP.", where the word "Corp" depends on "Castparts", ",", "Precision", "Portlan"; the word "Portlan" depends on ","; the word "begin" depends on "will", "trading", ".", "Corp"; the word "trading" depends on "with"; the word "with" depends on "symbol"; the word "symbol" depends on "the", "PCP".

[0080] The word dependency relationship includes the dependency relationship type, and the dependency relationship type can be a syntactic relationship, etc., such as a verb-object relationship, a subject-predicate relationship, a modifying relationship (who modifies who), and so on.

[0081] 2) Determine the word dependency relationship matrix according to the word dependency relationship.

[0082] In this step, the word dependency relationship is converted into the corresponding word dependency relationship matrix A, where A(i, j) represents that there is a dependency relationship between word i and word j.

[0083] 3) Determine the third text structure feature of the target word according to the matrix, the word vector of the target word, and the dependency relationship type vector, such as Figure 2 shown, the third text structure feature of the t-th target word is expressed as

[0084] 4) Through the structure information extraction module based on a multi-layer graph neural network, according to the third text structure feature of the target word, determine the first text structure feature of the target word

[0085] It can be found from the above graph neural network expression that a single-layer graph neural network can only exchange information between the target node and its adjacent nodes, but a multi-layer graph neural network can exchange information with its distant nodes. In specific implementation, a five-layer graph neural network can be adopted, which can capture the five-layer cascaded dependency relationship between words. For example Figure 4 in the word "Corp.", after five-level cascading, it depends on "PCP", so that the relationship between long-distance words can be captured

[0086] However, although the five-layer graph neural network can capture the relationship between long-distance words, there is relatively large noise. Through experiments, it is found that combining a two-layer graph neural network with the feature fusion method provided in the embodiments of the present application can achieve a better recognition effect. For example Figure 4 As shown, by combining two-layer graph neural networks, it can not only capture the cascaded dependency relationship between long-distance words above two layers. For example, the word "Corp." can be related to "PCP" through a two-layer cascading relationship in the mixed path, but also effectively reduce noise

[0087] For example Figure 5 As shown, in another example, the step of determining the first text structure feature of the target word through the structure information extraction module based on a multi-layer graph neural network according to the third text structure feature of the target word may include the following sub-steps: 1) Through the first-layer graph neural network included in the structure information extraction module, according to the third text structure feature (input vector ), determine the fourth text structure feature of the target word; 2) Through the second-layer graph neural network included in the structure information extraction module, according to the fourth text structure feature, determine the first text structure feature of the target word

[0088] After determining the first text feature and the first text structure feature of the target word through step S101, the second text feature of the target word can be determined through the unique feature fusion method provided in the embodiments of the present application

[0089] Step S103: Determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state

[0090] The named entity recognition model provided by the embodiment of the present application is different from the model stacking of ordinary graph neural networks and LSTM. It adopts a brand-new way to fuse the structural information of words and text information, that is, to fuse the first text feature and the first text structure feature of the target word. The model adopted in this embodiment can introduce the structural information of words (also known as the first text structure feature, that is, the structural information of the current state is introduced in each state) for each word, and can determine the amount of structural information input according to the usefulness of the structural information (that is, the second text structure feature of each word entering the memory state). Such a design can effectively extract the meaning of each word in the text to be processed (that is, the second text feature of the target word), so that the same word has different meanings in different texts.

[0091] The above-mentioned previous words may include multiple words in the text to be processed that are arranged in front of the target word. The above-mentioned previous words may include the previous word adjacent to the target word, or may not include the previous word adjacent to the target word. In this embodiment, through the text feature determination module in the named entity recognition model, the second text structure features of each word in the text to be processed entering the memory state can be determined. The second text structure features of the previous words entering the memory state may include the second text structure features of multiple previous words entering the memory state. This step can determine the second text structure feature of the target word entering the memory state, that is, determine the amount of structural information input of the target word, at least based on the first text structure feature of the target word and the second text structure features of the previous words entering the memory state.

[0092] In an example, through the feature memory sub-module included in the text feature determination module of the named entity recognition model, the second text structure feature of the target word entering the memory state is determined at least based on the first text structure feature of the target word and the second text structure features of the previous words entering the memory state.

[0093] The text feature determination module can use a bidirectional LSTM to fuse ordinary linear text features and structural information features (the first text structure feature of the target word) to obtain a better word feature vector based on text information. After the feature memory sub-module in the bidirectional LSTM is transformed, it can determine the amount of structural information input according to the usefulness of the structural information (the first text structure feature of the target word).

[0094] In this embodiment, through the feature memory sub-module, at least based on the second text feature h of the previous word of the target word that fuses text linear information and text structure information t-1 、the first text structure feature of the target word and the third text feature C of the previous words entering the memory state t-1 (including the second text structure feature m of the previous word entering the memory state t-1) Determine the third text feature C of the target word entering the memory state t (May include the second text structure feature m of the target word entering the memory state t ). Among them, the third text feature includes the second text structure feature entering the memory state, and may also include features such as text linearity entering the memory state. The text linear information belongs to a common concept in the prior art and will not be elaborated here.

[0095] Such as Figure 3 shown, the feature memory sub-module passes through the feature memory controller (newly designed m t gate), and at least determines the second text structure feature m of the target word entering the memory state according to the second text feature h of the previous word of the target word that fuses text linear information and text structure information t-1 、and the first text structure feature of the target word ; the feature memory sub-module passes through the feature memory, and at least determines the second text structure feature m of the target word entering the memory state according to the second text structure feature m of the target word entering the memory state t 、and the third text feature C of the previous word entering the memory state t 、and determines the third text feature C of the target word entering the memory state t-1 。 t .

[0096] After determining the second text structure feature of the target word that can enter the memory state, the next step can be carried out, and at least according to the first text feature and the second text structure feature of the target word, determine the second text feature of the target word.

[0097] Step S105: Determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word.

[0098] The method provided by the embodiment of the present application better fuses the two features of the text linear feature and the text structure feature of the target word through steps S103 and S105. In this embodiment, through the output sub-module included in the text feature determination module, at least according to the first text feature and the second text structure feature of the target word, determine the second text feature of the target word.

[0099] Such as Figure 3 shown, specifically in implementation, through the output sub-module, at least according to the first text feature X of the target word t 、the second text feature h of the previous word t-1 、the first text structure feature of the target word and the third text feature C of the target word entering the memory state t , determine the second text feature h of the target word t .

[0100] As Figure 3 shown, in specific implementation, the output sub-module passes through the first output controller O t gate, and determines the fourth text feature O at least according to the first text feature X t of the target word and the first text structure feature the second text feature h of the previous word t-1 ; the output sub-module passes through the second output controller, and determines the second text feature h of the target word at least according to the fourth text feature O t and the third text feature C of the target word entering the memory state t 。 t t

[0101] As Figure 3 shown, in this embodiment, the feature forgetting sub-module f included in the named entity recognition model can also be used t gate to determine the feature of the previous word losing memory according to the second text feature h t-1 of the previous word, the first text feature X t of the target word and the first text structure feature 。

[0102] Combined Figure 2 , Figure 3 and Figure 4 , the named entity recognition model provided in this embodiment adds a structure information extraction module (graph-encoded representation, g_t) based on a two-layer graph neural network on the basis of the original LSTM to input and capture the text structure features Corresponding to the newly added structure information extraction module, a new gate (feature memory controller) is also designed to control the amount of structure information entering the memory state (cell state). Figure 3 The f t , i t , m t and o t ​​They respectively represent the output data of the forget gate, input gate, newly added gate (feature memory controller), and output gate. Among them, the forget gate is used to forget some information of the previous state, only retaining the remaining information that may be valuable for subsequent states. In addition, since not all current text information is useful, the input gate can be used to control how much information of the current state is input. The newly added feature memory controller corresponds to the previous input gate. Since not all current structural information is useful, this gate controls how much structural information is input. The output gate is used to control how much of the current memory information can represent the second text feature of the target word. At the same time, the sizes of the forget gate and the output gate can be determined by the input X of the target word t , and the hidden state h of the previous layer t-1 . The model adopting this result can not only capture the relationship between words over long distances but also effectively reduce noise. Therefore, it can better integrate the text linear features and text structure features, thereby effectively improving the recognition accuracy.

[0103] Step S107: Determine the named entity type of the target word based on at least the second text feature of the target word.

[0104] After determining the meaning of the target word in the text to be processed (i.e., the second text feature), the named entity type of the target word can be determined by the classifier included in the named entity recognition model based on at least the second text feature of the target word.

[0105] As can be seen from the above embodiments, the named entity recognition method provided by the embodiments of the present application determines the first text feature and the first text structure feature of the target word in the text to be processed; determines the second text structure feature of the target word entering the memory state based on at least the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; determines the second text feature of the target word based on at least the first text feature of the target word and the second text structure feature of the target word entering the memory state; determines the named entity type of the target word based on at least the second text feature of the target word; this processing method enables better integration of these two features, namely, text linear features and text structure features; therefore, the recognition accuracy can be effectively improved.

[0106] Second Embodiment

[0107] In the above embodiment, a named entity recognition method is provided. Correspondingly, the present application also provides a named entity recognition device. This device corresponds to the embodiment of the above method. The parts of this embodiment that are the same as those in the first embodiment will not be described in detail again. Please refer to the corresponding parts in the first embodiment.

[0108] A named entity recognition device provided by the present application includes:

[0109] A first feature determination unit for determining a first text feature and a first text structure feature of a target word in a text to be processed;

[0110] A text structure feature control unit for determining a second text structure feature of the target word in a memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word in a memory state;

[0111] A second feature determination unit for determining a second text feature of the target word at least according to the first text feature of the target word and the second text structure feature;

[0112] A classification unit for determining the named entity type of the target word at least according to the second text feature of the target word.

[0113] The third embodiment

[0114] The present application also provides an electronic device. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For related parts, please refer to the corresponding description in the method embodiment. The device embodiment described below is only illustrative.

[0115] An electronic device according to this embodiment, the electronic device includes: a processor and a memory; the memory is used to store a program for implementing the named entity recognition method. After the device is powered on and runs the program of this method through the processor, the following steps are executed: determining a first text feature and a first text structure feature of a target word in a text to be processed; determining a second text structure feature of the target word in a memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word in a memory state; determining a second text feature of the target word at least according to the first text feature of the target word and the second text structure feature of the target word in a memory state; determining the named entity type of the target word at least according to the second text feature of the target word.

[0116] The fourth embodiment

[0117] In the above embodiment, a named entity recognition method is provided. Correspondingly, the present application also provides a named entity recognition model construction method. The execution subject of the method includes but is not limited to a server, and may also be any device capable of implementing the method. This method corresponds to the embodiment of the above method. The parts of this embodiment that are the same as those in the first embodiment will not be described in detail. Please refer to the corresponding parts in the first embodiment.

[0118] In this embodiment, the method may include the following steps:

[0119] Step 1: Obtain the set of corresponding relationships between the text and the named entity annotation data.

[0120] The text includes sentences, and the set of corresponding relationships serves as the training data set for the model.

[0121] Step 2: Construct the network structure of the named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of the target word; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module determines the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word.

[0122] The network structure of the model corresponds to the model in the first embodiment. For details, see the relevant description in the first embodiment, which will not be elaborated here.

[0123] Step 3: Learn to obtain the named entity recognition model according to the set of corresponding relationships.

[0124] Training the model parameters according to the training data belongs to the relatively mature prior art, so it will not be elaborated here.

[0125] As can be seen from the above embodiments, the method for constructing a named entity recognition model provided by the embodiments of the present application obtains the set of corresponding relationships between the text and the named entity annotation data; constructs the network structure of the named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of the target word; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module determines the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; learns to obtain the named entity recognition model according to the set of corresponding relationships; this processing method enables better integration of the two features of text linear features and text structure features, thereby enhancing the model prediction ability; therefore, the model accuracy can be effectively improved.

[0126] The Fifth Embodiment

[0127] In the above embodiments, a method for constructing a named entity recognition model is provided. Correspondingly, the present application also provides a device for constructing a named entity recognition model. This device corresponds to the embodiments of the above method. For the parts that are the same as those in the fourth embodiment, they will not be described again. Please refer to the corresponding parts in the fourth embodiment.

[0128] A device for constructing a named entity recognition model provided by the present application includes:

[0129] A training data acquisition unit, configured to acquire a set of corresponding relationships between texts and named entity annotation data;

[0130] A network construction unit, configured to construct a network structure of the named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is configured to determine a first text structure feature of a target word; the feature memory sub-module is configured to determine a second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module is configured to determine a second text feature of the target word at least according to the first text feature of the target word and the second text structure feature;

[0131] A training unit, configured to learn and obtain a named entity recognition model according to the set of corresponding relationships.

[0132] Sixth Embodiment

[0133] The present application also provides an electronic device. Since the device embodiments are basically similar to the method embodiments, the description is relatively simple. For the relevant parts, please refer to the partial description of the method embodiments. The device embodiments described below are only illustrative.

[0134] An electronic device according to this embodiment, the electronic device includes: a processor and a memory; the memory is used to store a program for implementing the method for constructing a named entity recognition model. After the device is powered on and runs the program of this method through the processor, the following steps are executed: obtaining a set of corresponding relationships between text and named entity annotation data; constructing a network structure of the named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of the target word; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module is used to determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; and learning to obtain a named entity recognition model according to the set of corresponding relationships.

[0135] Although this application is disclosed above with preferred embodiments, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the protection scope of this application should be determined by the scope defined by the claims of this application.

[0136] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0137] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0138] 1. A computer-readable medium includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information accessible by a computing device. As defined herein, a computer-readable medium does not include transitory media such as modulated data signals and carrier waves.

[0139] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as a method, system, or computer program product. Therefore, the present application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A named entity recognition method, characterized in that, Including: Determine the first text feature and the first text structure feature of the target word in the text to be processed; wherein, determining the first text structure feature includes: determining the first text structure feature of the target word according to the word dependency relationship in the text to be processed through a structure information extraction module based on a graph neural network; Determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; Determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; Determine the named entity type of the target word at least according to the second text feature of the target word.

2. The method according to claim 1, wherein Determine the first text structure feature of the target word through the structure information extraction module included in the named entity recognition model; Determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state through the feature memory sub-module included in the text feature determination module in the named entity recognition model; Determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word through the output sub-module included in the text feature determination module; Determine the named entity type of the target word at least according to the second text feature of the target word through the classifier included in the named entity recognition model.

3. The method according to claim 2, wherein Further including: Determine the third text structure feature of the target word entering the memory state through the feature memory sub-module at least according to the second text feature of the previous word of the target word that fuses text linear information and text structure information, the first text structure feature of the target word, and the third text structure feature of the previous word entering the memory state, wherein the third text feature includes the second text structure feature entering the memory state; Determine the second text feature of the target word through the output sub-module at least according to the first text feature of the target word, the second text feature of the previous word, the first text structure feature of the target word, and the third text structure feature of the target word entering the memory state.

4. The method according to claim 3, characterized in that, Further including: The feature memory sub-module determines the second text structure feature of the target word entering the memory state through the feature memory controller at least according to the second text feature of the previous word of the target word that fuses text linear information and text structure information and the first text structure feature of the target word; The feature memory sub-module determines the third text structure feature of the target word entering the memory state through the feature memory at least according to the second text structure feature of the target word entering the memory state and the third text structure feature of the previous word entering the memory state.

5. The method according to claim 3, wherein Further including: The output sub-module determines the fourth text feature through the first output controller at least according to the first text feature and the first text structure feature of the target word and the second text feature of the previous word; The output sub-module determines the second text feature of the target word through the second output controller at least according to the fourth text feature and the third text structure feature of the target word entering the memory state.

6. The method according to claim 2, wherein Determining the first text structure feature of the target word through the structure information extraction module included in the named entity recognition model, including: Determining the word dependency relationship of the text, where the word dependency relationship includes the dependency relationship type; Determining a word dependency relationship matrix according to the word dependency relationship; Determining the third text structure feature of the target word according to the matrix, the word vector of the target word, and the dependency relationship type vector; Determining the first text structure feature of the target word through the structure information extraction module based on a multi-layer graph neural network according to the third text structure feature of the target word.

7. The method according to claim 6, characterized in that The determining, through the structure information extraction module based on a multi-layer graph neural network, of the first text structure feature of the target word according to the third text structure feature of the target word includes: Determining the fourth text structure feature of the target word through the first-layer graph neural network included in the structure information extraction module according to the third text structure feature; Determining the first text structure feature of the target word through the second-layer graph neural network included in the structure information extraction module according to the fourth text structure feature.

8. The method according to claim 3, wherein It also includes: Determining the forgotten feature of the previous word through the feature forgetting sub-module included in the named entity recognition model according to the second text feature of the previous word, the first text feature of the target word, and the first text structure feature.

9. A method for constructing a named entity recognition model, characterized in that It includes: Obtaining a set of corresponding relationships between the text and the named entity annotation data; Constructing the network structure of the named entity recognition model; The model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of the target word through the structure information extraction module based on a graph neural network according to the word dependency relationship in the text to be processed; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module determines the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; Learning to obtain the named entity recognition model according to the set of corresponding relationships.

10. A named entity recognition device, characterized in that, It includes: A first feature determination unit for determining the first text feature and the first text structure feature of the target word in the text to be processed; where determining the first text structure feature includes: determining the first text structure feature of the target word through the structure information extraction module based on a graph neural network according to the word dependency relationship in the text to be processed; A text structure feature control unit for determining the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; A second feature determination unit for determining the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word; A classification unit for determining the named entity type of the target word at least according to the second text feature of the target word.

11. An apparatus for constructing a named entity recognition model, characterized in that, It includes: A training data acquisition unit for obtaining a set of corresponding relationships between the text and the named entity annotation data; A network construction unit for constructing the network structure of a named entity recognition model; the model includes: a structure information extraction module and a text feature determination module, and the text feature determination module includes: a feature memory sub-module and an output sub-module; the structure information extraction module is used to determine the first text structure feature of a target word according to the word dependency relationship in the text to be processed through a structure information extraction module based on a graph neural network; the feature memory sub-module is used to determine the second text structure feature of the target word entering the memory state at least according to the first text structure feature of the target word and the second text structure feature of the previous word entering the memory state; the output sub-module is used to determine the second text feature of the target word at least according to the first text feature and the second text structure feature of the target word. A training unit for learning a named entity recognition model according to the corresponding relationship set.

Images