Entity recognition method and apparatus, server, and storage medium

Abstract

The application is suitable for the technical field of neural networks, and provides an entity recognition method and device, a server and a storage medium. The method comprises the following steps: receiving to-be-recognized text, performing category mapping processing on each character in the to-be-recognized text based on a pre-set category label group, and obtaining the probability of each character under each category label; combining the category labels corresponding to each character into a label path, and obtaining multiple label paths corresponding to the to-be-recognized text; establishing an adjustment matrix of the corresponding character according to a pre-stored weight coefficient, and adjusting the probability of the corresponding character corresponding to each category label by using the adjustment matrix; and selecting a label path satisfying a pre-set selection condition from the multiple label paths according to the adjusted probability of each character under each category label, and recognizing a target entity from the to-be-recognized text. In the application, the recognition efficiency and accuracy of the target entity in the text data can be improved by using the method.

Description

Technical Field

[0001] This application belongs to the field of neural network technology, and in particular relates to entity recognition methods, devices, servers and storage media. Background Technology

[0002] Named Entity Recognition (NER) is a technique that identifies entities with specific meanings in text. In practice, NER primarily identifies entities such as names of people, places, organizations, and proper nouns. In real-world applications, NER is typically implemented using at least one method: rule-based or supervised learning.

[0003] In related technologies, when using NER technology to identify entities in text, if the format of the text data is not fixed, identifying specific entities, such as media name entities, journalist name entities, and organization name entities, usually requires manual extraction or proofreading, resulting in low recognition efficiency and high labor costs. Summary of the Invention

[0004] This application provides an entity recognition method, device, server, and storage medium, which can solve the problem in related technologies where, when using NER technology to recognize entities in text, if the format of the text data is not fixed, identifying specific entities, such as media name entities, journalist name entities, and organization name entities, usually requires manual extraction or proofreading, resulting in low recognition efficiency and high labor costs.

[0005] The first aspect of this application provides an entity recognition method, including:

[0006] The system receives the text to be recognized and performs category mapping processing on each character in the text based on a pre-defined category label group to obtain the probability of each character under each category label. The category label group is used to indicate at least one type of target entity, and an ordered combination of at least one label in the category label group indicates a target entity.

[0007] According to the order in which each character appears in the text to be recognized, the category labels corresponding to each character are combined into label paths to obtain multiple label paths corresponding to the text to be recognized.

[0008] For each character, an adjustment matrix is ​​established based on pre-stored weight coefficients, and the probability of the corresponding character corresponding to each category label is adjusted using the adjustment matrix. The adjustment matrix is ​​used to adjust the probability of the character corresponding to each category label.

[0009] Based on the adjusted probability of each character under each category label, a label path that meets the preset selection conditions is selected from multiple label paths and recorded as the target label path. Based on the category labels in the target label path, the target entity is identified from the text to be identified.

[0010] Furthermore, each character in the text to be recognized is subjected to category mapping to obtain the probability of each character under each category label, including:

[0011] Each character in the text to be recognized is converted into a character vector, and the character vectors are input into a pre-trained category mapping model to obtain the probability of the corresponding character under each category label. The category mapping model is used to represent the correspondence between the character vector and the probability of the character indicated by the character vector under each category label.

[0012] Furthermore, each character in the text to be recognized is subjected to category mapping to obtain the probability of each character under each category label, including:

[0013] Based on the positive order of each character in the text to be recognized, class mapping is performed on each character to obtain the positive probability of each character under each class label;

[0014] Based on the reverse order of each character in the text to be recognized, class mapping is performed on each character to obtain the reverse probability of each character under each category label;

[0015] The probability of a character under a category label includes the positive probability and the negative probability of the character under that category label.

[0016] Furthermore, based on the pre-stored weight coefficients, an adjustment matrix for the corresponding character is established, including:

[0017] Create a weight vector for each character based on its probability and weight coefficient under each category label;

[0018] For each character in the text to be recognized, the influence weight of the character on the corresponding character is determined based on the weight vector of the character and the weight vector of the corresponding character.

[0019] The influence weights of all characters on the corresponding characters are normalized, and the normalized influence weights are combined to generate the adjustment matrix for the corresponding characters.

[0020] Furthermore, based on the adjusted probability of each character under each category label, label paths that meet preset selection criteria are selected from multiple label paths, including:

[0021] Semantic parsing is performed on the text to be recognized to obtain the correlation degree of each character. The correlation degree is used to indicate the degree of association between a character and its adjacent characters.

[0022] Based on the adjusted probability of each character under each category label and the correlation degree of each character, select the label path that meets the preset selection conditions from multiple label paths.

[0023] Furthermore, semantic parsing is performed on the text to be recognized to obtain the correlation degree corresponding to each character, including:

[0024] Based on the positive order of each character in the text to be recognized, positive semantic parsing is performed on the text to be recognized to obtain the positive correlation degree corresponding to each character;

[0025] Based on the reverse order of each character in the text to be recognized, reverse semantic parsing is performed on the text to be recognized to obtain the reverse correlation degree corresponding to each character.

[0026] The relevance of a character includes both the positive relevance and the negative relevance of that character.

[0027] Furthermore, the preset selection criteria include at least one of the following:

[0028] Select the category label with the highest probability for each character;

[0029] Select the category label with the highest correlation for each character;

[0030] Select the category label with the highest weighted value of the corresponding probability and relevance for each character.

[0031] A second aspect of this application provides an entity recognition apparatus, comprising:

[0032] The category mapping unit is used to receive the text to be recognized, and perform category mapping processing on each character in the text to be recognized based on a pre-set category label group to obtain the probability of each character under each category label. The category label group is used to indicate at least one type of target entity, and an ordered combination of at least one label in the category label group indicates a target entity.

[0033] The tag generation unit is used to combine the category tags corresponding to each character into tag paths according to the order in which each character appears in the text to be recognized, thus obtaining multiple tag paths;

[0034] The label adjustment unit is used to establish an adjustment matrix for each character based on pre-stored weight coefficients, and to adjust the probability of the corresponding character corresponding to each category label using the adjustment matrix. The adjustment matrix is ​​used to adjust the probability of the character corresponding to each category label.

[0035] The entity extraction unit is used to select a label path that meets the preset selection conditions from multiple label paths based on the adjusted probability of each character under each category label, and denot it as the target label path. It also identifies the target entity from the text to be recognized based on the category labels in the target label path.

[0036] A third aspect of this application provides a server, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the entity recognition method provided in the first aspect.

[0037] A fourth aspect of this application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of the entity recognition method provided in the first aspect.

[0038] The entity recognition method, apparatus, server, and storage medium provided in this application have the following beneficial effects: In the process of entity recognition of text, by performing category mapping on each character in the text, the probability of each character under each category label is obtained. Then, according to the order of appearance of each character in the text to be recognized, the category labels corresponding to each character are combined into label paths, resulting in multiple label paths. This allows for automatic selection of label paths from multiple label paths based on the probability of each character under each category label, thereby obtaining the target entity. This enables automatic recognition of target entities in the text to be recognized, helping to improve the efficiency of target entity recognition. Furthermore, by establishing an adjustment matrix for each character and adjusting the probability of each character under each category label using this matrix, and then selecting a label path from multiple label paths based on the adjusted probability of each character under each category label, the target entity is obtained, which helps to improve the accuracy of target entity recognition. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart illustrating the implementation of an entity recognition method provided in an embodiment of this application;

[0041] Figure 2 This is a schematic diagram illustrating the acquisition of character category probability according to an embodiment of this application;

[0042] Figure 3 This is a flowchart illustrating the implementation of the influence weight matrix provided in one embodiment of this application;

[0043] Figure 4 This is a schematic diagram of establishing an influence weight matrix according to an embodiment of this application;

[0044] Figure 5 This is a structural block diagram of an entity recognition device provided in an embodiment of this application;

[0045] Figure 6 This is a structural block diagram of a server provided in one embodiment of this application. Detailed Implementation

[0046] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0047] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0048] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0049] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0050] To illustrate the technical solution of this application, the following embodiments will be used for explanation.

[0051] Please see Figure 1 , Figure 1 The following is a flowchart illustrating the implementation of an entity recognition method provided in an embodiment of this application, including:

[0052] Step 101: Receive the text to be recognized, and perform category mapping processing on each character in the text to be recognized based on the pre-set category label group to obtain the probability of each character under each category label.

[0053] The category label group is used to indicate at least one type of target entity, and an ordered combination of at least one label in the category label group can indicate a target entity.

[0054] The category label group typically consists of one or more category labels, which are usually predefined symbols, such as "B-MED", "I-MED", "O", etc. Here, B indicates the start of the target entity, I indicates the middle or end of the target entity, O indicates a non-target entity, and MED indicates that the target entity is a source media class.

[0055] In this context, the target entity is typically an entity in the text to be identified that corresponds to a predefined category. Categories can be names of people, places, organizations, proper nouns, etc. For example, if names of people and organizations are used as predefined categories, then for the text to be identified, "Zhang San is a student of XX University," the target entity corresponding to the name of the person is "Zhang San," and the target entity corresponding to the name of the organization is "XX University."

[0056] To give a further example, if the category of the target entity to be identified is only the source media category, then for the text to be identified, “Xinhua News Agency reporter Zhang San”, the category labels corresponding to each character can be: “B-MED, I-MED, I-MED, O, O, O, O”. In this case, the target entity can be obtained as the first three characters “Xinhua News Agency”.

[0057] In this embodiment, the entity recognition method described above is typically executed by a server. It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are imposed here.

[0058] In practice, the text to be recognized can be the text information directly input by the user received by the above-mentioned execution entity, or the text information extracted from a picture or photo input by the user.

[0059] Among them, each character in the text to be recognized is subjected to category mapping processing to obtain the probability of each character under each category label. Usually, there is a character annotation database in the above-mentioned execution entity. After the above-mentioned execution entity receives the text to be recognized, category mapping processing is performed based on the database to obtain the category labels corresponding to each character in the text to be recognized, and the probability of each character corresponding to each label in the category label group. For example, as Figure 2 shown, the preset category label group includes a category label of information source media. Using the BIO annotation method for annotation, the category labels can include "B-MED", "I-MED", "O", etc. Among them, MED is used to indicate that the target entity belongs to the information source media category. For the text to be recognized "Xinhua News Agency reported", the probability of the character "Xin" corresponding to each label in the category label group is 0.6 (B-MED), 0.3 (I-MED), 0.1 (O); the probability of the character "Hua" corresponding to each label in the category label group is 0.5 (B-MED), 0.3 (I-MED), 0.2 (O); the probability of the character "She" corresponding to each label in the category label group is 0.7 (B-MED), 0.2 (I-MED), 0.1 (O); the probability of the character "Bao" corresponding to each label in the category label group is 0.4 (B-MED), 0.3 (I-MED), 0.3 (O); the probability of the character "Dao" corresponding to each label in the category label group is 0.2 (B-MED), 0.3 (I-MED), 0.5 (O). In practice, the character annotation database can include existing corpus annotation libraries, such as the People's Daily annotation corpus and the NER corpus released by Microsoft Research Asia; it can also include annotation corpus data preset according to the category of the target entity; at the same time, the character annotation database can also include texts without target entities as auxiliary data.

[0060] Step 102: According to the order of appearance of each character in the text to be recognized, combine the category labels corresponding to each character into a label path to obtain multiple label paths corresponding to the text to be recognized.

[0061] Among them, the label path is usually a combination of the category labels of each character in the text to be recognized.

[0062] Among them, according to the order of appearance of each character in the text to be recognized, it usually means in the order from the first character to the last character in the text to be recognized. For example, for the text to be recognized "Xinhua News Agency reported", its order of appearance is: Xin - Hua - She - Bao - Dao.

[0063] In practice, combining the category labels corresponding to each character into a label path can be achieved by arranging the category labels of each character in the order of their appearance in the text to be recognized. For example, when using the BIO annotation method to annotate the text to be recognized, "Xinhua News Agency report", the combination of category labels for each character in the text, "B-MED, I-MED, I-MED, O, O", can serve as a label path. Obtaining multiple label paths corresponding to the text to be recognized usually means that each character in the text has a probability of corresponding to each preset category label; that is, each character corresponds to more than one category label. When combining the category labels of each character, multiple label paths can be obtained for the text to be recognized. For example, for the text to be recognized, "B-MED, I-MED, I-MED, O, O" can be one label path, and "B-MED, I-MED, O, B-MED, I-MED" can also be another label path.

[0064] In practice, models can also be used to implement steps 101-102 above. For example, a combination of the word2vec model and a Long Short-Term Memory (LSTM) model can be used to achieve the operations described in steps 101-102. The word2vec model is used to represent the correspondence between words and word vectors. The word2vec model is a commonly used neural network model for converting words into word vectors, and will not be elaborated upon here. Furthermore, to achieve better training results and improve training efficiency, a pre-trained language model (Bidirectional Encoder Representation from Transformers, BERT) can be used instead of word2vec, and a Bi-directional Long Short-Term Memory (BiLSTM) network can be used instead of LSTM.

[0065] Step 103: For each character, establish an adjustment matrix for the corresponding character based on the pre-stored weight coefficients, and use the adjustment matrix to adjust the probability of the corresponding character corresponding to each category label.

[0066] The weighting coefficients are typically pre-defined coefficients used to linearly transform the probability of the category label corresponding to the character. In practice, the probability of the category label is often represented by an embedding vector. To enable the execution entity to express the multiple meanings of the character, weights need to be assigned to the embedding vector, and a linear transformation needs to be performed on the embedding vector.

[0067] Among them, the adjustment matrix is used to adjust the probabilities of characters corresponding to various category labels.

[0068] Among them, according to the pre-stored weight coefficients, an adjustment matrix for the corresponding characters is established. Usually, the probabilities of the category labels corresponding to the characters are linearly transformed by the weight coefficients, and an adjustment matrix of the text to be recognized is obtained by adding or multiplying the probabilities of the category labels corresponding to the transformed characters.

[0069] Among them, the adjustment matrix is used to adjust the probabilities of the corresponding characters corresponding to various category labels. Usually, the correlation degree and influence size between characters are obtained through the adjustment matrix, and the probabilities of the category labels corresponding to the characters in the text to be recognized are adjusted. For example, if the category label corresponding to the character before the corresponding character is X, then the probability that the category of the corresponding character is X is also relatively large. Specifically, for the text to be recognized "Xinhua News Agency reported", before adjustment, the probabilities of the character "Hua" corresponding to each label in the category label group are 0.5 (B-MED), 0.3 (I-MED), 0.2 (O). Through the adjustment matrix, combined with the influence of other characters in the text to be recognized on the character "Hua", the category label probabilities of "Hua" are adjusted to 0.8 (B-MED), 0.1 (I-MED), 0.1 (O). Through the adjustment matrix, the recognition accuracy of the target entity in the text to be recognized can be improved.

[0070] Step 104: According to the adjusted probabilities of each character under each category label, select a label path that meets the preset selection conditions from multiple label paths, denoted as the target label path, and recognize the target entity from the text to be recognized according to the various category labels in the target label path.

[0071] Among them, selecting a label path that meets the preset selection conditions usually means selecting one that meets the preset selection conditions from multiple label paths.

[0072] Among them, recognizing the target entity from the text to be recognized according to the various category labels in the target label path usually means that after obtaining the target label path of the text to be recognized, extracting the characters in the text to be recognized corresponding to the target label. For example, when extracting the target entity of the source media category in the text to be recognized "Xinhua News Agency reported", the target label path corresponding to the text to be recognized is "B-MED, I-MED, I-MED, O, O". Among them, the characters corresponding to "B-MED, I-MED, I-MED" belong to the source media category, and the characters "Xinhua News Agency" corresponding to the category labels "B-MED, I-MED, I-MED" are extracted to achieve the purpose of extracting the target entity of the source media category.

[0073] The entity recognition method provided in this embodiment, during the process of text entity recognition, performs category mapping processing on each character in the text to obtain the probability of each character under each category label. Then, according to the order of appearance of each character in the text to be recognized, the category labels corresponding to each character are combined into label paths, resulting in multiple label paths. In this way, a label path can be automatically selected from multiple label paths based on the probability of each character under each category label, thereby obtaining the target entity. This enables automatic recognition of target entities in the text to be recognized, which helps to improve the efficiency of target entity recognition. In addition, an adjustment matrix is ​​established for each character, and the probability of each character under each category label is adjusted using the adjustment matrix. Then, based on the adjusted probability of each character under each category label, a label path is selected from multiple label paths to obtain the target entity, which helps to improve the accuracy of target entity recognition.

[0074] In some optional implementations of this embodiment, category mapping is performed on each character in the text to be recognized to obtain the probability of each character under each category label, including:

[0075] Each character in the text to be recognized is converted into a character vector, and the character vectors are input into a pre-trained category mapping model to obtain the probability of the corresponding character under each category label. The category mapping model is used to represent the correspondence between the character vector and the probability of the character indicated by the character vector under each category label.

[0076] The category mapping model typically categorizes the characters in the text to be recognized. The target entities to be extracted, along with their categories, can be stored in the category mapping model for training. When used, the text to be recognized is input into the category mapping model, which can then identify the target entities within the text. For example, after training and learning the names of various news source media, inputting the text "Xinhua News Agency report" into the category mapping model can extract the target entity "Xinhua News Agency" from the target news source media category.

[0077] The process involves converting each character in the text to be recognized into a word vector. This is typically achieved using existing models such as BERT and word2vec. In practice, BERT can be used as the underlying encoder to understand the semantics of the text and generate word vectors, producing a 768-dimensional word vector for each character. Alternatively, the text can be segmented into sentences, using predefined punctuation marks as technical identifiers. Each sentence is then treated as a unit for entity recognition, converting each character in the sentence into a word vector. Finally, a sentence vector of the same length as the sentence is generated based on the order in which the characters appear. For example, a sentence with 128 characters can generate a 768×128 sentence vector using BERT.

[0078] The process involves inputting character vectors into a pre-trained class mapping model to obtain the probability of each character under each class label. This can be achieved using multiple small models or a large model, each capable of performing class mapping and obtaining the probability of each character under each class label. In practice, BERT can be used to perform class mapping on the text, and then LSTM can be used to obtain the probability of each character under each class label.

[0079] In some optional implementations of this embodiment, category mapping is performed on each character in the text to be recognized to obtain the probability of each character under each category label, including:

[0080] Based on the forward order of each character in the text to be recognized, class mapping is performed on each character to obtain the forward probability of each character under each class label; based on the reverse order of each character in the text to be recognized, class mapping is performed on each character to obtain the reverse probability of each character under each class label.

[0081] The probability of a character under a category label includes the positive probability and the negative probability of the character under that category label.

[0082] In this text, the forward order of each character is from the first character to the last. For example, for the text "Xinhua News Agency report", the forward order is "Xin-Xia-She-Bao-Dao". The reverse order of each character is from the last character to the first character. For example, for the text "Xinhua News Agency report", the reverse order is "Dao-Bao-She-Hua-Xin". When performing category mapping on each character, the probability of the corresponding character is related to the probability of its preceding character. Performing category mapping on the text from both forward and reverse directions can improve the accuracy of entity recognition. In practice, BiLSTM can be used instead of the aforementioned LSTM to improve the accuracy of entity recognition. In some optional implementations of this embodiment, an adjustment matrix for the corresponding character is established based on pre-stored weight coefficients, such as... Figure 3 As shown, it includes:

[0083] Step 301: Create a weight vector for the corresponding character based on the probability and weight coefficient of the corresponding character under each category label.

[0084] The weighting coefficients are usually based on pre-trained coefficients of the category labels, which linearly transform the probability of the corresponding character under each category label.

[0085] Creating a weight vector for a given character typically involves calculating the probability and weight coefficient of that character under each category label. Each weight vector corresponds to a weight coefficient; for example, the probability X of a character's category label corresponds to a weight coefficient W. Q W k W v Create three weight vectors q, k, and v.

[0086] q = X × W Q k = X × W k v = X × W v

[0087] Step 302: For each character in the text to be recognized, determine the influence weight of the character on the corresponding character based on the weight vector of the character and the weight vector of the corresponding character.

[0088] Influence weight typically reflects the probability of the relationship between characters in the category label. The greater the influence weight between two characters, the stronger their mutual influence, and the higher the probability that they belong to the same category.

[0089] Specifically, the influence weight of a character on its corresponding character is determined based on the character's weight vector and the weight vector of the corresponding character. This influence weight is typically calculated using the weight vectors of the character and the corresponding character. For example, the influence between characters in the text is obtained by multiplying the weight vector q by the transpose of the weight vector k. Figure 4 As shown, w1, w2, and w3 are three character vectors corresponding to the characters in the text to be recognized. By multiplying the vectors, we can obtain the influence weight matrix between the characters in the text. Here, w1×w1 represents the influence of the character corresponding to w1 on its own category label probability, w1×w2 represents the influence of the character corresponding to w1 on the category label probability of the character corresponding to w2, and so on. The influence weight matrix reflects the mutual influence of the category label probabilities between the characters in the text.

[0090] Step 303: Normalize the influence weights of all characters on the corresponding characters, and combine the normalized influence weights to generate the adjustment matrix of the corresponding characters.

[0091] Normalization employs the softmax function to normalize the influence weights, considering the impact of each character on the same scale. In practice, to avoid excessively large influence weights before normalization, a reduction parameter is typically set to shrink the influence weights before normalization.

[0092] The normalized influence weights are combined to generate the adjustment matrix for the corresponding character. Usually, the normalized influence weights are arranged in the order of the influence weights of each character in the text to be recognized to obtain the adjustment matrix for the corresponding character.

[0093] In some optional implementations of this embodiment, based on the adjusted probability of each character under each category label, a label path that meets preset selection conditions is selected from multiple label paths, including:

[0094] Semantic parsing is performed on the text to be recognized to obtain the correlation degree of each character; based on the adjusted probability of each character under each category label and the correlation degree of each character, the label path that meets the preset selection conditions is selected from multiple label paths.

[0095] Among them, relevance refers to the degree of association between two characters.

[0096] Among them, to obtain the correlation degree corresponding to each character, it can be obtained by multiplying the word vectors corresponding to two adjacent characters. The larger the product, the higher the correlation degree. Here, by determining the correlation degree between adjacent characters, it can be determined whether two adjacent characters belong to the same category. For example, the probability that the two characters in the text "Zhang San" belong to the category label of a person's name is not high, but by determining the correlation degree of the two characters, it can be obtained that "Zhang San" belongs to the category label of a person's name. That is to say, by determining the correlation degree, it can be determined which character combinations in the text belong to the category labels in the category label group.

[0097] Among them, according to the adjusted probability of each character under each category label and the correlation degree corresponding to each character, from multiple label paths, select the label paths that meet the preset selection conditions. Usually, it means to further adjust the probability of the category label corresponding to the character of the text to be recognized based on the correlation degree corresponding to each character, determine which adjacent characters in the text to be recognized can form a target entity, and then combine the selection conditions to determine the path of the category label that best meets the requirements.

[0098] In some optional implementation manners of this embodiment, semantic parsing of the text to be recognized includes: performing forward semantic parsing on the text to be recognized in the forward appearance order of each character in the text to be recognized to obtain the forward correlation degree corresponding to each character; performing reverse semantic parsing on the text to be recognized in the reverse appearance order of each character in the text to be recognized to obtain the reverse correlation degree corresponding to each character.

[0099] Among them, the correlation degree corresponding to a character includes the forward correlation degree corresponding to the character and the reverse correlation degree corresponding to the character.

[0100] Among them, the forward correlation degree is usually the correlation degree between the corresponding character and the previous character obtained by performing semantic parsing on the text to be recognized in the forward appearance order of each character in the text to be recognized; the reverse correlation degree is usually the correlation degree between the corresponding character and the next character obtained by performing semantic parsing on the text to be recognized in the reverse appearance order of each character in the text to be recognized. Compared with the local optimal solution obtained by only performing forward parsing, performing forward parsing and reverse parsing on the text to be recognized can obtain the global optimal solution of the text to be recognized and improve the entity recognition accuracy.

[0101] In some optional implementation manners of this embodiment, the preset selection conditions may include at least one of the following:

[0102] The first item is to select the category label with the largest corresponding probability for each character.

[0103] Here, during the category mapping process, each character receives a probability corresponding to each preset category label. The probabilities are not equal, and the one with the highest probability is selected. For example, if the category label probabilities of the text to be recognized are "0.6(B-MED), 0.3(I-MED), 0.1(O)", then "0.6(B-MED)" with the highest probability is selected as the category label for that character.

[0104] The second step is to select the category label with the highest correlation for each character. The correlation is used to indicate the degree of association between a character and its adjacent characters.

[0105] Here, the greater the correlation between characters, the closer the connection. For example, for consecutive characters W1W2W3, the correlation of W1W2 is 0.6, and the correlation of W2W3 is -0.8. Therefore, the category label corresponding to W1W2 when the correlation is 0.6 is selected as the category label.

[0106] The third step is to select the category label with the highest weighted value of the corresponding probability and relevance for each character.

[0107] Here, selecting the category label with the highest weighted value of the probability and relevance of each character is a comprehensive consideration of the probability and relevance of the category labels corresponding to each character in the class.

[0108] Please see Figure 5 , Figure 5 This is a structural block diagram of an entity recognition device provided in an embodiment of this application, comprising:

[0109] The category mapping unit 501 is used to receive the text to be recognized, perform category mapping processing on each character in the text to be recognized based on a pre-set category label group, and obtain the probability of each character under each category label. The category label group is used to indicate at least one type of target entity, and an ordered combination of at least one label in the category label group can indicate a target entity.

[0110] The tag generation unit 502 is used to combine the category tags corresponding to each character into tag paths according to the order in which each character appears in the text to be recognized, so as to obtain multiple tag paths.

[0111] The label adjustment unit 503 is used to establish an adjustment matrix for each character based on pre-stored weight coefficients, and to adjust the probability of the corresponding character corresponding to each category label using the adjustment matrix. The adjustment matrix is ​​used to adjust the probability of the character corresponding to each category label.

[0112] The entity extraction unit 504 is used to select a label path that meets the preset selection conditions from multiple label paths based on the adjusted probability of each character under each category label, and denot it as the target label path, and to identify the target entity from the text to be recognized based on each category label in the target label path.

[0113] As an embodiment of this application, the category mapping unit 501 performs category mapping processing on each character in the text to be recognized to obtain the probability of each character under each category label, including:

[0114] Each character in the text to be recognized is converted into a character vector, and the character vectors are input into a pre-trained category mapping model to obtain the probability of the corresponding character under each category label. The category mapping model is used to represent the correspondence between the character vector and the probability of the character indicated by the character vector under each category label.

[0115] As an embodiment of this application, the category mapping unit 501 performs category mapping processing on each character in the text to be recognized to obtain the probability of each character under each category label, including:

[0116] Based on the positive order of each character in the text to be recognized, class mapping is performed on each character to obtain the positive probability of each character under each class label;

[0117] Based on the reverse order of each character in the text to be recognized, class mapping is performed on each character to obtain the reverse probability of each character under each category label;

[0118] The probability of a character under a category label includes the positive probability and the negative probability of the character under that category label.

[0119] As one embodiment of this application, the label adjustment unit 503 establishes an adjustment matrix for the corresponding character based on pre-stored weight coefficients, including:

[0120] Create a weight vector for each character based on its probability and weight coefficient under each category label;

[0121] For each character in the text to be recognized, the influence weight of the character on the corresponding character is determined based on the weight vector of the character and the weight vector of the corresponding character.

[0122] The influence weights of all characters on the corresponding characters are normalized, and the normalized influence weights are combined to generate the adjustment matrix for the corresponding characters.

[0123] As an embodiment of this application, based on the adjusted probability of each character under each category label, a label path that meets preset selection conditions is selected from multiple label paths, including:

[0124] Semantic analysis is performed on the text to be recognized to determine the degree of association between a character and its adjacent characters;

[0125] Based on the adjusted probability of each character under each category label and the correlation degree of each character, select the label path that meets the preset selection conditions from multiple label paths.

[0126] As one embodiment of this application, semantic parsing of the text to be identified includes:

[0127] Based on the forward order of each character in the text to be recognized, the text is subjected to forward semantic analysis to obtain the forward correlation degree.

[0128] Based on the reverse order of each character in the text to be recognized, reverse semantic parsing is performed on the text to be recognized to obtain the reverse correlation degree.

[0129] As an embodiment of this application, the preset selection conditions include at least one of the following:

[0130] Select the category label with the highest probability for each character;

[0131] Select the category label with the highest correlation for each character, where correlation indicates the degree of association between a character and its adjacent characters;

[0132] Select the category label with the highest weighted value of the corresponding probability and relevance for each character.

[0133] The device provided in this embodiment, in the process of entity recognition of text, performs category mapping processing on each character in the text to obtain the probability of each character under each category label, and adjusts the probability of the category label by establishing an adjustment matrix. Combined with preset selection conditions, the device can identify the target entity from the text, thereby improving the recognition efficiency of target entities in text data and reducing labor costs.

[0134] It should be understood that, Figure 5 In the structural block diagram of the entity recognition device shown, each unit is used to perform... Figure 1 , Figure 3 The steps in the corresponding embodiments, and for Figure 1 , Figure 3 The steps in the corresponding embodiments have been explained in detail in the above embodiments. Please refer to them for details. Figure 1 , Figure 3 as well as Figure 1 , Figure 3The relevant descriptions in the corresponding embodiments will not be repeated here.

[0135] Please see Figure 6 , Figure 6 This is a structural block diagram of a server provided in one embodiment of this application. The server 600 in this embodiment includes: a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601, such as an entity recognition program. When the processor 601 executes the computer program 603, it implements the steps in the various entity recognition embodiments described above, for example... Figure 1 Steps 101 to 104 shown, or Figure 3 As shown in 301 to 303. Alternatively, the processor 601 implements the above when executing computer program 603. Figure 5 The functions of each unit in the corresponding embodiments, for example, Figure 5 For details on the functions of units 501 to 504 shown, please refer to [link / reference needed]. Figure 5 The relevant descriptions in the corresponding embodiments are not repeated here.

[0136] For example, computer program 603 can be divided into one or more units, one or more of which are stored in memory 602 and executed by processor 601 to complete this application. The one or more units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 603 in server 600. For example, computer program 603 can be divided into a category mapping unit, a label generation unit, a label adjustment unit, and an entity extraction unit, with the specific functions of each unit as described above.

[0137] The server may include, but is not limited to, processor 601 and memory 602. Those skilled in the art will understand that... Figure 6 This is merely an example of server 600 and does not constitute a limitation on server 600. It may include more or fewer components than shown, or combine certain components, or different components. For example, the turntable device may also include input / output devices, network access devices, buses, etc.

[0138] The processor 601 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0139] The memory 602 can be an internal storage unit of the server 600, such as the server 600's hard drive or memory. The memory 602 can also be an external storage device of the server 600, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the server 600. Furthermore, the memory 602 can include both internal and external storage units of the server 600. The memory 602 is used to store computer programs and other programs and data required by the turntable device. The memory 602 can also be used to temporarily store data that has been output or will be output.

[0140] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] If an integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can also be implemented by a computer program instructing related hardware, and a computer program can be stored in a computer-readable storage medium. When executed by a processor, this computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0142] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0143] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An entity recognition method, characterized in that, include: The system receives text to be recognized and performs category mapping processing on each character in the text based on a pre-defined group of category labels to obtain the probability of each character under each category label. The group of category labels is used to indicate at least one type of target entity, and an ordered combination of at least one label in the group of category labels indicates a target entity. The category mapping processing is implemented using a bidirectional long short-term memory network and is processed according to the forward and reverse order of each character in the text to be recognized to obtain the forward and reverse probabilities of each character under each category label. The probability of a character under a category label includes both the forward and reverse probabilities. According to the order in which each character appears in the text to be identified, the category labels corresponding to each character are combined into a label path to obtain multiple label paths corresponding to the text to be identified; For each character, an adjustment matrix is ​​established based on pre-stored weight coefficients, and the probability of the corresponding character corresponding to each category label is adjusted using the adjustment matrix. The adjustment matrix is ​​used to adjust the probability of the character corresponding to each category label. Based on the adjusted probability of each character under each category label, a label path that meets the preset selection conditions is selected from the multiple label paths and recorded as the target label path. Based on each category label in the target label path, the target entity is identified from the text to be identified. The step of establishing the adjustment matrix for the corresponding character based on the pre-stored weight coefficients includes: Based on the probability of the corresponding character under each category label and the weight coefficient, a weight vector is created for the corresponding character; for each character in the text to be identified, the influence weight of the character on the corresponding character is determined based on the weight vector of the character and the weight vector of the corresponding character. The influence weights of all characters on the corresponding character are normalized, and the normalized influence weights are combined to generate the adjustment matrix of the corresponding character.

2. The entity recognition method according to claim 1, characterized in that, The process of performing category mapping on each character in the text to be identified to obtain the probability of each character under each category label includes: Each character in the text to be recognized is converted into a character vector, and the character vector is input into a pre-trained category mapping model to obtain the probability of the corresponding character under each category label. The category mapping model is used to characterize the correspondence between the character vector and the probability of the character indicated by the character vector under each category label.

3. The entity recognition method according to claim 1, characterized in that, The step of selecting a tag path that meets preset selection criteria from the multiple tag paths based on the adjusted probability of each character under each category tag includes: The text to be identified is semantically parsed to obtain the correlation degree of each character, wherein the correlation degree is used to indicate the degree of association between a character and its adjacent characters; Based on the adjusted probability of each character under each category label and the correlation degree of each character, a label path that meets the preset selection conditions is selected from the multiple label paths.

4. The entity recognition method according to claim 3, characterized in that, The semantic parsing of the text to be identified, to obtain the correlation degree corresponding to each character, includes: According to the positive order of each character in the text to be identified, the text to be identified is subjected to positive semantic parsing to obtain the positive correlation degree corresponding to each character; According to the reverse order of each character in the text to be identified, reverse semantic parsing is performed on the text to be identified to obtain the reverse correlation degree corresponding to each character; The relevance of a character includes both the positive relevance and the negative relevance of that character.

5. The entity recognition method according to any one of claims 1-4, characterized in that, The preset selection criteria include at least one of the following: Select the category label with the highest probability for each character; Select the category label with the highest correlation for each character; Select the category label with the highest weighted value of the corresponding probability and relevance for each character.

6. An entity recognition device, characterized in that, include: A category mapping unit is used to receive text to be recognized, and perform category mapping processing on each character in the text to be recognized based on a pre-set category label group to obtain the probability of each character under each category label. The category label group is used to indicate at least one type of target entity, and an ordered combination of at least one label in the category label group indicates a type of target entity. The category mapping processing is implemented using a bidirectional long short-term memory network, and the processing is performed according to the forward and reverse occurrence order of each character in the text to be recognized to obtain the forward probability and reverse probability of each character under each category label. The probability of a character under a category label includes the forward probability and the reverse probability. The tag generation unit is used to combine the category tags corresponding to each character into tag paths according to the order in which each character appears in the text to be recognized, so as to obtain multiple tag paths; The label adjustment unit is used to establish an adjustment matrix for each character based on pre-stored weight coefficients, and to adjust the probability of the corresponding character corresponding to each category label using the adjustment matrix, wherein the adjustment matrix is ​​used to adjust the probability of the character corresponding to each category label; The entity extraction unit is used to select a label path that meets the preset selection conditions from the multiple label paths according to the adjusted probability of each character under each category label, and denot it as the target label path; and to identify the target entity from the text to be identified according to each category label in the target label path. In the label adjustment unit, the step of establishing an adjustment matrix for the corresponding character based on pre-stored weight coefficients includes: creating a weight vector for the corresponding character based on the probability of the corresponding character under each category label and the weight coefficient; determining the influence weight of the character on the corresponding character for each character in the text to be recognized based on the weight vector of the character and the weight vector of the corresponding character; normalizing the influence weights of all characters on the corresponding character; and combining the normalized influence weights to generate the adjustment matrix for the corresponding character.

7. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the entity recognition method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the entity recognition method as described in any one of claims 1 to 5.

Images