Entity recognition method and related equipment

Abstract

The present application relates to the technical field of text analysis, and provides an entity recognition method and related equipment, the entity recognition method comprising: obtaining a target text including multiple words, and generating a part-of-speech embedding vector and a position embedding vector for each word; extracting a word vector for each word using a word vector extraction model; splicing the word's part-of-speech embedding vector, position embedding vector and word vector for each word respectively, to obtain the word's embedding features; using a two-layer bidirectional network, generating a global feature of the target text based on all embedding features; obtaining a local feature of the target text based on all word vectors; fusing the global feature and the local feature of the target text to obtain a fused feature, and identifying the fused feature to obtain an entity recognition result of the target text. The method of the present application can improve the accuracy of entity recognition.

Description

Technical Field

[0001] The present application relates to the technical field of text analysis, and in particular to an entity recognition method and related equipment. Background Art

[0002] Existing technologies face challenges in processing massive, heterogeneous, and real-time updated social media text data. Traditional content analysis methods often rely on keyword matching and simple sentiment analysis, which cannot deeply understand the semantics and context of the text, resulting in an inaccurate and incomplete grasp of public sentiment and discussion trends.

[0003] In addition, the diversity and complexity of social media texts require that analysis tools not only be able to identify and understand entities in the text, but also be able to capture the relationships between entities and the emotional tendencies in the overall context. For example, when analyzing user feedback on a product or event, existing systems may not be able to accurately identify users' sarcasm or puns, resulting in biased sentiment analysis. It can be seen that existing entity recognition methods have the problem of low accuracy in entity recognition. Summary of the invention

[0004] The present application provides an entity recognition method and related devices, which can solve the problem of low accuracy of entity recognition.

[0005] In a first aspect, an embodiment of the present application provides an entity recognition method, the entity recognition method comprising:

[0006] Obtain a target text including multiple words and generate a part-of-speech embedding vector and a position embedding vector for each word; the part-of-speech embedding vector is used to describe the part of speech of the word, and the position embedding vector is used to describe the position of the word in the target text;

[0007] The word vector extraction model is used to extract the word vector of each word; the word vector is used to describe the semantic and grammatical information of the word;

[0008] For each word, concatenate the word’s part-of-speech embedding vector, position embedding vector, and word vector to obtain the word’s embedding features;

[0009] A two-layer bidirectional network is used to generate global features of the target text based on all embedded features; the two-layer bidirectional network includes a forward two-layer model for extracting forward features of each word and a backward two-layer model for extracting backward features of each word, and both the forward two-layer model and the backward two-layer model include a plurality of gated neural units connected in sequence, and the gated neural unit adjusts the input data input to the gated neural unit by the previous gated neural unit by introducing a control state, and the global features are obtained by using the forward features and backward features of all words;

[0010] Obtain local features of the target text based on all word vectors;

[0011] The global features and local features of the target text are fused to obtain fused features, and the fused features are recognized to obtain entity recognition results of the target text; the entity recognition results are used to describe the categories of entities in the target text.

[0012] Optionally, the word's part-of-speech embedding vector, position embedding vector, and word vector are concatenated to obtain the word's embedding features, including:

[0013] By formula:

[0014]

[0015] Get the Embedding features of words ;

[0016] in, Indicates The word's part-of-speech embedding vector, Indicates The position embedding vector of each word, Indicates The word vector of each word, , Indicates the number of words.

[0017] Optionally, a two-layer bidirectional network is used to generate global features of the target text based on all embedded features, including:

[0018] A two-layer bidirectional network is used to process all embedded features to obtain the forward features and reverse features of each word, and the forward features and reverse features of each word are concatenated to obtain the final features of each word;

[0019] Calculate the global features of the target text based on all the final features.

[0020] Optionally, calculate the global features of the target text based on all final features, including:

[0021] By formula:

[0022]

[0023] Calculate the global features of the target text ;

[0024] in, Indicates weights, Indicates The final features of the words:

[0025]

[0026]

[0027] in, Indicates attention weight vector, represents the weight matrix, represents the bias term, , are activation functions, , Indicates the number of words.

[0028] Optionally, the input end of the forward two-layer model and the input end of the backward two-layer model are both input ends of the two-layer bidirectional network, and the output end of the forward two-layer model and the output end of the backward two-layer model are both output ends of the two-layer bidirectional network;

[0029] The forward two-layer model includes a first forward layer and a second forward layer;

[0030] The first forward layer includes a plurality of first forward neural units connected in sequence, the second forward layer includes a plurality of second forward neural units connected in sequence, the plurality of first forward neural units correspond to the plurality of second forward neural units one by one, the output end of each first forward neural unit is connected to the input end of the corresponding second forward neural unit, the input ends of all the first forward neural units are the input ends of the first forward layer, and the output ends of all the second forward neural units are the output ends of the first forward layer;

[0031] The backward two-layer model includes a first backward layer and a second backward layer;

[0032] The first backward layer includes a plurality of first backward neural units connected in sequence, the second backward layer includes a plurality of second backward neural units connected in sequence, the plurality of first backward neural units correspond to the plurality of second backward neural units one by one, the output end of each first backward neural unit is connected to the input end of the corresponding second backward neural unit, the input ends of all the first backward neural units are the input ends of the first backward layer, and the output ends of all the second backward neural units are the output ends of the first backward layer;

[0033] The first forward neural unit, the second forward neural unit, the first backward neural unit, and the second backward neural unit are all gated neural units;

[0034] The gated neural unit includes a Gaussian filter control subunit and a gated recurrent subunit connected in sequence;

[0035] The input end of the Gaussian filter control subunit is the input end of the gated neural unit, and the output end of the gated recurrent subunit is the output end of the gated neural unit;

[0036] The Gaussian filter control subunits in the first forward layer, the second forward layer, the first backward layer, and the second backward layer are all used to adjust the input data from the previous gated neural unit to the gated neural unit by introducing a control state.

[0037] Optionally, the fused features are identified to obtain entity recognition results of the target text, including:

[0038] Calculate the posterior probability that the target text belongs to each category based on the fused features;

[0039] The posterior probability with the largest value is taken as the final posterior probability, and the category corresponding to the final posterior probability is taken as the entity recognition result of the target text.

[0040] Optionally, the posterior probability that the target text belongs to each category is calculated based on the fused features, including:

[0041] By formula:

[0042]

[0043]

[0044] Calculate the target text belongs to the category The posterior probability ;

[0045] in, represents the probability density function, Indicates category The prior probability of express The probability of represents a feature set, Indicates category, represents the fusion features of the target text, represents the covariance matrix of the fused features, represents the dimension of fusion features, represents the determinant of the covariance matrix, Represents the mean of the fused features.

[0046] Optionally, the entity recognition method further includes:

[0047] According to the entity recognition result of the target text, data related to the target text is recommended to the target user; the target user is the user who inputs the target text.

[0048] In a second aspect, an embodiment of the present application provides an entity recognition device, including:

[0049] The first generation module obtains a target text including a plurality of words and generates a part-of-speech embedding vector and a position embedding vector for each word; the part-of-speech embedding vector is used to describe the part of speech of the word, and the position embedding vector is used to describe the position of the word in the target text;

[0050] The extraction module uses the word vector extraction model to extract the word vector of each word; the word vector is used to describe the semantic and grammatical information of the word;

[0051] The concatenation module concatenates the word’s part-of-speech embedding vector, position embedding vector, and word vector for each word to obtain the word’s embedding features.

[0052] The second generation module generates global features of the target text based on all embedded features using a two-layer bidirectional network; the two-layer bidirectional network includes a forward two-layer model for extracting forward features of each word and a backward two-layer model for extracting backward features of each word, the forward two-layer model and the backward two-layer model both include a plurality of gated neural units connected in sequence, the gated neural unit adjusts the input data input to the gated neural unit by the previous gated neural unit by introducing a control state, and the global features are obtained by using the forward features and backward features of all words;

[0053] The local feature acquisition module obtains the local features of the target text based on all word vectors;

[0054] The fusion module fuses the global features and local features of the target text to obtain fused features, and recognizes the fused features to obtain entity recognition results of the target text; the entity recognition results are used to describe the categories of entities in the target text.

[0055] In a third aspect, an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned entity recognition method when executing the above-mentioned computer program.

[0056] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program implements the above-mentioned entity recognition method when executed by a processor.

[0057] The above solution of the present application has the following beneficial effects:

[0058] In an embodiment of the present application, a target text including multiple words is obtained, and a part-of-speech embedding vector and a position embedding vector of each word are generated, and then the word vector of each word is extracted using a word vector extraction model, and then the word's part-of-speech embedding vector, position embedding vector and word vector are concatenated for each word to obtain the word's embedding features, and then a two-layer bidirectional network is used to generate the global features of the target text based on all embedded features, and then the local features of the target text are obtained according to all word vectors, and finally the global features and local features of the target text are fused to obtain fused features, and the fused features are identified to obtain the entity recognition result of the target text. Among them, the word's part-of-speech embedding vector, position embedding vector and word vector are concatenated to obtain the word's embedding features, so that the embedded features include the semantic information, position information, part-of-speech information and grammatical information of the word, improve the information richness of the embedded features, improve the accuracy of the global features generated according to the rich embedded features, and fuse and identify the global features and local features with high accuracy, so as to achieve an in-depth understanding of the target text and effectively improve the accuracy of entity recognition.

[0059] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A flowchart of an entity recognition method provided in one embodiment of the present application;

[0062] Figure 2 A schematic diagram of the structure of a double-layer bidirectional network provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of a gated neural unit provided in one embodiment of the present application;

[0064] Figure 4 A schematic diagram of the structure of an entity identification device provided in one embodiment of the present application;

[0065] Figure 5 A schematic diagram of the structure of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0066] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0067] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0068] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0069] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0070] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0071] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0072] In view of the problem of poor accuracy of existing entity recognition methods, an embodiment of the present application provides an entity recognition method, which obtains a target text including multiple words, generates a part-of-speech embedding vector and a position embedding vector for each word, and then uses a word vector extraction model to extract the word vector of each word, and then respectively for each word, the word's part-of-speech embedding vector, position embedding vector and word vector are spliced ​​to obtain the word's embedding features, and then a two-layer bidirectional network is used to generate the global features of the target text based on all embedded features, and then the local features of the target text are obtained according to all word vectors, and finally the global features and local features of the target text are fused to obtain fused features, and the fused features are identified to obtain the entity recognition result of the target text. Among them, the word's part-of-speech embedding vector, position embedding vector and word vector are spliced ​​to obtain the word's embedding features, so that the embedded features include the semantic information, position information, part-of-speech information and grammatical information of the word, improve the information richness of the embedded features, improve the accuracy of the global features generated according to the rich embedded features, and fuse and identify the global features and local features with high accuracy, so as to achieve an in-depth understanding of the target text and effectively improve the accuracy of entity recognition.

[0073] Next, the entity recognition method provided in this application is exemplified.

[0074] like Figure 1 As shown, the entity recognition method provided by this application includes the following steps:

[0075] Step 11, obtain a target text including multiple words, and generate a part-of-speech embedding vector and a position embedding vector for each word.

[0076] The above part-of-speech embedding vector is used to describe the part of speech of a word, and the position embedding vector is used to describe the position of a word in the target text.

[0077] In some embodiments of the present application, the expression Get the word's part-of-speech embedding vector, where represents the part-of-speech embedding vector, Indicates The part-of-speech tags of the words, Is a function that maps part-of-speech tags to vectors. In the sentence "I like to play table tennis", the part-of-speech embedding vector may be as follows: "I": pronoun, "like": verb, "play": verb, here as the object of liking, it may be regarded as an action or activity "table tennis": noun, indicating a sport or activity.

[0078] Through the expression Get the position embedding vector of the word, where represents the position embedding vector, Represents a function that generates a positional embedding vector, which gives each word in a sentence a vector representing its position. This enables understanding the order of words in a sentence, which is crucial to understanding the meaning of a sentence. For example: "I" will get an embedding at position 1. "Like" will get an embedding at position 2. "Play" will get an embedding at position 3. "Ping pong" will get an embedding at position 4.

[0079] Step 12: Use the word vector extraction model to extract the word vector of each word.

[0080] The above word vectors are used to describe the semantic and grammatical information of words.

[0081] In some embodiments of the present application, the above-mentioned word vector extraction model can be a pre-trained global vector (GloVe, Global Vectors) model. The expression is , Represents word vector, GloVe Representing the pre-trained GloVe model, for the sentence "I like to play table tennis", the GloVe word vector may be as follows: "I": The GloVe word vector will capture the semantic features of the first-person singular pronoun. "Like": The GloVe word vector will reflect the semantic features of a verb expressing preference or emotion. "Play": The GloVe word vector will represent the semantic features of a verb for an action or recreational activity. "Ping pong": The GloVe word vector will capture the semantic features of a noun for a specific sport, which may be related to words for other sports or leisure activities.

[0082] It should be noted that the GloVe model is an algorithm for generating word vectors. It captures the statistical regularity between words by analyzing the co-occurrence frequency of words in a text. To quantify the relevance between words. This metric is based on co-occurrence probability. For example, if there are three words and , It can be expressed as:

[0083]

[0084] in It's a word In words The co-occurrence probability in the context of It's a word In words The co-occurrence probability of appearing in the context of . The value of is between 0 and 1, and the larger the value, the greater the correlation:

[0085]

[0086] in, It's a word and The number of co-occurrences, It's a word The total number of times it occurs as a central word.

[0087] The GloVe model uses an exponential function to represent the co-occurrence probability, and the formula is:

[0088]

[0089] in, and The words and words The word vector of Representing words The word vector of is an exponential function. To simplify the problem, the GloVe model uses the logarithm of the co-occurrence probability as part of the objective function:

[0090]

[0091] in, Used to measure the similarity between two vectors. The meaning is that in the entire corpus, the word and words The number of times they appear together in a window, It's a word The total number of times it occurs as a central word.

[0092] The GloVe model is the sum of the squares of the difference between the logarithm of the predicted co-occurrence probability and the logarithm of the actual co-occurrence probability. The purpose of this loss function is to adjust the word vector so that the logarithm of the co-occurrence probability predicted by the model is as close as possible to the logarithm of the actual co-occurrence probability. The above loss function is updated and the weight function is introduced to obtain the final loss function:

[0093]

[0094] in, Represents the final value of the loss function, represents the weight function, The meaning is that in the entire corpus, the word and words The number of times they appear together in a window, It's a word The total number of times it appears as a central word, and Both represent bias terms.

[0095] The GloVe model uses the above final loss function to adjust the word vectors of all generated words, and takes the word vector when the final loss function value is less than the preset value as the final output.

[0096] Step 13, for each word, concatenate the word's part-of-speech embedding vector, position embedding vector and word vector to obtain the word's embedding feature.

[0097] Specifically, through the formula:

[0098]

[0099] Get the Embedding features of words .

[0100] in, Indicates The word's part-of-speech embedding vector, Indicates The position embedding vector of each word, Indicates The word vector of each word, , Indicates the number of words.

[0101] It is worth mentioning that the word’s part-of-speech embedding vector, position embedding vector and word vector are concatenated to obtain the word’s embedding features, so that the embedded features include the word’s semantic information, position information, part-of-speech information and grammatical information, thereby improving the information richness of the embedded features.

[0102] Step 14, using a two-layer bidirectional network, generates global features of the target text based on all embedded features.

[0103] The above-mentioned two-layer bidirectional network includes a forward two-layer model for extracting the forward features of each word and a backward two-layer model for extracting the backward features of each word. The forward two-layer model and the backward two-layer model both contain multiple gated neural units connected in sequence. The gated neural unit adjusts the input data input to the gated neural unit by the previous gated neural unit by introducing a control state. The global features are obtained using the forward features and backward features of all words.

[0104] In some embodiments of the present application, the step of using a two-layer bidirectional network to generate a global feature of the target text based on all embedded features specifically includes:

[0105] In the first step, a two-layer bidirectional network is used to process all embedded features to obtain the forward features and reverse features of each word, and the forward features and reverse features of each word are concatenated to obtain the final features of each word.

[0106] It should be noted that the structure of the above two-layer bidirectional network is as follows Figure 2 As shown, the input end of the forward two-layer model and the input end of the backward two-layer model are both the input end of the two-layer bidirectional network, and the output end of the forward two-layer model and the output end of the backward two-layer model are both the output end of the two-layer bidirectional network.

[0107] The forward two-layer model includes a first forward layer and a second forward layer.

[0108] The first forward layer includes multiple first forward neural units connected in sequence, and the second forward layer includes multiple second forward neural units connected in sequence. The multiple first forward neural units correspond one-to-one to the multiple second forward neural units. The output end of each first forward neural unit is connected to the input end of the corresponding second forward neural unit. The input ends of all first forward neural units are the input ends of the first forward layer, and the output ends of all second forward neural units are the output ends of the first forward layer.

[0109] The backward two-layer model includes a first backward layer and a second backward layer.

[0110] The first backward layer includes a plurality of first backward neural units connected in sequence, and the second backward layer includes a plurality of second backward neural units connected in sequence. The plurality of first backward neural units correspond one-to-one to the plurality of second backward neural units. The output end of each first backward neural unit is connected to the input end of the corresponding second backward neural unit. The input ends of all first backward neural units are the input ends of the first backward layer, and the output ends of all second backward neural units are the output ends of the first backward layer.

[0111] The first forward neural unit, the second forward neural unit, the first backward neural unit, and the second backward neural unit are all gated neural units (such as Figure 2 DBmGRU in .

[0112] The gated neural unit includes a Gaussian filter control subunit and a gated recurrent subunit connected in sequence;

[0113] The input end of the Gaussian filter control subunit is the input end of the gated neural unit, and the output end of the gated recurrent subunit is the output end of the gated neural unit;

[0114] The Gaussian filter control subunits in the first forward layer, the second forward layer, the first backward layer, and the second backward layer are all used to adjust the input data from the previous gated neural unit to the gated neural unit by introducing a control state.

[0115] Taking the gated neural unit as the first forward neural unit as an example, the embedded features of the word and the output data of the previous gated neural unit are simultaneously input into the Gaussian filter control subunit of the gated neural unit. The Gaussian filter control subunit utilizes the embedded features of the word and adjusts the output data of the previous gated neural unit by introducing a control state.

[0116] Specifically, the expression of the Gaussian filter control subunit is:

[0117]

[0118] Among them, the two-layer activation function tanh can significantly enhance the input features processed by the Gaussian filter control subunit. The effect will be becomes , introduce the parameter , yes The control state obtained after tanh processing is The introduction of is to increase the mutual influence between the GRU output at the previous moment and the input at this moment, which is conducive to discovering the external invisible features of the input features and is helpful to improve the accuracy of prediction. represents the smoothed word vector, Relative to the center point The offset, Indicates the Gaussian kernel corresponding to The weight of Indicates The offset embedding features of words, Indicates The control state of the word, represents the weight, represents the bias term, Indicates The final hidden features (i.e. the output data of the previous gated neural unit), Indicates the adjusted The final hidden features, represents the Hadamard product. hour, is the preset initial feature.

[0119] The above expression is the expression of the Gaussian filter control subunit in the first forward layer and the first backward layer. For the second forward layer, the Gaussian filter control subunit inputs the output data of the corresponding gated neural unit in the first forward layer, rather than the embedded features of the word. For the second backward layer, the Gaussian filter control subunit inputs the output data of the corresponding gated neural unit in the first backward layer, rather than the embedded features of the word. That is, in the expression It needs to be replaced by the output data of the corresponding gated neural unit in the first backward layer.

[0120] The gated recurrent subunit may specifically be a gated recurrent model (GRU, Gate Recurrent Unit), which is used to extract features from the output data of the previous gated neural unit after adjustment.

[0121] The complete computational flow of the gated recurrent subunit can be expressed as:

[0122] Compute update gate and reset gate :

[0123]

[0124] Compute candidate hidden states :

[0125]

[0126] Calculate the final hidden state :

[0127]

[0128] in, Indicates the input information at the current moment. , , Both represent bias terms.

[0129] like Figure 2 As shown, when the embedded features are input into the two-layer bidirectional network, each embedded feature has a corresponding first backward neural unit and first forward neural unit, as shown in Figure 2 The first embedding feature in Corresponding to the first ( Figure 2 In the forward two-layer model, the first forward neural unit and the first ( Figure 2 The first backward neural unit from left to right in the middle backward two-layer model embeds the first feature They are input into the first forward neural unit and the first backward neural unit respectively, and the second embedding feature Corresponding to the second first forward neural unit and the second first backward neural unit, the second embedding feature are input into the second first forward neural unit and the second first backward neural unit respectively. Embedded features Corresponding to The first feedforward neural unit and the The first backward neural unit, Embedded features Enter the The first feedforward neural unit and the When the double-layer bidirectional network outputs, the output of the first second forward neural unit is concatenated with the output of the first second backward neural unit to obtain the final feature of the first word. , the output of the second second forward neural unit is concatenated with the output of the second second backward neural unit to obtain the final feature of the second word , No. The output of the second feedforward neural unit is The outputs of the second backward neural units are concatenated to obtain the The final features of the words , the expression is:

[0130]

[0131]

[0132]

[0133] in, Indicates The final features of the words, Indicates splicing. Indicates The output of the first or second feedforward neural unit, Indicates The output of the first or second backward neural unit, represents the gated neural unit in the forward two-layer model, Represents the gated neural unit in the backward two-layer model, and is concatenated to obtain hour, Only indicates the The output of the second feedforward neural unit (i.e. words), Only indicates the The output of the second backward neural unit is the backward features of words).

[0134] In the second step, the global features of the target text are calculated based on all the final features.

[0135] Specifically, through the formula:

[0136]

[0137] Calculate the global features of the target text .

[0138] in, Indicates weights, Indicates The final features of the words:

[0139]

[0140]

[0141] in, Indicates attention weight vector, represents the weight matrix, represents the bias term, , are activation functions, , Indicates the number of words.

[0142] It is worth mentioning that the design of the gated neural unit introduces the control state in GRU To adjust the model's output for the previous moment This regulatory mechanism enables go through After weighting, it is converted into , and then participate in the calculation of the model as a new input. Compared with GRU, this design strengthens the model's dynamic adaptability to continuous time step input.

[0143] The above-mentioned gated neural unit is exemplarily described below with reference to a specific example.

[0144] like Figure 3 As shown, in the Gaussian filter control subunit, the embedding features of the word Perform Gaussian filtering to obtain smooth word vectors , the activation function tanh is used to calculate the smoothed word vector to obtain the control state , using the control state The output of the previous gated neural unit Calculate and get the output of the previous gated neural unit after adjustment , and Input to the gated recurrent unit (GRU) for calculation, Figure 3 middle represents the activation function, × represents the multiplication operation, + represents the addition operation, represents the output of the reset gate, represents the output of the update gate, represents the candidate hidden state of the GRU, Represents the final hidden features of the GRU, and the final hidden features output by the GRU are output to the next gated neural unit in the same layer and the corresponding gated neural units in other layers.

[0145] Step 15: Obtain local features of the target text based on all word vectors.

[0146] In some embodiments of the present application, all word vectors can be input into a text convolutional neural network (TextCNN) to obtain local features of the target text. The calculation formula is:

[0147]

[0148]

[0149] in, represents the output of the convolutional layer, Represents all word vectors, represents the convolution kernel, represents the bias term, represents the maximum pooling, Represents local features.

[0150] Step 16, the global features and local features of the target text are fused to obtain fused features, and the fused features are recognized to obtain entity recognition results of the target text.

[0151] The above entity recognition results are used to describe the categories of entities in the target text. Categories are user preference categories obtained by analyzing the user's search history, feedback, and current context information, such as time and location, such as beach activities (diving, sunbathing) or specific facilities (luxury hotels, children's parks). For example, for the text "beach vacation", the entity recognition result of the text is diving, indicating that the category of the entity in the text is diving.

[0152] In some embodiments of the present application, the above-mentioned steps of fusing the global features and local features of the target text to obtain the fused features, and identifying the fused features to obtain the entity recognition result of the target text specifically include:

[0153] In the first step, the global features and local features of the target text are fused to obtain the fused features.

[0154] Specifically, through the formula:

[0155]

[0156] Get fusion features .

[0157] in, Represents global features, Represents local features.

[0158] The second step is to identify the fused features and obtain the entity recognition results of the target text.

[0159] First, the posterior probability that the target text belongs to each category is calculated based on the fused features.

[0160] Specifically, through the formula:

[0161]

[0162]

[0163] Calculate the target text belongs to the category The posterior probability .

[0164] in, represents the probability density function, Indicates category The prior probability of express The probability of represents a feature set, Indicates category, represents the fusion features of the target text, represents the covariance matrix of the fused features, represents the dimension of the fusion feature, represents the determinant of the covariance matrix, Represents the mean of the fused features.

[0165] Then, the posterior probability with the largest value is taken as the final posterior probability, and the category corresponding to the final posterior probability is taken as the entity recognition result of the target text.

[0166] It should be noted that, ideally, the sum of the posterior probabilities of all categories should be very close to 1. Due to the precision of numerical calculations, there may be slight errors. If the sum is not equal to 1 (usually due to floating point precision issues), it can be adjusted by the following formula:

[0167]

[0168] in, represents the sum of all posterior probabilities, Indicates that the adjusted target text belongs to the category The posterior probability of .

[0169] Exemplarily, for a target user who inputs a target text, the search history, feedback, and current context information of the user in multiple searches are analyzed to obtain multiple categories corresponding to the user: children's activities, sunbathing, and diving. When the user inputs the target text "beach vacation", the posterior probabilities of the categories children's activities, sunbathing, and diving are 0.2, 0.5, and 0.3, respectively. Then 0.5 is used as the final posterior probability, and the corresponding category is sunbathing. The entity recognition result of the target text is sunbathing. After the target text is recognized, data related to the target text (such as news, videos, etc.) is recommended to the target user based on the entity recognition result of the target text. For example, if the user searches for "beach vacation", the system recommends news, tourist attractions, etc. related to sunbathing to the user based on the entity recognition result "sunbathing".

[0170] It is worth mentioning that the word's part-of-speech embedding vector, position embedding vector and word vector are concatenated to obtain the word's embedding features, so that the embedded features include the word's semantic information, position information, part-of-speech information and grammatical information, thereby improving the information richness of the embedded features. The accuracy of the global features generated based on the rich embedded features is improved, and the high-precision global features and local features are fused and identified, achieving an in-depth understanding of the target text and effectively improving the accuracy of entity recognition.

[0171] The entity identification device provided in this application is exemplarily described below.

[0172] like Figure 4 As shown, an embodiment of the present application provides an entity identification device, and the entity identification device 400 includes:

[0173] The first generation module 401 obtains a target text including a plurality of words, and generates a part-of-speech embedding vector and a position embedding vector for each word; the part-of-speech embedding vector is used to describe the part of speech of the word, and the position embedding vector is used to describe the position of the word in the target text;

[0174] Extraction module 402, extracting a word vector of each word using a word vector extraction model; the word vector is used to describe the semantic and grammatical information of the word;

[0175] The concatenation module 403 concatenates the word's part-of-speech embedding vector, position embedding vector and word vector for each word to obtain the word's embedding feature;

[0176] The second generation module 404 generates global features of the target text based on all embedded features using a two-layer bidirectional network; the two-layer bidirectional network includes a forward two-layer model for extracting forward features of each word and a backward two-layer model for extracting backward features of each word, the forward two-layer model and the backward two-layer model both include a plurality of gated neural units connected in sequence, the gated neural unit adjusts input data inputted to the gated neural unit by a previous gated neural unit by introducing a control state, and the global features are obtained by using the forward features and backward features of all words;

[0177] A local feature acquisition module 405 acquires local features of the target text according to all word vectors;

[0178] The fusion module 406 fuses the global features and local features of the target text to obtain fused features, and recognizes the fused features to obtain entity recognition results of the target text; the entity recognition results are used to describe the category of the target text.

[0179] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0180] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0181] like Figure 5 As shown, an embodiment of the present application provides a terminal device. The terminal device D10 of this embodiment includes: at least one processor D100 ( Figure 5 Only one processor is shown in the figure), a memory D101, and a computer program D102 stored in the memory D101 and executable on the at least one processor D100, wherein the processor D100 implements the steps of any of the above-mentioned method embodiments when executing the computer program D102.

[0182] Specifically, when the processor D100 executes the computer program D102, it obtains a target text including multiple words, generates a part-of-speech embedding vector and a position embedding vector for each word, then uses a word vector extraction model to extract the word vector of each word, and then splices the word's part-of-speech embedding vector, position embedding vector and word vector for each word to obtain the word's embedding features, and then uses a two-layer bidirectional network to generate the global features of the target text based on all embedded features, and then obtains the local features of the target text according to all word vectors, and finally fuses the global features and local features of the target text to obtain fused features, and identifies the fused features to obtain the entity recognition result of the target text. Among them, the word's part-of-speech embedding vector, position embedding vector and word vector are spliced ​​to obtain the word's embedding features, so that the embedded features include the semantic information, position information, part-of-speech information and grammatical information of the word, improve the information richness of the embedded features, improve the accuracy of the global features generated according to the rich embedded features, fuse and identify the global features and local features with high accuracy, achieve in-depth understanding of the target text, and effectively improve the accuracy of entity recognition.

[0183] The processor D100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0184] In some embodiments, the memory D101 may be an internal storage unit of the terminal device D10, such as a hard disk or memory of the terminal device D10. In other embodiments, the memory D101 may also be an external storage device of the terminal device D10, such as a plug-in hard disk, a smart memory card (SMC, SmartMedia Card), a secure digital (SD, Secure Digital) card, a flash card (Flash Card), etc. equipped on the terminal device D10. Further, the memory D101 may also include both an internal storage unit of the terminal device D10 and an external storage device. The memory D101 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory D101 may also be used to temporarily store data that has been output or is to be output.

[0185] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0186] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0187] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the entity recognition method device / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0188] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0189] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0190] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An entity recognition method, characterized in that: include: Obtain a target text including multiple words and generate a part-of-speech embedding vector and a position embedding vector for each word; The part-of-speech embedding vector is used to describe the part-of-speech of a word, and the position embedding vector is used to describe the position of a word in the target text; A word vector extraction model is used to extract a word vector for each word; the word vector is used to describe the semantic and grammatical information of the word; For each of the words, concatenate the word's part-of-speech embedding vector, position embedding vector, and word vector to obtain the embedding feature of the word; A two-layer bidirectional network is used to generate global features of the target text based on all embedded features; the two-layer bidirectional network includes a forward two-layer model for extracting forward features of each word and a backward two-layer model for extracting backward features of each word, the forward two-layer model and the backward two-layer model both include a plurality of gated neural units connected in sequence, the gated neural unit adjusts input data input to the gated neural unit by a previous gated neural unit by introducing a control state, and the global features are obtained by using the forward features and backward features of all words; Obtaining local features of the target text according to all word vectors; The global features and local features of the target text are fused to obtain fused features, and the fused features are identified to obtain entity recognition results of the target text; the entity recognition results are used to describe the categories of entities in the target text; The input end of the forward double-layer model and the input end of the backward double-layer model are both input ends of the double-layer bidirectional network, and the output end of the forward double-layer model and the output end of the backward double-layer model are both output ends of the double-layer bidirectional network. The forward two-layer model includes a first forward layer and a second forward layer; The first feed-forward layer includes a plurality of first feed-forward neural units connected in sequence, the second feed-forward layer includes a plurality of second feed-forward neural units connected in sequence, the plurality of first feed-forward neural units correspond to the plurality of second feed-forward neural units one by one, the output end of each of the first feed-forward neural units is connected to the input end of the corresponding second feed-forward neural unit, the input ends of all the first feed-forward neural units are the input ends of the first feed-forward layer, and the output ends of all the second feed-forward neural units are the output ends of the first feed-forward layer; The backward two-layer model includes a first backward layer and a second backward layer; The first backward layer includes a plurality of first backward neural units connected in sequence, the second backward layer includes a plurality of second backward neural units connected in sequence, the plurality of first backward neural units correspond to the plurality of second backward neural units one by one, the output end of each of the first backward neural units is connected to the input end of the corresponding second backward neural unit, the input ends of all the first backward neural units are the input ends of the first backward layer, and the output ends of all the second backward neural units are the output ends of the first backward layer; The first forward neural unit, the second forward neural unit, the first backward neural unit, and the second backward neural unit are all gated neural units; The gated neural unit includes a Gaussian filter control subunit and a gated loop subunit connected in sequence; The input end of the Gaussian filter control subunit is the input end of the gated neural unit, and the output end of the gated cycle subunit is the output end of the gated neural unit; The Gaussian filter control subunits in the first forward layer, the second forward layer, the first backward layer, and the second backward layer are all used to adjust the input data from the previous gated neural unit to the gated neural unit by introducing a control state.

2. The entity recognition method according to claim 1, characterized in that: The step of concatenating the word's part-of-speech embedding vector, position embedding vector, and word vector to obtain the word's embedding features includes: By formula: Get the Embedding features of words ; in, Indicates the The word's part-of-speech embedding vector, Indicates the The position embedding vector of each word, Indicates the The word vector of each word, , Indicates the number of words.

3. The entity recognition method according to claim 1, characterized in that: The method of using a two-layer bidirectional network to generate the global features of the target text based on all embedded features includes: Using a two-layer bidirectional network to process all embedded features, obtain forward features and reverse features of each word, and concatenate the forward features and reverse features of each word to obtain final features of each word; A global feature of the target text is calculated based on all the final features.

4. The entity recognition method according to claim 3, characterized in that: The calculating the global features of the target text based on all the final features includes: By formula: Calculate the global features of the target text ; in, Indicates weights, Indicates The final features of the words: in, Indicates attention weight vector, represents the weight matrix, represents the bias term, , are activation functions, , Indicates the number of words.

5. The entity recognition method according to claim 1, characterized in that: The step of identifying the fused features to obtain entity recognition results of the target text includes: Calculate the posterior probability that the target text belongs to each category according to the fusion feature; The posterior probability with the largest value is taken as the final posterior probability, and the category corresponding to the final posterior probability is taken as the entity recognition result of the target text.

6. The entity recognition method according to claim 5, characterized in that: The calculating the posterior probability that the target text belongs to each category according to the fusion feature includes: By formula: Calculate the target text belongs to the category The posterior probability ; in, represents the probability density function, Indicates category The prior probability of express The probability of represents a feature set, Indicates category, represents the fusion features of the target text, represents the covariance matrix of the fused features, represents the dimension of the fusion feature, represents the determinant of the covariance matrix, Represents the mean of the fused features.

7. The entity recognition method according to claim 1, characterized in that: The entity recognition method further includes: According to the entity recognition result of the target text, data related to the target text is recommended to a target user; the target user is a user who inputs the target text.

8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the entity recognition method according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the entity recognition method according to any one of claims 1 to 7 is implemented.

Images