Short text entity relevance identification method, system, electronic device and storage medium

By obtaining the short text training set and entity precoding matrix, it is converted into word vectors, position vectors and precoding vectors, and inputting them into the TD_LSTM network to generate a classification coded vector, which solves the problem of inaccurate recognition of the entity correlation of short text, and realizes the Highly accurate entity correlation recognition.

CN114444496BActive Publication Date: 2024-11-22BEIJING ZHIHUI XINGGUANG INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110439445.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-04-23
Publication Date
2024-11-22
Estimated Expiration
2041-04-23

AI Technical Summary

Technical Problem

In the prior art, short text entity correlation recognition is inaccurate, and machine learning-based methods require a large number of feature engineering. Although neural network-based methods can automatically learn features, entity correlation recognition is still not accurate enough.

Method used

A short text entity correlation recognition method is adopted. By obtaining the short text training set and entity precoding matrix, it is converted into word vectors, position vectors and precoding vectors, and input them into the TD_LSTM network to generate forward and backward encoding and splicing. The categorical coding vector is generated by the feedforward neural network, and through iterative optimization training of loss function, the optimal model is used to identify the correlation intensity of entities in short text.

Benefits of technology

Through multi-dimensional information fusion, the expression of entity features is enriched, the confidence in the calculation of entity-text correlation is improved, and the accurate identification of the correlation between short text entities is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114444496B_ABST
    Figure CN114444496B_ABST
Patent Text Reader

Abstract

The invention discloses a short text entity relevance recognition method, system, electronic device and storage medium. The method comprises: fusing a word vector containing entity context semantic information, a position vector containing entity position coding information and a precoding vector containing entity precoding information to obtain a fused word vector of a word in a training text; inputting the fused word vector of each word in the training text into a TD_LSTM model to obtain a forward vector coding and a backward vector coding, splicing the forward vector coding and the backward vector coding to obtain an entity splicing vector, passing the entity splicing vector through a feedforward neural network to obtain a classification coding vector of each training text, normalizing the classification coding vector, obtaining a loss function according to the normalized classification coding vector, iteratively optimizing the loss function to obtain an optimal model; inputting a short text to be recognized into the optimal model to obtain a recognition result of the text to be recognized; and achieving the accuracy of short text entity relevance recognition.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of text processing, and in particular to a short text entity relevance recognition method, system, electronic device and storage medium. Background Art

[0002] At present, the processing methods of text entity relevance are mainly divided into two categories: machine learning based and neural network based.

[0003] The machine learning-based method mainly constructs feature information of the text manually, and then uses machine learning classifiers to classify the feature information. Commonly used feature classifiers include support vector machine (SVM), naive Bayes, etc. In traditional machine learning methods, feature construction is very important and is the key to determining accuracy. A lot of feature engineering work is required, and the extraction and construction of feature information is not comprehensive, resulting in low accuracy in entity relevance recognition based on given feature information.

[0004] The neural network-based method represents the text with a deep semantic feature vector, and then predicts the relevance of the entity based on the given entity. Compared with manually constructing feature information, using neural networks for feature extraction can be more comprehensive, and it can learn the target feature representation from the data itself. Neural networks automatically learn the characteristics of features, avoiding the feature extraction process that requires a lot of domain knowledge. Commonly used neural networks include recurrent neural networks (RNN), long short-term memory networks (LSTM), etc. However, the entity features automatically learned based on contextual semantics are still incomplete, and there is a problem of inaccurate entity relevance recognition. Summary of the invention

[0005] In view of this, embodiments of the present invention provide a short text entity relevance identification method, system, electronic device and storage medium to solve the problem of inaccurate short text entity relevance identification in the prior art.

[0006] To this end, the following technical solutions are provided in the embodiments of the present invention:

[0007] According to the first aspect, an embodiment of the present invention provides a short text entity relevance identification method, including: obtaining a short text training set, the short text training set including multiple training texts and a named entity corresponding to each training text; obtaining an entity precoding matrix, the entity precoding matrix including precoding vectors corresponding to all entities, wherein the entity precoding is obtained based on the context information of the entity precoding; converting each word in each training text into a word vector, and converting the position of each word in the training text into a position vector; obtaining a precoding vector corresponding to each word according to each word in each training text and the entity precoding matrix, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is a zero vector; obtaining each word according to the word vector, position vector and precoding vector corresponding to each word in each training text. A fused word vector corresponding to each word in a training text; inputting the fused word vector of each training text into the TD_LSTM network respectively to obtain the forward vector code and the backward vector code corresponding to each training text; splicing the forward vector code and the backward vector code corresponding to each training text respectively to obtain the entity splicing vector corresponding to each training text; passing the entity splicing vector corresponding to each training text through a feedforward neural network to obtain the classification coding vector corresponding to each training text; normalizing the classification coding vector to obtain the normalized classification coding vector; obtaining the loss function between the classification vector of the model code and the actual entity classification vector according to the normalized classification coding vector; performing iterative optimization training according to the loss function to obtain the optimal model, the optimal model being used to identify the correlation strength between the short text and the entity in the short text; obtaining the short text to be identified; inputting the short text to be identified into the optimal model to obtain the recognition result of the text to be identified.

[0008] Optionally, the calculation formula for the fused word vector is as follows:

[0009] w_a=w_ta+w_pa+w_da

[0010] Among them, w_a is the fused word vector corresponding to word a; w_ta is the word vector corresponding to word a; w_pa is the position vector corresponding to word a, and w_da is the precoding vector corresponding to word a.

[0011] Optionally, after the step of inputting the fused word vector of each training text into the TD_LSTM network to obtain the forward vector encoding and the backward vector encoding corresponding to each training text, the method further includes: inputting the fused word vector of each training text into the TD_LSTM network to obtain the forward vector encoding and the backward vector encoding corresponding to each training text. The entity word encoding vector corresponding to the entity word in each training text in the network, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word; weighted average the entity word encoding vector, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word to obtain a new encoding vector corresponding to the entity word; compare the new encoding vector corresponding to the entity word with the precoding vector corresponding to the entity word in the entity precoding matrix to obtain a similarity calculation value; determine whether the similarity calculation value is greater than a preset similarity threshold; if the similarity calculation value is greater than the preset similarity threshold, the entity precoding matrix is ​​not updated; if the similarity calculation value is less than or equal to the preset similarity threshold, weighted average the new encoding vector corresponding to the entity word and the precoding vector corresponding to the entity word in the entity precoding matrix to obtain an updated precoding vector, and update the updated precoding vector to the entity precoding matrix.

[0012] Optionally, the calculation formula of the similarity calculation value is as follows:

[0013] Threshold=cosine(w_new,w_d)

[0014] Among them, Threshold is the similarity calculation value, w_new is the new encoding vector corresponding to the entity word, and w_d is the pre-encoding vector corresponding to the entity word;

[0015] The calculation formula for the new encoding vector corresponding to the entity word is as follows:

[0016]

[0017] Where s is the first preset number on the left side of the entity word, v is the second preset number on the right side of the entity word; w_t is the entity word encoding vector corresponding to the entity word in the training text, w_t l-1 is the encoding vector of the first word on the left side of the entity word in the training text, w_t l-2 is the encoding vector of the second word on the left side of the entity word in the training text, w_t l-s is the encoding vector of the sth word to the left of the entity word in the training text, w_t r+1 is the encoding vector of the first word to the right of the entity word in the training text, w_t r+2 is the encoding vector of the second word to the right of the entity word in the training text, w_t r+vis the encoding vector of the vth word to the right of the entity word in the training text.

[0018] Optionally, the calculation formula of the updated precoding vector is as follows:

[0019]

[0020] Among them, w_z is the updated pre-coding vector of the entity word; w_new is the new coding vector corresponding to the entity word, and w_d is the pre-coding vector corresponding to the entity word.

[0021] Optionally, before the step of inputting the short text to be recognized into the optimal model, it also includes: inputting the short text to be recognized into the TextRCNN model for text binary classification to obtain a classification result; if the classification result of the short text to be recognized is noise text, the text to be recognized is removed; if the classification result of the short text to be recognized is non-noise text, the text to be recognized is retained.

[0022] Optionally, the loss function is calculated as follows:

[0023]

[0024] Where n is the number of training samples, y i is the actual classification vector of the i-th training sample, a i is the classification vector of the i-th training sample encoded by the model.

[0025] According to a second aspect, an embodiment of the present invention provides a short text entity relevance identification system, including:

[0026] A first acquisition module is used to acquire a short text training set, wherein the short text training set includes a plurality of training texts and a named entity corresponding to each training text;

[0027] A second acquisition module is used to acquire an entity precoding matrix, where the entity precoding matrix includes precoding vectors corresponding to all entities, wherein the entity precoding is obtained according to context information of the entity precoding;

[0028] A first processing module is used to convert each word in each training text into a word vector, and convert the position of each word in the training text into a position vector;

[0029] A second processing module is used to obtain a precoding vector corresponding to each word according to each word and the entity precoding matrix in each training text, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is a zero vector;

[0030] The third processing module is used to obtain a fused word vector corresponding to each word in each training text according to the word vector, position vector and precoding vector corresponding to each word in each training text;

[0031] The fourth processing module is used to input the fused word vector of each training text into the TD_LSTM network to obtain the forward vector code and the backward vector code corresponding to each training text;

[0032] A fifth processing module is used to concatenate the forward vector code and the backward vector code corresponding to each training text to obtain an entity concatenation vector corresponding to each training text;

[0033] A sixth processing module, used for passing the entity concatenation vector corresponding to each training text through a feedforward neural network to obtain a classification coding vector corresponding to each training text;

[0034] A seventh processing module, used for normalizing the classification coding vector to obtain a normalized classification coding vector;

[0035] An eighth processing module, used to obtain a loss function between the model-encoded classification vector and the actual entity classification vector according to the normalized classification coding vector;

[0036] A ninth processing module, used for performing iterative optimization training according to the loss function to obtain an optimal model, wherein the optimal model is used for identifying the correlation strength between the short text and the entity in the short text;

[0037] The third acquisition module is used to acquire the short text to be recognized;

[0038] The tenth processing module is used to input the short text to be recognized into the optimal model to obtain the recognition result of the text to be recognized.

[0039] Optionally, the calculation formula for the fused word vector is as follows:

[0040] w_a=w_ta+w_pa+w_da

[0041] Among them, w_a is the fused word vector corresponding to word a; w_ta is the word vector corresponding to word a; w_pa is the position vector corresponding to word a, and w_da is the precoding vector corresponding to word a.

[0042] Optionally, it also includes: an eleventh processing module, which is used to input the fused word vector of each training text into the TD_LSTM network, respectively, the entity word encoding vector corresponding to each entity word in each training text, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word; a twelfth processing module, which is used to perform weighted averaging on the entity word encoding vector, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word to obtain a new encoding vector corresponding to the entity word; a thirteenth processing module, which is used to compare the new encoding vector corresponding to the entity word with the entity word. The precoding vectors corresponding to the entity words in the entity precoding matrix are compared for similarity to obtain a similarity calculation value; a judgment module is used to judge whether the similarity calculation value is greater than a preset similarity threshold; a fourteenth processing module is used to not update the entity precoding matrix if the similarity calculation value is greater than the preset similarity threshold; a fifteenth processing module is used to perform weighted averaging of the new coding vector corresponding to the entity word and the precoding vector corresponding to the entity word in the entity precoding matrix if the similarity calculation value is less than or equal to the preset similarity threshold to obtain an updated precoding vector, and update the updated precoding vector to the entity precoding matrix.

[0043] Optionally, the calculation formula of the similarity calculation value is as follows:

[0044] Threshold=cosine(w_new,w_d)

[0045] Among them, Threshold is the similarity calculation value, w_new is the new encoding vector corresponding to the entity word, and w_d is the pre-encoding vector corresponding to the entity word;

[0046] The calculation formula for the new encoding vector corresponding to the entity word is as follows:

[0047]

[0048] Where s is the first preset number on the left side of the entity word, v is the second preset number on the right side of the entity word; w_t is the entity word encoding vector corresponding to the entity word in the training text, w_t l-1 is the encoding vector of the first word on the left side of the entity word in the training text, w_t l-2 is the encoding vector of the second word on the left side of the entity word in the training text, w_t l-s is the encoding vector of the sth word to the left of the entity word in the training text, w_t r+1 is the encoding vector of the first word to the right of the entity word in the training text, w_t r+2 is the encoding vector of the second word to the right of the entity word in the training text, w_t r+vis the encoding vector of the vth word to the right of the entity word in the training text.

[0049] Optionally, the calculation formula of the updated precoding vector is as follows:

[0050]

[0051] Among them, w_z is the updated pre-coding vector of the entity word; w_new is the new coding vector corresponding to the entity word, and w_d is the pre-coding vector corresponding to the entity word.

[0052] Optionally, it also includes: a sixteenth processing module, which is used to input the short text to be identified into the TextRCNN model for text binary classification to obtain a classification result; a seventeenth processing module, which is used to remove the text to be identified if the classification result of the short text to be identified is a noise text; and an eighteenth processing module, which is used to retain the text to be identified if the classification result of the short text to be identified is a non-noise text.

[0053] Optionally, the loss function is calculated as follows:

[0054]

[0055] Where n is the number of training samples, y i is the actual classification vector of the i-th training sample, a i is the classification vector of the i-th training sample encoded by the model.

[0056] According to the third aspect, an electronic device is provided in an embodiment of the present invention, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the short text entity relevance identification method described in any one of the above-mentioned first aspects.

[0057] According to the fourth aspect, a computer-readable storage medium is provided in an embodiment of the present invention, and the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the short text entity relevance identification method described in any one of the above-mentioned first aspects.

[0058] The technical solution of the embodiment of the present invention has the following advantages:

[0059] The embodiment of the present invention provides a short text entity relevance recognition method, system, electronic device and storage medium, wherein the method comprises: obtaining a short text training set, wherein the short text training set comprises a plurality of training texts and a named entity corresponding to each training text; obtaining an entity precoding matrix, wherein the entity precoding matrix comprises precoding vectors corresponding to all entities, wherein the entity precoding is obtained based on the context information of the entity precoding; converting each word in each training text into a word vector, and converting the position of each word in the training text into a position vector; obtaining a precoding vector corresponding to each word according to each word in each training text and the entity precoding matrix, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is a zero vector; obtaining a word vector, a position vector and a precoding matrix according to the word vector, the position vector and the precoding matrix corresponding to each word in each training text ... The vector obtains the fused word vector corresponding to each word in each training text; the fused word vector of each training text is input into the TD_LSTM network respectively to obtain the forward vector code and the backward vector code corresponding to each training text; the forward vector code and the backward vector code corresponding to each training text are spliced ​​respectively to obtain the entity splicing vector corresponding to each training text; the entity splicing vector corresponding to each training text is passed through a feedforward neural network to obtain the classification coding vector corresponding to each training text; the classification coding vector is normalized to obtain a normalized classification coding vector; the loss function between the classification vector of the model code and the actual entity classification vector is obtained according to the normalized classification coding vector; iterative optimization training is performed according to the loss function to obtain the optimal model, and the optimal model is used to identify the correlation strength between the short text and the entity in the short text; the short text to be identified is obtained; the short text to be identified is input into the optimal model to obtain the recognition result of the text to be identified.The above steps use multi-dimensional information fusion. The first dimension of information is the word vector containing the semantic information of the entity context, the second dimension of information is the position vector containing the entity position encoding information, and the third dimension of information is the pre-coding vector containing the entity pre-coding information. The above three vectors are fused to obtain the fused word vector corresponding to the word in the training text. Not only the relevance of the entity to the context is considered from the local context, but also the relevance of the context information carried by the entity itself to the context is considered from the large context environment. The entity feature expression is enriched from multiple information dimensions, and the confidence of the calculation of the relevance between the entity and the text is improved. After that, the fused word vector corresponding to each word in the training text is input into TD_LST. The forward vector code and the backward vector code corresponding to each training text are obtained in the M model, and the forward vector code and the backward vector code are spliced ​​to obtain the entity splicing vector. The entity splicing vector is passed through the feedforward neural network to obtain the classification code vector corresponding to each training text. The classification code vector is normalized to obtain the normalized classification code vector. The loss function between the classification vector of the model code and the actual entity classification vector is obtained according to the normalized classification code vector. Iterative optimization training is performed according to the loss function to obtain the optimal model; finally, the short text to be recognized is input into the optimal model to obtain the recognition result of the text to be recognized, and the correlation between the entity in the short text to be recognized and the short text to be recognized is determined. The recognition accuracy of the short text entity correlation is achieved through the above steps. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A flowchart of a specific example of a short text entity relevance identification method according to an embodiment of the present invention;

[0062] Figure 2 A flowchart of another specific example of the short text entity relevance identification method according to an embodiment of the present invention;

[0063] Figure 3 A block diagram of a specific example of a short text entity relevance identification system according to an embodiment of the present invention;

[0064] Figure 4 Schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] The embodiment of the present invention provides a short text entity relevance identification method, such as Figure 1 As shown, the method includes steps S1-S13.

[0067] Step S1: Obtain a short text training set, where the short text training set includes a plurality of training texts and a named entity corresponding to each training text.

[0068] As an exemplary embodiment, short text is relative to long text. Specifically, short text refers to text with a word count within a preset number of words. In this embodiment, the preset word count is set to 350, which means that text with a word count of less than 350 words is a short text. This embodiment is only for illustrative purposes and is not limited to this. Of course, in other embodiments, the preset word count can also be set to other values, which can be reasonably set as needed in actual applications.

[0069] The short text training set includes multiple short text training texts and the named entities contained in each short text. Specifically, the named entity recognition method in the prior art can be used to identify the named entities of the training text to obtain the named entities corresponding to the training text. In this embodiment, the named entities include the names of institutions; of course, in other embodiments, the named entities may also include names of people, places, etc. The specific types can be reasonably determined according to actual needs. This embodiment is only for illustrative purposes and is not limited thereto.

[0070] Step S2: Acquire an entity precoding matrix, where the entity precoding matrix includes precoding vectors corresponding to all entities, wherein the entity precoding is obtained based on context information of entity precoding.

[0071] As an exemplary embodiment, the entity precoding matrix includes precoding vectors corresponding to all entities, and each entity corresponds to a precoding vector. The precoding vector is obtained based on the context information pre-encoded into the entity. The specific process is to first initialize an entity precoding matrix, and then train through TD_LSTM to obtain the entity vector, the word vector of the preset number on the left, and the word vector of the preset number on the right. The entity vector and the word vector of the word on the left and the word vector of the word on the right are weighted averaged to obtain a new entity vector of the entity that incorporates context information. The new entity vector is calculated similarity with the entity vector in the entity precoding matrix. If the similarity is less than the preset threshold, no operation is performed. If the similarity is greater than or equal to the preset threshold, the new entity vector and the original entity vector are weighted averaged and updated to the entity precoding matrix. Through continuous iterative training, new context information is continuously integrated into the entity precoding matrix, so that the entity itself carries rich context information. The context information is used to calculate the degree of relevance to the short text to help determine whether the entity is the main entity. Specifically, the entity precoding matrix can be obtained by initializing a vector matrix of all entities through a pre-encoding module.

[0072] When judging the relevance of an entity, we use the context information pre-encoded into the entity to calculate the relevance to the short text to help determine whether the entity is the main entity. The entity pre-coding is obtained through training data, and a large amount of context information is integrated into the entity pre-coding vector. For example, in the following text, four entities are mentioned, namely Vanke, Ping An of China, Kweichow Moutai, and CATL.

[0073] The context information of Vanke is generally related to real estate, the context information of Ping An of China is generally related to insurance, and the context information of Kweichow Moutai is generally related to liquor. Through rounds of iterative training, these context information are integrated into the entity vector, so that the entity itself carries a large amount of information, which can help determine the relevance of the entity and improve the confidence of the determination of entity relevance.

[0074] Step S3: Convert each word in each training text into a word vector, and convert the position of each word in the training text into a position vector.

[0075] As an exemplary embodiment, a word2vec word vector model with a preset dimension is pre-trained based on multi-domain corpora, and the word vector is obtained from the model. In this embodiment, the preset dimension is set to 300 dimensions. Of course, in other embodiments, the preset dimension can also be set to other values, which can be reasonably set as needed. The words in each short text in the training text are converted into word vectors w_t, and the word vectors represent the entity context semantic information.

[0076] The position of the word in the short text is converted into a position vector w_p, and the position vector represents the entity position encoding information. In this embodiment, the dimension of the position vector is also a preset dimension, and specifically, the preset dimension is set to 300 dimensions.

[0077] The position vector is encoded as follows: In this embodiment, sine-cosine position encoding is used, and the linear variation characteristics of sine and cosine functions are used to encode the position information of words in short texts. The specific formula is as follows:

[0078]

[0079]

[0080] In the above formula, pos represents the actual position of a word in a short text, i represents the i-th dimension of the word vector, and d model It represents the dimension of word vector.

[0081] Through the analysis and statistics of a large number of short texts, the main entity associated with the short text has a higher semantic affinity with the context, while the secondary entity has a lower semantic affinity with the context. The main entity is generally located at the head of the short text and rarely appears at the end of the short text, so position encoding can also improve the judgment of entity relevance.

[0082] Step S4: Obtain the precoding vector corresponding to each word according to each word and the entity precoding matrix in each training text, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is the zero vector.

[0083] As an exemplary embodiment, each word in a training text is matched with an entity in an entity precoding matrix. If the word is a word corresponding to an entity, then the word corresponding to the entity exists in the entity precoding matrix, that is, the entity corresponding to the word can be found in the entity precoding matrix, so the precoding vector corresponding to the entity in the entity precoding matrix is ​​used as the precoding vector of the entity word. If the word is a word corresponding to a non-named entity, then the word corresponding to the entity does not exist in the entity precoding matrix, that is, there is no entity corresponding to the word in the entity precoding matrix, then the precoding vector corresponding to the word is set to a zero vector, that is, the non-entity word does not add a precoding vector or keeps the original vector unchanged.

[0084] Step S5: Obtain the fused word vector corresponding to each word in each training text according to the word vector, position vector and precoding vector corresponding to each word in each training text.

[0085] As an exemplary embodiment, each word in the short text is converted into a word vector w_t, the position of the word in the short text is converted into a position vector w_p, and the precoding vectors w_d of the words in the short text obtained from the entity precoding matrix are added to obtain a fused word vector, and the fused word vector w_a is input into the BI_LSTM network.

[0086] Specifically, the calculation formula for the fused word vector is as follows:

[0087] w_a=w_ta+w_pa+w_da

[0088] Among them, w_a is the fused word vector corresponding to word a; w_ta is the word vector corresponding to word a; w_pa is the position vector corresponding to word a, and w_da is the precoding vector corresponding to word a. When word a is a named entity word, w_da is the corresponding precoding vector in the entity precoding matrix; when word a is a non-named entity word, w_da is a zero vector.

[0089] Step S6: Input the fused word vector of each training text into the TD_LSTM network to obtain the forward vector code and backward vector code corresponding to each training text.

[0090] As an exemplary embodiment, the TD_LSTM network uses two LSTM networks, namely, the LSTM_L network and the LSTM_R network. The input of the LSTM_L network is the first word of the short text to the entity word, which is input from left to right into the LSTM_L, and the output is the forward vector encoding that incorporates the context information of the short text. The input of the LSTM_R network is the last word of the short text to the entity word, which is input from right to left into the LSTM_R, and the output is the backward vector encoding that incorporates the context information of the short text.

[0091] The specific output results are as follows:

[0092] w_l=LSTM_L(w0,w1,…,wt)

[0093] w_r=LSTM_R(wn,wn-1,…,wt)

[0094] Among them, LSTM_L is a forward long short-term memory network (input words are fed to the network from left to right), LSTM_R is a backward long short-term memory network (input words are fed to the network from right to left); w0 is the first word from the left, w1 is the second word from the left, and so on, wt is the entity word; wn is the first word from the right, wn-1 is the second word from the right, and so on, wt is the entity word.

[0095] In this embodiment, the model used to identify the main entity or the secondary entity is TD-LSTM (Target-Dependent Long Short_Term Memory). The model is modeled according to the context before and after the entity word, and actually has two LSTMs, namely LSTM_L and LSTM_R.

[0096] As shown in the following sentence:

[0097] Why am I, a shopping mall employee, being chased around by the security guards of Suning Plaza every day!!!

[0098] The entity in this sentence is "Suning Plaza". The context before Suning Plaza is from "I" to "Suning Plaza", and the context after "Suning Plaza" is from "Suning Plaza" to the exclamation mark at the end of the sentence.

[0099] LSTM_L inputs the previous context of "Suning Plaza" into LSTM, that is, from the first word of the sentence to the entity word, from left to right, to the network for semantic information encoding, and obtains a forward vector encoding that incorporates the semantic information of the previous context.

[0100] LSTM_R inputs the context of "Suning Plaza" into LSTM, that is, from the last word of the sentence to the entity word, from right to left, and inputs the network in sequence to encode the semantic information to obtain the backward vector encoding that integrates the semantic information of the following context.

[0101] Step S7: respectively concatenate the forward vector code and the backward vector code corresponding to each training text to obtain an entity concatenation vector corresponding to each training text.

[0102] As an exemplary embodiment, the forward vector encoding that incorporates the previous information and the backward vector encoding that incorporates the following information are concatenated to obtain a concatenated vector. The specific concatenation method can be concatenation with a concat function, which integrates the previous semantic information and the following semantic information related to the entity into the entity vector, enriching the information content of the entity. Because the entity incorporates the contextual semantic information, the relevance of the entity to the context can be determined.

[0103] The calculation formula of the splicing vector is:

[0104] w_i=concat(w_l,w_r)

[0105] Among them, w_i is the entity concatenation vector corresponding to the training text, w_l is the forward vector encoding corresponding to the training text, and w_r is the backward vector encoding corresponding to the training text.

[0106] Step S8: The entity concatenation vector corresponding to each training text is passed through a feedforward neural network to obtain a classification coding vector corresponding to each training text.

[0107] As an exemplary embodiment, the entity concatenation vector corresponding to the obtained training text is passed through a feedforward neural network to obtain a classification coding vector, where the classification refers to strong correlation and weak correlation. In this embodiment, the classification coding includes two types, 0 and 1, respectively, 0 represents weak correlation and 1 represents strong correlation.

[0108] The calculation formula of the classification coding vector is:

[0109] c_i=FeedForward(w_i)

[0110] Among them, c_i is the classification encoding vector corresponding to the training text; w_i is the entity splicing vector corresponding to the training text.

[0111] Step S9: Normalize the classification coding vector to obtain a normalized classification coding vector.

[0112] As an exemplary embodiment, the role of normalization is to map the value range of the classification to the interval [0,1], so that the loss difference can be calculated with the actual classification values ​​0 and 1, and back propagation can be performed to update parameters and iteratively optimize the model. The classification encoding vector c_i is output to SoftMax for normalization to obtain a normalized classification encoding vector.

[0113] Step S10: Obtain the loss function between the model-encoded classification vector and the actual entity classification vector according to the normalized classification coding vector.

[0114] As an exemplary embodiment, the normalized classification coding vector is calculated to obtain a loss function between the classification vector encoded by the model and the actual entity classification vector.

[0115] The loss function is calculated as:

[0116]

[0117] Where n is the number of training samples, y i is the actual classification vector of the i-th training sample, a i is the classification vector of the i-th training sample encoded by the model.

[0118] Step S11: performing iterative optimization training according to the loss function to obtain an optimal model, wherein the optimal model is used to identify the correlation strength between short texts and entities in the short texts.

[0119] As an exemplary embodiment, the loss function is back-propagated into an iteration phase, the parameter weights are updated, and training is continued. This is to obtain the optimal parameters of the model through continuous training, and ultimately achieve the purpose of discerning the relevance of the entity. By continuously updating the parameters, the predicted value of the model is fitted to the true value of the sample as much as possible, thereby reducing the error between the predicted value and the true value, and training the optimal model that can predict the relevance of the entity.

[0120] Step S12: Obtain the short text to be recognized.

[0121] As an exemplary embodiment, a short text to be recognized is obtained according to a recognition task.

[0122] Step S13: input the short text to be recognized into the optimal model to obtain the recognition result of the text to be recognized.

[0123] As an exemplary embodiment, the short text to be recognized is input into the optimal model to obtain the recognition result output by the optimal model, and the recognition result includes 0 and 1, 0 indicates that the entity is weakly correlated with the short text, and 1 indicates that the entity is strongly correlated with the short text.

[0124] The above steps use multi-dimensional information fusion. The first dimension of information is the word vector containing the semantic information of the entity context, the second dimension of information is the position vector containing the entity position encoding information, and the third dimension of information is the pre-coding vector containing the entity pre-coding information. The above three vectors are fused to obtain the fused word vector corresponding to the word in the training text. Not only the relevance of the entity to the context is considered from the local context, but also the relevance of the context information carried by the entity itself to the context from the large context environment. It enriches the entity feature expression from multiple information dimensions and improves the confidence of the calculation of the relevance between the entity and the text. Then, the fused word vector corresponding to each word in the training text is used. The vector is input into the TD_LSTM model to obtain the forward vector code and backward vector code corresponding to each training text, and the forward vector code and the backward vector code are spliced ​​to obtain the entity splicing vector, and the entity splicing vector is passed through the feedforward neural network to obtain the classification code vector corresponding to each training text, and the classification code vector is normalized to obtain the normalized classification code vector. The loss function is obtained according to the normalized classification code vector, and iterative optimization training is performed according to the loss function to obtain the optimal model; finally, the short text to be recognized is input into the optimal model to obtain the recognition result of the text to be recognized, and the relevance between the entity in the short text to be recognized and the short text to be recognized is determined. The recognition accuracy of the short text entity relevance is achieved through the above steps.

[0125] As an exemplary embodiment, after the step S6 of inputting the fused word vector of each training text into the TD_LSTM network to obtain the forward vector encoding and backward vector encoding corresponding to each training text, steps S14-S19 are also included.

[0126] Step S14: Input the fused word vector of each training text into the TD_LSTM network to respectively input the entity word encoding vector corresponding to the entity word in each training text, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word.

[0127] In this embodiment, the first preset number is set to 3, and the second preset number is set to 3; of course, in other embodiments, the specific values ​​of the first preset number and the second preset number can also be reasonably set as needed. This embodiment is only described schematically and is not limited to this.

[0128] Specifically, the LSTM_L network and the LSTM_R network will also output the encoding vector of each word. In this embodiment, the entity word encoding vector w_t corresponding to the entity word output by the LSTM network and the encoding vectors corresponding to the words in the three windows on the left and right of the entity word encoding vector are taken. The encoding vectors of the three words on the left are respectively recorded as w_t l-1 、w_t l-2 and w_t l-3 , the encoding vectors of the three words on the right are recorded as w_t r+1 、w_t r+2 and w_t r+3 .

[0129] Step S15: Take a weighted average of the entity word encoding vector, the encoding vectors of the first preset number of words on the left side of the entity word, and the encoding vectors of the second preset number of words on the right side of the entity word to obtain a new encoding vector corresponding to the entity word.

[0130] As an exemplary embodiment, the calculation formula of the new encoding vector corresponding to the entity word is as follows:

[0131]

[0132] Where s is the first preset number on the left side of the entity word, v is the second preset number on the right side of the entity word; w_t is the entity word encoding vector corresponding to the entity word in the training text, w_t l-1 is the encoding vector of the first word on the left side of the entity word in the training text, w_t l-2 is the encoding vector of the second word on the left side of the entity word in the training text, w_t l-s is the encoding vector of the sth word to the left of the entity word in the training text, w_t r+1is the encoding vector of the first word to the right of the entity word in the training text, w_t r+2 is the encoding vector of the second word to the right of the entity word in the training text, w_t r+v is the encoding vector of the vth word to the right of the entity word in the training text.

[0133] Specifically, the encoding vector of the entity word, the encoding vector of the three words to the left of the entity word, and the encoding vector of the three words to the right of the entity word are weighted averaged, and the contextual information in the three context windows is integrated into the encoding vector of the entity word to obtain a new encoding vector corresponding to the entity word.

[0134] Step S16: perform a similarity comparison between the new encoding vector corresponding to the entity word and the pre-coding vector corresponding to the entity word in the entity pre-coding matrix to obtain a similarity calculation value.

[0135] As an illustrative embodiment, the similarity comparison uses the cosine function to calculate the cosine similarity. The Cosine function calculates the size of the angle between two vectors. Compared with the distance metric, the Cosine function pays more attention to the difference in direction between the two vectors rather than the distance or length. The Cosine function is not affected by the indicator scale. The value of the Cosine function falls in the interval [-1,1]. The larger the value, the smaller the difference.

[0136] In this embodiment, the calculation formula of the similarity calculation value is as follows:

[0137] Threshold=cosine(w_new,w_d)

[0138] Among them, Threshold is the similarity calculation value, w_new is the new encoding vector corresponding to the entity word, and w_d is the pre-encoding vector corresponding to the entity word.

[0139] Step S17: Determine whether the calculated similarity value is greater than a preset similarity threshold. If the calculated similarity value is greater than the preset similarity threshold, execute step S18; if the calculated similarity value is less than or equal to the preset similarity threshold, execute step S19.

[0140] As an exemplary embodiment, the preset similarity threshold may be 0.7; of course, in other embodiments, the specific value of the preset similarity threshold may also be other values, such as 0.8 or 0.6. This embodiment is only used for illustrative purposes and is not intended to be limiting. In actual applications, it may be reasonably set as needed.

[0141] Step S18: If the calculated similarity value is greater than the preset similarity threshold, the physical precoding matrix is ​​not updated.

[0142] As an exemplary embodiment, when the calculated similarity value is greater than the preset similarity threshold, it indicates that the new encoding vector corresponding to the entity word in the short text and the precoding vector corresponding to the entity word in the entity precoding matrix have a high similarity, which means that the context information carried by the short text has been integrated into the entity encoding vector before, so no further operation is required, and the entity precoding matrix does not need to be updated.

[0143] Step S19: If the calculated similarity value is less than or equal to the preset similarity threshold, the new coding vector corresponding to the entity word and the precoding vector corresponding to the entity word in the entity precoding matrix are weighted averaged to obtain an updated precoding vector, and the updated precoding vector is updated to the entity precoding matrix.

[0144] As an exemplary embodiment, when the calculated similarity value is less than or equal to the preset similarity threshold, it indicates that the new encoding vector corresponding to the entity word in the short text and the precoding vector corresponding to the entity word in the entity precoding matrix have low similarity, which means that the context information carried by the short text is relatively new information, and the new context information needs to be integrated into the original entity encoding vector. Therefore, the new encoding vector corresponding to the entity word and the precoding vector corresponding to the entity word in the entity precoding matrix are weighted averaged to obtain an updated precoding vector, and the updated precoding vector is updated to the entity precoding matrix.

[0145] The calculation formula of the updated precoding vector is as follows:

[0146]

[0147] Among them, w_z is the updated pre-coding vector of the entity word; w_new is the new coding vector corresponding to the entity word, and w_d is the pre-coding vector corresponding to the entity word.

[0148] The new encoding vector w_new and the original precoding vector w_d are weighted averaged to obtain the final precoding vector w_z, which is then updated to the entity precoding matrix to replace the original w_d. This is to continuously incorporate new contextual information into the entity, update the entity precoding vector, increase the context coverage of the entity precoding vector, and thus improve the accuracy of entity relevance recognition.

[0149] Specific as Figure 2 As shown, Figure 2 The function of the Entity emddings module is to generate a new entity vector. The new entity vector is obtained by weighted average of itself and the word vectors within a preset number on the left and right sides. The new entity vector produced is represented by e2', which is the new encoding vector.

[0150] In the above steps, the entity word encoding vector output by the LSTM network and the encoding vectors of the preset number of words around the entity word are taken, and these encoding vectors are weighted averaged, and the context information of multiple words in the context is integrated into the entity vector encoding to obtain a new encoding vector. The new encoding vector and the original precoding vector in the entity precoding matrix are similarly calculated. If the similarity calculation value is greater than the preset similarity threshold, it means that the context information carried by the short text has been integrated into the entity precoding vector before, and no further operation is required; if the similarity calculation value is less than or equal to the similarity threshold, it means that the context information carried by the short text is relatively new information, and the new context information needs to be integrated into the original entity precoding vector, and the new precoding vector and the original precoding vector are weighted averaged to obtain the final entity precoding vector and update it to the entity precoding matrix to replace the original entity precoding vector. By continuously integrating new context information into the entity, updating the entity precoding vector, and improving the judgment of entity relevance.

[0151] As an exemplary embodiment, before the step S13 of inputting the short text to be recognized into the optimal model, steps S20-S22 are also included.

[0152] Step S20: Input the short text to be identified into the TextRCNN model for text binary classification to obtain the classification result.

[0153] As an exemplary embodiment, TextRCNN first uses a bidirectional RNN to obtain the contextual semantics and grammatical information of the input short text to be recognized, then uses maximum pooling to automatically filter out the most important features, and then connects a fully connected layer for classification to obtain the classification result.

[0154] TextRCNN combines the advantages of RNN and CNN, using a bidirectional loop structure to obtain contextual information, which can reduce noise better than traditional window-based neural networks and can preserve word order to a large extent when learning text expressions. Secondly, the maximum pooling layer is used to obtain the important part of the text and automatically determine which feature plays a more important role in the information denoising process.

[0155] Specifically, TextRCNN combines the left context, right context, and word itself as word representation, and uses a bidirectional RNN to extract context information of sentences. After the convolution layer, the representation of all words is obtained. First, the maximum pooling operation is performed on them. The maximum pooling can help find the most important latent semantic information in the sentence. Then the text representation is obtained through the fully connected layer, and finally the softmax layer is used for classification.

[0156] Specifically, the classification results include two results, namely, noise text and non-noise text, wherein the noise text is a short text that does not contain an entity, and the non-noise text is a short text that contains an entity.

[0157] Step S21: If the classification result of the short text to be recognized is a noise text, the text to be recognized is removed.

[0158] Specifically, since the present embodiment identifies the relevance of entities in short texts, only short texts containing entities need to be identified, and short texts not containing entities do not need to be identified. Therefore, when the classification result of the text to be identified is noise text, the text to be identified is removed, that is, the noise text is not input into the optimal model in the next step.

[0159] Step S22: If the classification result of the short text to be recognized is non-noise text, the text to be recognized is retained.

[0160] Specifically, when the classification result of the short text to be identified is a non-noise text, that is, the short text to be identified contains one or more entities, the text to be identified needs to be retained, and then the text to be identified is input into the optimal model for entity relevance identification.

[0161] Through the above steps, the text that does not contain entities in the short text to be identified is removed, a large amount of noise information is removed, and only the text containing entities is retained, thereby improving the accuracy of entity relevance recognition.

[0162] The technical solution in this embodiment is target based correlation analysis (TBCA), whose main function is to identify the relevance of entities appearing in a short text, that is, when one or more entities appear in the text, it can identify whether the entity is a primary entity or a secondary entity, that is, to determine whether the entity has a strong or weak correlation with the text (the primary entity is a strongly correlated relationship, and the secondary entity is a weakly correlated relationship).

[0163] There are three entities in a text, namely A1, A2 and A3. These three entities are strongly correlated with the text. The optimal model will output 1 for strong correlation and 0 for weak correlation. The final output results are A1-1, A2-1, A3-1.

[0164] There are four entities in another text, namely B1, B2, B3, and B4. Among them, B1 is strongly correlated with the text, and B2, B3, and B4 are weakly correlated with the text. The final output results are B1-1, B2-0, B3-0, and B4-0.

[0165] Based on the network framework of deep learning, this method adopts a fusion scheme to identify the relevance of entities in short texts from the perspective of multi-dimensional information carrying. This method takes into account that entities (such as institutional entities) themselves also carry a large amount of information, so when designing the deep learning network framework, an entity precoding matrix is ​​added. During training, the entity information precoding is completed by continuously integrating new context information into the entity vector. During prediction, the word features plus the position features are encoded into the entity vector in the short text through the deep learning network, and then the entity precoding vector with context information is obtained from the entity information precoding matrix, and enters the forward feedback neural network together to calculate the relevance of the entity in the short text. The advantage of this method is that it utilizes the powerful encoding ability of the deep learning model and incorporates rich semantic and grammatical context information. Not only lexical information and syntactic information are designed, but also the context environment of the entity is designed. From multiple information dimensions, the entity feature expression is enriched to improve the confidence of the calculation of the relevance between the entity and the text. Not only the relevance of the entity and the context is considered from the local context, but also the relevance of the context information carried by the entity itself and the context is considered from the large context environment.

[0166] This method is very meaningful for entity monitoring and information retrieval in the field of business information. When one or more entities appear in a text, it can tell the information acquirer which entities are the main entities described by the text and which entities are the secondary entities described by the text, so that the most relevant information can be returned to the information acquirer; this method greatly improves the work efficiency of public opinion analysts.

[0167] In this embodiment, a short text entity relevance recognition system is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0168] This embodiment also provides a short text entity relevance recognition system, such as Figure 3 As shown, including:

[0169] A first acquisition module 1 is used to acquire a short text training set, wherein the short text training set includes a plurality of training texts and a named entity corresponding to each training text;

[0170] A second acquisition module 2 is used to acquire an entity precoding matrix, where the entity precoding matrix includes precoding vectors corresponding to all entities, wherein the entity precoding is obtained based on context information of the entity precoding;

[0171] The first processing module 3 is used to convert each word in each training text into a word vector, and convert the position of each word in the training text into a position vector;

[0172] The second processing module 4 is used to obtain a precoding vector corresponding to each word according to each word and the entity precoding matrix in each training text, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is a zero vector;

[0173] The third processing module 5 is used to obtain a fused word vector corresponding to each word in each training text according to the word vector, position vector and precoding vector corresponding to each word in each training text;

[0174] The fourth processing module 6 is used to input the fused word vector of each training text into the TD_LSTM network to obtain the forward vector code and the backward vector code corresponding to each training text;

[0175] The fifth processing module 7 is used to respectively concatenate the forward vector code and the backward vector code corresponding to each training text to obtain an entity concatenation vector corresponding to each training text;

[0176] The sixth processing module 8 is used to pass the entity concatenation vector corresponding to each training text through a feedforward neural network to obtain a classification coding vector corresponding to each training text;

[0177] The seventh processing module 9 is used to normalize the classification code vector to obtain a normalized classification code vector;

[0178] An eighth processing module 10, configured to obtain a loss function between the model-encoded classification vector and the actual entity classification vector according to the normalized classification coding vector;

[0179] A ninth processing module 11 is used for performing iterative optimization training according to the loss function to obtain an optimal model, wherein the optimal model is used to identify the correlation strength between the short text and the entity in the short text;

[0180] The third acquisition module 12 is used to acquire the short text to be recognized;

[0181] The tenth processing module 13 is used to input the short text to be recognized into the optimal model to obtain the recognition result of the text to be recognized.

[0182] As an exemplary embodiment, the calculation formula of the fused word vector is as follows:

[0183] w_a=w_ta+w_pa+w_da

[0184] Among them, w_a is the fused word vector corresponding to word a; w_ta is the word vector corresponding to word a; w_pa is the position vector corresponding to word a, and w_da is the precoding vector corresponding to word a.

[0185] As an exemplary embodiment, it also includes: an eleventh processing module, which is used to input the fused word vector of each training text into the TD_LSTM network, respectively, the entity word encoding vector corresponding to each entity word in the training text, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word; a twelfth processing module, which is used to perform weighted averaging on the entity word encoding vector, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word to obtain a new encoding vector corresponding to the entity word; a thirteenth processing module, which is used to input the new encoding vector corresponding to the entity word into the TD_LSTM network, respectively. The method comprises the following steps: performing a similarity comparison between the precoding vector corresponding to the entity word in the entity precoding matrix and the entity word in the entity precoding matrix to obtain a similarity calculation value; a judgment module for judging whether the similarity calculation value is greater than a preset similarity threshold; a fourteenth processing module for not updating the entity precoding matrix if the similarity calculation value is greater than the preset similarity threshold; and a fifteenth processing module for performing a weighted average of the new coding vector corresponding to the entity word and the precoding vector corresponding to the entity word in the entity precoding matrix if the similarity calculation value is less than or equal to the preset similarity threshold to obtain an updated precoding vector, and updating the updated precoding vector to the entity precoding matrix.

[0186] As an exemplary embodiment, the calculation formula of the similarity calculation value is as follows:

[0187] Threshold=cosine(w_new,w_d)

[0188] Among them, Threshold is the similarity calculation value, w_new is the new encoding vector corresponding to the entity word, and w_d is the pre-encoding vector corresponding to the entity word;

[0189] The calculation formula for the new encoding vector corresponding to the entity word is as follows:

[0190]

[0191] Where s is the first preset number on the left side of the entity word, v is the second preset number on the right side of the entity word; w_t is the entity word encoding vector corresponding to the entity word in the training text, w_t l-1 is the encoding vector of the first word on the left side of the entity word in the training text, w_t l-2 is the encoding vector of the second word on the left side of the entity word in the training text, w_t l-sis the encoding vector of the sth word to the left of the entity word in the training text, w_t r+1 is the encoding vector of the first word to the right of the entity word in the training text, w_t r+2 is the encoding vector of the second word to the right of the entity word in the training text, w_t r+v is the encoding vector of the vth word to the right of the entity word in the training text.

[0192] As an exemplary embodiment, the calculation formula of the updated precoding vector is as follows:

[0193]

[0194] Among them, w_z is the updated pre-coding vector of the entity word; w_new is the new coding vector corresponding to the entity word, and w_d is the pre-coding vector corresponding to the entity word.

[0195] As an exemplary embodiment, it also includes: a sixteenth processing module, which is used to input the short text to be recognized into the TextRCNN model for text binary classification to obtain a classification result; a seventeenth processing module, which is used to remove the text to be recognized if the classification result of the short text to be recognized is a noise text; and an eighteenth processing module, which is used to retain the text to be recognized if the classification result of the short text to be recognized is a non-noise text.

[0196] As an exemplary embodiment, the calculation formula of the loss function is as follows:

[0197]

[0198] Where n is the number of training samples, y i is the actual classification vector of the i-th training sample, a i is the classification vector of the i-th training sample encoded by the model.

[0199] The short text entity relevance recognition system in this embodiment is presented in the form of functional units, where the units refer to ASIC circuits, processors and memories that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0200] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0201] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, the electronic device includes one or more processors 71 and a memory 72. Figure 4 A processor 71 is taken as an example.

[0202] The controller may further include: an input device 73 and an output device 74 .

[0203] The processor 71, the memory 72, the input device 73 and the output device 74 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0204] The processor 71 may be a central processing unit (CPU). The processor 71 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. A general-purpose processor may be a microprocessor or any conventional processor.

[0205] The memory 72 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the short text entity relevance identification method in the embodiment of the present application. The processor 71 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 72, that is, the short text entity relevance identification method of the above method embodiment is implemented.

[0206] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of a processing device operated by the server, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely arranged relative to the processor 71, and these remote memories may be connected to a network connection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0207] The input device 73 can receive input digital or character information, and generate key signal input related to user settings and function control of the processing device of the server. The output device 74 can include display devices such as a display screen.

[0208] One or more modules are stored in the memory 72, and when executed by one or more processors 71, the execution is as follows: Figure 1-2 The method shown.

[0209] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the executed program can be stored in a computer-readable storage medium. When the program is executed, it can include the process of the embodiment of the above-mentioned short text entity relevance identification method. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0210] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A short text entity relevance identification method, It is characterized in that include: Acquire a short text training set, wherein the short text training set includes a plurality of training texts and a named entity corresponding to each training text; Acquire an entity precoding matrix, where the entity precoding matrix includes precoding vectors corresponding to all entities, wherein the entity precoding is obtained according to context information of entity precoding; Convert each word in each training text into a word vector, and convert the position of each word in the training text into a position vector; According to each word and entity precoding matrix in each training text, a precoding vector corresponding to each word is obtained, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is a zero vector; According to the word vector, position vector and precoding vector corresponding to each word in each training text, the fused word vector corresponding to each word in each training text is obtained; Input the fused word vector of each training text into the TD_LSTM network to obtain the forward vector encoding and backward vector encoding corresponding to each training text; Input the fused word vector of each training text into the entity word encoding vector corresponding to each entity word in the training text in the TD_LSTM network, the encoding vector of the first preset number of words on the left side of the entity word, and the encoding vector of the second preset number of words on the right side of the entity word; Taking a weighted average of the entity word encoding vector, the encoding vectors of the first preset number of words on the left side of the entity word, and the encoding vectors of the second preset number of words on the right side of the entity word, to obtain a new encoding vector corresponding to the entity word; Compare the new encoding vector corresponding to the entity word with the pre-coding vector corresponding to the entity word in the entity pre-coding matrix to obtain a similarity calculation value; Determine whether the calculated similarity value is greater than a preset similarity threshold; If the calculated similarity value is greater than the preset similarity threshold, the entity precoding matrix is ​​not updated; If the calculated similarity value is less than or equal to the preset similarity threshold, the new encoding vector corresponding to the entity word and the precoding vector corresponding to the entity word in the entity precoding matrix are weighted averaged to obtain an updated precoding vector, and the updated precoding vector is updated to the entity precoding matrix; The forward vector code and the backward vector code corresponding to each training text are concatenated to obtain the entity concatenation vector corresponding to each training text; The entity concatenation vector corresponding to each training text is passed through a feedforward neural network to obtain a classification encoding vector corresponding to each training text; Normalizing the classification coding vector to obtain a normalized classification coding vector; According to the normalized classification encoding vector, the loss function between the classification vector encoded by the model and the actual entity classification vector is obtained; Perform iterative optimization training according to the loss function to obtain an optimal model, wherein the optimal model is used to identify the correlation strength between short texts and entities in the short texts; Get the short text to be recognized; The short text to be recognized is input into the optimal model to obtain a recognition result of the text to be recognized.

2. The short text entity relevance identification method according to claim 1, It is characterized in that The calculation formula for the fused word vector is as follows: w_a = w_ta + w_pa + w_da Among them, w_a is the fused word vector corresponding to word a; w_ta is the word vector corresponding to word a; w_pa is the position vector corresponding to word a, and w_da is the precoding vector corresponding to word a.

3. The short text entity relevance identification method according to claim 1, It is characterized in that The calculation formula for the similarity calculation value is as follows: Threshold = cosine (w_new, w_d) Among them, Threshold is the similarity calculation value, w_new is the new encoding vector corresponding to the entity word, and w_d is the pre-encoding vector corresponding to the entity word; The calculation formula for the new encoding vector corresponding to the entity word is as follows: ; Among them, s is the first preset number on the left side of the entity word, and v is the second preset number on the right side of the entity word; w_t is the entity word encoding vector corresponding to the entity word in the training text, w_tl-1 is the encoding vector of the first word on the left side of the entity word in the training text, w_tl-2 is the encoding vector of the second word on the left side of the entity word in the training text, w_tl-s is the encoding vector of the s-th word on the left side of the entity word in the training text, w_tr+1 is the encoding vector of the first word on the right side of the entity word in the training text, w_tr+2 is the encoding vector of the second word on the right side of the entity word in the training text, and w_tr+v is the encoding vector of the v-th word on the right side of the entity word in the training text.

4. The short text entity relevance identification method according to claim 1, It is characterized in that The calculation formula of the updated precoding vector is as follows: w_z = (w_new + w_d) / 2 Among them, w_z is the updated pre-coding vector of the entity word; w_new is the new coding vector corresponding to the entity word, and w_d is the pre-coding vector corresponding to the entity word.

5. The short text entity relevance identification method according to claim 1, It is characterized in that Before the step of inputting the short text to be recognized into the optimal model, the method further includes: Input the short text to be identified into the TextRCNN model for text binary classification and obtain the classification result; If the classification result of the short text to be recognized is a noise text, the text to be recognized is removed; If the classification result of the short text to be recognized is a non-noise text, the text to be recognized is retained.

6. The short text entity relevance identification method according to any one of claims 1 to 5, It is characterized in that The loss function is calculated as follows: ; Among them, n is the number of training samples, yi is the actual classification vector of the i-th training sample, and ai is the classification vector of the i-th training sample encoded by the model.

7. A short text entity relevance identification system, used to implement the short text entity relevance identification method according to any one of claims 1 to 6, It is characterized in that include: A first acquisition module is used to acquire a short text training set, wherein the short text training set includes a plurality of training texts and a named entity corresponding to each training text; A second acquisition module is used to acquire an entity precoding matrix, where the entity precoding matrix includes precoding vectors corresponding to all entities, wherein the entity precoding is obtained according to context information of the entity precoding; A first processing module, for converting each word in each training text into a word vector, and converting the position of each word in the training text into a position vector; A second processing module is used to obtain a precoding vector corresponding to each word according to each word and the entity precoding matrix in each training text, wherein the precoding vector of the word corresponding to the named entity in each training text is the entity precoding vector, and the precoding vector of the word corresponding to the non-named entity in each training text is a zero vector; The third processing module is used to obtain a fused word vector corresponding to each word in each training text according to the word vector, position vector and precoding vector corresponding to each word in each training text; The fourth processing module is used to input the fused word vector of each training text into the TD_LSTM network to obtain the forward vector code and the backward vector code corresponding to each training text; A fifth processing module is used to concatenate the forward vector code and the backward vector code corresponding to each training text to obtain an entity concatenation vector corresponding to each training text; A sixth processing module, used for passing the entity concatenation vector corresponding to each training text through a feedforward neural network to obtain a classification coding vector corresponding to each training text; A seventh processing module, used for normalizing the classification coding vector to obtain a normalized classification coding vector; An eighth processing module, used to obtain a loss function between the model-encoded classification vector and the actual entity classification vector according to the normalized classification coding vector; A ninth processing module, used for performing iterative optimization training according to the loss function to obtain an optimal model, wherein the optimal model is used for identifying the correlation strength between the short text and the entity in the short text; The third acquisition module is used to acquire the short text to be recognized; The tenth processing module is used to input the short text to be recognized into the optimal model to obtain the recognition result of the text to be recognized.

8. An electronic device, It is characterized in that include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the short text entity relevance identification method described in any one of claims 1-6.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the short text entity relevance identification method described in any one of claims 1-6.