Data processing method, device, computer equipment and storage medium
By obtaining the target entity words and related entity words in the target text and using the neural network model for word vector generation and semantic encoding, the problem of in-depth mining of entity word related information in the existing technology is solved, and efficient prediction of user identity is achieved.
Patent Information
- Application Number
- CN202110793816.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-13
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2041-07-13
AI Technical Summary
Existing natural language processing technologies have difficulty in deeply mining other relevant information described by entity words, especially in effectively identifying and predicting user identity features.
By obtaining the target entity words and related entity words in the target text, using the neural network model for word vector generation and semantic encoding, combined with the identity discrimination network, the user identity of the target entity word is predicted.
It achieves label prediction for incoherent semantic entity words, improving the efficiency and accuracy of computer devices in identifying user identities.
Smart Images

Figure CN115618873B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment, and storage medium. Background Art
[0002] With the continuous in-depth development of Internet technology, natural language processing technology has also been continuously developing. With the assistance of natural language processing technology, the current text processing efficiency can be effectively improved. The natural language technologies that are currently used more frequently include Named Entity Recognition (NER). NER technology can be used to recognize proper nouns such as names of people, places, and institutions in the input text. In other words, NER technology can only realize the discrimination of the entity word type in the input text, and cannot further deeply mine other related information described by the entity word. Therefore, how to mine other related information described by the entity word has become a current research hotspot. Summary of the Invention
[0003] Embodiments of the present invention provide a data processing method, apparatus, computer device, and storage medium, which can realize vector representation of entity words and mine and predict user identities corresponding to entity words that serve as user names.
[0004] In one aspect, an embodiment of the present invention provides a data processing method, comprising:
[0005] Obtaining a target text, and obtaining a target entity word from the target text, wherein the target entity word is a user name of a target user;
[0006] Determining associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user;
[0007] Performing recognition processing on the target entity word to obtain a first word vector of the target entity word, and performing recognition processing on the associated entity word to obtain a second word vector of the associated entity word;
[0008] In combination with the first word vector and the second word vector, label prediction processing is performed on the target entity word to obtain a predicted label of the target entity word, and the predicted label is used to indicate whether the user identity of the target user is the target identity.
[0009] In another aspect, an embodiment of the present invention provides a data processing device, comprising:
[0010] An acquisition unit, configured to acquire a target text and acquire a target entity word from the target text, wherein the target entity word is a user name of a target user;
[0011] a determining unit, configured to determine associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user;
[0012] a processing unit, configured to perform recognition processing on the target entity word to obtain a first word vector of the target entity word, and perform recognition processing on the associated entity word to obtain a second word vector of the associated entity word;
[0013] The processing unit is also used to combine the first word vector and the second word vector to perform label prediction processing on the target entity word to obtain a predicted label of the target entity word, and the predicted label is used to indicate whether the user identity of the target user is the target identity.
[0014] In yet another aspect, an embodiment of the present invention provides a computer device, comprising a processor, an input device, an output device, and a memory, wherein the processor, input device, output device, and memory are interconnected, wherein the memory is used to store a computer program that supports the computer device in executing the above method, wherein the computer program includes program instructions, and the processor is configured to call the program instructions and execute the following steps:
[0015] Obtaining a target text, and obtaining a target entity word from the target text, wherein the target entity word is a user name of a target user;
[0016] Determining associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user;
[0017] Performing recognition processing on the target entity word to obtain a first word vector of the target entity word, and performing recognition processing on the associated entity word to obtain a second word vector of the associated entity word;
[0018] In combination with the first word vector and the second word vector, label prediction processing is performed on the target entity word to obtain a predicted label of the target entity word, and the predicted label is used to indicate whether the user identity of the target user is the target identity.
[0019] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are executed by a processor, the program instructions are used to execute the data processing method described in the first aspect.
[0020] In an embodiment of the present invention, after obtaining the target text, the computer device can obtain the username of the target user from the target text, and use the obtained username as the target entity word. After the computer device determines the target entity word, it can also determine the associated entity words of the target entity word from the target text. Then, based on the acquisition of the target entity word and the associated entity words in the target text, the computer device can obtain entity words with incoherent semantics. After obtaining the target entity word and the associated entity word, the computer device can respectively identify the target entity word to obtain the first word vector of the target entity word, and identify the associated entity word to obtain the second word vector of the associated entity word. Then, further, based on the obtained first word vector and second word vector, the computer device can combine the first word vector and the second word vector to predict the predicted label of the target entity word, so as to determine whether the target user indicated by the target entity word is the target identity according to the predicted label, so that the computer device can determine the label corresponding to the target entity word (that is, the refined entity type corresponding to the target entity word) based on the entity word with incoherent semantics. The method of using entity words with incoherent semantics to predict the label of the target entity word can effectively improve the convenience of the computer device in predicting the entity word label, thereby improving the efficiency of the computer device in predicting the entity word label. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0022] Figure 1a is a schematic structural diagram of a target network model provided by an embodiment of the present invention;
[0023] Figure 1b is a schematic structural diagram of a target network model provided by an embodiment of the present invention;
[0024] Figure 2 is a schematic flow chart of a data processing method provided by an embodiment of the present invention;
[0025] Figure 3 is a schematic flow chart of a data processing method provided by an embodiment of the present invention;
[0026] Figure 4 Schematic diagram of optimizing and training a target network model according to an embodiment of the present invention;
[0027] Figure 5a is a schematic diagram of performing an intent search based on a search word list provided by an embodiment of the present invention;
[0028] Figure 5b This is a schematic diagram of a scenario for performing intent search provided by an embodiment of the present invention;
[0029] Figure 6 is a schematic block diagram of a data processing device provided by an embodiment of the present invention;
[0030] Figure 7 This is a schematic block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0031] An embodiment of the present invention proposes a data processing method, so that a computer device can use the obtained username of a target user as a target entity word, and use the entity word that has a contextual relationship with the username of the target user as an associated entity word of the target entity word. After obtaining the target entity word and the associated entity word, the computer device can further perform recognition processing on the target entity word and the associated entity word respectively, and then obtain a first word vector corresponding to the target entity word, and a second word vector corresponding to the associated entity word. Then, based on the prediction processing of the obtained first word vector and the second word vector, the computer device can obtain a predicted label for the target entity word, and then determine whether the user identity of the target user is the target identity based on the indication of the predicted label, so that the computer device can realize label prediction for the target entity word based on non-semantically coherent entity words, and then determine whether the user identity of the target user is the target identity based on the obtained predicted label. In one embodiment, the context includes the preceding and following contexts. Then, the entity word that has a contextual relationship with a certain entity word (such as the target entity word mentioned above) includes the preceding and following contexts of the text where the entity word is located (i.e., the target text). The preceding (or following context) refers to the entity word in the target text that is located before (or after) the target entity word, and the entity word refers to a word that has a specific meaning or refers to a real object. Then it can be understood that the associated entity words that have an association relationship with the target entity word include: in the text where the target entity word is located, in addition to the target entity word, and pronouns, stop words, auxiliary words, etc., other words that have no specific meaning or cannot refer to real objects. For example, if the target text is "Xiao A explains wonderful game videos", then when the target entity word is "Xiao A", the associated entity words determined based on the contextual relationship include "explanation, game video", but do not include auxiliary words such as "wonderful" that have no specific reference. In addition, non-semantically coherent entity words are distinguished from semantically coherent entity words, where semantically coherent entity words are all words composed of segmented words obtained from the target text, while non-semantically coherent entity words refer to some words obtained from the segmented words of the target text.
[0032] In one embodiment, the computer device can call the target network model to identify the target entity word and the associated entity word respectively, thereby obtaining a first word vector of the target entity word and a second word vector of the associated entity word, wherein the target network model is a neural network for discriminating the specific user identity of the target user referred to by the target entity word, that is, the target network model can be used to discriminate whether the target user is a specific user identity, wherein the specific user identity can be, for example, a game commentator identity, or a live broadcaster identity, or a host identity, etc. In the embodiment of the present invention, the specific user identity is mainly described in detail by taking the game commentator identity as an example. When the specific user identity is other identities, please refer to the embodiment of the present invention. Among them, the model structure of the target network model may specifically include a word vector generation network, a semantic encoding network and an identity discrimination network, wherein the word vector generation network is used to convert each character (or word) in the above-mentioned entity words (including the above-mentioned target entity words, and the associated entity words of the target entity words) into a vector representation of a fixed length; and the semantic encoding network can be used to perform semantic analysis on the entity words, so as to obtain the vector representation corresponding to each character (or word) in the associated entity words, and combine the semantic analysis results of the entity words to obtain the vector representation of the corresponding entity words. The identity discrimination network can also be called an identity discrimination module, and the identity discrimination module can discriminate the user identity of the target user indicated by the target entity word based on the vector representation of the entity word (including the vector representation of the above-mentioned target entity word, and the vector identification of the associated entity word). In one embodiment, the model structure of the target network model can be as follows Figure 1a As shown, the target network model can perform semantic analysis on the target entity word and the associated entity word when obtaining the first word vector of the target entity word and the second word vector of the associated entity word, so that the first word vector of the target entity word and the second word vector of the associated entity word both contain the entity word semantics; In addition, the model structure of the target network model can also be as follows Figure 1b As shown, the target network model can be used only to perform semantic analysis on associated entity words, so that the second word vector representation corresponding to the associated entity word introduces the entity word semantics, without performing semantic analysis on the target entity word, that is, the first word vector of the target entity word is obtained by splicing the vector representation corresponding to each character in the target entity word.
[0033] After obtaining the first word vector corresponding to the target entity word and the second word vector of the associated entity word, the identity discrimination network (i.e., the identity discrimination module) can perform label prediction processing on the target entity word based on the first word vector and the second word vector, and then obtain the predicted label of the target entity word, so that the computer device can obtain the identity prediction result of the target user based on the obtained predicted label for the target entity word. In one embodiment, the word vector generation network and the semantic encoding network included in the target network model can be multiple, that is, different entity words can use different word vector generation networks to generate vector representations corresponding to the characters in the corresponding entity words, and different semantic encoding networks can be used to perform semantic analysis on the entity words to introduce the semantics of the entity words into the vector representation of the entity words, or, the word vector generation network and the semantic encoding network included in the neural network model can also be only one. Then, after the target entity word and the associated entity word are input into the target network model, the neural network model will reuse the word vector generation network and the semantic encoding network to obtain the vector representation of the target entity word and the vector representation of the associated entity word. In addition, the semantic encoding network can also introduce an attention mechanism when performing semantic recognition on entity words, wherein the word vector generation network can be implemented using an algorithm based on word embedding vectors, or word2vec (a generation algorithm for character vectors in entity words), and the semantic encoding network can be implemented using a long short-term memory network (Long Short-Term Memory, LSTM), or other semantic encoding networks, such as Bi-LSTM (a bidirectional long short-term memory network). That is to say, in the embodiment of the present invention, the specific implementation method of each neural network in the neural network model is not limited.
[0034] In one embodiment, the target text obtained by the computer device can be any training text in the training text set, and when the target text obtained by the computer device is a training text, the computer device can call the target network model to obtain the predicted label of the target entity word, and then optimize the target network model based on the real label corresponding to the target entity word, and then obtain a trained target network model. After the computer device obtains the target network model, it can use the obtained target network model to predict the label of the target entity word in other texts (i.e., texts without real labels), thereby realizing the mining prediction of the user identity of the user corresponding to the target entity word of the unknown label based on the prediction result of the label corresponding to the target entity word, thereby improving the recognition and recall of whether the user identity of the user corresponding to the user name in the text is the target identity, and thereby improving the accuracy of the computer device in judging whether the user identity of the user corresponding to the user name in the text is the target identity. It can also be understood that when the real label of the target entity word is unknown, the computer device can use the predicted label predicted by the target network model to judge whether the target user is the user identity based on the obtained predicted label, thereby realizing the mining prediction of the user identity corresponding to the target entity word in other texts.
[0035] See Figure 2 , is a schematic flow chart of a data processing method proposed in an embodiment of the present invention, and the data processing method can be specifically executed by the above-mentioned computer device, such as Figure 2 As shown, the method may include:
[0036] S201, obtaining a target text, and obtaining a target entity word from the target text, where the target entity word is the username of a target user.
[0037] S202 , determining associated entity words of the target entity word from the target text, where the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user.
[0038] In step S201 and step S202, the target text is a text that includes entity words of the username type, and the entity words of the username type included in the target text are the usernames of the target users. Then, after the computer device obtains the target text, it can use the entity words of the username type corresponding to the entity words in the target text as the target entity words. In a specific implementation, after obtaining the target text, the computer device may first determine the text segmentation corresponding to the target text. When determining the text segmentation of the target text, the computer device may first call the word segmentation service and the entity word recognition service to obtain the word segmentation result (i.e., the word segmentation set) of the target text, as well as the entity word recognition result. In one embodiment, the computer device may use a natural language processing algorithm (NLP) to implement word segmentation processing on the target text to obtain the word segmentation set corresponding to the target text. When the computer device performs entity word recognition on the target text, it may use a named entity recognition (NER) algorithm to perform entity word recognition processing on the target text to obtain the entity word recognition result corresponding to the target entity word. NLP is an important direction in the fields of computer science and artificial intelligence. Natural language processing (NLP) studies the theories and methods that enable effective communication between humans and computers using natural language. Natural language processing (NLP) integrates linguistics, computer science, and mathematics. Therefore, research in this field involves natural language—the language we use everyday—and is closely related to linguistics. NLP technologies typically include text processing, semantic understanding, machine translation, robotic question-answering, and knowledge graphs.
[0039] In one embodiment, if the target text is "Xiao Xi explains the highlights of the game video Honor of Kings", then after the computer device performs word segmentation processing on the target text, the obtained word segmentation set can be "Xiao Xi / Explanation / Game / Video / Honor of Kings / Of / Highlights", and the entity word recognition result obtained after performing entity word recognition processing on the target text can be "Xiao Xi_Username / Honor of Kings_IP-Game Name". Then, after the computer device obtains the word segmentation set and entity word recognition result of the target text, the computer device can also filter the word segmentation set so that the word segmentation set only includes those word segmentations that do not constitute entity words, and filters out label symbols and stop words, etc. Then, based on the word segmentation set of the above target text, the word segmentation set after filtering processing can be "Xiao Xi / Explanation / Game / Video / Honor of Kings / Highlights". The word segmentation set after filtering is equivalent to the original word segmentation set, with the character "Of" deleted.
[0040] After the computer device obtains the filtered word segmentation set, the filtered word segmentation set and the entity word recognition result can be merged to obtain the text segmentation corresponding to the target text, wherein, in the text segmentation, the segmentation and entity words will not be distinguished, and in the embodiment of the present invention, the words in the segmentation are collectively referred to as entity words. Based on the filtered word segmentation set "Xiao Xi / Explanation / Game / Video / King of Glory / Highlights" obtained by the word segmentation processing of the above-mentioned target text, and the entity word recognition result "Xiao Xi_Username / King of Glory_IP-Game Name" obtained by the entity word recognition processing, the computer device can finally obtain the text segmentation of the target text, namely "Xiao Xi_Username / King of Glory_IP-Game Name / Explanation / Game / Video / Highlights". Among them, IP refers to a proper name category returned after calling the NER service. IP includes game names, film and television drama names, book names, music names, competition names, etc. Among them, when the NER service is called, it can correctly identify the IP name, or it can further identify the IP's sub-category information when identifying the IP name. For example, when the input text is "Xiao Xi explains the game video Honor of Kings", calling the NER service can identify Honor of Kings as an IP category, and its sub-category is the game category. Then, after the computer device determines the text segment corresponding to the target text, since the entity words of the user name type will be marked in the text segment, the computer device can select the entity words marked as the user name type (such as the above-mentioned Xiao Xi) as the target entity words based on the text segment. Then it can be understood that the target entity word can be used to refer to the target user named Xiao Xi.
[0041] After obtaining the text segment corresponding to the target text, the computer device can also determine the associated entity words of the target entity word based on the text segment corresponding to the target text, wherein the associated entity words can be any one or more other entity words in the text segment corresponding to the target text except the target entity word. Similarly, if the text segment of the target text is "Xiao Xi_Username / King of Glory_IP-Game Name / Explanation / Game / Video / Highlights", after the computer device obtains the target entity word "Xiao Xi" from the target text, it will use any one or more other entity words (such as King of Glory, or explanation, etc.) in the text segment of the target text except "Xiao Xi" as the associated entity words of the target entity word. It can be understood that the associated entity words are some descriptive words related to the identity characteristics of the target user, or the associated entity words can also be any other entity words that have no specific meaning and only have a contextual relationship with the target entity word. In an embodiment of the present invention, the computer device may also use other methods to obtain the target entity words in the target text, such as only performing NER processing on the target text, and then based on the processing results, taking the entity words in the target text marked as the user name category as the target entity words, and then selecting any other words from the target text except the target entity words as associated entity words. That is to say, in an embodiment of the present invention, the method by which the computer device obtains the target entity words of the target text and the corresponding associated entity words is not limited.
[0042] After the computer device obtains the target entity word and the associated entity word of the target entity word from the target text, the computer device can call the target network model to identify the target entity word and the associated entity word, and then obtain the first word vector corresponding to the target entity word and the second word vector corresponding to the associated entity word, that is, execute step S203.
[0043] S203: Perform recognition processing on the target entity word to obtain a first word vector of the target entity word, and perform recognition processing on the associated entity word to obtain a second word vector corresponding to the associated entity word.
[0044] S204: Combine the first word vector and the second word vector to perform label prediction processing on the target entity word to obtain a predicted label of the target entity word. The predicted label is used to indicate whether the user identity of the target user is the target identity.
[0045] In step S203 and step S204, when the computer device identifies and processes the target entity word and the associated entity word and obtains the first word vector of the target entity word and the second word vector of the associated entity word respectively, the computer device can process the target entity word and the associated entity word respectively to obtain the vector representation of each character in the target entity word and the vector representation of each character in the associated entity word. In a specific implementation, the computer device can obtain a vector representation of each character by calling a target network model, wherein when the target network model is called to generate a vector representation of each character, the target network model will be based on the word vector generation network in the target network model. After the computer device calls the word vector generation network to obtain the vector representation of each character in the target entity word and the vector representation of each character in the associated entity word, the vector representation of each character in the target entity word can be directly spliced to obtain the first word vector of the target entity word. After obtaining the vector of each character in the associated entity word, the computer device can further call the semantic coding network to semantically encode the associated entity word, and obtain the second word vector of the associated entity word based on the semantic coding and the vector of each character in the associated entity word. It can be understood that the semantic coding processing of the associated entity word based on the semantic coding network can realize the introduction of the semantics of the associated entity word into the corresponding second word vector.
[0046] After the computer device obtains the first word vector corresponding to the target entity word and the second word vector corresponding to the associated entity word, the first word vector and the second word vector can be input into the output layer of the target network model. The output layer is a softmax (a logistic regression algorithm) layer including an identity discrimination network. It can be understood that after the output layer obtains the first word vector and the second word vector, it can perform label prediction processing on the target entity word to obtain the predicted label of the target entity word. In a specific implementation, when the output layer performs label prediction processing on the target entity word based on the first word vector and the second word vector, the output layer first generates two candidate labels for the target entity word and the confidence corresponding to each candidate label, wherein the two generated candidate labels include a first candidate label for indicating that the target user is the target identity, and a second candidate label for indicating that the target user is not the target identity, then the computer device can determine the predicted label corresponding to the target entity word based on the generated candidate labels and the corresponding confidence. In one embodiment, if the confidence corresponding to the first candidate label included in the generated candidate labels is greater than a preset threshold, and the confidence corresponding to the second candidate label included in the generated candidate labels is less than the preset threshold, then the computer device can determine that the predicted label of the target entity word is the first candidate label, that is, it can determine that the target user identity is the target identity. Alternatively, in another implementation, if the confidence of the first candidate label included in the generated candidate labels is a, and the confidence of the second candidate label included in the generated candidate labels is b, and a>b, then the computer device can also determine that the predicted label of the target entity word is the first candidate label. In an embodiment of the present invention, there is no limitation on how the computer device determines the predicted label of the target entity word based on the generated candidate labels and the corresponding confidence levels.
[0047] In one embodiment, the target network model can be a trained network model. Then, based on the trained network model, it is possible to predict the label of the username based on the username and the corresponding context, thereby determining the user identity of the user corresponding to the username, thereby realizing the mining of the user identity corresponding to the unknown username, and improving the data expansion of the user corresponding to the username under the target user identity. When the target network model is an untrained network model, the computer device can also try to determine the true label of the target entity word after obtaining the predicted label of the target entity word, so as to optimize the training of the target network model based on the label difference between the predicted label and the true label to obtain the trained target network model, so as to use the trained target network model to predict the label of the target entity word with an unknown true label, thereby enhancing the recall of the user corresponding to the target user with the target identity.
[0048] In an embodiment of the present invention, after obtaining the target text, the computer device can obtain the username of the target user from the target text, and use the obtained username as the target entity word. After the computer device determines the target entity word, it can also determine the associated entity words of the target entity word from the target text. Then, based on the acquisition of the target entity word and the associated entity words in the target text, the computer device can obtain entity words with incoherent semantics. After obtaining the target entity word and the associated entity word, the computer device can respectively identify the target entity word to obtain the first word vector of the target entity word, and identify the associated entity word to obtain the second word vector of the associated entity word. Then, further, based on the obtained first word vector and second word vector, the computer device can combine the first word vector and the second word vector to predict the predicted label of the target entity word, so as to determine whether the target user indicated by the target entity word is the target identity according to the predicted label, so that the computer device can determine the label corresponding to the target entity word (that is, the refined entity type corresponding to the target entity word) based on the entity word with incoherent semantics. The method of using entity words with incoherent semantics to predict the label of the target entity word can effectively improve the convenience of the computer device in predicting the entity word label, thereby improving the efficiency of the computer device in predicting the entity word label.
[0049] See Figure 3 , is a schematic flow chart of a data processing method proposed in an embodiment of the present invention. The method can also be executed by the above-mentioned computer device. The embodiment of the present invention mainly elaborates on the training process of the target network model, such as Figure 3 As shown, the method may include:
[0050] S301, obtaining a target text, and obtaining a target entity word from the target text, where the target entity word is the username of the target user.
[0051] S302, determining associated entity words of the target entity word from the target text, where the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user.
[0052] In one embodiment, the target text can be any training text selected from a training text set, and when the target text is a training text in the training text set, model training of the target network model can be implemented, wherein each training text included in the training text set includes a target entity word, that is, when the computer device obtains the target entity word from the target text, it can first obtain the training text set, and then obtain the user name of the target user from the training text set as the target entity word, wherein the training text set refers to a text set including the same user name, that is, each training text in the training text set includes the same user name, and the included same user name is the user name of the target user. In one embodiment, the computer device can obtain the training text set based on the search log of the user's historical search. In a specific implementation, the computer device can first obtain the search log and determine the text segment of each search text in the search log. The text segment includes one or more entity words, and one or more entity words include entity words marked with entity word categories. Then, after the computer device determines the text segment of each search text in the search log, the text segment corresponding to the entity word category of each search text segment is filtered out from the text segment according to the entity words marked with entity word categories. The text segment with the same user name is used to form the training text set.
[0053] In one embodiment, the search log is the relevant data generated by a computer device when one or more users conduct data searches on the Internet within a certain time range. The search log includes at least the search text. It is understood that when any user conducts a data search, they can first enter the relevant text, so that the Internet can provide search feedback based on the user's input text. In other words, the search text in the search log obtained based on the data search statistics is the text entered by the user when conducting the data search. Among them, the search text can be, for example, "Xiao A's game commentary video", "Xiao A's game commentary live broadcast", or "Xiao B's live broadcast selling goods", etc. The time range of the search log obtained by the computer device through the search text statistics can be, for example, one day or one hour, etc., which is not limited in the embodiments of the present invention. After obtaining the search log, the computer device can determine the text segmentation of each search text in the search log. When determining the text segmentation of any search text, the computer device can first perform word segmentation and entity word recognition on any search text to obtain a word segmentation set of any search text, and an entity word recognition result, wherein the entity word recognition result includes one or more entity words and a category of each entity word; then the computer device can filter the word segmentation set, and use the filtered word segmentation set and the entity word recognition result to form a text segmentation of any search text. The way in which the computer device obtains the text segmentation of any search text can be referred to the way in which the segmentation is obtained in the above embodiment, and will not be repeated here.
[0054] If the search log obtained by the computer device is denoted as a set Q, and each search text included in the search log is denoted as a search query, for any search query in the search log Q, assuming it is query_i, the above-mentioned method for generating text fragments can be used to obtain the text fragments of query_i, where the text fragments of query_i can be denoted as query_i_segments. Assume query_i_segments = {t_1, t_2, t_3, ..., t_n}, where t_i represents the i-th word segment in the text fragment of query_i. After the computer device obtains the text fragments of each search text in the set Q corresponding to the search log, if the text fragment query_i_segments does not contain entity words of the username category, the search text is not further processed and can be directly discarded. In another implementation method, assuming that there are entity words in the username category in the text segment query_i_segments, and t_j in the text segment query_i_segments is the entity word in the username category (i.e., the username), then the computer device can filter out queries with entity words in the username category from the set Q corresponding to the search log, and add the text segments of the filtered queries that include queries with the same username to the training text set, so that the same username appears in all training text sets.
[0055] Based on the training text set determined by the computer device, the computer device may use the user name that appears in each text segment in the training text set as a target entity word and determine the associated entity words of the target entity word. In a specific implementation, if t_j in the text segment query_i_segments included in the training text set is the target entity word, then the context set of the target entity word in the text segment query_i_segments can be recorded as person_query_i_context, and specifically, person_query_i_context = query_i_segments-{t_j}, that is, the context set of the target entity word in the text segment may include the difference between the text segment query_i_segments and the target entity word t_j. The computer device can then count the difference between each text fragment in the training text set and the target entity word. After obtaining the context of the target entity word at the granularity of each text fragment, the computer device can select some or all entity words from the obtained context as associated entity words of the target entity word, and record the number of occurrences of each context at the same time. In this way, all global contexts of the target entity word (i.e., associated entity words) and the corresponding number of occurrences of each context are obtained.
[0056] If the training text set includes three text segments corresponding to queries, they can be as follows: ① to ③:
[0057] ①Xiaoxi's game commentary video: Xiaoxi_name / commentary / game / video;
[0058] ②Xiaoxi's commentary on the highlights of the game video: Xiaoxi_name / commentary / highlights;
[0059] ③Xiao Xi explains King of Glory: Xiao Xi_name / King of Glory_IP-game / commentary;
[0060] It can be understood that the target entity word obtained by the computer device is "Xiao Xi," and its associated entity words and corresponding occurrence counts are: Honor of Kings | 1 time / Commentary | 3 times / Game | 1 time / Video | 1 time / Highlights | 1 time. Based on the occurrence count of each associated entity word obtained by the computer device, the computer device can determine an importance score for each associated entity word. The computer device can then determine the true label of the target entity word based on the importance score, and use the true label to optimize and train the target network model.
[0061] S303: Call the target network model to perform recognition processing on the target entity word to obtain the first word vector of the target entity word.
[0062] S304: Call the target network model to perform recognition processing on the associated entity words corresponding to the target entity words included in each training text to obtain a second word vector corresponding to each associated entity word.
[0063] S305, calling the target network model, and combining the first word vector and the second word vector to perform label prediction processing on the target entity word to obtain a predicted label of the target entity word, and the predicted label is used to indicate whether the user identity of the target user is the target identity.
[0064] After the computer device obtains the target entity word, it can call the target network model to identify the target entity word and obtain the first word vector of the target entity word. It can also call the target network model to identify the associated entity words corresponding to the target entity word included in each training text and obtain the second word vector corresponding to each associated entity word. Then, after the computer device obtains the predicted label of the target entity word based on the first word vector and the second word vector, it can also optimize the training of the target network model in combination with the real label of the target entity word. In one embodiment, the computer device can determine the real label of the target entity word based on the importance score corresponding to each associated entity word obtained; wherein the importance score is used to indicate the accuracy of using the corresponding associated entity word to describe the identity characteristics of the target user. If the number of associated entity words determined by the computer device is N, N≥1 and is an integer; then the computer device can obtain the number of occurrences of any associated entity word in the N associated entity words, and thus can determine the importance score corresponding to any associated entity word based on the number of occurrences. In one embodiment, the computer device may use the number of occurrences of any associated entity word in N associated entity words as the importance score of any associated entity word, or the computer device may first normalize the number of occurrences of any associated entity word in N associated entity words, and then use the normalized number as the importance score of any associated entity word. For example, if the associated entity words and the corresponding number of occurrences determined by the computer device are: Honor of Kings|1 time / Commentary|3 times / Game|1 time / Video|1 time / Highlights|1 time, then the computer device may determine that the importance score corresponding to the associated entity word Honor of Kings is 1 point, the importance score corresponding to the associated entity word Commentary is 3 points, etc.; or it may determine that the importance score corresponding to the associated entity word Honor of Kings is 1 / 7 points, the importance score corresponding to the associated entity word Commentary is 3 / 7 points, etc.
[0065] After determining the importance score of each associated entity word, the computer device may determine the true label of the target entity word based on the importance score. The true label is used to indicate whether the target entity word (or target text) is a positive example or a negative example. A positive example means that the user identity of the target user indicated by the target entity word is the target identity, and a negative example means that the user identity of the target user indicated by the target entity word is not the target identity. When determining the true label of the target entity word based on the importance score, the computer device may first arrange the N associated entity words in descending order according to the corresponding importance scores, and select L associated entity words from the first place in the arrangement in the order of arrangement; 1≤L≤N, and L is an integer; and if the selected L associated entity words include the reference entity word, the true label of the target entity word is determined to be the first label, and the first label is used to indicate that the user identity of the target user is the target identity, wherein the general value of N can be 10, 15, etc., and the general value of L is 5 or 8, etc. In one embodiment, the reference entity words are preset according to the target identity to be determined as needed, and are descriptive words related to the identity characteristics of the target identity. The target identity includes: game commentary identity; the reference entity words preset for the game commentary identity include any one or more of the following: commentary, and entity words of any game name category. The user of the game commentary identity refers to a user who helps listeners and viewers follow the progress of the game, describes what happens in the game, and expounds on the commentator's own point of view. On video vertical websites, there will be a large number of commentary users uploading their own game commentary videos, and a large number of fans will search for game videos explained by game commentators. The target identity can also be a live broadcast identity, then the reference entity words preset for the live broadcast identity include any one or more of the following: live broadcast, and entity words of product name category, etc.
[0066] In one embodiment, associated entity words with the same importance score can be randomly arranged, wherein, if the associated entity words determined by the computer device and the corresponding number of occurrences are: Honor of Kings|1 time / Commentary|3 times / Game|2 times / Video|1 time / Highlights|1 time / Live|1 time, then the computer device can sort the associated entity words in order of importance scores from high to low, and the resulting order can be: Commentary, Honor of Kings, Game, Video, Highlights, Live, and then the computer device can select L associated entity words in sequence. Assuming L is 5, the selected L associated entity words include: Commentary, Honor of Kings, Game, Video, Highlights. If the computer device determines that the reference entity word is the commentary and the entity word that appears in the game name (or game category), then the computer device can determine that the selected L associated entity words include the reference entity word, then the computer device can determine that the target entity word is a positive example, and determine that the true label of the target entity word is the first label. If the selected L associated entity words do not include the reference entity word, that is, do not include the explanation and the game name does not appear, the computer device can determine that the target entity word is a negative example and determine that the true label of the target entity word is the second label.
[0067] In one embodiment, if the L associated entity words selected by the computer device do not include the reference entity word, the true label of the target entity word can be determined to be the second label, and the second label is used to indicate that the user identity of the target user is not the target identity; or, if the L associated entity words selected by the computer device do not include the reference entity word, the computer device can further select J associated entity words from the L+1th associated entity word in sequence, and when the reference entity word is not included in the selected L+J associated entity words, the target entity word is determined to be a negative example, and the true label of the target entity word is determined to be the second label, 1≤J≤N, and J is an integer. For example, if the 5 associated entity words selected by the computer device do not include the reference entity word, the true label of the target entity word can be directly determined to be the second label, or the computer device can further select J (assuming 5) associated entity words, so that when the reference entity word is not included in the selected 10 (L+J) associated entity words, the target entity word is determined to be a negative example, and its corresponding true label is the second label. That is to say, if the target identity to be determined is a game commentator, the computer device can first determine the related entity words ranked in the top L (such as TOP5) of the corresponding importance scores based on the importance scores of the related entity words, and determine whether "commentary" appears in the related entity words ranked in the top 5 of the corresponding importance scores, and include game category entity words. If it is determined that "commentary" appears in the related entity words ranked in the top 5 of the corresponding importance scores, and includes game category entity words, and if it is determined that the target entity word is a positive example, and the true label of the target entity word is determined to be the first label. If it is determined that "commentary" does not appear in the related entity words ranked in the top L+J (such as TOP10) of the corresponding importance scores, and does not include game category entity words, then the target entity word is determined to be a negative example, and the true label of the target entity word is determined to be the second label.
[0068] After the computer device determines the real label of the target entity word, it can determine the label difference between the predicted label and the real label based on the predicted label and the real label of the target entity word, so that the computer device can adjust the model parameters of the target network model according to the label difference, and stop training the target network model when the label difference between the predicted label and the real label is less than the preset difference, and obtain the trained target network model. In one embodiment, the target network model is a TOP_n Context Entity Typing Model (an entity word processing model), which can be specifically as follows Figure 4As shown, after the computer device obtains the target entity word and the associated entity word of the target entity word, it inputs the target entity word into the target network model to obtain the first word vector of the target entity word, and inputs the associated entity word into the target network model to obtain the second word vector of the associated entity word, wherein the associated entity word input into the target network model by the computer device is the associated entity word ranked in the top n positions (such as the top 10 or top 15) in terms of corresponding importance scores. After the target network model obtains the input associated entity word, it can call a corresponding semantic encoding network to process an associated entity word respectively, and then obtain the second word vector of each associated entity word, so that the target network model can obtain the predicted label of the target entity word based on the first word vector and the second word vector. When the obtained predicted label is different from the true label of the target entity word, the model parameters of the target network model can be iteratively optimized, and when the obtained predicted label is the same as the true label of the target entity word, the optimization of the model parameters is stopped to obtain the trained target network model. Practice has shown that the trained target network model can achieve an F1 score of 90% on the test set. The trained target network model is used to predict usernames and corresponding contexts not covered by the rules. Finally, the prediction results are manually inspected, and it is found that the trained target network model is highly accurate in predicting whether the username indicates the target user is the target identity.
[0069] After the computer device obtains the predicted label of the target entity word, if the predicted label indicates that the user identity of the target user is the target identity, the computer device may add the target entity word to the search word list, and the user identity of each entity word included in the search word list is the target identity. Further, the computer device may add the search word list to the intent recognition module so that the intent recognition module performs intent search feedback based on the search word list to perform intent search based on the search word list. In one embodiment, the computer device may obtain the intended search word from the user side and obtain the search user name from the intended search word. The intended search word is an entity word that does not point to a specific search content. Further, the computer device may search the search word list to see whether the search user name is included, and when it is determined that the search user name is included, the data of the search user corresponding to the search user name under the target identity is used as search feedback data, thereby outputting the search feedback data on the user side. In a specific implementation, if the target identity is a game commentator identity, the search word list obtained by the computer device is a game commentator word list, which records the user name of the corresponding user whose corresponding prediction label indicates that the user identity of the corresponding user is a game commentator identity. The determination of the game commentator word list can be used to expand the game commentator knowledge base of the existing NER service, thereby increasing the recognition recall of the game commentator by the NER service.
[0070] In another implementation, the mined game commentator vocabulary can also be applied to the intent recognition module. Intent query refers to the keyword type in which the user search term does not specifically point to a certain content, but generally refers to a type of video content. For example, query = "Xiaobai commentary" is a short video intent query. The intent recognition module is to identify the search intent of the search term and the key slot information. For example, for query = "Xiaobai commentary", the intent recognition module will output the intent result that the game commentator slot is "Xiaobai", thereby outputting relevant videos when Xiaobai is the game commentator, and feeding back to the user side, such as Figure 5a As shown, the mined game commentators can be used to expand the recall of intent recognition. In one embodiment, the intent search term can be sent to the server by the search user through the terminal device, wherein when the search user searches for data based on the intent search term, he can refer to Figure 5b , wherein the search user can input the intended search term through the terminal device 50, and after the terminal device obtains the intended search term, it can send the intended search term to the server 51, then the server 51 can obtain the search user name based on the intended search term, and when determining that the search user name is included in the search term table, the data of the search user corresponding to the search user name under the target identity is sent as search feedback data to the terminal device 50, then the terminal device 50 can display the search feedback data obtained based on the intended search term to the search user, thereby completing the intended search process of the search user, wherein, if the intended search term input by the search user is the above-mentioned "Xiaobai's Explanation", then it can be understood that the search feedback data finally displayed in the terminal device 50 can be as follows Figure 5a An example interface for intent areas is shown in .
[0071] When a search user enters an intended search term through the terminal device 50, the intended search term can be entered into the target application. That is, the intended search term obtained by the terminal device 50 is the intended search term entered into the target application by the search user. The target application can be a search engine, or other application that supports search functions, etc., wherein the application can be an independent client in the terminal device, or it can also be a small program in a client, etc., which is not limited in the embodiments of the present invention. It should be noted that after obtaining the intended search term, the terminal device can also directly determine the search feedback data and output and display the search feedback data.
[0072] In an embodiment of the present invention, the computer device may first pre-process the search log based on the NER technology and the word segmentation technology to obtain the target entity word, and then determine the real label of the target entity word according to the context of the target entity word. In addition, the computer device may also perform recognition processing on the discrete context (i.e., the associated entity word of the target entity word) and the target entity word to obtain the first word vector of the target entity word and the second word vector of each associated entity word. Then, the target network model may be called to obtain the predicted label of the target entity word based on the first word vector and the second word vector, so that the target network model may be trained based on the predicted label and the real label. Finally, the search log of the part not covered by the rule may be predicted based on the trained target network model to expand the recognition recall of the game commentator, and then a user vocabulary corresponding to the target identity may be generated, so that the computer device can subsequently perform intent search based on the user vocabulary to improve the accuracy of the search.
[0073] Based on the description of the above data processing method embodiment, the embodiment of the present invention also proposes a data processing device, which can be a computer program (including program code) running on the above computer device. The data processing device can be used to perform the following steps: Figure 2 and Figure 3 For the data processing method described, please refer to Figure 6 The data processing device includes: an acquisition unit 601, a determination unit 602 and a processing unit 603.
[0074] An acquisition unit 601 is configured to acquire a target text and acquire a target entity word from the target text, where the target entity word is a username of a target user.
[0075] A determining unit 602 is configured to determine associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user;
[0076] The processing unit 603 is configured to perform recognition processing on the target entity word to obtain a first word vector of the target entity word, and perform recognition processing on the associated entity word to obtain a second word vector of the associated entity word;
[0077] The processing unit 603 is further used to combine the first word vector and the second word vector to perform label prediction processing on the target entity word to obtain a predicted label of the target entity word, and the predicted label is used to indicate whether the user identity of the target user is the target identity.
[0078] In one embodiment, if the target text is any training text in a training text set, each training text in the training text set includes the target entity word; the processing unit 603 is specifically configured to:
[0079] The associated entity words corresponding to the target entity word included in each training text are identified to obtain a second word vector corresponding to each associated entity word.
[0080] In one embodiment, the first word vector, the second word vector, and the predicted label are all obtained by calling a target network model, and the target network model is a neural network for user identity discrimination; the device also includes: an adjustment unit 604.
[0081] The acquisition unit 601 is further configured to acquire an importance score corresponding to each associated entity word;
[0082] The determining unit 602 is further configured to determine a true label of the target entity word according to the importance score; the importance score is used to indicate the accuracy of using the corresponding associated entity word to describe the identity characteristics of the target user;
[0083] An adjusting unit 604 is configured to determine a label difference between the predicted label and the true label of the target entity word, and adjust a model parameter of the target network model according to the label difference;
[0084] The adjusting unit 604 is further configured to stop training the target network model when the label difference is less than a preset difference, thereby obtaining a trained target network model.
[0085] In one embodiment, the number of the associated entity words is N, where N is ≥ 1 and is an integer; the determining unit 602 is specifically configured to:
[0086] Arrange the N related entity words in descending order of their corresponding importance scores, and select L related entity words from the first one in the arrangement order; 1≤L≤N, and L is an integer;
[0087] If the selected L associated entity words include a reference entity word, the true label of the target entity word is determined to be the first label, and the first label is used to indicate that the user identity of the target user is the target identity. The reference entity word is a descriptive word preset based on the target identity and related to the identity characteristics of the target identity.
[0088] In one embodiment, the determining unit 602 is further configured to determine, if the selected L associated entity words do not include the reference entity word, that the true label of the target entity word is a second label, the second label being used to indicate that the user identity of the target user is not the target identity; or
[0089] The determination unit 602 is also used to select J related entity words from the L+1th related entity word in sequence if the selected L related entity words do not include the reference entity word, and when the reference entity word is not included in the selected L+J related entity words, determine that the true label of the target entity word is the second label, 1≤J≤N, and J is an integer.
[0090] In one embodiment, the number of the associated entity words is N, where N is ≥ 1 and is an integer; the acquiring unit 601 is specifically configured to:
[0091] Obtaining the number of occurrences of any associated entity word in the N associated entity words, and determining an importance score corresponding to the any associated entity word according to the number of occurrences;
[0092] Among them, the importance score corresponding to any associated entity word is the number of times the any associated entity word appears in the N associated entity words, or the importance score corresponding to any associated entity word is the normalized number of times the any associated entity word appears in the N associated entity words.
[0093] In one embodiment, the processing unit 603 is specifically configured to:
[0094] Obtaining a search log and determining a text segment of each search text in the search log, wherein the text segment includes one or more entity words, and an entity word marked with an entity word category is included in the one or more entity words;
[0095] According to the entity words of the entity word category marked in the text fragment of each search text, the text fragments whose corresponding entity word category is the username category and which have the same username are filtered out from the text fragments, and the text fragments with the same username constitute the training text set.
[0096] In one embodiment, the processing unit 603 is specifically configured to:
[0097] For any search text, perform word segmentation processing and entity word recognition processing on the search text to obtain a word segmentation set of the search text and an entity word recognition result, wherein the entity word recognition result includes one or more entity words and a category of each entity word;
[0098] The word segmentation set is filtered, and the filtered word segmentation set and the entity word recognition result constitute the text segment of any search text.
[0099] In an embodiment of the present invention, after acquiring the target text, the acquisition unit 601 can acquire the user name of the target user from the target text, and use the acquired user name as the target entity word. After the determination unit 602 determines the target entity word, it can also determine the associated entity words of the target entity word from the target text. Then, based on the acquisition of the target entity word and the associated entity words in the target text, entity words with incoherent semantics can be acquired. After obtaining the target entity word and the associated entity word, the processing unit 603 can respectively perform recognition processing on the target entity word to obtain the first word vector of the target entity word, and perform recognition processing on the associated entity word to obtain the second word vector of the associated entity word. Then, further, based on the obtained first word vector and second word vector, the processing unit 603 can combine the first word vector and the second word vector to predict the predicted label of the target entity word, so as to determine whether the target user indicated by the target entity word is the target identity according to the predicted label, and thus determine the label corresponding to the target entity word (that is, the refined entity type corresponding to the target entity word) based on the entity word with incoherent semantics. The method of using entity words with incoherent semantics to predict the label of the target entity word can effectively improve the convenience in predicting the entity word label, thereby improving the efficiency of predicting the label of the entity word.
[0100] See Figure 7 , is a schematic block diagram of the structure of a computer device provided by an embodiment of the present invention. The computer device can be a server, which can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; in addition, the computer device can also be a terminal device, which can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. Figure 7The computer device in the illustrated embodiment may include: one or more processors 701; one or more input devices 702; one or more output devices 703; and memory 704. The processors 701, input devices 702, output devices 703, and memory 704 are connected via a bus 705. The memory 704 is used to store computer programs, which include program instructions. The processor 701 is used to execute the program instructions stored in the memory 704.
[0101] The memory 704 may include volatile memory, such as random-access memory (RAM); the memory 704 may also include non-volatile memory, such as flash memory, solid-state drive (SSD), etc.; the memory 704 may also include a combination of the above types of memory.
[0102] The processor 701 may be a central processing unit (CPU). The processor 701 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The PLD may be a field-programmable gate array (FPGA), a generic array logic (GAL), etc. The processor 701 may also be a combination of the above structures.
[0103] In the embodiment of the present invention, the memory 704 is used to store a computer program, and the computer program includes program instructions. The processor 701 is used to execute the program instructions stored in the memory 704 to implement the above-mentioned Figure 2 and Figure 3 Steps of the corresponding method.
[0104] In one embodiment, the processor 701 is configured to call the program instructions to execute:
[0105] Obtaining a target text, and obtaining a target entity word from the target text, wherein the target entity word is a user name of a target user;
[0106] Determining associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user;
[0107] Performing recognition processing on the target entity word to obtain a first word vector of the target entity word, and performing recognition processing on the associated entity word to obtain a second word vector of the associated entity word;
[0108] In combination with the first word vector and the second word vector, label prediction processing is performed on the target entity word to obtain a predicted label of the target entity word, and the predicted label is used to indicate whether the user identity of the target user is the target identity.
[0109] An embodiment of the present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, so that the computer device performs the above-mentioned Figure 2 or Figure 3 The computer-readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).
[0110] The above disclosure is only a partial embodiment of the present invention, which certainly cannot be used to limit the scope of the rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.
Claims
1. A data processing method, characterized in that: include: Obtaining a target text, and obtaining a target entity word from the target text, wherein the target entity word is a user name of a target user; Determining associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user; Performing recognition processing on the target entity word to obtain a first word vector of the target entity word, and performing recognition processing on the associated entity word to obtain a second word vector of the associated entity word; Combining the first word vector and the second word vector, performing label prediction processing on the target entity word to obtain a predicted label of the target entity word, where the predicted label is used to indicate whether the user identity of the target user is the target identity; If the predicted tag indicates that the user identity of the target user is the target identity, then the target entity word is added to the search word list, and the user identity of the user corresponding to each entity word included in the search word list is the target identity; The search word list is added to the intention recognition module so that the intention recognition module performs intention search feedback based on the search word list.
2. The method according to claim 1, wherein If the target text is any training text in a training text set, each training text in the training text set includes the target entity word; the identifying process for the associated entity word to obtain a second word vector corresponding to the associated entity word includes: The associated entity words corresponding to the target entity word included in each training text are identified to obtain a second word vector corresponding to each associated entity word.
3. The method according to claim 1, wherein The first word vector, the second word vector, and the predicted label are all obtained by calling a target network model, and the target network model is a neural network for user identity discrimination; The method further comprises: Obtaining an importance score corresponding to each associated entity word, and determining a true label of the target entity word based on the importance score; the importance score is used to indicate the accuracy of using the corresponding associated entity word to describe the identity characteristics of the target user; Determining a label difference between the predicted label and the true label according to the target entity word, and adjusting model parameters of the target network model according to the label difference; When the label difference is less than the preset difference, the training of the target network model is stopped to obtain a trained target network model.
4. The method according to claim 3, wherein The number of the associated entity words is N, where N is ≥ 1 and is an integer; and determining the true label of the target entity word according to the importance score includes: Arrange the N related entity words in descending order of their corresponding importance scores, and select L related entity words from the first one in the arrangement order; 1≤L≤N, and L is an integer; If the selected L associated entity words include a reference entity word, the true label of the target entity word is determined to be the first label, and the first label is used to indicate that the user identity of the target user is the target identity. The reference entity word is a descriptive word preset based on the target identity and related to the identity characteristics of the target identity.
5. The method according to claim 4, wherein The method further comprises: If the selected L associated entity words do not include the reference entity word, then the true label of the target entity word is determined to be the second label, and the second label is used to indicate that the user identity of the target user is not the target identity; or If the selected L associated entity words do not include the reference entity word, then J associated entity words are selected from the L+1th associated entity word in sequence, and when the reference entity word is not included in the selected L+J associated entity words, the true label of the target entity word is determined to be the second label, 1≤J≤N, and J is an integer.
6. The method according to claim 4 or 5, characterized in that The target identities include: game commentator identity; The reference entity words preset for the game commentator identity include any one or more of the following: commentary, and entity words of any game name category.
7. The method according to claim 3, wherein The number of the associated entity words is N, where N is ≥ 1 and is an integer; and obtaining the importance score corresponding to each associated entity word includes: Obtaining the number of occurrences of any associated entity word in the N associated entity words, and determining an importance score corresponding to the any associated entity word according to the number of occurrences; Among them, the importance score corresponding to any associated entity word is the number of times the any associated entity word appears in the N associated entity words, or the importance score corresponding to any associated entity word is the normalized number of times the any associated entity word appears in the N associated entity words.
8. The method according to claim 2, wherein The method of obtaining the training text set includes: Obtaining a search log and determining a text segment of each search text in the search log, wherein the text segment includes one or more entity words, and an entity word marked with an entity word category is included in the one or more entity words; According to the entity words of the entity word category marked in the text fragment of each search text, the text fragments whose corresponding entity word category is the username category and which have the same username are filtered out from the text fragments, and the text fragments with the same username constitute the training text set.
9. The method according to claim 8, wherein Determining the text fragment of each search text in the search log includes: For any search text, perform word segmentation processing and entity word recognition processing on the search text to obtain a word segmentation set of the search text and an entity word recognition result, wherein the entity word recognition result includes one or more entity words and a category of each entity word; The word segmentation set is filtered, and the filtered word segmentation set and the entity word recognition result constitute the text segment of any search text.
10. The method according to claim 1, wherein The method further comprises: Obtaining an intended search term, and obtaining a search user name from the intended search term, wherein the intended search term is an entity word that does not point to specific search content; The search word table is searched to see whether the search user name is included. If it is determined that the search user name is included, the data of the search user corresponding to the search user name under the target identity is used as search feedback data, and the search feedback data is output.
11. A data processing device, characterized in that: include: An acquisition unit, configured to acquire a target text and acquire a target entity word from the target text, wherein the target entity word is a user name of a target user; a determining unit, configured to determine associated entity words of the target entity word from the target text, wherein the associated entity words are any one or more entity words other than the target entity word in the target text, and the associated entity words include descriptive words related to the identity characteristics of the target user; a processing unit, configured to perform recognition processing on the target entity word to obtain a first word vector of the target entity word, and perform recognition processing on the associated entity word to obtain a second word vector of the associated entity word; The processing unit is further configured to perform label prediction processing on the target entity word in combination with the first word vector and the second word vector to obtain a predicted label for the target entity word, where the predicted label is used to indicate whether the user identity of the target user is the target identity; The processing unit is further configured to add the target entity word to a search word list if the predicted tag indicates that the user identity of the target user is a target identity, wherein the user identity of each entity word included in the search word list corresponds to the target identity; The search word list is added to the intention recognition module so that the intention recognition module performs intention search feedback based on the search word list.
12. A computer device, characterized in that: The method comprises a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Method for extracting relations among named entities in Internet massive data and system thereof
CN104933164A
Method and device for training entity recognition and relation classification model, equipment and medium
CN111159407A