Social user location prediction method based on pre-trained language model and knowledge graph

Abstract

The application provides a social user location prediction method based on a pre-trained language model and a knowledge graph, which comprises the following steps: obtaining a valid text set and a valid entity set published by a first social user; using a pre-trained language model to encode the valid text and the valid entity respectively to obtain semantic representations of the valid text and the valid entity; calculating an anonymous relationship representation between the valid entity and the corresponding valid text according to the semantic representations of the valid text and the valid entity; extracting a k-hop subgraph of the valid entity from a geographic encyclopedia knowledge graph; constructing a joint graph of the first social user according to the entity, the valid text, the valid entity, the anonymous relationship representation and the k-hop subgraph of the first social user; and predicting the location of the first social user by using a progressive relationship graph neural network according to the joint graph. The application can mine the potential correlation between the user and the location, has strong generalization, and can perform interpretable reasoning.

Description

Technical Field

[0001] This application relates to the field of social user location prediction technology, and in particular to a method and apparatus for social user location prediction based on a pre-trained language model and knowledge graph. Background Technology

[0002] Several methods for predicting social media user location information already exist. Existing methods include using machine learning to predict user location by dividing a geographic area into grids, using Naive Bayes to estimate the probability of each word in a tweet falling into a different grid, using Kullback-Leibler divergence (KLD) to find tweets with the word distribution most similar to the grid, and finally using density estimation instead of count-based estimation. Because this technique involves geographic grid division, it is only suitable for small-scale, coarse-grained estimation and requires a large amount of training data. Existing methods also include using neural networks to predict location, by introducing Long Short-Term Memory (LSTM) networks to encode tweets, and by introducing Bidirectional Encoder Representations from Transformers (BERT) to encode tweets. While these methods have achieved some success, they generally rely on self-descriptions of user location and are not good at predicting the location of users lacking such information. Existing methods also include inference methods that use Graph Convolutional Networks (GCNs) to fuse the entity-to-entity relationship graphs in tweets and use Gaussian mixture models to learn the Gaussian distribution of entity locations. These methods suffer from drawbacks such as densely connected relationship graphs and the need for large amounts of training data. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in the related art.

[0004] Therefore, the first objective of this application is to propose a social user location prediction method based on a pre-trained language model and knowledge graph. This method solves the technical problem that existing social user location prediction methods require a large amount of training data and rely on self-descriptions of user location. By extracting entities from the text posted by social users and combining them with an external geographic knowledge graph, a user-centric joint graph of user, text, and knowledge graph is constructed. The method then uses a pre-trained language model and a progressive relational graph neural network to predict the user's location. This method can perform location reasoning for users who lack self-descriptions, uncover the potential relationship between users and locations, has strong generalization ability, and can perform interpretable reasoning.

[0005] The second objective of this application is to propose a social user location prediction device based on a pre-trained language model and knowledge graph.

[0006] The third objective of this application is to propose a computer device.

[0007] The fourth objective of this application is to provide a non-transitory computer-readable storage medium.

[0008] To achieve the above objectives, the first aspect of this application proposes a method for predicting the location of social users based on a pre-trained language model and a knowledge graph, comprising: obtaining a set of valid texts and a set of valid entities published by a first social user, wherein the set of valid texts includes multiple valid texts, and each valid text includes at least one valid entity; encoding the valid texts and valid entities using a pre-trained language model to obtain semantic representations of the valid texts and valid entities, and calculating the representation of the anonymous relationship between the valid entities and the corresponding valid texts based on the semantic representations of the valid texts and valid entities; extracting a k-hop subgraph from a geographical encyclopedia knowledge graph based on the valid entities, and constructing a joint graph of the first social user based on the entities, valid texts, valid entities, the representation of anonymous relationships, and the k-hop subgraph; and predicting the location of the first social user using a progressive relational graph neural network based on the joint graph.

[0009] Optionally, in one embodiment of this application, obtaining the set of valid texts and the set of valid entities published by the first social user includes:

[0010] Obtain the text set published by the first social user, extract named entities from the text set, link the entities to obtain the valid entity set, and obtain the text corresponding to the valid entities from the text set to obtain the valid text set;

[0011] Specifically, named entities are connected to entities in the geographic knowledge base using an end-to-end entity linking method, and named entities without linking relationships are filtered out to obtain a valid set of entities.

[0012] Optionally, in one embodiment of this application, the representation of the anonymous relationship between the valid entity and the corresponding valid text is calculated based on the semantic representation of the valid text and the semantic representation of the valid entity, including:

[0013] Set a preset number of anonymous relation factors, and use the first loss function to separate the anonymous relation factors;

[0014] The semantic representations of the trained anonymous relation factors, the semantic representations of the effective entities, and the semantic representations of the effective text corresponding to the effective entities are concatenated, and the attention values ​​of the effective entities to the anonymous relation factors are calculated based on the concatenation results. The trained semantic representations of the anonymous relation factors are obtained by initializing and training the semantic representations of the anonymous relation factors.

[0015] Based on the attention values ​​of valid entities to the anonymous relation factors and the semantic representations of the trained anonymous relation factors, the representation of the anonymous relation between valid entities and the corresponding valid text is calculated.

[0016] Optionally, in one embodiment of this application, extracting a k-hop subgraph from a geographic encyclopedia knowledge graph based on valid entities includes:

[0017] Step S1: Initialize the first candidate entity set based on the valid entities, and initialize the candidate fact set to an empty set;

[0018] Step S2: Query the geographic knowledge base for triples with entities in the first candidate entity set as head or tail entities, add the triples to the candidate fact set, and add the entities in the triples to the first candidate entity set.

[0019] Step S3: Repeat step S2 k times to obtain a k-jump subgraph.

[0020] Optionally, in one embodiment of this application, a joint graph of the first social user is constructed based on the entities, valid text, valid entities, representations of anonymous relationships, and k-hop subgraphs of the first social user, including:

[0021] Connect the entities and valid text of the first social user according to the relationship between the entities and valid text of the first social user, wherein the relationship between the entities and valid text of the first social user includes posting relationship and forwarding relationship;

[0022] Connect the valid text and the valid entities in the valid text according to the representation of the anonymous relationship between the valid entities and the corresponding valid text;

[0023] Connect all valid entities corresponding to the valid text using association relationships, and connect the valid entities and entities in the k-hop subgraph according to the k-hop subgraph to obtain the joint graph of the first social user.

[0024] Optionally, in one embodiment of this application, the location of the first social user is predicted using a progressive relational graph neural network based on the joint graph, including:

[0025] The query is constructed based on the entities and relationships of the first social user, where "user is located" is used as a relationship;

[0026] The joint graph and the query are input into the progressive relational graph neural network to obtain the inverse triples of the triples in the candidate fact set, and the inverse triples are added to the joint graph.

[0027] The second candidate entity set is initialized to a set of entities containing the first social user, and the edge set is initialized to an empty set;

[0028] According to the joint graph, select triples starting from entities in the second candidate entity set and add them to the edge set. Then add the entities involved in the triples in the edge set to the second candidate entity set. Calculate the message representation of the entities in the second candidate entity set. Iterate multiple times until all entities in the joint graph are added to the second candidate entity set and the message representation of all entities in the second candidate entity set is obtained.

[0029] Filter out candidate entities that are not geographical entities from the second candidate entity set, calculate the score of the filtered candidate entities according to the message representation, and select the candidate entity with the highest score as the first social user location.

[0030] Optionally, in one embodiment of this application, before predicting the location of the first social user using a progressive relational graph neural network based on the joint graph, the following steps are included:

[0031] Get a collection of texts posted by multiple social media users, as well as the actual locations of these users;

[0032] Construct a joint graph for each social user based on the text set;

[0033] Based on the joint graph and the actual location, the progressive relational graph neural network is trained using a joint loss function;

[0034] The joint loss function is expressed as:

[0035] Loss = Loss prd +λLoss dis

[0036] Where Loss represents the joint loss function, Loss dis Let Loss represent the first loss function. prd Let λ represent the second loss function, and let λ represent the preset combination coefficients.

[0037] The first loss function is expressed as:

[0038]

[0039] Among them, Loss dis Let m represent the first loss function, m be the number of anonymous relation factors, and s be the number of anonymous relation factors. i，j Let i be the cosine similarity between anonymous relation factor i and anonymous relation factor j;

[0040] The second loss function is expressed as:

[0041]

[0042] Among them, Loss prd For the second loss function, s q For social users, q represents the relationship "user is located", o q For social users q The actual location of G1 for social users q And the corresponding actual position o q The triple (s) consisting of the relation "user is located" and q q , q, o q The set of f(s) q , q, o q E represents the score of a social user's actual location. sq For social users q The set of all filtered candidate entities, f(s) q (q, o) represents the triples (s, q, o) output by the progressive relational graph neural network. q The score of , q, o).

[0043] To achieve the above objectives, a second aspect of this application proposes a social user location prediction device based on a pre-trained language model and knowledge graph, comprising:

[0044] The acquisition module is used to acquire the set of valid texts and the set of valid entities published by the first social user. The set of valid texts includes multiple valid texts, and each valid text includes at least one valid entity.

[0045] The anonymous relation representation module is used to encode the effective text and effective entities using a pre-trained language model to obtain the semantic representation of the effective text and the semantic representation of the effective entities. Based on the semantic representation of the effective text and the semantic representation of the effective entities, the module calculates the representation of the anonymous relation between the effective entities and the corresponding effective text.

[0046] The module is used to extract k-hop subgraphs from the geographical encyclopedia knowledge graph based on valid entities, and to construct a joint graph of the first social user based on the entities, valid text, valid entities, the representation of anonymous relations, and the k-hop subgraphs of the first social user.

[0047] The prediction module is used to predict the location of the first social user based on the joint graph using a progressive relational graph neural network.

[0048] To achieve the above objectives, a third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the social user location prediction method based on a pre-trained language model and knowledge graph described in the above embodiment.

[0049] To achieve the above objectives, a fourth aspect of this application proposes a non-transitory computer-readable storage medium that, when instructions in the storage medium are executed by a processor, enables the execution of a social user location prediction method based on a pre-trained language model and a knowledge graph.

[0050] The social user location prediction method, apparatus, computer device, and non-transitory computer-readable storage medium based on pre-trained language models and knowledge graphs in this application solve the technical problems of existing social user location prediction methods requiring a large amount of training data and relying on self-descriptions of user location. By extracting entities from the text posted by social users and combining them with external geographic knowledge graphs to construct a user-centric joint graph of user, text, and knowledge graph, and using pre-trained language models and progressive relational graph neural networks to predict user location, it can perform location reasoning for users who lack self-descriptions, explore potential associations between users and locations, has strong generalization ability, and can perform interpretable reasoning.

[0051] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0052] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0053] Figure 1 A flowchart illustrating a social user location prediction method based on a pre-trained language model and knowledge graph, provided in Embodiment 1 of this application;

[0054] Figure 2 This is another flowchart of the social user location prediction method based on pre-trained language models and knowledge graphs according to an embodiment of this application;

[0055] Figure 3 This is an example diagram of the user, text, and knowledge graph joint graph of the social user location prediction method based on pre-trained language models and knowledge graphs in this application embodiment;

[0056] Figure 4 This is a structural diagram of the progressive relational graph neural network of the social user location prediction method based on pre-trained language models and knowledge graphs in an embodiment of this application.

[0057] Figure 5 This is a schematic diagram of the structure of a social user location prediction device based on a pre-trained language model and knowledge graph, provided in Embodiment 2 of this application. Detailed Implementation

[0058] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0059] Most existing methods for predicting the geolocation of social users rely on publicly available location descriptions and directly use TF-IDF, RNN, BERT pre-trained models, and other methods to calculate text features to predict the user's location. These methods can only use shallow semantics when predicting location and lack interpretability of prediction results, and cannot effectively use external knowledge for reasoning.

[0060] This application proposes a social user location prediction method based on pre-trained language models and knowledge graphs. It extracts valuable entities from tweets, generates anonymous relation representations between entities and tweets using pre-trained language models, and combines related subgraphs extracted from the knowledge base with a progressive relation graph neural network reasoning method to infer the target location entity.

[0061] The following description, with reference to the accompanying drawings, describes a method and apparatus for predicting the location of social users based on pre-trained language models and knowledge graphs, according to embodiments of this application.

[0062] Figure 1 This is a flowchart illustrating a social user location prediction method based on a pre-trained language model and knowledge graph, as provided in Embodiment 1 of this application.

[0063] like Figure 1 As shown, this social user location prediction method based on pre-trained language models and knowledge graphs includes the following steps:

[0064] Step 101: Obtain the set of valid texts and the set of valid entities published by the first social user, wherein the set of valid texts includes multiple valid texts and each valid text includes at least one valid entity;

[0065] Step 102: Encode the effective text and effective entities using the pre-trained language model to obtain the semantic representation of the effective text and the semantic representation of the effective entities. Based on the semantic representation of the effective text and the semantic representation of the effective entities, calculate the representation of the anonymous relationship between the effective entities and the corresponding effective text.

[0066] Step 103: Extract k-hop subgraphs from the geographical encyclopedia knowledge graph based on valid entities, and construct a joint graph of the first social user based on the entities, valid text, valid entities, representations of anonymous relations, and k-hop subgraphs of the first social user.

[0067] Step 104: Based on the joint graph, predict the location of the first social user using a progressive relational graph neural network.

[0068] The social user location prediction method based on pre-trained language models and knowledge graphs in this application involves obtaining a set of valid texts and a set of valid entities published by a first social user. The set of valid texts includes multiple valid texts, and each valid text includes at least one valid entity. The valid texts and entities are encoded using a pre-trained language model to obtain semantic representations of the valid texts and entities. Based on these semantic representations, the anonymous relation between the valid entities and their corresponding valid texts is calculated. A k-hop subgraph is extracted from the geographical encyclopedia knowledge graph based on the valid entities. A joint graph of the first social user is constructed based on the entities, valid texts, the representation of anonymous relations, and the k-hop subgraph. Finally, the location of the first social user is predicted using a progressive relational graph neural network based on the joint graph. Therefore, this method can solve the technical problem that existing social user location information prediction methods require a large amount of training data and rely on users' self-provided location descriptions. By extracting entities from the text posted by social users and combining them with external geographic knowledge graphs, a user-centric joint graph of user, text, and knowledge graph is constructed. Then, a pre-trained language model and progressive relational graph neural network are used to predict the user's location. This method can perform location reasoning for users who lack self-provided location descriptions, explore the potential relationship between users and locations, has strong generalization ability, and can perform interpretable reasoning.

[0069] This application proposes a social user geolocation prediction method combining knowledge graphs. It extracts entities from tweets and constructs a user-centric joint graph of users, tweets, and knowledge graphs by combining them with an external geographic knowledge graph. A pre-trained language model and a progressive relational graph neural network are then used to predict the user's location. This application does not rely on publicly available user geolocation information, thus enabling location reasoning for users lacking self-descriptions. Furthermore, the use of an external knowledge graph allows for the uncovering of potential relationships between users and locations, rather than simply predicting location based on tweet grammatical features or shallow semantics. The progressive relational graph neural network exhibits strong generalization capabilities and enables interpretable reasoning.

[0070] The problem this application aims to solve can be formally defined as follows: the input is a set of texts posted by a user and a geographical encyclopedia knowledge graph; the output is the city where the user is located. The overall process of user location reasoning is as follows: Figure 2 As shown, the system takes text posted by a user as input, extracts entities from the text, calculates the relationship between the text and the corresponding entities, constructs a user-centric joint graph of user, text, and knowledge graph by combining it with an external geographic knowledge graph, and uses a progressive relational graph neural network for reasoning to predict the user's location.

[0071] Furthermore, in this embodiment of the application, obtaining the set of valid texts and the set of valid entities published by the first social user includes:

[0072] Obtain the text set published by the first social user, extract named entities from the text set, link the entities to obtain the valid entity set, and obtain the text corresponding to the valid entities from the text set to obtain the valid text set;

[0073] Specifically, named entities are connected to entities in the geographic knowledge base using an end-to-end entity linking method, and named entities without linking relationships are filtered out to obtain a valid set of entities.

[0074] In this embodiment of the application, entities are extracted from the text published by the first social user and linked together. Irrelevant entities are filtered out to obtain a set of valid entities contained in the text. An end-to-end entity linking method is used to connect entities to entities in a knowledge base, and stop words and entities not included in the knowledge base are filtered out to obtain a set of valid entities. The knowledge base is a geographical encyclopedia knowledge graph.

[0075] The geographical encyclopedia knowledge graph used in this application is the wiki geographical knowledge graph, which is a subset of wikidata and contains only entities with geographical coordinates and triples associated with these entities.

[0076] For example, a user might post text 1 as: "My phone died, so I had my friend film this billboard in Times Square!" From this example text, we can obtain the entities "Times Square" and "billboard". After filtering out stop words and entities not included in the wiki's geographical knowledge graph, the actual entity obtained is "Times Square". Entities included in the wiki's geographical knowledge graph are called valid entities, and "Times Square" is the obtained valid entity. Therefore, text 1 corresponding to "Times Square" is valid text.

[0077] Further, in this embodiment of the application, based on the semantic representation of the valid text and the semantic representation of the valid entity, the representation of the anonymous relationship between the valid entity and the corresponding valid text is calculated, including:

[0078] Set a preset number of anonymous relation factors, and use the first loss function to separate the anonymous relation factors;

[0079] The semantic representations of the trained anonymous relation factors, the semantic representations of the effective entities, and the semantic representations of the effective text corresponding to the effective entities are concatenated, and the attention values ​​of the effective entities to the anonymous relation factors are calculated based on the concatenation results. The trained semantic representations of the anonymous relation factors are obtained by initializing and training the semantic representations of the anonymous relation factors.

[0080] Based on the attention values ​​of valid entities to the anonymous relation factors and the semantic representations of the trained anonymous relation factors, the representation of the anonymous relation between valid entities and the corresponding valid text is calculated.

[0081] In this embodiment, m anonymous relation factors are set. First, a pre-trained language model is used to encode each of the user's tweets and the semantic representation of the entities therein. Then, the semantic representation of the tweets, the representation of the anonymous relation factors, and the entity representation are used to calculate the attention of the entity to each anonymous relation factor. Finally, the attention mechanism is used to integrate all the anonymous relation factors to calculate the representation of the anonymous relation.

[0082] In this embodiment of the application, it is necessary to calculate the anonymous relationship between each entity and its corresponding text.

[0083] First, this application uses a pre-trained language model BERT to encode the text representation h corresponding to the entity. t The entity representation h is obtained from BERT using the token corresponding to the entity in the text. e Among them, h t It is a length of n t The vector, h e It is a length of n e The vector.

[0084] In this process, the pre-trained language model BERT encodes text by first dividing the sentence into several tokens according to words, transforming it into a token sequence, and finally outputting a global semantic representation of the text and the semantic representation corresponding to each token. This application uses the global semantic representation of the text as the text representation, and the semantic representation of the token corresponding to the entity as the entity representation.

[0085] For example, for text 1 and the entity "Times Square", text 1 is encoded as a representation h of the text. t The physical "Times Square" is coded as h e .

[0086] Then, this application combines multiple anonymity relation factors through an attention mechanism to calculate the representation h of the anonymity relation between the text and entities in the text. r At the same time, constraint methods are used to minimize the correlation between each anonymous relation factor.

[0087] By combining the entity representation h e The text representation h corresponding to the entity t The representation of the anonymity factor h i Calculate the attention value 'a' of the entity for each anonymous relation factor. i :

[0088] c i =W tei [h t ||h e ||h i ]

[0089]

[0090] Among them, c i W represents the temporary attention value, || represents the vector concatenation operation, and W represents the vector concatenation value. tei For a shape of m×(n) e +n t +n f The weight matrix, where m represents the number of anonymous relations (usually 4), and n... e n is the length of the entity representation. t n is the length of the text representation corresponding to the entity. f h is the length of the representation of the anonymous relation factor. t h represents the text corresponding to the entity. e For the representation of an entity, h i Let h be the i-th anonymous relation factor. i It is a length of n f The vector, t is the temperature coefficient (usually taken as 5), a i Let be the attention value of the i-th anonymous relation factor.

[0091] Then use the attention value a for each anonymity factor. i and the representation of the anonymity factor h i The anonymous relation representation h is calculated. r :

[0092]

[0093] Among them, h r A representation of an anonymous relationship between entities and their corresponding text. i h is the attention value of the i-th anonymity factor. iLet be the representation of the i-th anonymous factor.

[0094] The semantic representation of the anonymous relation factors in this application is optimized simultaneously during the optimization of the progressive relation graph neural network, resulting in a trained semantic representation of the anonymous relation factors.

[0095] Furthermore, this application applies separation constraints to each anonymous relation factor to reduce their correlation and enhance the semantic richness of the anonymous relation. Many schemes can be chosen for separation constraints, all based on the principle of reducing the similarity / correlation between anonymous relation factors. This application uses cosine constraints for separation constraints.

[0096] Cosine constraints use cosine similarity to measure the similarity between anonymized factors. The cosine similarity s between two different anonymized factors is... i，j The calculation method is as follows:

[0097]

[0098] Among them, s i，j h is the cosine similarity between the i-th and j-th anonymization factors. i and h j These are the representations of two different anonymous factors, where ||x||2 is the L2 norm of x.

[0099] The loss function used in cosine constraints is Loss. dis The calculation is as follows:

[0100]

[0101] Among them, Loss dis Let m represent the first loss function, m be the number of anonymous relation factors, and s be the number of anonymous relation factors. i，j Let i be the cosine similarity between anonymous relation factors i and j. It is worth noting that the above formula should satisfy i≠j.

[0102] Furthermore, in this embodiment of the application, extracting a k-hop subgraph from the geographical encyclopedia knowledge graph based on valid entities includes:

[0103] Step S1: Initialize the first candidate entity set based on the valid entities, and initialize the candidate fact set to an empty set;

[0104] Step S2: Query the geographic knowledge base for triples with entities in the first candidate entity set as head or tail entities, add the triples to the candidate fact set, and add the entities in the triples to the first candidate entity set.

[0105] Step S3: Repeat step S2 k times to obtain a k-jump subgraph.

[0106] This application starts with entities extracted from text and extracts a k-hop subgraph from a geographical encyclopedia knowledge graph. By combining the extracted valid entities, text, and social user entities with the representation of anonymous relationships and the k-hop subgraph extracted from the geographical encyclopedia knowledge graph, a user-centric joint graph of users, text, and knowledge graph is constructed.

[0107] In this embodiment of the application, (1) the effective entities extracted and connected from all the texts published by the user are used as the first candidate entity set, and the candidate fact set is initialized to an empty set; (2) all triples (i.e., neighbor triples) with the entities of the first candidate entity set as the head entity or tail entity are queried from the knowledge base, these triples are added to the candidate fact set, and the entities involved in these triples are added to the first candidate entity set; (3) repeat (2) k times to obtain a k-hop association subgraph.

[0108] Furthermore, in this embodiment of the application, a joint graph of the first social user is constructed based on the entities, valid text, valid entities, representations of anonymous relationships, and k-hop subgraphs of the first social user, including:

[0109] Connect the entities and valid text of the first social user according to the relationship between the entities and valid text of the first social user, wherein the relationship between the entities and valid text of the first social user includes posting relationship and forwarding relationship;

[0110] Connect the valid text and the valid entities in the valid text according to the representation of the anonymous relationship between the valid entities and the corresponding valid text;

[0111] Connect all valid entities corresponding to the valid text using association relationships, and connect the valid entities and entities in the k-hop subgraph according to the k-hop subgraph to obtain the joint graph of the first social user.

[0112] This application treats users and user-posted text as entities. It combines extracted valid entities, text entities, user entities, representations of anonymous relationships, and k-hop subgraphs extracted from a geographical encyclopedia knowledge graph to construct a user-centric joint graph of users, text, and knowledge graphs.

[0113] In the embodiments of this application, such as Figure 3As shown, firstly, user entities and user tweet entities with valid entities are connected according to the relationships between users and their texts. These relationships generally include two types: tweeting and retweeting. A tweeting relationship indicates that a user posted a tweet, while a retweeting relationship indicates that a user retweeted a tweet posted by another user. Then, tweets and entities within them are connected using calculated anonymous relationships. All entities within the same tweet are connected using "related" relationships. Finally, the entities involved in a tweet and their k-hop subgraphs are connected through the entities involved in the tweet, resulting in a joint graph of users, tweets, and the knowledge graph.

[0114] Furthermore, in this embodiment of the application, the location of the first social user is predicted using a progressive relational graph neural network based on the joint graph, including:

[0115] The query is constructed based on the entities and relationships of the first social user, where "user is located" is used as a relationship;

[0116] The joint graph and the query are input into the progressive relational graph neural network to obtain the inverse triples of the triples in the candidate fact set, and the inverse triples are added to the joint graph.

[0117] The second candidate entity set is initialized to a set of entities containing the first social user, and the edge set is initialized to an empty set;

[0118] According to the joint graph, select triples starting from entities in the second candidate entity set and add them to the edge set. Then add the entities involved in the triples in the edge set to the second candidate entity set. Calculate the message representation of the entities in the second candidate entity set. Iterate multiple times until all entities in the joint graph are added to the second candidate entity set and the message representation of all entities in the second candidate entity set is obtained.

[0119] Filter out candidate entities that are not geographical entities from the second candidate entity set, calculate the score of the filtered candidate entities according to the message representation, and select the candidate entity with the highest score as the first social user location.

[0120] This application enhances the joint text entity-knowledge graph by adding reverse edges. Using the user entity as the query head entity, it employs a Progressive Relational Graph Neural Network (PRGNN) to infer the user's possible geographical location. Based on the labels in the dataset, this application adds a special relation "user located at" between the user and the target location, using it as a supervisory signal to train the model. The inference model constructs queries based on the first social user entity and the "user located at" relation.

[0121] Progressive relational graph neural networks (PRNNs) are a special type of inference method based on graph neural networks. They use graph neural networks to encode sequences of relational directed graphs (r-digraphs) for inference, with a typical example being the relational digraphgraph neural network (RED-GNN). PRNNNs offer advantages such as good inference performance, good generalization ability, and strong interpretability.

[0122] The overall structure of the Progressive Graph Neural Network (PRGNN) used in this application is as follows: Figure 4 As shown, it is divided into two main parts: an encoder and a decoder. The encoder consists of 3-6 basic PRGNN blocks connected in series. Each basic PRGNN block contains an r-digraph sampler, a GNN layer, and a GRU-based entity representation update module. The user's joint graph is input into the progressive relational graph neural network for prediction and inference, obtaining scores for different locations, and selecting the highest score as the user's location.

[0123] In this embodiment, after inputting the joint graph and query into the Progressive Graph Neural Network (PRGNN), the PRGNN adds inverse triples of all existing triples in the candidate fact set to the k-hop subgraph portion of the association graph. For example, if there is already a triple (a, r, b), then an inverse triple (b, r) needs to be added. -1 ,a).

[0124] The R-digraph sampler of the Progressive Relational Graph Neural Network (PRGNN) used in this application can generate r-digraph graph sequences. Before all r-digraph samplers operate, the second candidate entity set is first initialized to a set containing only user entities from the question triples, and the edge set is initialized to an empty set. Each time the sampler samples the r-digraph, it selects triples originating from entities in the second candidate entity set, adds them to the edge set, and then adds all entities involved in the triples in the edge set to the second candidate entity set.

[0125] Understandably, each r-digraph sampler takes a second set of candidate entities and a set of edges as input, and outputs an expanded set of second candidate entities and a set of edges following the workflow. Throughout the progressive relational graph neural network, except for the first sampler, each sampler takes the output of the previous sampler as input.

[0126] In this application, the graph expansion process of adding entities involved in all triples in the edge set to the second candidate entity set and the computation of entity message representations are performed alternately. One round of expansion is performed, followed by one round of message representation computation. This alternation is repeated continuously until all entities in the joint graph are added to the second candidate entity set, and the message representations of all entities in the second candidate entity set are obtained.

[0127] The Progressive Relational Graph Neural Network (PRGNN) used in this application is an artificial neural network that works on graph structures, generally consisting of a message passing function and an aggregation mechanism. The GNN layer used in PRGNN is also a message passing network, but the initial representation of all entities is initialized to a zero vector, and each relation (i.e., the type of edge) has its own unique representation.

[0128] This application selects a summation operation with attention as the message passing function, which can be described as follows:

[0129] a m =sigmoid(W1[h q ||h h ||h r ]+b1)

[0130] h h，r =W2(a m h h +(1-a m )h r )+b2

[0131] Among them, a m It is the combined attention value, h h，rThis represents the message; sigmoid is the sigmoid activation function; W1 and W2 are weight matrices; b1 and b2 are offset vectors; h... h h is the head entity representation of the triple corresponding to the edge. r h is the representation of the relation of the triples corresponding to the edges. q It is the representation of the relation "user is located" in the entire problem triple <user, user is located, target location>, h h and h r Used to calculate the message body, h q Used to calculate h t and h r Weight allocation.

[0132] The aggregation mechanism here combines multiple aggregation methods, including attention aggregation, min pooling aggregation, max pooling aggregation, and average pooling aggregation.

[0133] Attention aggregation is defined as follows:

[0134] c h，r，q =W3[h h，r ||h q ]

[0135] a h，r，q =softmax(c h，r，q )

[0136]

[0137] Max pooling aggregation is:

[0138] h max =max(h h，r )

[0139] Minimum pooling aggregation is:

[0140] h min =min(h) h，r )

[0141] Average pooling aggregation is:

[0142]

[0143] The method for combining various aggregation methods is as follows:

[0144]

[0145] Among them, c h，r，q It is the temporary attention value of the attention convergence part, h h，r It represents a message, h q It is a representation of the relationship "the user is located in", ah，r，q It is the attention value of the attention aggregation part, softmax is the softmax activation function, h a This is the attention aggregation representation obtained from attention aggregation computation, where o is the central entity of this aggregation, (h, r, o) are the neighbor triples of the entity with o as the tail, N(o) is the set of neighbor triples of the entity with o as the tail, and h max It is the maximum aggregation representation obtained by max pooling aggregation calculation, where max is the element-wise maximum value operation, and h is the maximum aggregation value. min It is the minimum aggregation representation obtained by minimum pooling aggregation calculation. min is the element-wise minimum operation, h avg It is average pooling aggregation, where n is the length of the set of neighbor triplets of the entity ending in o. W3 and W4 are the aggregated message representations of entity o.

[0146] This application uses the aggregated entity representation generated by the GNN part to update the representation of each candidate entity based on the GRU entity representation update module.

[0147] This application represents the aggregated message of entity o. As the current sequence input to the GRU, the old message representation of entity o is used. As a historical hidden representation, the new message representation of entity o. The calculation method is as follows:

[0148]

[0149] For the decoder part, this application uses a mask to block out candidate entities that are not geographic entities (the score of the blocked candidate entities is assigned a value of 0), and then uses a fully connected layer (fclayer) to calculate the score for the remaining entities. The score of each candidate entity is calculated as follows:

[0150] score = f(s) q ,q,o)=W5h o

[0151] Where, score is the score of the candidate entity, f(s) q (q, o) represents the triples (s, q, o) output by the progressive relational graph neural network. q The scores of (q, o), W5 is the weight matrix, h o This represents the entity o output by the last GNN layer.

[0152] This application selects the candidate entity with the highest score as the final predicted location, and can also query other information about the entity, such as latitude and longitude coordinates, through the wiki geographic knowledge graph.

[0153] Furthermore, in this embodiment of the application, before predicting the location of the first social user using a progressive relational graph neural network based on the joint graph, the following steps are included:

[0154] Get a collection of texts posted by multiple social media users, as well as the actual locations of these users;

[0155] Construct a joint graph for each social user based on the text set;

[0156] Based on the joint graph and the actual location, the progressive relational graph neural network is trained using a joint loss function;

[0157] The joint loss function is expressed as:

[0158] Loss = Loss prd +λLoss dis

[0159] Where Loss represents the joint loss function, Loss dis Let Loss represent the first loss function. prd Let λ represent the second loss function, and let λ represent the preset combination coefficients.

[0160] The first loss function is expressed as:

[0161]

[0162] Among them, Loss dis Let m represent the first loss function, m be the number of anonymous relation factors, and s be the number of anonymous relation factors. i，j Let i be the cosine similarity between anonymous relation factor i and anonymous relation factor j;

[0163] The second loss function is expressed as:

[0164]

[0165] Among them, Loss prd For the second loss function, s q For social users, q represents the relationship "user is located", o q For social users q The actual location of G1 for social users q And the corresponding actual position o q The triple (s) consisting of the relation "user is located" and q q , q, o q The set of f(s) q , q, o q E represents the score of a social user's actual location. sq For social usersq The set of all filtered candidate entities, f(s) q (q, o) represents the triples (s, q, o) output by the progressive relational graph neural network. q The score of , q, o).

[0166] The parameters of the anonymized relation representation and the anonymized relation factor representation in this application are optimized using a multi-label, multi-class log loss function, and the PRGNN-based prediction module in this application also uses a multi-label, multi-class log loss function as the objective function, i.e.:

[0167]

[0168] Among them, Loss prd For the second loss function, s q For social users, q represents the relationship "user is located", o q For social users q The actual location of G1 for social users q And the corresponding actual position o q The triple (s) consisting of the relation "user is located" and q q , q, o q The set of f(s) q , q, o q E represents the score of a social user's actual location. sq For social users q The set of all filtered candidate entities, f(s) q (q, o) represents the triples (s, q, o) output by the progressive relational graph neural network. q The score of , q, o).

[0169] The final loss of the model optimization is:

[0170] Loss = Loss prd +λLoss dis

[0171] Where Loss represents the joint loss function, Loss dis This represents the first loss function, i.e., the loss function used for cosine constraints. prd This represents the second loss function, namely the multi-label multi-class log loss function, and λ represents the preset combination coefficient.

[0172] The optimizer for the model was Adam, with a learning rate of 0.001.

[0173] Figure 5 This is a schematic diagram of the structure of a social user location prediction device based on a pre-trained language model and knowledge graph, provided in Embodiment 2 of this application.

[0174] like Figure 5 As shown, this social user location prediction device based on a pre-trained language model and knowledge graph includes:

[0175] The acquisition module 10 is used to acquire a set of valid texts and a set of valid entities published by the first social user, wherein the set of valid texts includes multiple valid texts and each valid text includes at least one valid entity;

[0176] The anonymous relation representation module 20 is used to encode the effective text and effective entities using a pre-trained language model to obtain the semantic representation of the effective text and the semantic representation of the effective entities, and to calculate the representation of the anonymous relation between the effective entities and the corresponding effective text based on the semantic representation of the effective text and the semantic representation of the effective entities.

[0177] Module 30 is used to extract k-hop subgraphs from the geographical encyclopedia knowledge graph based on valid entities, and to construct a joint graph of the first social user based on the entities, valid text, valid entities, the representation of anonymous relations, and the k-hop subgraphs of the first social user.

[0178] Prediction module 40 is used to predict the location of the first social user based on the joint graph using a progressive relational graph neural network.

[0179] The social user location prediction device based on a pre-trained language model and knowledge graph according to embodiments of this application includes an acquisition module for acquiring a set of valid texts and a set of valid entities published by a first social user, wherein the set of valid texts includes multiple valid texts and each valid text includes at least one valid entity; an anonymous relation representation module for encoding the valid texts and valid entities using a pre-trained language model to obtain semantic representations of the valid texts and valid entities, and calculating the representation of anonymous relations between valid entities and corresponding valid texts based on the semantic representations of the valid texts and valid entities; a construction module for extracting k-hop subgraphs from a geographical encyclopedia knowledge graph based on the valid entities, and constructing a joint graph of the first social user based on the entities, valid texts, valid entities, the representation of anonymous relations, and the k-hop subgraphs of the first social user; and a prediction module for predicting the location of the first social user using a progressive relational graph neural network based on the joint graph. Therefore, this method can solve the technical problem that existing social user location information prediction methods require a large amount of training data and rely on users' self-provided location descriptions. By extracting entities from the text posted by social users and combining them with external geographic knowledge graphs, a user-centric joint graph of user, text, and knowledge graph is constructed. Then, a pre-trained language model and progressive relational graph neural network are used to predict the user's location. This method can perform location reasoning for users who lack self-provided location descriptions, explore the potential relationship between users and locations, has strong generalization ability, and can perform interpretable reasoning.

[0180] To implement the above embodiments, this application also proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the social user location prediction method based on pre-trained language models and knowledge graphs described in the above embodiments.

[0181] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the social user location prediction method based on a pre-trained language model and knowledge graph as described in the above embodiments.

[0182] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0183] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0184] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0185] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0186] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0187] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0188] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0189] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A method for predicting the location of social users based on pre-trained language models and knowledge graphs, characterized in that, Includes the following steps: Obtain the set of valid texts and the set of valid entities published by the first social user, wherein the set of valid texts includes multiple valid texts and each valid text includes at least one valid entity; by using an end-to-end entity linking method to connect named entities to entities in the geographic knowledge base, and filtering out named entities without linking relationships, the set of valid entities is obtained; The effective text and the effective entity are encoded using a pre-trained language model to obtain the semantic representation of the effective text and the semantic representation of the effective entity. Based on the semantic representation of the effective text and the semantic representation of the effective entity, the representation of the anonymous relationship between the effective entity and the corresponding effective text is calculated. Based on the valid entities, a k-jump subgraph is extracted from the geographical encyclopedia knowledge graph. A joint graph of the first social user is constructed based on entities with geographical coordinates in the geographical knowledge base, the valid text, the valid entities, the representation of anonymous relationships, and the k-jump subgraph. The method for determining the k-jump subgraph includes: Step S1: Initializing a first candidate entity set based on valid entities, and initializing the candidate fact set to an empty set; Step S2: Querying from the geographical knowledge base for triples with entities in the first candidate entity set as head or tail entities, adding the triples to the candidate fact set, and adding the entities in the triples to the first candidate entity set; Step S3: Repeating step S2 k times to obtain the k-jump subgraph. Based on the joint graph, the location of the first social user is predicted using a progressive relational graph neural network. Specifically, a query is constructed based on the entities and relations of the first social user, with "user located at" as a relation. The joint graph and the query are input into the progressive relational graph neural network to obtain inverse triples from the candidate fact set, and these inverse triples are added to the joint graph. The second candidate entity set is initialized to contain entities of the first social user, and the edge set is initialized to an empty set. Based on the joint graph, triples originating from entities in the second candidate entity set are added to the edge set, and entities involved in the triples in the edge set are added to the second candidate entity set. The message representation of the entities in the second candidate entity set is calculated, and this process is iterated multiple times until all entities in the joint graph are added to the second candidate entity set, and the message representation of all entities in the second candidate entity set is obtained. Candidate entities in the second candidate entity set that are not geographical entities are filtered, and the scores of the filtered candidate entities are calculated based on the message representations. The candidate entity with the highest score is selected as the location of the first social user.

2. The method as described in claim 1, characterized in that, The acquisition of the set of valid texts and the set of valid entities published by the first social user includes: Obtain the text set published by the first social user, extract named entities from the text set, link the entities to obtain a valid entity set, and obtain the text corresponding to the valid entities from the text set to obtain a valid text set.

3. The method as described in claim 2, characterized in that, The step of calculating the representation of the anonymous relationship between the valid entity and the corresponding valid text based on the semantic representation of the valid text and the semantic representation of the valid entity includes: Set a preset number of anonymous relation factors, and use a first loss function to separate the anonymous relation factors; The semantic representations of the trained anonymous relation factors, the semantic representations of the effective entities, and the semantic representations of the effective text corresponding to the effective entities are concatenated, and the attention value of the effective entities to the anonymous relation factors is calculated based on the concatenation result. The trained semantic representations of the anonymous relation factors are obtained by initializing and training the semantic representations of the anonymous relation factors. Based on the attention values ​​of the effective entities to the anonymous relation factors and the semantic representations of the trained anonymous relation factors, the representation of the anonymous relation between the effective entities and the corresponding effective text is calculated.

4. The method as described in claim 3, characterized in that, The step of constructing a joint graph of the first social user based on the entities of the first social user, the valid text, the valid entities, the representation of the anonymous relationship, and the k-hop subgraph includes: Connect the entities and valid text of the first social user according to the relationship between the entities and valid text of the first social user, wherein the relationship between the entities and valid text of the first social user includes posting relationship and forwarding relationship; Connect the valid text and the valid entities in the valid text according to the representation of the anonymous relationship between the valid entities and the corresponding valid text; Connect all valid entities corresponding to the valid text using association relationships, and connect the valid entities with the entities in the k-hop subgraph according to the k-hop subgraph to obtain the joint graph of the first social user.

5. The method as described in claim 4, characterized in that, Before predicting the location of the first social user using a progressive relational graph neural network based on the joint graph, the process includes: Obtain a collection of texts posted by multiple social media users, as well as the actual locations of the multiple social media users; Based on the text set, construct a joint graph for each social user; The progressive relational graph neural network is trained using a joint loss function based on the joint graph and the actual location. The joint loss function is expressed as: in, Represented as a joint loss function, Denotes the first loss function. This represents the second loss function. Indicates the preset combination coefficients; The first loss function is expressed as: in, This represents the first loss function, where m is the number of anonymous relation factors. Let i be the cosine similarity between anonymous relation factor i and anonymous relation factor j; The second loss function is expressed as: in, For the second loss function, For social users, For the relationship "user is located", For social users The actual location, For social users and the corresponding actual location The relationship between "users located" The triplet The set, A score representing the actual location of social users. For social users The set of all filtered candidate entities. Triples output by an asymptotic graph neural network The score.

6. A social user location prediction device based on a pre-trained language model and knowledge graph, characterized in that, include: The acquisition module is used to acquire a set of valid texts and a set of valid entities published by the first social user. The set of valid texts includes multiple valid texts, and each valid text includes at least one valid entity. The set of valid entities is obtained by connecting named entities to entities in the geographic knowledge base using an end-to-end entity linking method and filtering out named entities without linking relationships. The anonymous relation representation module is used to encode the effective text and the effective entity using a pre-trained language model to obtain the semantic representation of the effective text and the semantic representation of the effective entity, and to calculate the representation of the anonymous relation between the effective entity and the corresponding effective text based on the semantic representation of the effective text and the semantic representation of the effective entity. A construction module is used to extract k-hop subgraphs from the geographical encyclopedia knowledge graph based on the valid entities, and to construct a joint graph of the first social user based on entities with geographical coordinates in the geographical knowledge base, the valid text, the valid entities, the representation of the anonymous relationship, and the k-hop subgraphs; wherein, the method for determining the k-hop subgraphs includes: step S1: initializing a first candidate entity set based on valid entities, and initializing the candidate fact set as an empty set; step S2: querying the geographical knowledge base for triples with entities in the first candidate entity set as head entities or tail entities, adding the triples to the candidate fact set, and adding the entities in the triples to the first candidate entity set; step S3: repeating step S2 k times to obtain the k-hop subgraphs; The prediction module is used to predict the location of a first social user using a progressive relational graph neural network based on the joint graph. Specifically, it constructs a query based on the entities and relationships of the first social user, with "user located" as a relationship; inputs the joint graph and the query into the progressive relational graph neural network to obtain inverse triples from the candidate fact set, and adds these inverse triples to the joint graph; initializes the second candidate entity set to a set containing entities of the first social user, and initializes the edge set to an empty set; based on the joint graph, it selects triples originating from entities in the second candidate entity set and adds them to the edge set, then adds entities involved in the triples in the edge set to the second candidate entity set, calculates the message representation of the entities in the second candidate entity set, iterates multiple times until all entities in the joint graph are added to the second candidate entity set, and obtains the message representation of all entities in the second candidate entity set; filters candidate entities in the second candidate entity set that are not geographical entities, calculates the score of the filtered candidate entities based on the message representation, and selects the candidate entity with the highest score as the location of the first social user.

7. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-5.

Images