Heterogeneous point of interest matching method and device based on graph neural network

Abstract

This application relates to a method and apparatus for heterogeneous interest point matching based on graph neural networks. The method includes: acquiring a first set of interest points and a second set of interest points within a target geographical area; constructing a first interest point map based on the first set of interest points and a second interest point map based on the second set of interest points; selecting multiple pairs of preliminary interest point matching pairs from the first and second sets of interest points; labeling these pairs of preliminary interest point matching pairs to obtain multiple pairs of seed interest point matching pairs; iteratively training a graph neural network model based on the first and second interest point maps and the multiple pairs of seed interest point matching pairs to obtain a trained graph neural network model; processing the first and second interest point maps using the trained model; and determining all interest point matching pairs in the first and second sets of interest points based on the processing results. This application can improve the accuracy and speed of heterogeneous interest point matching.

Description

Technical Field

[0001] This application relates to the field of electronic maps, and in particular to a method, apparatus, computer device, and storage medium for matching heterogeneous points of interest based on graph neural networks. Background Technology

[0002] Points of Interest (POIs), typically containing information such as name, address, latitude and longitude, and category, are the most important content of online electronic maps and the foundation of internet location services. Because POI data on the internet comes from diverse sources and involves varying collection and processing methods, these data exhibit differences in spatial location, attribute information, and richness. Therefore, effectively eliminating these inconsistencies and organizing them into an accurate and usable dataset has become a current research hotspot.

[0003] Point of interest (POI) matching is the process of identifying relationships between heterogeneous data sources that differ in their descriptions but correspond to the same point of interest in the real world. By using matching methods to fuse information from different POI data sources, the information contained within the POI data can be enriched, and inconsistencies between data points can be eliminated.

[0004] Currently, the main methods for heterogeneous point of interest matching are as follows:

[0005] 1. Unsupervised similarity computation scheme:

[0006] Interest points are extracted from two heterogeneous interest point datasets. The similarity between the names and addresses of these two interest points is calculated. Similarity algorithms include edit distance and TF-IDF (term frequency–inverse document frequency). The calculated name and address text similarities are weighted to calculate the overall similarity, which is used as the similarity score between the two interest points. When the score is higher than a certain threshold, the two interest points are considered to have a matching relationship, thus enabling the matching of heterogeneous interest point datasets.

[0007] 2. Text matching scheme based on traditional machine learning models:

[0008] From two heterogeneous interest point data sets, interest point pairs with matching relationships are extracted as training data. Features are constructed by calculating the name text similarity, physical distance, and category similarity of the interest point pairs. A text matching model is trained using traditional machine learning methods such as Gradient Boosting Decision Tree (GBDT) and XGBoost to determine whether two interest points have a matching relationship, thereby matching heterogeneous interest point data.

[0009] 3. Text matching scheme based on pre-trained deep learning model:

[0010] Using pairs of interest points with matching relationships as training data, a text matching model is trained by fine-tuning commonly used pre-trained deep models such as BERT (Bidirectional Encoder Representation from Transformers) and A LBERT (A LiteBERT) to determine whether two interest points have a matching relationship.

[0011] The inventors discovered that the above solutions all have some drawbacks in practical applications.

[0012] For example, the disadvantages of the above scheme 1:

[0013] (1) Based on the unsupervised similarity scoring method, the matching effect is poor for scenarios where two points of interest have a real matching relationship but are very different in text.

[0014] (2) Mismatches may occur when two points of interest are textually very similar but do not actually have a matching relationship.

[0015] (3) The threshold for similarity scores is not easy to set.

[0016] The disadvantages of Scheme 2 above:

[0017] (1) A lot of feature engineering work is required to construct features, which is a rather complicated process.

[0018] (2) The model is shallow, has limited expressive power, and has a low ceiling for text matching effect.

[0019] (3) The matching process involves matching one-to-one points of interest, and then traversing all data to match the whole data. The matching efficiency is low, and it is not possible to match directly at the overall data level.

[0020] (4) This method assumes that the points of interest are independent of each other. However, there is a certain spatial relationship between the points of interest. Therefore, the information on the relationship between the points of interest is not used for matching. The information used is limited and the effect is not good.

[0021] The disadvantages of the above scheme 3:

[0022] (1) Pre-trained deep models are generally input with plain text information and have poor compatibility with non-text features.

[0023] (2) Like traditional machine learning models, pre-trained deep models match one-to-one interest points and then traverse all data to match the whole data. The matching efficiency is low and it cannot be matched directly at the overall data level.

[0024] (3) This method assumes that the points of interest are independent of each other. However, there is a certain spatial relationship between the points of interest. Therefore, the information on the relationship between the points of interest is not used for matching. The information used is limited and the effect is not good. Summary of the Invention

[0025] To address the aforementioned shortcomings or drawbacks, this application provides a method, apparatus, computer device, and storage medium for heterogeneous interest point matching based on graph neural networks. The embodiments of this application can improve the accuracy and speed of heterogeneous interest point matching.

[0026] This application provides a heterogeneous interest point matching method based on graph neural networks according to a first aspect. In one embodiment, the method includes:

[0027] Obtain a first set of points of interest and a second set of points of interest within the target geographical area. Construct a first point of interest map based on the first set of points of interest and a second point of interest map based on the second set of points of interest. The first set of points of interest and the second set of points of interest are heterogeneous data.

[0028] Multiple pairs of preliminary interest point matching are selected from the first set of interest points and the second set of interest points. These multiple pairs of preliminary interest point matching are labeled to obtain multiple pairs of seed interest point matching.

[0029] Based on the first interest point map, the second interest point map, and the multiple pairs of seed interest point matching pairs, the graph neural network model is iteratively trained to obtain the trained graph neural network model.

[0030] The first and second interest point maps are processed by a trained graph neural network model, and all interest point matching pairs in the first and second interest point sets are determined based on the processing results.

[0031] In one embodiment, constructing a first interest point map based on a first interest point set includes:

[0032] Obtain the geographic location information of each point of interest in the first set of points of interest;

[0033] Calculate the distance between each pair of points of interest based on their geographical location information in the first set of points of interest.

[0034] The weight of the edge between each pair of interest points is determined based on the distance between each pair of interest points in the first interest point set, and the first interest point map is obtained.

[0035] The weight of the edge between any two points of interest is determined based on the distance between any two points of interest in the first set of points of interest, including:

[0036] When the distance between any two interest points in the first interest point set is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1.

[0037] When the distance between any two interest points in the first interest point set is greater than or equal to a preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0.

[0038] In one embodiment, constructing a second interest point map based on a second interest point set includes:

[0039] Obtain the geographic location information of each point of interest in the second set of points of interest;

[0040] Calculate the distance between each pair of points of interest based on their geographical location information in the second set of points of interest.

[0041] The weight of the edge between each pair of interest points is determined based on the distance between each pair of interest points in the second interest point set, thus obtaining the second interest point map;

[0042] The weight of the edge between any two points of interest is determined based on the distance between them in the second set of points of interest, including:

[0043] When the distance between any two interest points in the second interest point set is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1.

[0044] When the distance between any two interest points in the second interest point set is greater than or equal to a preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0.

[0045] In one embodiment, any training process of a graph neural network model includes:

[0046] The adjacency matrix and node attribute feature matrix of the first interest point map and the second interest point map are respectively input into the graph neural network model to obtain the output data of the graph neural network model. The output data includes the embedding of each interest point in the first interest point map and the second interest point map.

[0047] From the multiple pairs of seed interest point matching pairs, determine the target seed interest point matching pairs for this training, use the target seed interest point matching pairs as the positive matching pairs for this training, and construct the negative matching pairs for this training based on the target seed interest point matching pairs.

[0048] Obtain the embeddings of positive and negative matching pairs from the output data, and calculate the loss for this training based on the embeddings of positive and negative matching pairs.

[0049] Determine whether the conditions for stopping training are met based on the loss.

[0050] If the conditions are met, stop training and use the graph neural network model as the trained graph neural network model.

[0051] If the conditions are not met, update the network parameters of the graph neural network model based on the loss, and then train the updated graph neural network model again.

[0052] In one embodiment, a trained graph neural network model is used to process the first interest point map and the second interest point map, and based on the processing result, all interest point matching pairs in the first interest point set and the second interest point set are determined, including:

[0053] Input the first interest point map into the trained graph neural network model, and obtain the embedding of each interest point in the first interest point set based on the output of the trained graph neural network model;

[0054] The second interest point map is input into the trained graph neural network model, and the embedding of each interest point in the second interest point set is obtained based on the output of the trained graph neural network model.

[0055] Each point of interest in the first set of points of interest is paired with each point of interest in the second set of points of interest to obtain multiple pairs of points of interest;

[0056] Calculate the embedding distance for each pair of interest points; the embedding distance for each pair of interest points refers to the distance between the embeddings of the two interest points included in each pair of interest points.

[0057] Each pair of interest points whose embedding distance is less than a preset threshold is identified as an interest point matching pair.

[0058] In one embodiment, the number of first and second interest point maps is the same and there are multiple such maps; correspondingly, constructing the first interest point map based on the first interest point set and constructing the second interest point map based on the second interest point set includes:

[0059] The target geographical area is divided into multiple spatial grids;

[0060] The latitude and longitude attributes of each point of interest in the first set of points of interest and the second set of points of interest are traversed separately to determine the first subset of points of interest and the second subset of points of interest corresponding to each spatial grid.

[0061] Based on the first and second subsets of interest points corresponding to each spatial grid, construct the corresponding first and second interest point maps for each spatial grid.

[0062] In one embodiment, a trained graph neural network model is used to process the first interest point map and the second interest point map, and based on the processing result, all interest point matching pairs in the first interest point set and the second interest point set are determined, including:

[0063] Input the first interest point map and the second interest point map corresponding to each spatial grid into the trained graph neural network model respectively. Based on the output of the trained graph neural network model, obtain the embedding of each interest point in the first interest point subset corresponding to each spatial grid, and the embedding of each interest point in the second interest point subset corresponding to each spatial grid.

[0064] Pair the first and second subsets of interest points corresponding to each spatial grid to obtain the set of interest point pairs for each spatial grid.

[0065] Calculate the embedding distance of each interest point pair in the set of interest point pairs for each spatial grid; the embedding distance of each interest point pair refers to the distance between the embeddings of the two interest points included in each interest point pair;

[0066] Each pair of interest points in the set of interest point pairs of each spatial grid, whose embedding distance is less than a preset threshold, is identified as an interest point matching pair.

[0067] This application provides a heterogeneous interest point matching device based on a graph neural network according to a second aspect. In one embodiment, the device includes:

[0068] The map construction module is used to obtain a first set of points of interest and a second set of points of interest within the target geographical area, construct a first point of interest map based on the first set of points of interest, and construct a second point of interest map based on the second set of points of interest; the first set of points of interest and the second set of points of interest are heterogeneous data.

[0069] The seed matching pair construction module is used to select multiple pairs of preliminary interest point matching pairs from the first interest point set and the second interest point set, and to label the multiple pairs of preliminary interest point matching pairs to obtain multiple pairs of seed interest point matching pairs.

[0070] The model training module is used to iteratively train the graph neural network model based on the first interest point map, the second interest point map, and the multiple pairs of seed interest point matching pairs to obtain the trained graph neural network model.

[0071] The interest point matching module is used to process the first interest point map and the second interest point map using a trained graph neural network model, and to determine all interest point matching pairs in the first interest point set and the second interest point set based on the processing results.

[0072] This application provides a computer device according to a third aspect, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods.

[0073] According to a fourth aspect, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods.

[0074] Compared with the prior art, the embodiments of this application can bring the following beneficial effects:

[0075] Traditional machine learning and pre-trained deep models make the assumption that two interest points are unrelated when dealing with the problem of deduplication of interest points. The embodiments of this application, however, assume that two interest points are related. Based on this, heterogeneous interest point matching is performed by combining the attribute features of the interest points themselves and the spatial positional relationship between interest points.

[0076] Specifically, the inventors discovered that mutually matching pairs of interest points (POPs) typically have similar neighbors. Based on this discovery, they chose a graph convolutional neural network (GCNN) model with a graph structure for heterogeneous POP matching. The model inputs the textual information of the POP names and the spatial relationships between POPs. By transmitting information about aggregated neighbor nodes, the GCNN has the ability to identify isomorphic graphs and can better identify similar neighbor structures. In an POP graph, matching POP pairs usually have similar neighbors; that is, the neighboring POPs of two matching POPs often contain other equivalent POP pairs. The node embedding representation is generated by aggregating neighbor information, making it easier to match other equivalent POP pairs among neighboring nodes. This approach can more effectively solve the matching problem between two heterogeneous POP graphs.

[0077] On the other hand, traditional machine learning and pre-trained deep models can generally only be trained based on seed interest point pairs. However, seed interest point pairs require manual annotation, which is costly to obtain, and therefore the number is generally small, leading to insufficient training. The embodiments of this application utilize graph convolutional neural network models, which allow all data to participate in training, thus enabling more thorough model training and resulting in better model performance.

[0078] Furthermore, traditional point-of-interest (POI) matching methods match POIs one-to-one and then traverse the entire graph for graph-level matching, resulting in low matching efficiency and the inability to perform matching directly at the graph level. This case, however, utilizes a graph convolutional neural network model to achieve layer-level matching, improving both performance and efficiency. Attached Figure Description

[0079] Figure 1 This is a flowchart illustrating a heterogeneous interest point matching method based on a graph neural network in one embodiment.

[0080] Figure 2 This is a structural block diagram of a heterogeneous interest point matching device based on a graph neural network in one embodiment;

[0081] Figure 3 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0082] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0083] This application provides a heterogeneous interest point matching method based on graph neural networks. In one embodiment, the method includes as follows: Figure 1 The steps shown are described below, and the method will be explained in detail below.

[0084] S110: Obtain a first set of points of interest (POIs) and a second set of POIs within the target geographical area. Construct a first POI map based on the first POI set and a second POI map based on the second POI set. The first and second POI sets are heterogeneous data.

[0085] S120: Select multiple pairs of preliminary interest point matching pairs from the first interest point set and the second interest point set, and label these multiple pairs of preliminary interest point matching pairs to obtain multiple pairs of seed interest point matching pairs;

[0086] S130: Iteratively train the graph neural network model based on the first interest point map, the second interest point map, and the multiple pairs of seed interest point matching pairs to obtain a trained graph neural network model;

[0087] S140: Process the first and second interest point maps using the trained graph neural network model, and determine all interest point matching pairs in the first and second interest point sets based on the processing results.

[0088] This embodiment offers the following advantages over existing technologies:

[0089] Traditional machine learning and pre-trained deep models make the assumption that two interest points are unrelated when dealing with the problem of deduplication of interest points. The embodiments of this application, however, assume that two interest points are related. Based on this, heterogeneous interest point matching is performed by combining the attribute features of the interest points themselves and the spatial positional relationship between interest points.

[0090] Specifically, the inventors discovered that mutually matching pairs of interest points (POPs) typically have similar neighbors. Based on this discovery, they chose a graph convolutional neural network (GCNN) model with a graph structure for heterogeneous POP matching. The model inputs the textual information of the POP names and the spatial relationships between POPs. By transmitting information about aggregated neighbor nodes, the GCNN has the ability to identify isomorphic graphs and can better identify similar neighbor structures. In an POP graph, matching POP pairs usually have similar neighbors; that is, the neighboring POPs of two matching POPs often contain other equivalent POP pairs. The node embedding representation is generated by aggregating neighbor information, making it easier to match other equivalent POP pairs among neighboring nodes. This approach can more effectively solve the matching problem between two heterogeneous POP graphs.

[0091] On the other hand, traditional machine learning and pre-trained deep models can generally only be trained based on seed interest point pairs. However, seed interest point pairs require manual annotation, which is costly to obtain, and therefore the number is generally small, leading to insufficient training. The embodiments of this application utilize graph convolutional neural network models, which allow all data to participate in training, thus enabling more thorough model training and resulting in better model performance.

[0092] Furthermore, traditional point-of-interest (POI) matching methods match POIs one-to-one and then traverse the entire graph for graph-level matching, resulting in low matching efficiency and the inability to perform matching directly at the graph level. This case, however, utilizes a graph convolutional neural network model to achieve layer-level matching, improving both performance and efficiency.

[0093] In one embodiment, the above-mentioned construction of a first point of interest (POI) map based on a first set of POIs includes: obtaining the geographical location information of each POI in the first POI set; calculating the distance between each pair of POIs based on their geographical location information; and determining the weight of the edge between each pair of POIs based on their distance, thereby obtaining the first POI map. The distance between two POIs refers to spatial distance (or physical distance), such as 200 meters, 100 meters, 50 meters, etc.

[0094] The aforementioned determination of the weight of the edge between any two interest points based on the distance between any two interest points in the first interest point set includes: when the distance between any two interest points in the first interest point set is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1; when the distance between any two interest points in the first interest point set is greater than or equal to the preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0. The preset threshold can be set according to actual needs, such as 50 meters, etc., and this embodiment does not specifically limit it.

[0095] In one embodiment, the above-described construction of a second point of interest (POI) map based on a second set of POIs includes: obtaining the geographic location information of each POI in the second POI set; calculating the distance between each pair of POIs based on their geographic location information; and determining the weight of the edge between each pair of POIs based on their distance, thereby obtaining the second POI map. The preset threshold is the same as the preset threshold in the above embodiment; please refer to the description of the above embodiment for details.

[0096] The aforementioned determination of the weight of the edge between any two interest points based on the distance between any two interest points in the second set of interest points includes: when the distance between any two interest points in the second set of interest points is less than a preset threshold, determining that an edge relationship is formed between the two interest points and setting the weight of the edge between the two interest points to 1; when the distance between any two interest points in the second set of interest points is greater than or equal to the preset threshold, determining that no edge relationship is formed between the two interest points and setting the weight of the edge between the two interest points to 0.

[0097] In one embodiment, any training process of a graph neural network model includes:

[0098] The adjacency matrix and node attribute feature matrix of the first interest point map and the second interest point map are respectively input into the graph neural network model to obtain the output data of the graph neural network model. The output data includes the embedding of each interest point in the first interest point map and the second interest point map.

[0099] From the multiple pairs of seed interest point matching pairs, determine the target seed interest point matching pairs for this training, use the target seed interest point matching pairs as the positive matching pairs for this training, and construct the negative matching pairs for this training based on the target seed interest point matching pairs.

[0100] Obtain the embeddings of positive and negative matching pairs from the output data, and calculate the loss for this training based on the embeddings of positive and negative matching pairs.

[0101] Determine whether the conditions for stopping training are met based on the loss.

[0102] If the conditions are met, stop training and use the graph neural network model as the trained graph neural network model.

[0103] If the conditions are not met, update the network parameters of the graph neural network model based on the loss, and then train the updated graph neural network model again.

[0104] The first interest point map or the second interest point map is constructed by using the interest points in the first interest point set or the second interest point set as nodes.

[0105] Accordingly, in one embodiment, the above-described processing of the first and second interest point maps using a trained graph neural network model, and the determination of all interest point matching pairs in the first and second interest point sets based on the processing results, includes:

[0106] The first interest point map is input into a trained graph neural network model. Based on the output of the trained graph neural network model, the embedding of each interest point in the first interest point set is obtained. The second interest point map is input into a trained graph neural network model. Based on the output of the trained graph neural network model, the embedding of each interest point in the second interest point set is obtained. Each interest point in the first interest point set is paired with each interest point in the second interest point set to obtain multiple interest point pairs. The embedding distance of each interest point pair is calculated. The embedding distance of each interest point pair refers to the distance between the embeddings of the two interest points included in each interest point pair. Each interest point pair with an embedding distance less than a preset threshold is determined as an interest point matching pair.

[0107] In another embodiment, the number of first and second interest point maps is the same and there are multiple such maps; correspondingly, the above-described construction of the first interest point map based on the first interest point set and the construction of the second interest point map based on the second interest point set includes:

[0108] The target area is divided into multiple spatial grids; the latitude and longitude attributes of each point of interest in the first and second sets of points of interest are traversed to determine the first and second subsets of points of interest corresponding to each spatial grid; based on the first and second subsets of points of interest corresponding to each spatial grid, a corresponding first and second point of interest map is constructed for each spatial grid.

[0109] This embodiment considers scenarios where the target geographical area is large, such as the entirety of China. In such cases, the data to be matched (i.e., point of interest matching) is the entire dataset of China, resulting in a massive amount of data. Constructing two graphs (point of interest maps) directly from all this data would create enormous node attribute feature matrices and adjacency matrices for each graph, placing extremely high demands on computational resources. Therefore, with limited computational resources, the two graphs are divided into many smaller graphs based on a spatial grid. Deduplication is then performed on each smaller graph, reducing the number of nodes and adjacency matrices in each smaller graph, allowing data matching to be performed with less computational resources. A specific method is as follows:

[0110] Based on China's geospatial coordinates, the area is divided into 1-kilometer by 1-kilometer square spatial grids from the westernmost to the easternmost and from the northernmost to the southernmost point (the grid size can be flexibly adjusted according to actual needs). Each of the four vertices of the grid has corresponding latitude and longitude coordinates, and each point of interest has latitude and longitude attributes. By traversing all points of interest according to latitude and longitude, the points of interest can fall into the corresponding grids. In this way, each grid will fall into the nearest points of interest, and subsequent matching of points of interest only needs to be performed in each spatial grid.

[0111] Accordingly, in one embodiment, the above-described processing of the first and second interest point maps using a trained graph neural network model, and the determination of all interest point matching pairs in the first and second interest point sets based on the processing results, includes:

[0112] The first and second interest point maps corresponding to each spatial grid are input into a trained graph neural network model. Based on the output of the trained graph neural network model, the embeddings of each interest point in the first interest point subset and the second interest point subset corresponding to each spatial grid are obtained. The first and second interest point subsets corresponding to each spatial grid are paired to obtain the interest point pair set for each spatial grid. The embedding distance of each interest point pair in the interest point pair set of each spatial grid is calculated. The embedding distance of each interest point pair refers to the distance between the embeddings of the two interest points included in each interest point pair. Each interest point pair in the interest point pair set of each spatial grid whose embedding distance is less than a preset threshold is identified as an interest point matching pair.

[0113] The above embodiments will be illustrated by a specific application example below.

[0114] This application example is specifically divided into six parts: spatial grid division, seed interest point matching pair acquisition, data preprocessing, feature engineering, model training, and model prediction.

[0115] This application example uses China as the target geographical area. First, based on China's geospatial coordinates, the geographical space of China is divided into 1km x 1km square grids. Then, corresponding interest points are placed into their respective grids to form subgraphs. This application example uses supervised learning to train the graph neural network model. Since it is a supervised learning task, seed interest point matching pairs need to be manually labeled for model training. After obtaining the seed interest point matching pairs, the data needs to be preprocessed, including full-width to half-width conversion, removal of special characters, conversion of uppercase to lowercase, and conversion of traditional Chinese to simplified Chinese, to clean the data. Before training the model, feature engineering is performed to obtain the attribute features of each node in each interest point graph and the adjacency matrix of each interest point graph. The node attribute feature matrix and adjacency matrix of each interest point graph are input into the graph convolutional neural network (GCN). The model is trained by minimizing the loss function through backpropagation to obtain the weight matrix W, which is the parameter that the model needs to learn. After obtaining the parameters W, graph-level interest point matching prediction can be performed through forward propagation.

[0116] The following is an explanation of each of the above parts.

[0117] 1. Spatial grid division

[0118] Since the two interest point maps (which can be simply referred to as maps) that need to be matched are all the data from across China, the total amount of data is enormous. If all the data were used to construct a large map directly, the nodes and adjacency matrix would be extremely large, placing excessive demands on computational resources. Therefore, with limited computational resources, the large map is divided into many smaller maps according to a spatial grid, and the two maps are matched on their respective smaller maps. This reduces the number of nodes and the adjacency matrix, allowing matching to be performed with less computational resources.

[0119] Specific method: According to China's geographic spatial coordinates, from the westernmost to the easternmost point, and from the northernmost to the southernmost point, divide the area into 1 km x 1 km square spatial grids. Each of the four vertices of the grid has corresponding latitude and longitude coordinates, and each point of interest has latitude and longitude attributes. By traversing all points of interest according to latitude and longitude, the points of interest can fall into the corresponding grid. In this way, each grid will fall into the nearest points of interest.

[0120] After performing the above operations on both interest point maps, subsequent interest point matching only needs to be performed in their respective spatial grids.

[0121] 2. Seed Interest Point Matching Pair Acquisition

[0122] This application example requires seed interest point pairs to train the model, so some data needs to be labeled as seed interest point pairs. By traversing all the aforementioned spatial grids and using simple interest point name text similarity (such as edit distance), potentially matching interest point pairs are roughly found in each spatial grid. These are then handed over to the labelers to identify the actual interest point pairs. This method is used to construct seed interest point matching pairs.

[0123] 3. Data Preprocessing

[0124] The names of points of interest may contain special symbols and traditional Chinese characters, requiring preprocessing before constructing the feature input model. Furthermore, to ensure consistency in the distribution of labeled and unlabeled data, both need to undergo the same preprocessing operations. The data preprocessing process includes the following four steps:

[0125] (1) Convert full-width characters to half-width characters

[0126] (2) Remove special characters

[0127] (3) Converting English capital letters to lowercase

[0128] (4) Converting Traditional Chinese to Simplified Chinese

[0129] 4. Feature Engineering

[0130] (1) Attribute characteristics of generated graph nodes

[0131] The input to a Graph Convolutional Neural Network (GCN) includes the graph's topology, i.e., the adjacency matrix, and the attribute features of all nodes in the graph. Each node's attribute feature is a multi-dimensional feature vector. In this application example, a specified algorithm is used to process each interest point into a 512-dimensional embedding vector, which is then used as the attribute feature of that node. This specified algorithm can be any existing algorithm capable of mapping interest points to embedding vectors, and therefore will not be elaborated upon here.

[0132] (2) Generate the edges and adjacency matrices of all subgraph structures.

[0133] This application example treats all points of interest within 50 meters as having edge relationships. All points of interest in a spatial grid are extracted to form a set. A single point of interest is selected from this set, and its distance to all other points in the set (excluding itself) is calculated. Points of interest within 50 meters are then edged to the selected point, with an edge weight of 1. This process is repeated until all points of interest in the set are selected, thus forming the edge relationships for all points of interest in this spatial grid subgraph. According to the definition of a graph structure, the adjacency matrix of this subgraph is obtained. The adjacency matrix is ​​represented as follows:

[0134]

[0135] Finally, by traversing all the spatial grids using the method described above, the adjacency matrix of all spatial grid subgraphs is generated.

[0136] 5. Model Training

[0137] Given two corresponding spatial grid subgraphs of interest points KG1 and KG2, and a set of pre-matched interest point seed pairs S = {(e i1 ,e i2 )} m i＝1 .

[0138] This application example uses node embeddings from a GCN to find new interest point matching pairs. The basic idea is to use a GCN to embed interest points from different graphs into a unified vector space, while aiming to make matched interest points closer together and unmatched interest points farther apart.

[0139] (1) Input to GCN:

[0140] GCNs (Graph-based Networks) are a type of neural network that operates directly on the graph. Their inputs are the node attribute features and adjacency matrix of the graph, and their goal is to output node-level interest point embeddings for subsequent interest point matching. The model uses two two-layer GCNs, each processing one graph (KG), with GCN1 and GCN2 processing KG1 and KG2 respectively. The node attribute features input to the model are derived from feature engineering in step 4; both GCN1 and GCN2 have 512-dimensional node attribute features. The adjacency matrix input to the model is also obtained through four steps of feature engineering. After the node attribute features and adjacency matrix are input into the GCN model, subsequent GCN operations are performed.

[0141] (2) Operations of GCN:

[0142] A GCN model contains multiple GCN layers. This application example uses two layers, with the input H of the l-th layer being... (l) ∈R n×d(l) is a node attribute feature matrix (i.e., all node attribute features), where n is the number of nodes, d (l) The number of features in the l-th layer is the number of features, and the output of the l-th layer is the new feature matrix.

[0143]

[0144] Where σ is the ReLU activation function (used for linear activation transformation), and A is an n*n adjacency matrix. Where I is the identity matrix. yes The degree matrix of the diagonal nodes, W (l) ∈R d(l)×d(l+1) It is the weight matrix between the two layers, used for convolution operations, d (l+1) It is a new dimension.

[0145] (3) Output of GCN

[0146] After passing through two two-layer GCNs, the node feature vectors output by GCN1 and GCN2 have the same dimension, which is a 512-dimensional embedding representation. This embedding representation can be used for subsequent interest point matching.

[0147] (4) Loss function of GCN

[0148] This application example aims to minimize the distance between matched interest points and maximize the distance between unmatched interest points, thus constructing the loss function accordingly. The distance between interest points is the embedding distance between them. For an interest point pair e1 and e2, where e1∈KG1 and e2∈KG2, e1 and e2 are the node embeddings output by GCN1 and GCN2, respectively. The distance between them is calculated as follows:

[0149] D(e1,e2)=||h(e1)-h(e2)||1

[0150] The model is trained by minimizing the following margin-based loss function:

[0151]

[0152] Where, [x]+=max{0,x},S' (e1,e2) The negative matching pair is obtained by randomly replacing an interest point with (e1,e2), and γ is the interval that distinguishes positive and negative matching pairs. The model is trained by minimizing the loss function through backpropagation, updating the weight matrix W in each layer. After several rounds of training, the model can finally learn the weight matrix W to predict the matching of interest point pairs.

[0153] 6. Model Prediction

[0154] This application example is suitable for offline graph-level matching prediction, where the prediction involves finding more new interest point matching pairs within a constructed graph. During training, a weight matrix W is learned. Using input node attribute features and an adjacency matrix, and through GCN operations, each node outputs an embedding representation.

[0155] For a specific output embedding e1∈KG1, calculate its embedding distance with all points of interest in KG2, and select the one with the smallest embedding distance among all points of interest. If this embedding distance is less than a certain threshold, the two points of interest are considered to match; otherwise, they are considered not to match. Following the above method, by traversing all points of interest in KG1 that are not in the seed pair, the corresponding matching points of interest in KG2 can be obtained, thus allowing the matching prediction results to be obtained directly at the layer level.

[0156] Figure 1 This is a flowchart illustrating a heterogeneous interest point matching method based on graph neural networks in one embodiment. It should be understood that, although... Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0157] Based on the same inventive concept, this application also provides a heterogeneous interest point matching device based on graph neural networks. In this embodiment, as... Figure 2 As shown, the heterogeneous interest point matching device based on graph neural networks includes the following modules:

[0158] Map construction module 110 is used to obtain a first set of points of interest and a second set of points of interest within the target geographical area, construct a first point of interest map based on the first set of points of interest, and construct a second point of interest map based on the second set of points of interest; the first set of points of interest and the second set of points of interest are heterogeneous data.

[0159] The seed matching pair construction module 120 is used to select multiple pairs of preliminary interest point matching pairs from the first interest point set and the second interest point set, and to label the multiple pairs of preliminary interest point matching pairs to obtain multiple pairs of seed interest point matching pairs.

[0160] The model training module 130 is used to iteratively train the graph neural network model based on the first interest point map, the second interest point map and the multiple pairs of seed interest point matching pairs to obtain the trained graph neural network model.

[0161] The interest point matching module 140 is used to process the first interest point map and the second interest point map through a trained graph neural network model, and determine all interest point matching pairs in the first interest point set and the second interest point set based on the processing results.

[0162] In one embodiment, when the map construction module is used to construct a first interest point map based on a first interest point set, it is specifically used for:

[0163] Obtain the geographic location information of each point of interest in the first set of points of interest;

[0164] Calculate the distance between each pair of points of interest based on their geographical location information in the first set of points of interest.

[0165] The weight of the edge between each pair of interest points is determined based on the distance between each pair of interest points in the first interest point set, and the first interest point map is obtained.

[0166] Furthermore, when determining the weight of the edge between any two interest points based on the distance between any two interest points in the first interest point set, the graph construction module is specifically used for:

[0167] When the distance between any two interest points in the first interest point set is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1.

[0168] When the distance between any two interest points in the first interest point set is greater than or equal to a preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0.

[0169] In one embodiment, when the map construction module is used to construct a second interest point map based on the second interest point set, it is specifically used for:

[0170] Obtain the geographic location information of each point of interest in the second set of points of interest;

[0171] Calculate the distance between each pair of points of interest based on their geographical location information in the second set of points of interest.

[0172] The weight of the edge between each pair of interest points is determined based on the distance between each pair of interest points in the second interest point set, thus obtaining the second interest point map;

[0173] Furthermore, when determining the weight of the edge between any two interest points based on the distance between any two interest points in the second interest point set, the graph construction module is specifically used for:

[0174] When the distance between any two interest points in the second interest point set is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1.

[0175] When the distance between any two interest points in the second interest point set is greater than or equal to a preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0.

[0176] The model training module, used for training any single training iteration of the graph neural network model, includes:

[0177] The adjacency matrix and node attribute feature matrix of the first interest point map and the second interest point map are respectively input into the graph neural network model to obtain the output data of the graph neural network model. The output data includes the embedding of each interest point in the first interest point map and the second interest point map.

[0178] From the multiple pairs of seed interest point matching pairs, determine the target seed interest point matching pairs for this training, use the target seed interest point matching pairs as the positive matching pairs for this training, and construct the negative matching pairs for this training based on the target seed interest point matching pairs.

[0179] Obtain the embeddings of positive and negative matching pairs from the output data, and calculate the loss for this training based on the embeddings of positive and negative matching pairs.

[0180] Determine whether the conditions for stopping training are met based on the loss.

[0181] If the conditions are met, stop training and use the graph neural network model as the trained graph neural network model.

[0182] If the conditions are not met, update the network parameters of the graph neural network model based on the loss, and then train the updated graph neural network model again.

[0183] In one embodiment, the interest point matching module, when processing the first interest point map and the second interest point map using a trained graph neural network model, and determining all interest point matching pairs in the first interest point set and the second interest point set based on the processing result, is specifically used for:

[0184] Input the first interest point map into the trained graph neural network model, and obtain the embedding of each interest point in the first interest point set based on the output of the trained graph neural network model;

[0185] The second interest point map is input into the trained graph neural network model, and the embedding of each interest point in the second interest point set is obtained based on the output of the trained graph neural network model.

[0186] Each point of interest in the first set of points of interest is paired with each point of interest in the second set of points of interest to obtain multiple pairs of points of interest;

[0187] Calculate the embedding distance for each pair of interest points; the embedding distance for each pair of interest points refers to the distance between the embeddings of the two interest points included in each pair of interest points.

[0188] Each pair of interest points whose embedding distance is less than a preset threshold is identified as an interest point matching pair.

[0189] In another embodiment, the number of the first interest point map and the second interest point map are the same and there are multiple such maps;

[0190] Accordingly, when constructing a first interest point map based on a first set of interest points and a second interest point map based on a second set of interest points, the map construction module is specifically used for:

[0191] The target geographical area is divided into multiple spatial grids;

[0192] The latitude and longitude attributes of each point of interest in the first set of points of interest and the second set of points of interest are traversed separately to determine the first subset of points of interest and the second subset of points of interest corresponding to each spatial grid.

[0193] Based on the first and second subsets of interest points corresponding to each spatial grid, construct the corresponding first and second interest point maps for each spatial grid.

[0194] Accordingly, the interest point matching module, when processing the first and second interest point maps using a trained graph neural network model, and determining all interest point matching pairs in the first and second interest point sets based on the processing results, is specifically used for:

[0195] Input the first interest point map and the second interest point map corresponding to each spatial grid into the trained graph neural network model respectively. Based on the output of the trained graph neural network model, obtain the embedding of each interest point in the first interest point subset corresponding to each spatial grid, and the embedding of each interest point in the second interest point subset corresponding to each spatial grid.

[0196] Pair the first and second subsets of interest points corresponding to each spatial grid to obtain the set of interest point pairs for each spatial grid.

[0197] Calculate the embedding distance of each interest point pair in the set of interest point pairs for each spatial grid; the embedding distance of each interest point pair refers to the distance between the embeddings of the two interest points included in each interest point pair;

[0198] Each pair of interest points in the set of interest point pairs of each spatial grid, whose embedding distance is less than a preset threshold, is identified as an interest point matching pair.

[0199] Specific limitations regarding the graph neural network-based heterogeneous interest point matching device can be found in the limitations of the graph neural network-based heterogeneous interest point matching method described above, and will not be repeated here. Each module in the aforementioned graph neural network-based heterogeneous interest point matching device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0200] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a heterogeneous interest point matching method based on a graph neural network.

[0201] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0202] This embodiment also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method provided in any of the above method embodiments.

[0203] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method provided in any of the above method embodiments.

[0204] Those skilled in the art will understand that implementing all or part of the processes in the above method embodiments can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0205] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0206] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for heterogeneous interest point matching based on graph neural networks, characterized in that, The method includes: Obtain a first set of points of interest and a second set of points of interest within the target geographical area; construct a first point of interest map based on the first set of points of interest and a second point of interest map based on the second set of points of interest; the first set of points of interest and the second set of points of interest are heterogeneous data. Multiple pairs of preliminary interest point matching are selected from the first set of interest points and the second set of interest points, and the multiple pairs of preliminary interest point matching are labeled to obtain multiple pairs of seed interest point matching pairs; The graph neural network model is iteratively trained based on the first interest point map, the second interest point map, and the multiple pairs of seed interest point matching pairs to obtain a trained graph neural network model. The trained graph neural network model is used to process the first interest point map and the second interest point map, and all interest point matching pairs in the first interest point set and the second interest point set are determined based on the processing results. Any training process of the graph neural network model includes: The adjacency matrix and node attribute feature matrix of the first interest point map and the second interest point map are respectively input into the graph neural network model to obtain the output data of the graph neural network model. The output data includes the embedding of each interest point in the first interest point map and the second interest point map. From the multiple pairs of seed interest point matching pairs, determine the target seed interest point matching pairs for this training, use the target seed interest point matching pairs as the positive matching pairs for this training, and construct the negative matching pairs for this training based on the target seed interest point matching pairs. Obtain the embeddings of the positive matching pairs and the negative matching pairs from the output data, and calculate the loss for this training based on the embeddings of the positive matching pairs and the negative matching pairs; Determine whether the training stop condition is met based on the loss. If the conditions are met, training is stopped, and the graph neural network model is used as the trained graph neural network model. If the conditions are not met, the network parameters of the graph neural network model are updated according to the loss, and the updated graph neural network model is trained again.

2. The method as described in claim 1, characterized in that, The step of constructing the first interest point map based on the first interest point set includes: Obtain the geographic location information of each point of interest in the first set of points of interest; Calculate the distance between each pair of interest points based on the geographical location information of each pair of interest points in the first set of interest points; The weight of the edge between each pair of interest points is determined based on the distance between each pair of interest points in the first set of interest points, and the first interest point map is obtained. The step of determining the weight of the edge between any two interest points based on the distance between any two interest points in the first set of interest points includes: When the distance between any two interest points in the first set of interest points is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1. When the distance between any two interest points in the first set of interest points is greater than or equal to a preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0.

3. The method as described in claim 1, characterized in that, The step of constructing a second interest point map based on the second interest point set includes: Obtain the geographic location information of each point of interest in the second set of points of interest; Calculate the distance between each pair of interest points based on the geographical location information of each pair of interest points in the second set of interest points; The weight of the edge between each pair of interest points is determined based on the distance between each pair of interest points in the second set of interest points, and the second interest point map is obtained. The step of determining the weight of the edge between any two interest points based on the distance between any two interest points in the second set of interest points includes: When the distance between any two interest points in the second set of interest points is less than a preset threshold, it is determined that an edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 1. When the distance between any two interest points in the second set of interest points is greater than or equal to a preset threshold, it is determined that no edge relationship is formed between the two interest points, and the weight of the edge between the two interest points is set to 0.

4. The method as described in claim 1, characterized in that, The step of processing the first interest point map and the second interest point map using the trained graph neural network model, and determining all interest point matching pairs in the first interest point set and the second interest point set based on the processing result, includes: The first interest point map is input into the trained graph neural network model, and the embedding of each interest point in the first interest point set is obtained according to the output of the trained graph neural network model. The second interest point map is input into the trained graph neural network model, and the embedding of each interest point in the second interest point set is obtained according to the output of the trained graph neural network model. Each point of interest in the first set of interest points is paired with each point of interest in the second set of interest points to obtain multiple pairs of interest points; Calculate the embedding distance for each pair of interest points; the embedding distance for each pair of interest points refers to the distance between the embeddings of the two interest points included in each pair of interest points. Each pair of interest points whose embedding distance is less than a preset threshold is identified as an interest point matching pair.

5. The method as described in claim 1, characterized in that, The number of the first interest point map and the second interest point map are the same, both being multiple; The step of constructing a first interest point map based on the first interest point set and constructing a second interest point map based on the second interest point set includes: The target geographical area is divided into multiple spatial grids; The latitude and longitude attributes of each point of interest in the first set of points of interest and the second set of points of interest are traversed separately to determine the first subset of points of interest and the second subset of points of interest corresponding to each spatial grid. Based on the first and second subsets of interest points corresponding to each spatial grid, construct the corresponding first and second interest point maps for each spatial grid.

6. The method as described in claim 5, characterized in that, The step of processing the first interest point map and the second interest point map using the trained graph neural network model, and determining all interest point matching pairs in the first interest point set and the second interest point set based on the processing result, includes: The first interest point map and the second interest point map corresponding to each spatial grid are respectively input into the trained graph neural network model. Based on the output of the trained graph neural network model, the embedding of each interest point in the first interest point subset corresponding to each spatial grid and the embedding of each interest point in the second interest point subset corresponding to each spatial grid are obtained. Pair the first and second subsets of interest points corresponding to each spatial grid to obtain the set of interest point pairs for each spatial grid. Calculate the embedding distance of each interest point pair in the set of interest point pairs for each spatial grid; the embedding distance of each interest point pair refers to the distance between the embeddings of the two interest points included in each interest point pair; Each pair of interest points in the set of interest point pairs of each spatial grid, whose embedding distance is less than a preset threshold, is identified as an interest point matching pair.

7. A multi-source data matching device for points of interest, characterized in that, The device includes: The map construction module is used to obtain a first set of points of interest and a second set of points of interest within a target geographical area, construct a first point of interest map based on the first set of points of interest, and construct a second point of interest map based on the second set of points of interest; the first set of points of interest and the second set of points of interest are heterogeneous data. The seed matching pair construction module is used to select multiple pairs of preliminary interest point matching pairs from the first interest point set and the second interest point set, and to label the multiple pairs of preliminary interest point matching pairs to obtain multiple pairs of seed interest point matching pairs. The model training module is used to iteratively train the graph neural network model based on the first interest point map, the second interest point map and the multiple pairs of seed interest point matching pairs to obtain a trained graph neural network model. The interest point matching module is used to process the first interest point map and the second interest point map through the trained graph neural network model, and determine all interest point matching pairs in the first interest point set and the second interest point set according to the processing result. Any training process of the graph neural network model includes: The adjacency matrix and node attribute feature matrix of the first interest point map and the second interest point map are respectively input into the graph neural network model to obtain the output data of the graph neural network model. The output data includes the embedding of each interest point in the first interest point map and the second interest point map. From the multiple pairs of seed interest point matching pairs, determine the target seed interest point matching pairs for this training, use the target seed interest point matching pairs as the positive matching pairs for this training, and construct the negative matching pairs for this training based on the target seed interest point matching pairs. Obtain the embeddings of the positive matching pairs and the negative matching pairs from the output data, and calculate the loss for this training based on the embeddings of the positive matching pairs and the negative matching pairs; Determine whether the training stop condition is met based on the loss. If the conditions are met, training is stopped, and the graph neural network model is used as the trained graph neural network model. If the conditions are not met, the network parameters of the graph neural network model are updated according to the loss, and the updated graph neural network model is trained again.

8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images