Model training, interest point deduplication methods, devices, electronic equipment and storage media

Abstract

This disclosure provides a model training method, apparatus, electronic device, and storage medium for point of interest (POI) deduplication. The model training method includes: acquiring sample data, which includes information about POIs to be processed and information about multiple candidate POIs; performing feature fusion analysis on a preset model to be trained based on the information of the POIs to be processed and the information of the multiple candidate POIs to obtain the sample similarity between each candidate POI and the POI to be processed; and, in response to determining that training is complete based on the sample similarity, using the trained model as the POI deduplication model. In this disclosure, the POI deduplication model can simultaneously analyze information from multiple candidate POIs, and during feature fusion analysis, it can integrate factors influencing the interactions between different POIs, making the analysis process more comprehensive and reasonable, thereby improving the accuracy of the POI deduplication model.

Description

Technical Field

[0001] This disclosure relates to the field of Internet technology, and in particular to a model training, interest point deduplication method, device, electronic device and storage medium. Background Technology

[0002] Points of Interest (POIs) emerged as Geographic Information Systems (GIS) developed to a certain stage, driven by users' demand for personalized services. Comprehensive POI information is essential for enriching navigation maps. Timely POIs can alert users to road junctions and detailed information about surrounding buildings, making it easier to find the places you need during navigation and choose the most convenient and efficient route for path planning.

[0003] In the POI mining process, newly added POIs typically need to undergo deduplication to determine if any duplicate POIs exist in the database. Current technologies usually employ a POI deduplication model to compare the POI to be deduplicated with multiple recalled POIs one-to-one to determine if any duplicates exist.

[0004] Existing POI deduplication models use a one-to-one analysis method, meaning they analyze information for only one POI at a time. However, different POIs may influence each other, making this one-to-one analysis method somewhat one-sided and resulting in low accuracy for POI deduplication models. Summary of the Invention

[0005] In view of the above problems, this disclosure provides a model training, interest point deduplication method, apparatus, electronic device, and storage medium to overcome or at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of this disclosure, a model training method is provided, comprising:

[0007] Acquire sample data, which includes information about the POI to be processed and information about multiple candidate POIs;

[0008] In the preset training model, feature fusion analysis is performed based on the information of the sample POI to be processed and the information of the multiple sample candidate POIs to obtain the sample similarity between each of the sample candidate POIs and the sample POI to be processed.

[0009] In response to determining that training is complete based on the sample similarity, the trained model is used as the POI deduplication model.

[0010] Optionally, the feature fusion analysis based on the information of the sample POI to be processed and the information of the multiple sample candidate POIs includes: for each sample candidate POI, extracting a similarity feature vector of the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed; performing feature fusion analysis on the similarity feature vector to obtain the interaction feature vector of each of the sample candidate POIs; and for each sample candidate POI, determining the sample similarity between the current sample candidate POI and the sample POI to be processed based on the interaction feature vector of the current sample candidate POI.

[0011] Optionally, the step of extracting the similarity feature vector of the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed includes: for each piece of information of the current sample candidate POI, obtaining the similarity information of the current information based on the corresponding information of the current information and the sample POI to be processed, and extracting the similarity feature vector of the current information based on the similarity information; concatenating the similarity feature vectors of each piece of information to obtain a first concatenated vector, and performing feature fusion analysis on the first concatenated vector to obtain the similarity feature vector of the current sample candidate POI.

[0012] Optionally, the step of extracting the similarity feature vector of the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed includes: for each piece of information of the current sample candidate POI belonging to a first type, extracting the similarity feature vector of the current information based on the corresponding information of the current information and the sample POI to be processed; for each piece of information of the current sample candidate POI belonging to a second type, obtaining the similarity information of the current information based on the corresponding information of the current information and the sample POI to be processed, and extracting the similarity feature vector of the current information based on the similarity information; concatenating the similarity feature vectors of each piece of information belonging to the second type to obtain a second concatenated vector, performing feature fusion analysis on the second concatenated vector to obtain a fused feature vector; and concatenating the similarity feature vector of the information belonging to the first type with the fused feature vector to obtain the similarity feature vector of the current sample candidate POI.

[0013] Optionally, the step of performing feature fusion analysis on the similarity feature vectors to obtain the interaction feature vectors of each of the candidate POIs includes: concatenating the similarity feature vectors to obtain a concatenated similarity feature vector; performing feature fusion analysis on the concatenated similarity feature vectors to obtain a concatenated interaction feature vector; and restoring the interaction feature vectors of each of the candidate POIs based on the concatenated interaction feature vectors.

[0014] Optionally, the step of performing feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector includes: using at least one fully connected layer to perform feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector.

[0015] Optionally, the step of performing feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector includes: using a combination network of at least one fully connected layer and a dropout layer to perform feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector.

[0016] According to a second aspect of the embodiments of this disclosure, a method for determining duplicate points of interest is provided, comprising:

[0017] Obtain information about the POI to be processed and information about multiple candidate POIs;

[0018] The information of the POI to be processed and the information of the multiple candidate POIs are input into a pre-trained POI deduplication model to obtain the similarity between each candidate POI and the POI to be processed, as output by the POI deduplication model; wherein, the POI deduplication model is trained by the model training method according to any one of claims 1 to 7.

[0019] Based on the similarity, it is determined whether there is a POI among the candidate POIs that is duplicated with the POI to be processed.

[0020] According to a third aspect of the embodiments of this disclosure, a model training apparatus is provided, comprising:

[0021] The first acquisition module is used to acquire sample data, which includes information on the point of interest (POI) to be processed and information on multiple candidate POIs.

[0022] The training module is used to perform feature fusion analysis on the information of the sample POI to be processed and the information of the multiple sample candidate POIs in the preset training model, so as to obtain the sample similarity between each of the sample candidate POIs and the sample POI to be processed.

[0023] The first determining module is used to determine that training is complete based on the sample similarity and to use the trained model as the POI deduplication model.

[0024] Optionally, the training module includes: a feature extraction unit, configured to extract a similarity feature vector of the current candidate POI based on the information of the current candidate POI and the information of the POI to be processed for each candidate POI; a feature interaction unit, configured to perform feature fusion analysis on the similarity feature vector to obtain an interaction feature vector of each candidate POI; and a similarity determination unit, configured to determine the sample similarity between the current candidate POI and the POI to be processed for each candidate POI based on the interaction feature vector of the current candidate POI.

[0025] Optionally, the feature extraction unit includes: a first acquisition subunit, configured to acquire similarity information of the current information based on the corresponding information of the current information and the POI to be processed in the sample for each piece of information of the current sample candidate POI, and extract the similarity feature vector of the current information based on the similarity information; and a second acquisition subunit, configured to concatenate the similarity feature vectors of each piece of information to obtain a first concatenated vector, and perform feature fusion analysis on the first concatenated vector to obtain the similarity feature vector of the current sample candidate POI.

[0026] Optionally, the feature extraction unit includes: a third acquisition subunit, configured to extract a similarity feature vector of the current information for each piece of information belonging to the first type of the current sample candidate POI, based on the corresponding information of the current information and the sample POI to be processed; a fourth acquisition subunit, configured to obtain similarity information of the current information for each piece of information belonging to the second type of the current sample candidate POI, based on the corresponding information of the current information and the sample POI to be processed, and extract a similarity feature vector of the current information based on the similarity information; a fifth acquisition subunit, configured to concatenate the similarity feature vectors of each piece of information belonging to the second type to obtain a second concatenated vector, and perform feature fusion analysis on the second concatenated vector to obtain a fused feature vector; and a sixth acquisition subunit, configured to concatenate the similarity feature vector of the information belonging to the first type with the fused feature vector to obtain a similarity feature vector of the current sample candidate POI.

[0027] Optionally, the feature interaction unit includes: a splicing subunit, used to splice the similarity feature vectors to obtain a spliced ​​similarity feature vector; a fusion subunit, used to perform feature fusion analysis on the spliced ​​similarity feature vector to obtain a spliced ​​interaction feature vector; and a restoration subunit, used to restore the interaction feature vector of each of the sample candidate POIs based on the spliced ​​interaction feature vector.

[0028] Optionally, the fusion subunit is specifically used to perform feature fusion analysis on the splicing similarity feature vector using at least one fully connected layer to obtain the splicing interaction feature vector.

[0029] Optionally, the fusion subunit is specifically used to perform feature fusion analysis on the splicing similarity feature vector using a combination network of at least one fully connected layer and a dropout layer to obtain the splicing interaction feature vector.

[0030] According to a fourth aspect of the embodiments of this disclosure, an interest point deduplication device is provided, comprising:

[0031] The second acquisition module is used to acquire information about the POI to be processed and information about multiple candidate POIs;

[0032] A prediction module is used to input the information of the POI to be processed and the information of the plurality of candidate POIs into a pre-trained POI deduplication model to obtain the similarity between each candidate POI and the POI to be processed, as output by the POI deduplication model; wherein, the POI deduplication model is trained by the model training method according to any one of claims 1 to 7.

[0033] The second determining module is used to determine, based on the similarity, whether there is a POI among the candidate POIs that is duplicated with the POI to be processed.

[0034] According to a fifth aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: one or more processors; and one or more computer-readable storage media having instructions stored thereon; wherein, when the instructions are executed by the one or more processors, the processors cause the processors to perform a model training method as described in any of the preceding claims, or to perform an interest point deduplication method as described in any of the preceding claims.

[0035] According to a sixth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the model training method as described in any of the preceding claims, or to perform the interest point deduplication method as described in any of the preceding claims.

[0036] This disclosure provides a model training method, apparatus, electronic device, and storage medium for point of interest (POI) deduplication. During model training, sample data is acquired, including information on the POI to be processed and information on multiple candidate POIs. In a pre-defined training model, feature fusion analysis is performed based on the information of the POI to be processed and the information of the multiple candidate POIs to obtain the sample similarity between each candidate POI and the POI to be processed. In response to determining training completion based on the sample similarity, the trained model is used as the POI deduplication model. Therefore, in this disclosure, the POI deduplication model can simultaneously analyze information from multiple candidate POIs, and during feature fusion analysis, it can integrate factors influencing the interactions between different POIs, making the analysis more comprehensive and reasonable, thereby improving the accuracy of the POI deduplication model. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the description of the embodiments of this disclosure will be briefly introduced below. Obviously, the drawings described below are only some drawings of the embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of a POI location according to an embodiment of this disclosure.

[0039] Figure 2 This is a flowchart of the steps of a model training method according to an embodiment of the present disclosure.

[0040] Figure 3 This is a schematic diagram of the structure of a POI deduplication model according to an embodiment of this disclosure.

[0041] Figure 4 This is a schematic diagram of the structure of a feature extraction part according to an embodiment of the present disclosure.

[0042] Figure 5 This is a schematic diagram of another feature extraction part according to an embodiment of the present disclosure.

[0043] Figure 6 This is a flowchart of the steps of an interest point deduplication method according to an embodiment of the present disclosure.

[0044] Figure 7 This is a structural block diagram of a model training device according to an embodiment of the present disclosure.

[0045] Figure 8 This is a structural block diagram of an interest point deduplication device according to an embodiment of the present disclosure. Detailed Implementation

[0046] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of the embodiments of this disclosure. Based on the embodiments of this disclosure, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this disclosure.

[0047] The embodiments disclosed herein can be applied to scenarios such as POI deduplication. Optionally, a POI can be a shop, a shopping mall, an office building, a park, a residential area, a square, a bus stop, etc.

[0048] In existing technologies, POI deduplication models typically use a one-to-one comparison between the POI to be deduplicated and multiple recalled POIs to determine if any POIs duplicate the one to be deduplicated. Because it's a one-to-one comparison, the POI deduplication model can only acquire and analyze information from one POI at a time, failing to capture information from surrounding POIs. However, surrounding POIs often influence the analysis of the POI, so the results obtained from a one-to-one comparison are often biased, especially when there are multiple similar POIs in the vicinity. An example follows to illustrate this.

[0049] Reference Figure 1 This diagram illustrates a POI location according to an embodiment of the present disclosure. Figure 1 As shown, the Affiliated Primary School of Beijing Petroleum Institute has three gates. Figure 1 One of the gates is a POI (Point of Interest), namely Southwest Gate 3, Southwest Gate 2, and South Gate 1. The POI to be deduplicated is named Southwest Gate, and its location is... Figure 1 The location of the dots between the three gates in the southwest and central areas and the two gates in the southwest. If we compare the southwest gate with each of these three gates one-to-one, the results are shown in the table below (the scores in the table represent similarity):

[0050] Southwest Gate 3 0.81 Southwest Gate 2 0.93 South Gate 1 0.12

[0051] Based on the scores in the table above, it would be assumed that Southwest Gate and Southwest Gate 2 are the same gate. However, the actual situation is that Southwest Gate and Southwest Gate 3 are the same gate. The root cause of this error is that the POI deduplication model, when judging Southwest Gate 2, is unaware of the closer Southwest Gate 3. When judging Southwest Gate 3, because it is unaware of Southwest Gate 2's existence, it cannot guarantee that it will give a higher score than Southwest Gate 2, thus leading to an error in the judgment result.

[0052] To address the aforementioned issues, this disclosure proposes an improved POI deduplication model. This model can simultaneously acquire and analyze information from multiple candidate POIs during POI deduplication, incorporating factors that influence the interactions between candidate POIs. This makes the analysis process more rational and thus improves the accuracy of the POI deduplication model. The details are as follows.

[0053] Reference Figure 2 The diagram illustrates a flowchart of a model training method according to an embodiment of the present disclosure.

[0054] like Figure 2 As shown, the model training method may include the following steps:

[0055] Step 201: Obtain sample data, which includes information on the POI to be processed and information on multiple candidate POIs.

[0056] In the implementation, sample data can be constructed based on existing POI information. Each POI has corresponding related information. Optionally, the POI information may include, but is not limited to: name, location, type, address, etc.

[0057] In one optional implementation, if an existing POI is used as a sample POI to be processed, then based on the location information of the sample POI to be processed and the location information of each POI, POIs whose distance from the sample POI to be processed is less than a set threshold can be recalled, and the recalled POIs can be used as the sample candidate POIs corresponding to the sample POI to be processed.

[0058] In another optional implementation, if an existing POI is used as a sample POI to be processed, then based on the location information of the sample POI and the location information of each POI, POIs whose distance from the sample POI to be processed is less than a set threshold can be recalled. Considering that some of the recalled POIs may be unrelated to the sample POI to be processed, the recalled POIs can also be filtered to select POIs related to the sample POI to be processed, and the selected POIs can be used as the sample candidate POIs corresponding to the sample POI to be processed.

[0059] Optionally, the recalled POIs can be filtered based on text similarity. Specifically, for each recalled POI, the text similarity between the POI to be processed and the recalled POI can be calculated based on the information of the POI to be processed (such as name, type, address, etc.) and the information of the recalled POI. Then, the recalled POIs are sorted in descending order based on the text similarity, and the top-ranked POIs are selected as the candidate POIs corresponding to the POI to be processed.

[0060] Optionally, the recalled POIs can be filtered based on the longest common substring. Specifically, for each recalled POI, the longest common substring between the POI to be processed and the recalled POI can be calculated based on the information of the POI to be processed (such as name, type, address, etc.) and the information of the recalled POI. Then, the recalled POIs are sorted in descending order based on the longest common substring, and the top-ranked POIs are selected as the candidate POIs corresponding to the POI to be processed.

[0061] Step 202: In the preset training model, feature fusion analysis is performed based on the information of the sample POI to be processed and the information of the multiple sample candidate POIs to obtain the sample similarity between each of the sample candidate POIs and the sample POI to be processed.

[0062] During model training, the information of the POI to be processed and the information of multiple candidate POIs corresponding to the POI to be processed are input into the preset training model. In the preset training model, feature fusion analysis can be performed based on the information of the POI to be processed and the information of multiple candidate POIs to obtain the sample similarity between each candidate POI and the POI to be processed. The training model outputs the sample similarity between each candidate POI and the POI to be processed.

[0063] Step 203: In response to determining that training is complete based on the sample similarity, the trained model is used as the POI deduplication model.

[0064] In one alternative implementation, the actual similarity between each candidate POI and the POI to be processed can be labeled based on whether the candidate POI and the POI to be processed are actually duplicates. Based on the sample similarity between each candidate POI and the POI to be processed output by the model to be trained, and the labeled actual similarity between each candidate POI and the POI to be processed, the loss function of the model to be trained can be calculated.

[0065] A loss function measures the degree of discrepancy between a model's predictions and the true values. A small loss function indicates that the model closely approximates the true data distribution, indicating good model performance; conversely, a large loss function indicates a significant difference between the model and the true data distribution, indicating poor model performance. The task of training a model is to use optimization methods to find the model parameters that minimize the loss function. Therefore, the completion of training can be determined based on the loss function of the model to be trained. Specifically, training can be considered complete when the loss function meets preset conditions (e.g., the loss function is less than a certain threshold).

[0066] Optionally, the loss function may include, but is not limited to: cross-entropy loss, zero-one loss, logistic loss, hinge loss, exponential loss, etc.

[0067] If training is confirmed to be complete, the trained model is used as the POI deduplication model. If training is confirmed to be incomplete, the relevant parameters of the model to be trained are adjusted and training continues.

[0068] In this embodiment of the disclosure, the POI deduplication model can analyze the information of multiple candidate POIs simultaneously. During the feature fusion analysis process, it can integrate the factors that influence the interaction between different POIs, making the analysis process more comprehensive and reasonable, thereby improving the accuracy of the POI deduplication model.

[0069] The structure of the POI deduplication model in the embodiments of this disclosure will be described in detail below, with examples.

[0070] Reference Figure 3 A schematic diagram of the structure of a POI deduplication model according to an embodiment of the present disclosure is shown. Figure 3 In this example, we will use P0 as the POI to be processed and P1 to P100 as the 100 candidate POIs corresponding to P0.

[0071] The specific processing procedure of the POI deduplication model is as follows:

[0072] 1. Input the information of the POI to be processed and the information of each candidate POI into the POI deduplication model.

[0073] like Figure 3 As shown, the information of P0, P1, P2, P3, ..., P100 is input into the POI deduplication model.

[0074] POI information may include, but is not limited to: name, location (i.e., coordinates), type, address, etc. The type may include, but is not limited to: shop, shopping mall, residential area, office building, park, square, bus stop, etc.

[0075] 2. In the POI deduplication model, for each candidate POI, the similarity feature vector of the current candidate POI is extracted based on the information of the current candidate POI and the information of the POI to be processed.

[0076] like Figure 3As shown, based on the information of P0 and P1, the similarity feature vector F1 (d = 256) of P1 relative to P0 can be extracted; based on the information of P0 and P2, the similarity feature vector F2 (d = 256) of P2 relative to P0 can be extracted; based on the information of P0 and P3, the similarity feature vector F3 (d = 256) of P3 relative to P0 can be extracted; ... based on the information of P0 and P100, the similarity feature vector F100 (d = 256) of P100 relative to P0 can be extracted. Here, d = 256 indicates that the dimension of the vector is 256. This can be understood as... Figure 3 The dimensions in the example are for illustrative purposes only and are not intended as a limitation on the dimensions of similarity feature vectors.

[0077] The POI deduplication model includes a feature extraction part. The process of extracting the similarity feature vector of each candidate POI based on the information of the current sample candidate POI and the information of the sample to be processed is performed by this feature extraction part.

[0078] Since the input of the POI deduplication model contains information from multiple POIs, the feature extraction part for two POIs should be designed with performance in mind, and the structure should be as simple and efficient as possible.

[0079] In one optional implementation, the process of extracting the similarity feature vector of the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed may include: for each piece of information of the current sample candidate POI, obtaining the similarity information of the current information based on the corresponding information of the current information and the sample POI to be processed; extracting the similarity feature vector of the current information based on the similarity information; concatenating the similarity feature vectors of each piece of information to obtain a first concatenated vector; and performing feature fusion analysis on the first concatenated vector to obtain the similarity feature vector of the current sample candidate POI.

[0080] Reference Figure 4 The diagram illustrates a structural schematic of a feature extraction section according to an embodiment of this disclosure. Figure 4 As shown, the information of a Point of Interest (POI) includes name, location, type, and address.

[0081] First, calculate the name similarity between two POIs (one sample POI to be processed and one sample candidate POI); calculate the distance between two POIs based on their locations; calculate the type similarity between two POIs based on their types; and calculate the address similarity between two POIs based on their addresses.

[0082] Optionally, for the name similarity between two POIs, a model for calculating name similarity can be pre-trained. The input to this model is the names of the two POIs (specifically, a concatenated string of their names), and the output is the name similarity between the two POIs. The training process of this model for calculating name similarity will not be discussed in detail in this embodiment.

[0083] Optionally, the distance between two POIs can be calculated based on their coordinate positions, which will not be discussed in detail in this embodiment.

[0084] Optionally, for the type similarity between two POIs, a model for calculating type similarity can be pre-trained. The input of this model is the type of the two POIs (specifically, a concatenated string of the types of the two POIs), and the output of the model is the type similarity between the two POIs. The training process of this model for calculating type similarity will not be discussed in detail in this embodiment.

[0085] Optionally, the address similarity between two POIs can be calculated based on text similarity, which will not be discussed in detail in this embodiment.

[0086] Then, the name similarity is input into the embedding layer for mapping (the embedding is used to convert discrete variables into continuous vector representations) to obtain the similarity feature vector corresponding to the name similarity; the distance is input into the embedding layer for mapping to obtain the distance similarity feature vector; the type similarity is input into the embedding layer for mapping to obtain the type similarity feature vector; and the address similarity is input into the embedding layer for mapping to obtain the address similarity feature vector.

[0087] Then, the similarity feature vectors corresponding to name similarity, distance, type similarity, and address similarity are input into the concatenation layer (Connect) for concatenation processing to obtain the first concatenated vector.

[0088] Finally, the first concatenated vector is input into the fully connected layer (Dense) for feature fusion processing to obtain the similarity feature vector F (d=256) between the current sample candidate POI and the sample to be processed POI.

[0089] Figure 4 The feature extraction section shown does not incorporate complex deep networks, ensuring a simple and efficient processing procedure.

[0090] In another optional implementation, the process of extracting the similarity feature vector of the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed may include: for each piece of information of the current sample candidate POI belonging to a first type, extracting the similarity feature vector of the current information based on the corresponding information of the current information and the sample POI to be processed; for each piece of information of the current sample candidate POI belonging to a second type, obtaining the similarity information of the current information based on the corresponding information of the current information and the sample POI to be processed, and extracting the similarity feature vector of the current information based on the similarity information; concatenating the similarity feature vectors of each piece of information belonging to the second type to obtain a second concatenated vector, performing feature fusion analysis on the second concatenated vector to obtain a fused feature vector; and concatenating the similarity feature vector of the information belonging to the first type with the fused feature vector to obtain the similarity feature vector of the current sample candidate POI.

[0091] Optionally, the first type may include, but is not limited to, name, type, etc. The second type may include, but is not limited to, location, type, address, etc.

[0092] Reference Figure 5 The diagram illustrates a structural schematic of another feature extraction section according to an embodiment of this disclosure. Figure 5 As shown, the information of a Point of Interest (POI) includes name, location, type, and address.

[0093] On one hand, the names of two POIs (one sample POI to be processed and one sample candidate POI) (specifically, the concatenated string of the two POI names, name A + name B) are input into the embedding layer. After the embedding layer performs mapping processing, the features are extracted by the convolutional neural network (CNN) to obtain the similarity feature vector corresponding to the name similarity of the two POIs.

[0094] On the other hand, the distance between two POIs is calculated based on their locations; the type similarity between the two POIs is calculated based on their types; and the address similarity between the two POIs is calculated based on their addresses. Then, the distance is input into the embedding layer for mapping to obtain the distance-related similarity feature vector; the type similarity is input into the embedding layer for mapping to obtain the type-related similarity feature vector; and the address similarity is input into the embedding layer for mapping to obtain the address-related similarity feature vector. Then, the distance-related, type-related, and address-related similarity feature vectors are input into the concatenation layer for concatenation to obtain a second concatenated vector. Finally, the second concatenated vector is input into the fully connected layer (Dense) for feature fusion to obtain the fused feature vectors for the two POIs.

[0095] Then, the similarity feature vectors corresponding to the name similarity of the two POIs and the corresponding fusion feature vectors of the two POIs are input into the concatenation layer (Connect) for concatenation processing to obtain the similarity feature vector F (d=256) between the current sample candidate POI and the sample POI to be processed.

[0096] Figure 5 The feature extraction section shown does not use the more complex self-attention method for semantic feature extraction of names, but instead uses a more effective CNN network. Furthermore, no particularly complex deep networks are added to the processing of other information, thus ensuring the efficiency of the entire model.

[0097] 3. Perform feature fusion analysis on the similarity feature vectors of each candidate POI to obtain the interaction feature vectors of each candidate POI.

[0098] Optionally, the process of performing feature fusion analysis on the similarity feature vectors of each candidate POI to obtain the interaction feature vectors of each candidate POI may include the following steps A1 to A3:

[0099] Step A1: Concatenate the similarity feature vectors of each candidate POI to obtain the concatenated similarity feature vector.

[0100] In one alternative implementation, the similarity feature vectors of each candidate POI can be input into the concatenation layer, and the similarity feature vectors of each candidate POI can be concatenated together in the concatenation layer to obtain a concatenated similarity feature vector.

[0101] like Figure 3As shown, the 100 similarity feature vectors of 256 dimensions, F1, F2, F3, ..., F100, are concatenated through a concatenation layer (Connect) to form a vector with d = 25600 (i.e., 1 × 25600 dimensions). In this embodiment, this vector is called the concatenated similarity feature vector. This concatenated similarity feature vector represents the combined feature vector of P0 and the surrounding 100 POIs, P1, P2, P3, ..., P100.

[0102] Step A2: Perform feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector.

[0103] After obtaining the spliced ​​similarity feature vector, feature fusion analysis is performed on the spliced ​​similarity feature vector to enable deep fusion and generalization of features, and to fuse and interact relatively independent features so that the features of each POI can incorporate the features of other POIs.

[0104] In one optional implementation, the process of performing feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector may include: using at least one fully connected layer to perform feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector.

[0105] A fully connected layer essentially performs a linear transformation from one feature space to another. Each dimension of the target space is considered to be influenced by every dimension of the source space. Without being overly rigorous, we can say that the target vector is a weighted sum of the source vectors. Therefore, a fully connected layer allows for deep feature fusion of concatenated similarity feature vectors, enabling features from other POIs to be incorporated into the features of each POI. Furthermore, considering model performance, a fully connected layer can also reduce the dimensionality of the concatenated similarity feature vectors, thereby reducing feature complexity. In practice, fully connected layers can be implemented using convolutional operations.

[0106] In another optional implementation, the process of performing feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector may include: using a combination network of at least one fully connected layer and a dropout layer to perform feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector.

[0107] In this embodiment, not only is a fully connected layer used to perform feature fusion and dimensionality reduction operations on the concatenated similarity feature vector, but a discarding layer can also be added to randomly discard features in the vector, thereby avoiding overfitting caused by too many parameters.

[0108] like Figure 3As shown, the splicing similarity feature vector output by the splicing layer (Connect) is first passed through a combination network of fully connected layers and dropout layers (Dense+Drop) for feature fusion, forming a vector with d=12800 (i.e., 1×12800 dimensions); then this vector with d=12800 is passed through a combination network of fully connected layers and dropout layers (Dense+Drop (i.e., Dropout)) for feature fusion, forming a vector with d=6400 (i.e., 1×6400 dimensions). In this embodiment, this vector is called the splicing interaction feature vector.

[0109] Step A3: Based on the spliced ​​interactive feature vector, the interactive feature vector of each of the candidate POIs is restored.

[0110] The concatenated interactive feature vector includes features from each candidate POI. By splitting the concatenated interactive feature vector equally according to the number of candidate POIs, the interactive feature vector corresponding to each candidate POI can be reconstructed. It is important to note that the order of the candidate POIs during concatenation is the same as the order of the candidate POIs during splitting.

[0111] In one alternative implementation, the concatenated interactive feature vector can be input into the reshaping layer, and the concatenated interactive feature vector can be restored in the reshaping layer to obtain the interactive feature vector of each candidate POI.

[0112] like Figure 3 As shown, the spliced ​​interactive feature vector output by the combined network of the last fully connected layer and the dropout layer (Dense+Drop) is input into the reshape layer for restoration. The spliced ​​interactive feature vector of d=6400 is restored to a dimension of 100×64, representing the interactive feature vector of 100 candidate sample POIs. This interactive feature vector will include the features of the surrounding POIs.

[0113] 4. For each candidate POI, determine the sample similarity between the current candidate POI and the POI to be processed based on the interaction feature vector of the current candidate POI.

[0114] In one alternative implementation, the interaction feature vectors of each candidate POI can be input into a normalization layer. Based on the interaction feature vectors of the current candidate POI, the normalization layer determines the sample similarity between the current candidate POI and the POI to be processed.

[0115] like Figure 3As shown, the interaction feature vectors of the 100 candidate POIs output by the Reshape layer are input into the Softmax normalization layer for processing. Softmax maps the input to real numbers between 0 and 1, and normalization ensures that the sum is 1, so the sum of the probabilities of multi-class classification is also exactly 1. Therefore, Softmax can obtain a score (i.e., sample similarity) based on the interaction feature vector of each candidate POI, thus outputting the sample similarity of the 100 candidate POIs.

[0116] Based on the POI deduplication model obtained from the above training, POI deduplication can be performed.

[0117] Reference Figure 6 The diagram illustrates a flowchart of a method for determining duplicate points of interest according to an embodiment of this disclosure.

[0118] like Figure 6 As shown, the method for identifying duplicate points of interest may include the following steps:

[0119] Step 601: Obtain information about the POI to be processed and information about multiple candidate POIs.

[0120] The information for a Point of Interest (POI) may include, but is not limited to: name, location, type, address, etc.

[0121] In one alternative implementation, based on the location information of the POI to be processed and the location information of existing POIs in the database, POIs whose distance from the POI to be processed is less than a set threshold can be recalled, and the recalled POIs can be used as candidate POIs corresponding to the POI to be processed.

[0122] In another optional implementation, based on the location information of the POI to be processed and the location information of existing POIs in the database, POIs whose distance from the POI to be processed is less than a set threshold can be recalled. Considering that some of the recalled POIs may be unrelated to the POI to be processed, the recalled POIs can also be filtered to select those related to the POI to be processed, and these selected POIs can be used as candidate POIs corresponding to the POI to be processed. The filtering process is described in step 201 above.

[0123] Step 602: Input the information of the POI to be processed and the information of the multiple candidate POIs into a pre-trained POI deduplication model to obtain the similarity between each candidate POI and the POI to be processed, as output by the POI deduplication model.

[0124] The information of the POI to be processed and the information of the multiple candidate POIs are input into the POI deduplication model trained above. In the POI deduplication model, feature fusion analysis is performed based on the information of the POI to be processed and the information of the multiple candidate POIs to obtain the similarity between each candidate POI and the POI to be processed. For the specific processing procedures inside the POI deduplication model, please refer to the relevant descriptions in the above embodiments, which will not be discussed in detail here.

[0125] Step 603: Based on the similarity, determine whether there is a POI among the candidate POIs that is duplicated with the POI to be processed.

[0126] The higher the similarity between the candidate POI and the POI to be processed, the greater the possibility that the candidate POI and the POI to be processed overlap. Therefore, the similarity can be used to determine whether there is a POI that overlaps with the POI to be processed among the candidate POIs.

[0127] Optionally, candidate POIs can be sorted in descending order based on similarity. Starting from the first candidate POI after sorting, the similarity of the candidate POI is compared with a preset first similarity threshold. If the similarity of the candidate POI is greater than the first similarity threshold, it can be determined that the candidate POI is a duplicate of the POI to be processed. If the similarity of the candidate POI is less than or equal to the first similarity threshold, it can be compared with a preset second similarity threshold. If the similarity of the candidate POI is less than the second similarity threshold, it can be determined that the candidate POI is not a duplicate of the POI to be processed. If the similarity of the candidate POI is greater than or equal to the second similarity threshold, it can be considered that whether the candidate POI is a duplicate of the POI to be processed is uncertain.

[0128] The embodiments disclosed herein improve the accuracy of POI deduplication, especially when there are multiple similar POIs in the vicinity. The POI deduplication model is closer to the processing procedure in POI deduplication, which is more reasonable in principle and has better interpretability.

[0129] Reference Figure 7 The diagram shows a structural block diagram of a model training apparatus according to an embodiment of the present disclosure.

[0130] like Figure 7 As shown, the model training device may include the following modules:

[0131] The first acquisition module 701 is used to acquire sample data, which includes information on the point of interest (POI) to be processed and information on multiple candidate POIs.

[0132] Training module 702 is used to perform feature fusion analysis on the information of the sample POI to be processed and the information of the multiple sample candidate POIs in a preset model to be trained, so as to obtain the sample similarity between each of the sample candidate POIs and the sample POI to be processed.

[0133] The first determining module 703 is used to determine that training is complete based on the sample similarity and to use the trained model as the POI deduplication model.

[0134] Optionally, the training module 702 includes: a feature extraction unit, configured to extract a similarity feature vector of the current candidate POI based on the information of the current candidate POI and the information of the POI to be processed for each candidate POI; a feature interaction unit, configured to perform feature fusion analysis on the similarity feature vector to obtain an interaction feature vector of each candidate POI; and a similarity determination unit, configured to determine the sample similarity between the current candidate POI and the POI to be processed for each candidate POI based on the interaction feature vector of the current candidate POI.

[0135] Optionally, the feature extraction unit includes: a first acquisition subunit, configured to acquire similarity information of the current information based on the corresponding information of the current information and the POI to be processed in the sample for each piece of information of the current sample candidate POI, and extract the similarity feature vector of the current information based on the similarity information; and a second acquisition subunit, configured to concatenate the similarity feature vectors of each piece of information to obtain a first concatenated vector, and perform feature fusion analysis on the first concatenated vector to obtain the similarity feature vector of the current sample candidate POI.

[0136] Optionally, the feature extraction unit includes: a third acquisition subunit, configured to extract a similarity feature vector of the current information for each piece of information belonging to the first type of the current sample candidate POI, based on the corresponding information of the current information and the sample POI to be processed; a fourth acquisition subunit, configured to obtain similarity information of the current information for each piece of information belonging to the second type of the current sample candidate POI, based on the corresponding information of the current information and the sample POI to be processed, and extract a similarity feature vector of the current information based on the similarity information; a fifth acquisition subunit, configured to concatenate the similarity feature vectors of each piece of information belonging to the second type to obtain a second concatenated vector, and perform feature fusion analysis on the second concatenated vector to obtain a fused feature vector; and a sixth acquisition subunit, configured to concatenate the similarity feature vector of the information belonging to the first type with the fused feature vector to obtain a similarity feature vector of the current sample candidate POI.

[0137] Optionally, the feature interaction unit includes: a splicing subunit, used to splice the similarity feature vectors to obtain a spliced ​​similarity feature vector; a fusion subunit, used to perform feature fusion analysis on the spliced ​​similarity feature vector to obtain a spliced ​​interaction feature vector; and a restoration subunit, used to restore the interaction feature vector of each of the sample candidate POIs based on the spliced ​​interaction feature vector.

[0138] Optionally, the fusion subunit is specifically used to perform feature fusion analysis on the splicing similarity feature vector using at least one fully connected layer to obtain the splicing interaction feature vector.

[0139] Optionally, the fusion subunit is specifically used to perform feature fusion analysis on the splicing similarity feature vector using a combination network of at least one fully connected layer and a dropout layer to obtain the splicing interaction feature vector.

[0140] Reference Figure 8 The diagram shows a structural block diagram of an interest point deduplication device according to an embodiment of the present disclosure.

[0141] like Figure 8 As shown, the point of interest deduplication device may include the following modules:

[0142] The second acquisition module 801 is used to acquire information about the POI to be processed and information about multiple candidate POIs;

[0143] The prediction module 802 is used to input the information of the POI to be processed and the information of the plurality of candidate POIs into a pre-trained POI deduplication model to obtain the similarity between each candidate POI and the POI to be processed, as output by the POI deduplication model; wherein, the POI deduplication model is trained by the model training method according to any one of claims 1 to 7.

[0144] The second determining module 803 is used to determine, based on the similarity, whether there is a POI among the candidate POIs that is duplicated with the POI to be processed.

[0145] In this embodiment of the disclosure, the POI deduplication model can analyze the information of multiple candidate POIs simultaneously. During the feature fusion analysis process, it can integrate the factors that influence the interaction between different POIs, making the analysis process more comprehensive and reasonable, thereby improving the accuracy of the POI deduplication model.

[0146] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0147] In embodiments of this disclosure, an electronic device is also provided. The electronic device may include one or more processors and one or more computer-readable storage media storing instructions thereon, such as an application program. When the instructions are executed by the one or more processors, the processors cause the processors to perform a model training method as described in any of the above embodiments, or to perform an interest point deduplication method as described in any of the above embodiments.

[0148] In embodiments of this disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored, which can be executed by a processor of an electronic device. When the computer program is executed by the processor, the processor performs either the model training method as described in any of the preceding embodiments or the interest point deduplication method as described in any of the preceding embodiments.

[0149] The processors mentioned above can be general-purpose processors, including but not limited to: Central Processing Unit (CPU), Network Processor (NP), Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0150] The computer-readable storage media mentioned above may include, but are not limited to: Read Only Memory (ROM), Random Access Memory (RAM), Compact Disc Read Only Memory (CD-ROM), Electronic Erasable Programmable Read Only Memory (EEPROM), Hard Disk, Floppy Disk, Flash Memory, etc.

[0151] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this disclosure are not directed to any particular programming language. It should be understood that the embodiments of this disclosure described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the embodiments of this disclosure.

[0152] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0153] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various inventive aspects, in the foregoing description of exemplary embodiments of this disclosure, various features of embodiments of this disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed embodiments of this disclosure require more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the present disclosure.

[0154] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose.

[0155] The various component embodiments of this disclosure can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the motion picture generation device according to the embodiments of this disclosure. Embodiments of this disclosure can also be implemented as device or apparatus programs for performing some or all of the methods described herein. Such programs implementing embodiments of this disclosure can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0156] It should be noted that the above embodiments are illustrative of embodiments of this disclosure and not restrictive of embodiments of this disclosure, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of this disclosure can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0157] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0158] The above description is merely a specific implementation of the embodiments of this disclosure, but the protection scope of the embodiments of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this disclosure should be covered within the protection scope of the embodiments of this disclosure.

Claims

1. A model training method, characterized in that, include: Acquire sample data, which includes information on the point of interest (POI) to be processed and information on multiple candidate POIs. In the preset training model, feature fusion analysis is performed based on the information of the sample POI to be processed and the information of the multiple sample candidate POIs to obtain the sample similarity between each of the sample candidate POIs and the sample POI to be processed. In response to determining that training is complete based on the sample similarity, the trained model is used as the POI deduplication model; The feature fusion analysis based on the information of the POI to be processed in the sample and the information of the multiple candidate POIs in the sample includes: For each candidate POI, a similarity feature vector is extracted based on the information of the current candidate POI and the information of the POI to be processed. The similarity feature vectors are subjected to feature fusion analysis to obtain the interaction feature vectors of each candidate POI of the samples; For each candidate POI, the sample similarity between the current candidate POI and the POI to be processed is determined based on the interaction feature vector of the current candidate POI.

2. The method according to claim 1, characterized in that, The step of extracting a similarity feature vector for the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed includes: For each piece of information of the current sample candidate POI, obtain the similarity information of the current information based on the corresponding information of the current information and the sample POI to be processed, and extract the similarity feature vector of the current information based on the similarity information; The similarity feature vectors of each piece of information are concatenated to obtain the first concatenated vector. Feature fusion analysis is then performed on the first concatenated vector to obtain the similarity feature vector of the current sample candidate POI.

3. The method according to claim 1, characterized in that, The step of extracting a similarity feature vector for the current sample candidate POI based on the information of the current sample candidate POI and the information of the sample POI to be processed includes: For each piece of information belonging to the first type of candidate POI in the current sample, a similarity feature vector is extracted based on the corresponding information between the current information and the POI to be processed in the sample. For each piece of information belonging to the second type of candidate POI in the current sample, the similarity information of the current information is obtained based on the corresponding information of the current information and the POI to be processed in the sample, and the similarity feature vector of the current information is extracted based on the similarity information. The similarity feature vectors of each type of information are concatenated to obtain a second concatenated vector. Feature fusion analysis is then performed on the second concatenated vector to obtain a fused feature vector. The similarity feature vector of the information belonging to the first type is concatenated with the fused feature vector to obtain the similarity feature vector of the current sample candidate POI.

4. The method according to claim 1, characterized in that, The step of performing feature fusion analysis on the similarity feature vectors to obtain the interaction feature vectors of each candidate POI includes: The similarity feature vectors are concatenated to obtain the concatenated similarity feature vector; The splicing similarity feature vector is subjected to feature fusion analysis to obtain the splicing interaction feature vector; The interaction feature vectors of each candidate POI are obtained by reconstructing the spliced ​​interaction feature vectors.

5. The method according to claim 4, characterized in that, The step of performing feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector includes: The splicing similarity feature vector is analyzed by using at least one fully connected layer to obtain the splicing interaction feature vector.

6. The method according to claim 4, characterized in that, The step of performing feature fusion analysis on the splicing similarity feature vector to obtain the splicing interaction feature vector includes: The splicing similarity feature vector is analyzed by using a combination network of at least one fully connected layer and a dropout layer to obtain the splicing interaction feature vector.

7. A method for determining duplicate points of interest, characterized in that, include: Obtain information about the POI to be processed and information about multiple candidate POIs; The information of the POI to be processed and the information of the multiple candidate POIs are input into a pre-trained POI deduplication model to obtain the similarity between each candidate POI and the POI to be processed, as output by the POI deduplication model; wherein, the POI deduplication model is trained by the model training method according to any one of claims 1 to 6. Based on the similarity, it is determined whether there is a POI among the candidate POIs that is duplicated with the POI to be processed.

8. A model training device, characterized in that, include: The first acquisition module is used to acquire sample data, which includes information on the POI to be processed and information on multiple candidate POIs. The training module is used to perform feature fusion analysis on the sample POI to be processed and the information of multiple sample candidate POIs in a preset model to be trained, to obtain the sample similarity between each of the sample candidate POIs and the sample POI to be processed. The feature fusion analysis based on the information of the sample POI to be processed and the information of the multiple sample candidate POIs includes: For each candidate POI, a similarity feature vector is extracted based on the information of the current candidate POI and the information of the POI to be processed. The similarity feature vectors are subjected to feature fusion analysis to obtain the interaction feature vectors of each candidate POI of the samples; For each candidate POI, the sample similarity between the current candidate POI and the POI to be processed is determined based on the interaction feature vector of the current candidate POI. The first determining module is used to determine that training is complete based on the sample similarity and to use the trained model as the POI deduplication model.

9. A point of interest deduplication device, characterized in that, include: The second acquisition module is used to acquire information about the POI to be processed and information about multiple candidate POIs; The prediction module is used to input the information of the POI to be processed and the information of the multiple candidate POIs into a pre-trained POI deduplication model to obtain the similarity between each candidate POI and the POI to be processed, as output by the POI deduplication model; wherein, the POI deduplication model is trained by the model training method according to any one of claims 1 to 6. The second determining module is used to determine, based on the similarity, whether there is a POI among the candidate POIs that is duplicated with the POI to be processed.

10. An electronic device, characterized in that, include: One or more processors; and one or more computer-readable storage media on which instructions are stored; When the instruction is executed by the one or more processors, the processor performs the model training method as described in any one of claims 1 to 6, or performs the interest point deduplication method as described in claim 7.

11. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, causes the processor to perform the model training method as described in any one of claims 1 to 6, or to perform the interest point deduplication method as described in claim 7.

Images