A POI classification method based on multi-source data and related device

Abstract

The application discloses a POI classification method based on multi-source data and a related device. The method combines text editing distance, discrete degree mechanism and natural language processing to perform POI classification, so that the cost of manual marking can be reduced, and the classification accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic map, in particular to a POI classification method and device based on multi-source data. BACKGROUND

[0002] POI (Point of Interest) refers to point data in an Internet electronic map, which basically includes four attributes of name, address, coordinate and category. Since POI covers a large amount of information such as address, latitude and longitude, contact information, name, etc., with the rapid development of electronic maps, POI data also includes picture data, comment data and score data. Since POI data comes from various sources and all dimensions and data are quite different, the POI classification of each platform is also different. Therefore, when performing big data analysis on POI, there is a problem of insufficient accuracy in classifying POI by using the name of the point of interest. SUMMARY

[0003] In view of the above problems of the prior art, the present application aims to provide a POI classification method based on multi-source data, which combines text edit distance, dispersion mechanism and natural language processing to classify POI, thereby reducing the cost of manual labeling and improving the accuracy of classification.

[0004] To achieve the above-mentioned application purposes, the present application provides the following technical solutions.

[0005] A POI classification method based on multi-source data comprises the following steps:

[0006] S1: obtaining POI data provided by an Internet platform, and performing word segmentation processing on the text information of the obtained POI data to obtain feature words corresponding to each POI data;

[0007] S2: calculating the edit distance between the feature words of each POI data and the feature words of a target category, and when the calculated edit distance score reaches a preset threshold, taking the classification category corresponding to the target category as the classification category of the POI data;

[0008] S3: taking the POI data with a determined classification category as a training set, calculating the prior probability of the feature words of each POI data, and then calculating the posterior probability of each classification category according to the POI data with an undetermined classification category, and taking the classification category with the maximum posterior probability as the classification category of the POI data;

[0009] S4: According to the feature words of the POI data of the determined classification category, and using the mean shift clustering algorithm, it is judged whether there is POI data with an outlying degree exceeding a preset threshold in each classification category, if there is, it is marked as abnormal data, and step S5 is entered, if not, the classification category corresponding to all POI data is output;

[0010] S5: The POI data not marked as abnormal data is used as a training set of a machine learning model, the machine learning model is trained, and after the training is successful, the POI data marked as abnormal data is input into the machine learning model, the corresponding classification category is output, and the output classification category is re-determined as the classification category of the POI data, and step S4 is re-entered.

[0011] According to a specific embodiment, in step S1 of the POI classification method based on multi-source data, before the word segmentation processing is performed, POI data with high similarity is fused into one POI data according to the similarity of POI accessory information in the POI data; wherein the POI accessory information includes the name, address, contact information and latitude and longitude of the POI.

[0012] According to a specific embodiment, in step S1 of the POI classification method based on multi-source data, the text information of the obtained POI data is subjected to word segmentation processing, which includes:

[0013] After the first word segmentation processing is performed on each POI data, the initial edit distance of the feature words of the POI data and the text information of the POI data is calculated, if the initial edit distance is lower than a preset threshold, the word segmentation processing mode is adjusted to re-perform the word segmentation processing on the POI data, until the initial edit distance is not lower than the preset threshold.

[0014] According to a specific embodiment, in step S3 of the POI classification method based on multi-source data, the probability of each feature word appearing is used as a prior probability, and the prior probability and the probability of each feature word in the corresponding classification category are substituted into a Bayes formula to calculate the posterior probability.

[0015] According to a specific embodiment, in step S5 of the POI classification method based on multi-source data, before the machine learning model is trained, each POI data not marked as abnormal data is constructed as a training feature containing three channel feature encodings; wherein the POI name, feature words and comment data in each POI data are subjected to feature encoding processing to obtain three channel feature encodings.

[0016] Further, when the training feature is constructed, the feature words are enhanced by adding the synonyms of the feature words.

[0017] In another aspect of the specific implementation, the present invention further provides an electronic device, comprising:

[0018] Data acquisition module, used to obtain POI data provided by the Internet platform;

[0019] The word segmentation processing module is used to perform word segmentation processing on the text information of the acquired POI data to obtain the feature words corresponding to each POI data;

[0020] A first classification module is configured to calculate the edit distance between the feature word of each POI data and the feature word of the target classification. When the calculated edit distance score reaches a preset threshold, the classification category corresponding to the target classification is used as the classification category of the POI data;

[0021] A second classification module is configured to use the POI data with determined classification categories in the first classification module as a training set, calculate the prior probability of the feature word corresponding to each POI data, and then calculate the posterior probability corresponding to each classification category based on the POI data with undetermined classification categories, and use the classification category with the maximum posterior probability as the classification category of the POI data;

[0022] The anomaly analysis module is used to determine whether there is POI data with an outlier degree exceeding a preset threshold in each classification category based on the feature words of the POI data of the determined classification category and using the mean shift clustering algorithm. If so, it is marked as an anomaly data and fed into the machine learning model for processing; if not, the classification category corresponding to all POI data is output;

[0023] The machine learning module is used to use POI data that is not marked as abnormal data as a training set for the machine learning model, train the machine learning model, and after successful training, input the POI data marked as abnormal data into the machine learning model, output the corresponding classification category, and re-determine the output classification category as the classification category of the POI data, and re-enter the abnormality analysis module for processing.

[0024] According to a specific embodiment, in an electronic device of the present invention, the word segmentation processing module is used to merge POI data with higher similarity into one POI data based on the similarity of POI ancillary information in the POI data before entering the word segmentation processing module for the word segmentation processing; wherein the POI ancillary information includes the name, address, contact information, and latitude and longitude of the POI.

[0025] According to a specific embodiment, in the electronic device of the present application, the word segmentation processing module is configured to, after performing first word segmentation processing on each POI data, calculate an initial edit distance between the feature words of the POI data and the text information of the POI data, and if the initial edit distance is lower than a preset threshold, adjust the word segmentation processing mode to re-perform word segmentation processing on the POI data until the initial edit distance is not lower than the preset threshold.

[0026] According to a specific embodiment, in the electronic device of the present application, the machine learning module is configured to, before training the machine learning model, construct each POI data that is not labeled as abnormal data into a training feature containing three channel feature encodings; wherein the POI name, feature words and comment data in each POI data are subjected to feature encoding processing to obtain three channel feature encodings.

[0027] Compared with the prior art, the present application has the following beneficial effects:

[0028] The POI classification method based on multi-source data of the present application can reduce the cost of manual labeling and improve the classification accuracy by combining text edit distance, dispersion mechanism and natural language processing for POI classification. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a flowchart of the method of the present application;

[0030] Figure 2 is a structural schematic diagram of the electronic device of the present application. DETAILED DESCRIPTION

[0031] The present application will be further described in detail below in combination with the drawings and specific embodiments. However, it should not be understood that the above-mentioned subject matter of the present application is limited to the following embodiments only, and any technology realized based on the content of the present application falls within the scope of the present application.

[0032] As shown in a POI classification method based on multi-source data, comprising the following steps: Figure 1

[0033] S1: obtaining POI data provided by an Internet platform, and performing word segmentation processing on the text information of the obtained POI data to obtain feature words corresponding to each POI data;

[0034] S2: calculating an edit distance between the feature words of each POI data and the feature words of a target classification, and when the calculated edit distance score reaches a preset threshold, taking the classification category corresponding to the target classification as the classification category of the POI data;

[0035] ​S3: taking the POI data of the determined classification category as a training set, calculating the prior probability of the feature word corresponding to each POI data, and then calculating the posterior probability of each classification category according to the POI data of the undetermined classification category, and taking the classification category with the maximum posterior probability as the classification category of the POI data; specifically, taking the probability of the occurrence of each feature word as the prior probability, and substituting the prior probability and the probability of each feature word in the corresponding classification category into the Bayes formula to calculate the posterior probability.

[0036] S4: judging whether there is POI data with an outlying degree exceeding a preset threshold in each classification category according to the feature words of the POI data of the determined classification category and using the mean shift clustering algorithm, if there is, marking it as abnormal data and entering step S5, and if not, outputting the classification category corresponding to all POI data.

[0037] S5: taking the POI data not marked as abnormal data as a training set of a machine learning model, training the machine learning model, and after the training is successful, inputting the POI data marked as abnormal data into the machine learning model, outputting the corresponding classification category, and re-determining the output classification category as the classification category of the POI data and re-entering step S4.

[0038] In an embodiment, before the word segmentation processing in step S1 of the POI classification method based on multi-source data, POI data with high similarity is fused into one POI data according to the similarity of POI accessory information in the POI data; wherein the POI accessory information includes the name, address, contact information and latitude and longitude of the POI. Specifically, in step S1 of the POI classification method based on multi-source data, the text information of the obtained POI data is subjected to word segmentation processing, including:

[0039] After the first word segmentation processing of each POI data, the initial edit distance between the feature word of the POI data and the text information of the POI data is calculated, and if the initial edit distance is lower than a preset threshold, the word segmentation processing mode is adjusted to re-process the word segmentation of the POI data until the initial edit distance is not lower than the preset threshold.

[0040] In an embodiment, in step S5 of the POI classification method based on multi-source data, before training the machine learning model, each POI data not marked as abnormal data is constructed into a training feature containing three channel feature encodings; wherein the POI name, feature word and comment data in each POI data are subjected to feature encoding processing to obtain three channel feature encodings. Further, when constructing the training feature, the feature word is enhanced by adding its synonyms.

[0041] As Figure 2 The application further provides an electronic device, which comprises: A data acquisition module, configured to acquire POI data provided by an Internet platform;

[0042] A data acquisition module, configured to acquire POI data provided by an Internet platform; A data acquisition module, configured to acquire POI data provided by an Internet platform;

[0043] A word segmentation processing module, configured to perform word segmentation processing on text information of the acquired POI data, and obtain feature words corresponding to each piece of POI data; A first classification module, configured to calculate an edit distance between the feature words of each piece of POI data and feature words of a target classification, and when the calculated edit distance score reaches a preset threshold, take the classification category corresponding to the target classification as the classification category of the POI data;

[0044] A second classification module, configured to take POI data whose classification category has been determined in the first classification module as a training set, calculate a prior probability of the feature words of each piece of POI data, and then calculate a posterior probability of each classification category according to POI data whose classification category has not been determined, and take the classification category with the maximum posterior probability as the classification category of the POI data; A second classification module, configured to take POI data whose classification category has been determined in the first classification module as a training set, calculate a prior probability of the feature words of each piece of POI data, and then calculate a posterior probability of each classification category according to POI data whose classification category has not been determined, and take the classification category with the maximum posterior probability as the classification category of the POI data;

[0045] An anomaly analysis module, configured to determine, according to the feature words of POI data whose classification category has been determined, whether there is POI data with an outlying degree exceeding a preset threshold in each classification category by using a mean shift clustering algorithm, and if there is, mark the POI data as abnormal data and input the POI data into a machine learning model for processing; or if there is not, output the classification category corresponding to all POI data; An anomaly analysis module, configured to determine, according to the feature words of POI data whose classification category has been determined, whether there is POI data with an outlying degree exceeding a preset threshold in each classification category by using a mean shift clustering algorithm, and if there is, mark the POI data as abnormal data and input the POI data into a machine learning model for processing; or if there is not, output the classification category corresponding to all POI data;

[0046] A machine learning module, configured to take POI data not marked as abnormal data as a training set of a machine learning model, train the machine learning model, and after the training is successful, input POI data marked as abnormal data into the machine learning model, output the corresponding classification category, and re-determine the output classification category as the classification category of the POI data and re-input the POI data into the anomaly analysis module for processing. A machine learning module, configured to take POI data not marked as abnormal data as a training set of a machine learning model, train the machine learning model, and after the training is successful, input POI data marked as abnormal data into the machine learning model, output the corresponding classification category, and re-determine the output classification category as the classification category of the POI data and re-input the POI data into the anomaly analysis module for processing.

[0047] In some embodiments, the word segmentation processing module is further configured to fuse POI data with high similarity into one piece of POI data according to the similarity of POI accessory information in the POI data before the POI data enters the word segmentation processing module for the word segmentation processing; wherein the POI accessory information includes the name, address, contact information and longitude and latitude of the POI. In some embodiments, the word segmentation processing module is further configured to fuse POI data with high similarity into one piece of POI data according to the similarity of POI accessory information in the POI data before the POI data enters the word segmentation processing module for the word segmentation processing; wherein the POI accessory information includes the name, address, contact information and longitude and latitude of the POI.

[0048] In some embodiments, the word segmentation processing module is further configured to fuse POI data with high similarity into one piece of POI data according to the similarity of POI accessory information in the POI data before the POI data enters the word segmentation processing module for the word segmentation processing; wherein the POI accessory information includes the name, address, contact information and longitude and latitude of the POI. In some embodiments, the word segmentation processing module is further configured to fuse POI data with high similarity into one piece of POI data according to the similarity of POI accessory information in the POI data before the POI data enters the word segmentation processing module for the word segmentation processing; wherein the POI accessory information includes the name, address, contact information and longitude and latitude of the POI.

[0049] In some embodiments, the electronic device of the present application, the word segmentation processing module is configured to, after performing the first word segmentation processing on each POI data, calculate an initial edit distance between the feature words of the POI data and the text information of the POI data, and if the initial edit distance is lower than a preset threshold, adjust the word segmentation processing mode to re-perform the word segmentation processing on the POI data until the initial edit distance is not lower than the preset threshold.

[0050] Specifically, the machine learning module is configured to, before training the machine learning model, construct each POI data that is not labeled as abnormal data as a training feature containing three channel feature encodings; wherein the POI name, feature words and comment data in each POI data are subjected to feature encoding processing to obtain three channel feature encodings.

[0051] It should be understood that the device disclosed by the present application can be implemented in other ways. For example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the communication connection between the modules can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical or other forms.

[0052] In addition, each function module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present alone, or two or more modules can be integrated in one processing unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0053] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0054] The above merely describes preferred embodiments of the present application, and is not used to limit the present application, any modification, equivalent replacement and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A POI classification method based on multi-source data, comprising the following steps: S1: obtaining POI data provided by an Internet platform, and performing word segmentation on text information of the obtained POI data to obtain feature words corresponding to each piece of POI data; S2: calculating an edit distance between the feature words of each piece of POI data and feature words of a target classification, and when the calculated edit distance score reaches a preset threshold, taking a classification category corresponding to the target classification as the classification category of the POI data; S3: taking the POI data with the determined classification category as a training set, calculating a prior probability of the feature words corresponding to each piece of POI data, and then calculating a posterior probability of each classification category according to the POI data with an undetermined classification category, and taking a classification category with the maximum posterior probability as the classification category of the POI data; S4: judging whether there is POI data with an outlying degree exceeding a preset threshold in each classification category according to the feature words of the POI data with the determined classification category and using a mean shift clustering algorithm, and if there is, marking the POI data as abnormal data and entering step S5; if not, outputting the classification category corresponding to all POI data; S5: taking the POI data not marked as abnormal data as a training set of a machine learning model, training the machine learning model, and after the training is successful, inputting the POI data marked as abnormal data into the machine learning model to output the corresponding classification category, and re-determining the output classification category as the classification category of the POI data and re-entering step S4.

2. The POI classification method based on multi-source data according to claim 1, wherein in step S1, before the word segmentation processing, POI data with high similarity are fused into one POI data according to the similarity of POI accessory information in the POI data. The POI attached information includes the name, address, contact information and latitude and longitude of the POI. 3.The POI classification method based on multi-source data according to claim 1, wherein in step S1, the word segmentation on the text information of the obtained POI data comprises: After first word segmentation is performed on each piece of POI data, an initial edit distance between the feature words of the POI data and the text information of the POI data is calculated, and if the initial edit distance is lower than a preset threshold, the word segmentation mode is adjusted to re-perform word segmentation on the POI data until the initial edit distance is not lower than the preset threshold. 4.The POI classification method based on multi-source data according to claim 1, wherein in step S3, the probability of each feature word appearing is taken as a prior probability, and the prior probability and the probability of each feature word appearing in the corresponding classification category are substituted into a Bayes formula to calculate the posterior probability. 5.The method of claim 1, wherein, In step S5, before the machine learning model is trained, each piece of POI data not marked as abnormal data is constructed into a training feature containing three channel feature encodings; wherein the POI name, feature words and comment data in each piece of POI data are subjected to feature encoding processing to obtain three channel feature encodings. 6.The method of claim 5, wherein, When the training feature is constructed, the feature words are enhanced by adding synonyms of the feature words.

7. An electronic device, comprising: It comprises: a data acquisition module configured to acquire POI data provided by an Internet platform; The word segmentation processing module is configured to perform word segmentation processing on the text information of the obtained POI data to obtain feature words corresponding to each piece of POI data. The first classification module is configured to calculate an edit distance between the feature words of each piece of POI data and feature words of a target category, and when the calculated edit distance score reaches a preset threshold, take the classification category corresponding to the target category as the classification category of the POI data. The second classification module is configured to take the POI data whose classification category has been determined in the first classification module as a training set, calculate a prior probability of the feature words of each piece of POI data, and then calculate a posterior probability of each classification category according to the POI data whose classification category has not been determined, and take the classification category with the maximum posterior probability as the classification category of the POI data. The anomaly analysis module is configured to determine, according to the feature words of the POI data whose classification category has been determined, whether there is POI data with an outlying degree exceeding a preset threshold in each classification category by using a mean shift clustering algorithm, and if there is, mark the POI data as abnormal data and input the POI data into a machine learning model for processing, and if there is not, output the classification category corresponding to all POI data. The machine learning module is configured to take the POI data that is not marked as abnormal data as a training set of a machine learning model, train the machine learning model, and after the training is successful, input the POI data marked as abnormal data into the machine learning model to output the corresponding classification category, and then take the output classification category as the classification category of the POI data and re-input the POI data into the anomaly analysis module for processing.

8. An electronic device as claimed in claim 7, characterized in that The word segmentation processing module is further configured to, before the word segmentation processing, fuse POI data with high similarity into one piece of POI data according to the similarity of POI accessory information in the POI data, and the POI accessory information includes the name, address, contact information, and longitude and latitude of the POI.

9. An electronic device as claimed in claim 8, characterized in that The word segmentation processing module is configured to, after performing the first word segmentation processing on each piece of POI data, calculate an initial edit distance between the feature words of the POI data and the text information of the POI data, and if the initial edit distance is lower than a preset threshold, adjust the word segmentation processing mode to perform word segmentation processing on the POI data again until the initial edit distance is not lower than the preset threshold.

10. An electronic device as claimed in claim 9, characterized in that The machine learning module is configured to, before training the machine learning model, construct each piece of POI data that is not marked as abnormal data into a training feature containing three channel feature encodings, and obtain the three-channel feature encodings by performing feature encoding processing on the POI name, feature words, and comment data in each piece of POI data.

Images