A method for cleaning and classifying POI data based on industry classification
By using predetermined industry classification mapping and grading rules and the semantic vectorization technology of the BERT model, combined with the LightGBM classifier, the problem of corresponding map POI data with national standard industry categories was solved, achieving accurate classification and cross-platform comparability of POI data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- INST OF URBAN ENVIRONMENT CHINESE ACAD OF SCI
- Filing Date
- 2026-01-22
- Publication Date
- 2026-05-26
Smart Images
Figure CN122086871A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of geographic information data processing technology, and in particular to a method for cleaning and classifying POI data for industry classification. Background Technology
[0002] In fields such as geographic information services, smart city construction, and business data analysis, map POI (Point of Interest) data is one of the core basic data, and the accuracy and standardization of its industry classification directly determine the application value of the data.
[0003] Currently, mainstream map service providers all use their own defined POI industry classification systems for data collection and labeling. Due to differences in business positioning and data collection criteria among these providers, their classification standards exhibit significant fragmentation in terms of category hierarchy, name definitions, and attribution rules. Furthermore, existing POI data classification methods struggle to effectively correspond with national industry standards such as the "National Economic Industry Classification," leading to ambiguity in the industry attributes of POI data and a lack of authoritative and cross-platform comparability in classification results. For instance, in cross-provider data fusion applications, the inconsistent classification systems prevent the effective integration of large amounts of data, resulting in a waste of data resources. Summary of the Invention
[0004] The embodiments of the present invention provide a method for cleaning and classifying POI data based on industry classification, which aims to solve the problem that the industry classification of POI data in existing technology maps is mostly a custom system defined by each map provider, which is difficult to correspond with the national standard industry categories, resulting in fuzzy industry classification of POI data and a lack of authority and comparability.
[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for cleaning and classifying POI data for industry-specific classification, comprising the following steps: Obtain the original classification system table for all POI data on the map, and remove POI data that do not have a clear industry attribute from the original classification system table to obtain the POI data with original industry classification; the original classification system table includes the POI data name, the district / county to which it belongs, and the original major-minor industry classification of the POI data; Using predetermined industry classification mapping and grading rules, predetermined national standard industry categories are mapped to POI data that conform to the mapping rules; For POI data of companies and enterprises that are not mapped to any category, candidate companies and enterprises of the district or county to which they belong are screened from the enterprise registration information. From the candidate companies and enterprises, similar companies and enterprises whose company names or former names have a similarity score greater than a predetermined similarity threshold with the company names or former names of the companies and enterprises that are not mapped to any category are matched. The known national standard industry categories of the similar companies and enterprises are then mapped to the corresponding POI data. The BERT language model is used to uniformly convert the company names in the company and enterprise POI data into semantic vectors of a predetermined length; Using the successfully matched company / enterprise POI data, an industry semantic library is formed with name semantic vectors and industry categories, and the industry semantic library is used as samples to train the LightGBM classifier model. For company / enterprise POI data that fails to match, its semantic vector is input into the trained LightGBM classifier model, which outputs the corresponding national standard industry category, thus completing the industry mapping of the company / enterprise POI data.
[0006] Furthermore, the step of mapping predetermined national standard industry categories to POI data that conforms to the mapping rules using predetermined industry classification mapping and grading rules includes the following steps: Pre-obtain the business definition scope of each major category in the map POI data and compare it with the business definition scope of the national standard industry categories; If the business definition scope of a certain POI data category is completely or highly overlapped with a certain national standard industry category, then a unique mapping relationship between the category and the national standard industry category will be established, and the national standard industry category will be directly mapped to the POI data category. If the business definition scope of a certain POI data category covers multiple national standard industry categories, then no mapping will be established at the category level, and the processing will be left to the intermediate category level. If the business definition scope of a class in the POI data completely belongs to or highly overlaps with a certain national standard industry category, then the national standard industry category shall be used to map the class in the POI data. If the class mapping in the POI data fails, then the process will revert to matching the POI data category with the national standard industry category. If the POI data category still cannot be matched and mapped to the national standard industry category, then the POI data is marked as "unmapped category".
[0007] Furthermore, the company names in the business registration information are also standardized simultaneously with the company names in the POI data. The standardization process includes removing company suffixes, removing parentheses, and / or removing leading and trailing whitespace.
[0008] Further, the step of matching similar companies from the candidate companies whose company names or former names have a similarity score greater than a predetermined similarity threshold with unmapped category companies includes: using string similarity based on word ordering to compare the company names or former names of each candidate company in the business registration information with the company names in the POI data; when the string similarity score of the company is not lower than the preset similarity threshold, the candidate company is selected as a similar company in the POI data.
[0009] Furthermore, the step of using the BERT language model to transform the company names in the company / enterprise POI data into semantic vectors of a predetermined length includes: After segmenting, padding, and / or truncating the company names in the POI data, they are input into the BERT language model. The BERT language model sorts the processed company name word vectors by using position vectors composed of predefined classification tags and delimiter tags to obtain sentence vectors of a predetermined length, i.e. semantic vectors.
[0010] Furthermore, the LightGBM classifier model is trained using a hierarchical K-fold cross-validation method.
[0011] Secondly, the present invention provides a map POI data industry classification device, including a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement the POI data cleaning and classification method for industry classification as described above.
[0012] Thirdly, the present invention provides a computer-readable storage medium storing at least one program, which is executed by a processor to implement the industry-oriented POI data cleaning and classification method as described above.
[0013] Fourthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the industry-oriented POI data cleaning and classification method as described above.
[0014] The above technical solution has the following technical effects: The process involves: obtaining the original classification system table for all POI data on the map and removing POI data without clear industry attributes to obtain POI data with original industry classifications; using predetermined industry classification mapping rules to map predetermined national standard industry categories to POI data that conform to the mapping rules; for company POI data without mapping classifications, selecting candidate companies from the company's business registration information for their respective districts and counties, and matching similar companies whose company names or former names have a similarity score greater than a predetermined similarity threshold with the unmapped company names, and mapping the known national standard industry categories of the similar companies to the corresponding POI data; using the BERT language model to convert the company names of the company POI data into semantic vectors of a predetermined length, and constructing an industry semantic library using the semantic vectors of successfully matched company POI data and their corresponding national standard industry category labels, and training a LightGBM classifier model; for company POI data that failed to match, inputting the semantic vectors into the pre-trained LightGBM classifier model, and outputting the corresponding national standard industry categories to map the company POI data. This invention solves the problem that existing map POI data industry classifications are mostly custom systems defined by each map provider, making it difficult to correspond with national standard industry categories, resulting in ambiguous POI data industry classifications and a lack of authority and comparability. Attached Figure Description
[0015] Figure 1 This is a flowchart illustrating a method for cleaning and classifying POI data for industry-specific classification according to an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the process of classifying company POI data that failed to match using the BERT model and the LightGBM model in one embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a map POI data industry classification device according to an embodiment of the present invention. Detailed Implementation
[0016] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments and to explain the operating principles of the embodiments in conjunction with the relevant descriptions in the specification. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention. Components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0017] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.
[0018] Example 1: Figure 1This is a flowchart illustrating an industry-oriented POI data cleaning and classification method according to an embodiment of the present invention, as shown below. Figure 1 As shown, the method of this embodiment includes the following steps: Obtain the original classification system table for all POI data on the map to obtain the original industry-classified POI data. In one specific implementation, the attribute tables from the POI data vector files are exported and merged in batches. The exported data table must completely include the POI data name, the secondary structure of "major category - intermediate category", and the district / county to which it belongs. Categories without clear industry attributes are uniformly marked as "other" and removed from the original classification system table. Only categories with clear industry attributes are used as categories for subsequent industry mapping. In one specific implementation, "without clear industry attributes" means that the POI point exists only as a geographic marker, physical facility, temporary activity, or virtual information, does not involve continuous production, operation, or service activities, and cannot be found in the corresponding industry category in the "National Industrial Classification of Economic Activities (GB / T 4754)" standard.
[0019] Using predetermined industry classification mapping and grading rules, predetermined national standard industry categories are mapped to POI data that conform to the mapping rules; in one specific implementation, the following steps are included: Pre-obtain the business definition scope of each major category in the map POI data and compare it with the business definition scope of the national standard industry categories; If the business definition scope of a certain POI data category is completely or highly overlapped with a certain national standard industry category, then a unique mapping relationship between the category and the national standard industry category will be established, and the national standard industry category will be directly mapped to the POI data category. If the business definition scope of a certain POI data category covers multiple national standard industry categories, then no mapping will be established at the category level, and the processing will be left to the intermediate category level. If the business definition scope of a class in the POI data completely belongs to or highly overlaps with a certain national standard industry category, then the national standard industry category shall be used to map the class in the POI data. If the class mapping in the POI data fails, then the process will revert to matching the POI data category with the national standard industry category. If the POI data category still cannot be matched and mapped to the national standard industry category, then the POI data is marked as "unmapped category".
[0020] For unmapped POI data, the industry of the company / enterprise POIs is precisely matched based on their names using the company / enterprise's business registration information. The BERT model is used to semantically vectorize the company / enterprise names. An industry semantic database is established based on the matched POI data, and a LightGBM (Gradient Boosting Tree) classifier model is trained on this database. The semantic vectors of the company / enterprise names from the unmatched POI data are then input into the LightGBM classifier model to predict the industry of each POI, thus achieving industry mapping and completing global classification. Company registration information can come from either or a combination of government-published company / enterprise credit information systems and commercial paid databases. When relying solely on government information is insufficient for fine-grained industry mapping, commercial databases are supplemented to obtain fields such as industry codes and business scope. When commercial databases are unavailable, only government information can be used, and industry prediction can be performed using a semantic model. Any compliant data source that meets the availability requirements of core fields including company / enterprise name, former name, district / county, and corresponding national economic industry category can be used as the data source for this invention.
[0021] In one specific implementation, for POI data of companies without mapped categories, candidate companies belonging to their respective districts and counties are screened from the company registration information. From these candidate companies, similar companies whose names or former names have a similarity score greater than a predetermined similarity threshold with the unmapped categories are matched. The known national standard industry categories of these similar companies are then mapped to the corresponding POI data. In another specific implementation, during matching, both the company names in the company registration information and the company names in the POI data are standardized simultaneously, including removing company suffixes, removing parentheses, and / or removing leading and trailing whitespace. In one specific implementation, the similarity threshold is 85.
[0022] In one specific implementation, matching candidate companies with those whose company names or former names have a similarity score greater than a predetermined similarity threshold to unmapped category companies includes: using string similarity based on lexical sorting to compare the company names or former names of each candidate company in the business registration information with the company names in the POI data; when the string similarity score of a company is not lower than the preset similarity threshold, the candidate company is selected as a similar company in the POI data.
[0023] Figure 2 This is a schematic diagram illustrating the process of classifying company / enterprise POI data that failed to match into an industry, using the BERT and LightGBM models in one embodiment of the present invention. Figure 2As shown, the company names in the POI data are segmented, padded, and / or truncated before being input into the BERT language model. The BERT language model sorts the processed company name word vectors by using a position vector composed of predefined classification tags [CLS] and delimiter tags [SEP] to obtain a sentence vector of a predetermined length, i.e., a semantic vector. Based on the POI data of companies and enterprises that have been successfully matched to the industry, an industry semantic library is constructed by obtaining their semantic vectors and their corresponding national standard industry category tags. We used samples from an industry semantic library as training data and employed hierarchical K-fold cross-validation to train the LightGBM classifier model. Input the semantic vectors of companies that failed to match into the trained LightGBM classifier model, and output the corresponding national standard industry category to map the POI data of companies.
[0024] In one specific implementation, an industry semantic library is constructed by matching semantic vectors of companies in the industry with paired samples of industry categories. This includes a feature matrix X, a vector file of the form N×d (e.g., .npy), where N is the number of samples and d is the vector dimension; and a label sequence y, an industry category label file corresponding one-to-one with X (e.g., .csv).
[0025] In one specific implementation, the semantic vectors of company names and their industry labels are used as the training set. The sample size of each industry category is counted, and categories with insufficient sample size are removed; only the cleaned samples are retained for training and evaluation. Hierarchical K-fold cross-validation is used to divide the data multiple times to ensure that the category ratio of the training / test set is consistent in each fold. At each fold, the LightGBM classifier model is used for fitting and prediction on the corresponding test set to obtain the classification prediction results for each sample. The arithmetic mean of the probabilities of each industry under the K sub-models is taken, and the industry with the highest probability is taken as the final predicted industry. At the same time, the model performance evaluation index is output.
[0026] In one specific implementation, this embodiment uses the data "POI.shp of a certain city in 2020" as an example to illustrate the POI data cleaning and classification method oriented towards industry classification, including the following steps: 1) Obtain the original classification system table of a certain map POI data. The original classification of the map POI data contains 24 major categories, 272 medium categories, and 915 minor categories, as shown in Table 1.
[0027] Table 1. Original Classification System of POIs
[0028] 2) Export the attribute table from the vector file “2020 City POI Map POI.shp” to form “2020 City POI Data Attribute Table.csv”. The exported data table contains 8 fields: POI data name, major category, intermediate category, minor category (optional), province (optional), city (optional), district / county, and district / county to which it belongs, with a total of 1,048,575 data records.
[0029] 3) In the "2020 POI Data Attribute Table of a Certain City.csv", categories with no clear industry attributes such as "Place Name and Address Information", "Event Activities", "Indoor Facilities", "Access Facilities" and "Virtual Data" were uniformly marked as "Other" and removed. Only categories with clear industry attributes were used as the categories for subsequent industry mapping. A total of 745,005 data entries were included.
[0030] 4) Iterate through all attribute values contained in the "Major Category" field of "2020 POI Data Attribute Table of a Certain City.csv" and set the corresponding national standard industry category when mapping the major category. If there is a clear and unique correspondence between the major category and the national standard industry category, the major category direct mapping is used and the value is assigned according to "Major Category - Industry Category". For example, "Catering Services" corresponds to the accommodation and catering industries, as shown in Table 2. If the industry attribute corresponding to the major category is not unique, the intermediate category fine mapping is enabled and the value is assigned according to "Intermediate Category - Industry Category", as shown in Table 3.
[0031] Table 2 POI Category Mapping Rules
[0032] Table 3 Class Mapping Rules in POI
[0033] 5) Industry classification and hierarchical mapping was performed on the "2020 POI Data Attribute Table of a Certain City.csv" file, using a "hierarchical matching-backoff judgment" mechanism. A field "Mapping National Standard Industry Category" was added. When executing the mapping rules, values were assigned preferentially based on the fine mapping of the intermediate category. If a match failed, it was back to direct mapping and assignment to the major category. 637,683 matching data entries were completed, achieving a matching completion rate of 85.60%. Data entries that failed to match during hierarchical mapping were assigned the "Unclassified" value and entered the matching prediction model for company / enterprise POI data, totaling 107,023 data entries.
[0034] 6) In the National Corporate Credit Information Disclosure System: Visit https: / / www.gsxt.gov.cn / corp-query-homepage.html or the commercial paid database Qichacha (https: / / www.qcc.com / ) to retrieve basic business registration information for companies in a certain city. The information includes the following fields: company name, former name, province, city, district / county, and national standard industry category. The result is a "2020 Business Registration Information Table for Companies in a Certain City.csv" file containing 2,511,096 records.
[0035] 7) Filter out the data in the "Mapping National Standard Industry Category" field of "2020 City POI Data Attribute Table.csv" that is "Uncategorized", forming "2020 City Company POI Data Attribute Table.csv". Simultaneously perform name standardization processing on both "2020 City Enterprise Business Registration Information Table.csv" and "2020 City Company POI Data Attribute Table.csv", including: Remove common company and enterprise suffixes, including Limited Liability Company, Limited Company, Joint Stock Company, Group, Company, Holding Company, Branch, and Head Office; Remove parentheses from both Chinese and English text, including parentheses () and parentheses (). Remove leading and trailing whitespace.
[0036] After standardization, the fields "clean_name", "clean_company_name", and "clean_former_name" are obtained, which can be used in subsequent matching steps.
[0037] Records where the enterprise's business registration information [district / county] = POI [district / county] are used as a candidate subset of company POIs for subsequent comparison, thereby improving matching efficiency. Within the candidate subset, the following two types of matching rules are executed sequentially: If POI[clean_name] = Enterprise Business Information[clean_Company Name] or POI[clean_name] = Enterprise Business Information[clean_Former Name], then the match is considered successful.
[0038] When the above rules are not matched, a word-order-based string similarity matching rule is applied to each company in the candidate set. The POI[clean_name] is compared with the company's business registration information[clean_company_name] and the company's business registration information[clean_former_name], respectively. The RapidFuzz function is used to split the name string into words, sort them, and then reassemble them into a string. The edit distance similarity between the two strings is then calculated and recorded as the similarity score. A similarity score ≥ 85 is considered a successful match.
[0039] After successful matching, POI[Mapped National Standard Industry Category] = Enterprise Business Registration Information[National Standard Industry Category]. If the matching is unsuccessful, it is recorded as "Unmatched". The data with successful matching is filtered out to form the "2020 POI Data Attribute Table of Companies in a Certain City_Matching Successful.csv" as the subsequent prediction model training sample set, which contains a total of 35,940 data records, with a matching completion rate of 90.42%. The "Unmatched" data is filtered out to form the "2020 POI Data Attribute Table of Companies in a Certain City_Unmatched.csv", which contains a total of 71,083 data records, and is reserved for determining its所属 industry in the subsequent semantic vectorization and classification prediction links.
[0040] 8) Use the "2020 POI Data Attribute Table of Companies in a Certain City_Matching Successful.csv" as the paired sample set for extracting the "Name Semantic Vector - Industry Category". Use POI[Name] as the text source of the semantic vector and POI[Mapped National Standard Industry Category] as the label; perform preprocessing on the company names for BERT encoding, including: Add the classification marker [CLS] at the beginning of the sequence and the separation marker [SEP] at the end of the sequence; In this example, 32 is set as the maximum sequence length. Truncate the overly long sequences and pad the insufficient sequences according to the longest sample in this batch (padding=True, truncation=True, max_length=32); Set the batch size to 64 for batch encoding.
[0041] After encoding, use the BERT Chinese pre-trained language model to extract the position vector at the [CLS] position as the sentence vector, convert POI[Name] into a fixed-length semantic vector, using the configuration dimension parameters自带 by the pre-trained weights of the bert-base model, to obtain a name semantic vector with d = 768; stack all the vectors in order of the samples to form the feature matrix X, and save it as "X_2020 A Certain City_Companies and Enterprises_POI_Industry Matching Successful_vectors.npy", with the matrix dimension of N×d. In this example, N = 35,939 and d = 768. At the same time, generate the national economic industry label sequence y corresponding to X one by one, and save it as the industry category label file of "y_A Certain City_Companies and Enterprises_POI_Industry Matching Successful_labels.csv". In this example, the label sequence length N = 35,939, ensuring that the i-th row of X strictly corresponds to the i-th item of y. The feature matrix file and the label sequence together form the industry semantic library.
[0042] 9) Load the feature matrix file "X_2020 a certain city_company enterprise POI_industry matching success_vectors.npy" and the industry category label file of the label sequence "y_a certain city_company enterprise POI_industry matching success_labels.csv" included in the industry semantic library. Conduct a category frequency statistics on y, eliminate the categories with insufficient sample numbers (set the minimum sample threshold m_min, for example, in this example, m_min = 100, not limited to this), and only retain the cleaned samples for training and evaluation; synchronously update the label mapping dictionary and sample index to ensure that X, y are consistent with the set of national economic industry categories. Use a label encoder to discretize and encode the cleaned y to obtain the numerical label y, and save the one-to-one mapping from the national economic industry categories to integers, so as to inverse-transform the numerical label back to the national economic industry categories in the prediction stage.
[0043] Perform multiple divisions on the cleaned samples using stratified K-fold cross-validation. For example, in this example, K = 5, not limited to this, to ensure that the category proportions of each fold's training / test set are consistent: set a random seed, for example, seed = 2020, not limited to this. Ensure that the results are reproducible; for the k-th fold, obtain: training index train_idx(k), test index test_idx(k); slice from X and y to get X_train, y_train, X_test, y_test. On each fold, construct a multi-class LightGBM model and fit it on the training set. When fitting, use X_train and y_train to fit the model on each fold to obtain the sub-model of this fold and its optimal state on the training set. On the k-th fold, use the best model of this fold to predict the corresponding test set, retain the original index corresponding to each sample for alignment and summarization between folds. Align and summarize the test outputs of the K sub-models according to the sample index, take the arithmetic mean of the class probabilities of the same sample on the K sub-models, and the national economic industry category with the highest probability is used as the final predicted industry category of this sample to achieve a 100% matching completion rate.
[0044] Example Two: Figure 3 It is a schematic structural diagram of the map POI data industry classification device in an embodiment of the present invention, as Figure 3 shown. The device includes a processor 301, a memory 302, a bus 303, and a computer program stored in the memory 302 and operable on the processor 301. The processor 301 includes one or more processing cores. The memory 302 is connected to the processor 301 through the bus 303. The memory 302 is used to store program instructions. When the processor executes the computer program, it implements the steps in the above method embodiment of Embodiment One of the present invention.
[0045] Furthermore, as an executable solution, the map POI data industry classification device can be a computer unit, which can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer unit may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above-described computer unit structure is merely an example and does not constitute a limitation on the computer unit. It may include more or fewer components, or combine certain components, or use different components. For example, the computer unit may also include input / output devices, network access devices, buses, etc., and this embodiment of the invention does not limit this.
[0046] Furthermore, as an executable solution, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, connecting various parts of the entire computer unit via various interfaces and lines.
[0047] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer unit by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.
[0048] Example 3: The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the embodiments of the present invention.
[0049] If the modules / units integrated in the computer unit are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.
[0050] Example 4: The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the industry-oriented POI data cleaning and classification method as described above.
[0051] Although the invention has been specifically shown and described in conjunction with preferred embodiments, those skilled in the art should understand that various changes in form and detail may be made to the invention without departing from the spirit and scope of the invention as defined in the appended claims, all of which shall be within the scope of protection of the invention.
Claims
1. A method for cleaning and classifying POI data based on industry classification, characterized in that, Includes the following steps: Obtain the original classification system table for all POI data on the map, and remove POI data that do not have a clear industry attribute from the original classification system table to obtain the POI data with original industry classification; the original classification system table includes the POI data name, the district / county to which it belongs, and the original major-minor industry classification of the POI data; Using predetermined industry classification mapping and grading rules, predetermined national standard industry categories are mapped to POI data that conform to the mapping rules; For unmapped POI data of companies and enterprises, candidate companies and enterprises of their respective districts and counties are screened from the business registration information of the companies. Similar companies and enterprises with a similarity score greater than a predetermined similarity threshold to the company and enterprise names of the unmapped POI data are matched from the company and enterprise names or former names of the candidate companies and enterprises. The known national standard industry categories of the similar companies and enterprises are mapped to the corresponding company and enterprise POI data. The BERT language model is used to uniformly convert the company names in the company and enterprise POI data into semantic vectors of a predetermined length; Using the successfully matched company / enterprise POI data, an industry semantic library is formed with name semantic vectors and industry categories, and the industry semantic library is used as samples to train the LightGBM classifier model. For company / enterprise POI data that fails to match, its semantic vector is input into the trained LightGBM classifier model, which outputs the corresponding national standard industry category, thus completing the industry mapping of the company / enterprise POI data.
2. The POI data cleaning and classification method oriented towards industry classification according to claim 1, characterized in that, The process of mapping predetermined national standard industry categories to POI data that conforms to the mapping rules using predetermined industry classification mapping and grading rules includes the following steps: Pre-obtain the business definition scope of each major category in the map POI data and compare it with the business definition scope of the national standard industry categories; If the business definition scope of a certain POI data category is completely or highly overlapped with a certain national standard industry category, then a unique mapping relationship between the category and the national standard industry category will be established, and the national standard industry category will be directly mapped to the POI data category. If the business definition scope of a certain POI data category covers multiple national standard industry categories, then no mapping will be established at the category level, and the processing will be left to the intermediate category level. If the business definition scope of a class in the POI data completely belongs to or highly overlaps with a certain national standard industry category, then the national standard industry category shall be used to map the class in the POI data. If the class mapping in the POI data fails, then the process will revert to matching the POI data category with the national standard industry category. If the POI data category still cannot be matched and mapped to the national standard industry category, then the POI data is marked as "unmapped category".
3. The POI data cleaning and classification method oriented towards industry classification according to claim 1, characterized in that, The company names in the business registration information are also standardized along with the company names in the POI data. The standardization process includes removing company suffixes, removing parentheses, and / or removing leading and trailing whitespace.
4. The POI data cleaning and classification method oriented towards industry classification according to claim 1, characterized in that, The step of matching similar companies from the candidate companies whose company names or former names have a similarity score greater than a predetermined similarity threshold with unmapped category companies includes: using string similarity based on word ordering to compare the company names or former names of each candidate company in the business registration information with the company names in the POI data; when the string similarity score of the company is not lower than the preset similarity threshold, the candidate company is selected as a similar company in the POI data.
5. The POI data cleaning and classification method oriented towards industry classification according to claim 1, characterized in that, The process of using the BERT language model to transform company names in company-type POI data into semantic vectors of a predetermined length includes: After segmenting, padding, and / or truncating the company names in the POI data, they are input into the BERT language model. The BERT language model sorts the processed company name word vectors by using position vectors composed of predefined classification tags and delimiter tags to obtain sentence vectors of a predetermined length, i.e. semantic vectors.
6. The POI data cleaning and classification method oriented towards industry classification according to claim 1, characterized in that, The LightGBM classifier model is trained using a hierarchical K-fold cross-validation method.
7. A map POI data industry classification device, characterized in that, It includes a memory and a processor, the memory storing at least one program, the at least one program being executed by the processor to implement the industry-oriented POI data cleaning and classification method as described in any one of claims 1-6.
8. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, which is executed by a processor to implement the industry-oriented POI data cleaning and classification method as described in any one of claims 1-6.
9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the POI data cleaning and classification method for industry-oriented classification as described in any one of claims 1-6.