A method and device for processing multi-source place name data

Abstract

The present application provides a method and device for processing multi-source place name data, the method comprising: obtaining an initial place name dataset, an incremental place name dataset, and an administrative division dataset within the mission area; pre-processing the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet the general format requirements; taking the lowest administrative unit level as the basic matching fusion unit, fusing the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset to obtain a third place name dataset; performing translation processing on the place name data in the third place name dataset to obtain a candidate place name dataset of the target language type; performing spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain a target place name dataset. In this way, the technical solution of the present application can effectively improve the processing efficiency of multi-source place name data and the accuracy of the processing results.

Description

Technical Field

[0001] The present application relates to the field of geographic information technology, and in particular to a method and device for processing multi-source place name data. Background Art

[0002] Place names are proprietary names given to natural or human geographical entities at specific spatial locations. They are important basic geographic information and social public information resources, and play a significant role in national defense construction, economic development, and other areas. With the increasingly close global cooperation in data and the rapid development of Internet technology, the demand for place name data from all walks of life is increasing. Place name data is also an important form of surveying and mapping geographic information data. The construction and development of global geographic information resources requires the continuous and efficient acquisition and updating of place name data both domestically and internationally. A large amount of open source place name data and other data containing administrative place names are available on the Internet, including GeoNames, OSM (Open Street Map), GADM (Database of Global Administrative Areas), Geocode Earth, etc., which are freely available.

[0003] However, due to the different sources of Internet place name data, there are overlaps and redundancies among multi-source place names. The existing technology only performs simple acquisition, cleaning, classification mapping and fusion of multi-source place names, and there are still objective problems such as poor fusion results. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method and device for processing multi-source place name data. The technical solution of this application can effectively improve the processing efficiency of multi-source place name data and the accuracy of processing results.

[0005] The present invention provides a method for processing multi-source place name data, the method comprising:

[0006] Obtain the initial place name dataset, incremental place name dataset, and administrative division dataset within the mission area;

[0007] preprocessing the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet general format requirements;

[0008] Taking the lowest administrative unit level as a basic matching and fusion unit, fusing the place name data belonging to the same basic matching and fusion unit in the first and second place name datasets to obtain a fused third place name dataset; wherein the lowest administrative unit level is determined based on the administrative division dataset;

[0009] performing translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type;

[0010] The candidate place name dataset is processed in sequence for spatial compliance and edge integrity to obtain a target place name dataset.

[0011] Optionally, the preprocessing includes at least one of the following processing methods: data format conversion, spatial reference system unification, structure normalization processing, data cleaning, place name classification mapping, field mapping, romanization transcription and attribute encoding.

[0012] Optionally, the method of taking the lowest administrative unit level as a basic matching and fusion unit and fusing the place name data belonging to the same basic matching and fusion unit in the first place name dataset and the second place name dataset includes:

[0013] For the place name data in the first place name data set and the place name data in the second place name data set under the same basic matching fusion unit, determining whether the category codes of the two place name data are consistent, and determining whether the foreign name attribute values ​​of the two place name data are consistent;

[0014] If both are consistent, the two place name data are fused to form the place name data in the third place name data set.

[0015] Optionally, if the category codes of the two place name data are consistent but the foreign name attribute values ​​of the two place name data are inconsistent, fusing the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset includes:

[0016] According to the place name common name synonym comparison rule base, identify whether two place name data are synonymous variants;

[0017] If yes, the two place name data are merged;

[0018] If not, the edit distance algorithm is used to determine the place name similarity between the two place name data;

[0019] If the place name similarity between two place name data is greater than the similarity threshold, the two place name data will be fused.

[0020] Optionally, the calculation formula for the place name similarity is:

[0021] S=1-E / L

[0022] Among them, S is the place name similarity, E is the Levenshtein edit distance between two place name data, and L is the maximum length of the foreign name string of the two place name data.

[0023] Optionally, performing translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type includes:

[0024] For each place name data in the third place name data set, identifying the language type of the place name data;

[0025] Determine target translation rules based on the language type of the place name data and the target language type;

[0026] According to the target transliteration rule, firstly the proper noun part in the place name data is segmented into place name syllables and transliterated, and then the common noun part in the place name data is translated into common name.

[0027] The word order of the translated proper nouns and common nouns is adjusted to obtain candidate place name data of the target language type, and a candidate place name dataset is formed based on all the candidate place name data.

[0028] Optionally, the step of sequentially performing spatial compliance processing and edge integrity processing on the candidate place name dataset to obtain a target place name dataset includes:

[0029] For each candidate place name data in the candidate place name dataset, the spatial topological relationship of the candidate place name data is verified according to the corresponding core vector element data, and the abnormality correction and abnormality record of the verification result are performed;

[0030] Obtain the edge reference data corresponding to the candidate place name dataset;

[0031] According to the edge reference data and a preset edge processing strategy, the edge place name data in the candidate place name dataset after spatial compliance processing is processed to obtain a target place name dataset.

[0032] The present application also provides a device for processing multi-source place name data, the device comprising:

[0033] The acquisition module is used to obtain the initial place name dataset, incremental place name dataset, and administrative division dataset within the mission area;

[0034] A preprocessing module, configured to preprocess the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet general format requirements;

[0035] a fusion module, configured to fuse the place name data belonging to the same basic matching fusion unit in the first and second place name datasets, using the lowest administrative unit level as a basic matching fusion unit, to obtain a fused third place name dataset; wherein the lowest administrative unit level is determined based on the administrative division dataset;

[0036] a translation module, configured to perform translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type;

[0037] The collaborative processing module is used to perform spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain the target place name dataset.

[0038] Optionally, the preprocessing includes at least one of the following processing methods: data format conversion, spatial reference system unification, structure normalization processing, data cleaning, place name classification mapping, field mapping, romanization transcription and attribute encoding.

[0039] Optionally, when the fusion module is used to fuse the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset using the lowest administrative unit level as the basic matching fusion unit, the fusion module is used to:

[0040] For the place name data in the first place name data set and the place name data in the second place name data set under the same basic matching fusion unit, determining whether the category codes of the two place name data are consistent, and determining whether the foreign name attribute values ​​of the two place name data are consistent;

[0041] If both are consistent, the two place name data are fused to form the place name data in the third place name data set.

[0042] Optionally, if the category codes of the two place name data are consistent but the foreign name attribute values ​​of the two place name data are inconsistent, when the fusion module is used to fuse the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset, the fusion module is used to:

[0043] According to the place name common name synonym comparison rule base, identify whether two place name data are synonymous variants;

[0044] If yes, the two place name data are merged;

[0045] If not, the edit distance algorithm is used to determine the place name similarity between the two place name data;

[0046] If the place name similarity between two place name data is greater than the similarity threshold, the two place name data will be fused.

[0047] Optionally, the calculation formula for the place name similarity is:

[0048] S=1-E / L

[0049] Among them, S is the place name similarity, E is the Levenshtein edit distance between two place name data, and L is the maximum length of the foreign name string of the two place name data.

[0050] Optionally, when the translation module is used to perform translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type, the translation module is used to:

[0051] For each place name data in the third place name data set, identifying the language type of the place name data;

[0052] Determine target translation rules based on the language type of the place name data and the target language type;

[0053] According to the target transliteration rule, firstly the proper noun part in the place name data is segmented into place name syllables and transliterated, and then the common noun part in the place name data is translated into common name.

[0054] The word order of the translated proper nouns and common nouns is adjusted to obtain candidate place name data of the target language type, and a candidate place name dataset is formed based on all the candidate place name data.

[0055] Optionally, when the collaborative processing module is used to sequentially perform spatial compliance processing and edge integrity processing on the candidate place name dataset to obtain the target place name dataset, the collaborative processing module is used to:

[0056] For each candidate place name data in the candidate place name dataset, the spatial topological relationship of the candidate place name data is verified according to the corresponding core vector element data, and the abnormality correction and abnormality record of the verification result are performed;

[0057] Obtain the edge reference data corresponding to the candidate place name dataset;

[0058] According to the edge reference data and a preset edge processing strategy, the edge place name data in the candidate place name dataset after spatial compliance processing is processed to obtain a target place name dataset.

[0059] An embodiment of the present application also provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the processing method described above are performed.

[0060] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the processing method described above are executed.

[0061] An embodiment of the present application provides a method and device for processing multi-source place name data, the processing method comprising: obtaining an initial place name dataset, an incremental place name dataset, and an administrative division dataset within a mission area; preprocessing the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet general format requirements; taking the lowest administrative unit level as a basic matching fusion unit, fusing the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset to obtain a fused third place name dataset; wherein the lowest administrative unit level is determined according to the administrative division dataset; translating the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type; and performing spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain a target place name dataset.

[0062] In this way, the technical solution provided by this application solves the problems of low matching efficiency, non-standard translation, and low quality and accuracy of place name fusion in the existing technology by improving matching algorithms, standardizing place name translation methods, and adding collaborative processing and verification. At the same time, it provides a reasonable and feasible solution for global multi-source place name fusion updates and collaborative verification.

[0063] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 A flowchart of a method for processing multi-source place name data provided in an embodiment of the present application;

[0066] Figure 2 Provides an example of a transliteration table for Russian;

[0067] Figure 3 A schematic diagram of the process of translating and writing place name data provided by this application;

[0068] Figure 4 A schematic diagram of the structure of a device for processing multi-source place name data provided in an embodiment of the present application;

[0069] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for which protection is claimed, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work falls within the scope of protection of the present application.

[0071] Place names are proprietary names given to natural or human geographical entities at specific spatial locations. They are important basic geographic information and social public information resources, and play a significant role in national defense construction, economic development, and other areas. With the increasingly close global cooperation in data and the rapid development of Internet technology, the demand for place name data from all walks of life is increasing. Place name data is also an important form of surveying and mapping geographic information data. The construction and development of global geographic information resources requires the continuous and efficient acquisition and updating of place name data both domestically and internationally. A large amount of open source place name data and other data containing administrative place names are available on the Internet, including GeoNames, OSM (Open Street Map), GADM (Database of Global Administrative Areas), Geocode Earth, etc., which are freely available.

[0072] However, due to the different sources of Internet place name data, there are overlaps and redundancies among multi-source place names. The existing technology only performs simple acquisition, cleaning, classification mapping and fusion of multi-source place names. There are still objective problems such as low matching and fusion efficiency, non-standard translation, low accuracy of results, and lack of multi-source collaborative verification.

[0073] Based on this, the embodiments of the present application provide a method and device for processing multi-source place name data to solve objective problems such as low matching and fusion efficiency, non-standard translation, low accuracy of results, and lack of multi-source collaborative verification.

[0074] See also Figure 1 , Figure 1 This is a flow chart of a method for processing multi-source place name data provided by an embodiment of the present application. Figure 1 As shown in , the processing method provided in the embodiment of the present application includes:

[0075] S101. Obtain an initial place name dataset, an incremental place name dataset, and an administrative division dataset within the mission area.

[0076] S102 : Preprocess the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet general format requirements.

[0077] S103: Taking the lowest administrative unit level as the basic matching and fusion unit, fuse the place name data belonging to the same basic matching and fusion unit in the first and second place name datasets to obtain a fused third place name dataset.

[0078] S104 : performing translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type.

[0079] S105 , performing spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain a target place name dataset.

[0080] Regarding step S101, in this step, the initial place name dataset is generally a dataset containing all or most of the place name data within the scope of the task; the incremental place name dataset generally includes new or updated place name data, which is used to supplement the deficiencies of the initial dataset; the administrative division dataset is used to describe the boundary information of administrative regions at all levels to ensure that the data can be classified and processed according to administrative divisions.

[0081] For example, in one embodiment provided in this application, an initial GeoNames place name dataset Name1 is obtained based on the scope of the mission area. Name1 contains multiple types of initial place name elements N1i. Multi-source place name data from other Internet place name websites, map websites, real estate websites, cultural and tourism websites, diplomatic websites, international organizations, and other websites are collected and acquired using algorithms such as breadth-based theme-first web crawlers and depth-based theme-first web crawlers to form an incremental place name dataset Name2. Name2 contains multiple types of incremental place name elements N2j. N1i and N2j record the latitude and longitude coordinates, place name classification, place name name, collection / update date, and other field attributes of each place name.

[0082] Obtain the corresponding GADM administrative division dataset Bound1 based on the Name1 range. In this way, extract the multi-level administrative division surface layer in Bound1 as the basic unit for place name matching and fusion.

[0083] The elements in Bound1 record the name, administrative unit level, geographic location range, official language, and other attribute fields for each country in the world. This paper uses the lowest available administrative unit level as the basic matching and fusion unit. Thus, within this basic matching and fusion unit, operations such as matching, fusion, updating, and translation are performed on the Name1 and Name2 place name elements one by one.

[0084] Regarding step S102, in this step, the initial place name dataset is preprocessed to obtain a first place name dataset, and the incremental place name dataset is preprocessed to obtain a second place name dataset.

[0085] In one embodiment provided in the present application, the preprocessing includes at least one of the following processing methods: data format conversion, spatial reference system unification, structure normalization processing, data cleaning, place name classification mapping, field mapping, romanization transcription and attribute encoding.

[0086] The data format conversion process can uniformly convert data formats (such as CSV, shapefile, JSON, etc.) from different sources (the sources of the initial place name dataset and the incremental place name dataset are generally different) into a common geospatial database format (such as Geodatabase, *.gdb), and ensure that all place name data use the same geometry type (such as point features).

[0087] The spatial reference system unification processing may be to unify the spatial reference systems of all data into a universal geographic coordinate system "GCS_WGS_1984".

[0088] The structure normalization process can unify the data structure and ensure that the names, types, and lengths of all data source fields remain consistent.

[0089] The data cleaning process can be used to clean up redundancies and errors in the data to ensure the accuracy and completeness of the data.

[0090] The place name classification mapping process can be used to classify place names according to their type (such as city, river, mountain, etc.), mapping the classification fields of different data sources to a unified category code. Specifically, the place name types in each data source can be mapped to standard category fields through a lookup table or preset rules.

[0091] The field mapping is used to uniformly map attribute fields from different data sources to the merged data structure to ensure field consistency. For example, the attribute fields of place names (such as foreign names, Chinese names, etc.) from different data sources are uniformly mapped to the target field.

[0092] The romanization is used to perform standardized romanization on place names in non-Latin alphabets to ensure the international applicability of the place names.

[0093] The attribute code is used to encode foreign language names and Chinese names to ensure that data does not appear garbled in a multilingual environment.

[0094] In addition, it can also include incremental data source processing (such as adding additional fields according to the characteristics of the incremental data source).

[0095] For example, an implementation method for preprocessing the initial place name dataset and the incremental place name dataset provided in this application includes: converting the place name data format in the initial place name dataset Name1 and the incremental place name dataset Name2 into a common geographic spatial database Geodatabase (*.gdb) format for storage, uniformly converting the geometry type into the shapefile vector point feature data format, and uniformly converting the spatial reference into the common "GCS_WGS_1984" geographic coordinate system, and performing structural normalization, data cleaning, place name classification mapping, field mapping, romanization transcription, and attribute encoding operations in turn to ensure structural consistency between data sources.

[0096] Tables 1 and 2 show the partial place name classification mapping table and the basic fields of the data attribute structure of the processed dataset (the fused third place name dataset is also shown in Table 2). Table 1 shows the partial place name classification mapping table, and Table 2 shows the place name data attribute field structure. Tables 1 and 2 contain fields such as the unique data source identifier (SRCID), category code, foreign name, Chinese name, administrative region code, and collection / update date for each place name. Additional fields can be added based on the incremental data source. Other attribute data not included in this table from the initial data source should be retained and grouped into a single attribute in the "SRCATTR" field. The "ENAME1" field contains the place name's standard Romanized spelling. Attribute values ​​in foreign name fields are uniformly encoded in UTF-8 (Unicode Transformation format - 8-bit) to eliminate garbled text caused by special characters in different foreign languages ​​(e.g., Autovía de Andalucía in Spanish). Attribute values ​​in Chinese name fields are uniformly converted to Unicode to facilitate subsequent name query matching and collaborative processing of map annotation display.

[0097] Table 1:

[0098]

[0099] Table 2:

[0100]

[0101]

[0102] Among them, Romanization transliteration: For some countries or regions, the place names are displayed in the local language of the country. In this case, it is necessary to perform Romanization transliteration according to the local language transliteration rules formulated by the country. For an example, please refer to Figure 2 , Figure 2 It should be noted that the full name of the Russian language is Russian, which is referred to as Russian in this application.

[0103] Then, after performing operations such as unifying the upper and lower case letters and processing special characters on each place name point feature in the Name1 and Name2 datasets after the above processing, the place name matching and fusion processing can be performed.

[0104] For step S103, the lowest administrative unit level is determined according to the administrative division dataset.

[0105] The administrative division dataset includes administrative unit names and administrative unit levels. The administrative unit level defines the level of each administrative region, such as national, provincial, county, township, etc. Administrative unit levels help identify the hierarchical relationships between regions. The lowest administrative unit typically refers to the smallest administrative district. Using this lowest unit as the basis for matching ensures the spatial consistency and accuracy of place name data.

[0106] Here, the lowest administrative unit level is used as the basic matching fusion unit, and the place name data in the first place name data set and the second place name data set that belong to the same basic matching fusion unit are fused, including: dividing the first place name data set and the second place name data set according to the lowest administrative unit, and fusing the place name data in the two place name data sets that belong to the same administrative unit.

[0107] For example, if a district is the lowest administrative unit, all place name data belonging to a district are extracted from the first place name dataset and the second place name dataset and fused separately.

[0108] With respect to step S103, in one embodiment provided in the present application, the lowest administrative unit level is used as a basic matching and fusion unit, and the place name data belonging to the same basic matching and fusion unit in the first place name dataset and the second place name dataset are fused, including:

[0109] S1031. For the place name data in the first place name data set and the place name data in the second place name data set under the same basic matching fusion unit, determine whether the category codes of the two place name data are consistent, and determine whether the foreign name attribute values ​​of the two place name data are consistent.

[0110] S1032: If both are consistent, the two place name data are merged to form the place name data in the third place name data set.

[0111] With respect to step S1031, the category codes and foreign name attribute values ​​of the two place name data can be determined by looking up a table.

[0112] Regarding step S1032, when fusing the two place name data, the fusion can be performed based on the place name data belonging to the first place name data set, or based on the place name data with the most attribute fields.

[0113] In another embodiment provided by the present application, if the category codes of the two place name data are consistent but the foreign name attribute values ​​of the two place name data are inconsistent, the fusion processing of the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset includes:

[0114] S1033, identifying whether two place name data are synonymous variants according to a place name common name synonym comparison rule base;

[0115] S1034: If yes, merge the two place name data.

[0116] S1035: If not, use the edit distance algorithm to determine the place name similarity between the two place name data.

[0117] S1046: If the place name similarity between the two place name data is greater than the similarity threshold, the two place name data are fused.

[0118] For step S1033, the place name synonym replacement rule base is pre-established, for example, a GSRI that statistically summarizes common common name synonym comparison rule bases with specific semantics and high frequency of occurrence, such as Raod, Rd, Highway, hyway, Hwy, Square, Sq, Drive, etc., all represent roads and are considered to represent the same place name during algorithm matching judgment.

[0119] The identification of whether the two place name data are synonymous variants is to determine whether the two place name data have word abbreviations, abbreviations, synonyms, etc.

[0120] Regarding step S1034, the fusion rules are the same as step S1032 and will not be repeated here.

[0121] With respect to step S1035, in one embodiment provided in this application, the calculation formula for the place name similarity is:

[0122] S=1-E / L

[0123] Where S is the place name similarity, E is the Levenshtein edit distance between two place name data, and L is the maximum length of the foreign name string of the two place name data. The larger S is, the higher the similarity between the two place names.

[0124] Regarding step S1036, the similarity threshold is pre-set or dynamically determined, which is not limited here.

[0125] As an example, the following content illustrates the fusion processing of place name data. This solution innovatively proposes an improved edit distance algorithm (I_Levenshtein Distance) based on the lowest administrative unit, which associates and matches each place name data in the first place name dataset N1i (originally belonging to the initial place name dataset) and the second place name dataset N2j (originally belonging to the incremental place name dataset) within the lowest administrative unit in the mission area. The specific algorithm is as follows:

[0126] 1) First, a place name synonym replacement rule base is constructed, and statistics are summarized from the GSRI rule base of common common name synonyms with specific semantics and high occurrence frequency. For example, Raod, Rd, Highway, hyway, Hwy, Square, Sq, Drive, etc. all represent roads and are considered to represent the same place name when the algorithm matches.

[0127] 2) When the CLSID category codes of the two place name data in the N1i and N2j datasets are consistent and the foreign name ENAME1 attribute values ​​are exactly the same, they are directly fused.

[0128] 3) When the CLSID category codes of two place name data in the N1i and N2j datasets are consistent, but the foreign name ENAME1 attribute values ​​are not completely consistent, the common name synonym comparison rule library GSRI matching search is added to determine whether they are word abbreviations, abbreviations, synonyms, etc. If the judgment result is true, they are directly merged. If the judgment result is false, the Levenshtein distance algorithm is used to calculate the character similarity of the foreign name ENAME1 of the two place name elements, and the elements with a similarity of more than 90% are merged.

[0129] For example, please refer to Table 3, which is a comparison table of common place name synonyms provided by this application. According to this table, it can be determined whether two place name data are synonymous variants.

[0130] Table 3:

[0131]

[0132] Regarding step S104, in this step, the fused place name dataset is transliterated, i.e., the place names are converted into the spelling of the target language, which may specifically involve romanization, phonetic conversion of Chinese place names, or standardized translation of foreign place names.

[0133] In one embodiment provided in the present application, the step of performing translation processing on the place name data in the third place name dataset based on a preset translation rule to obtain a candidate place name dataset of the target language type includes:

[0134] S1041. For each place name data in the third place name data set, identify the language type of the place name data.

[0135] S1042: Determine a target translation rule according to the language type of the place name data and the target language type.

[0136] S1043: First, according to the target transliteration rule, perform syllable segmentation and transliteration on the proper noun part of the place name data, and then perform free translation on the common noun part of the place name data.

[0137] S1044. Adjust the word order of the translated proper nouns and common nouns to obtain candidate place name data of the target language type, and form a candidate place name data set based on all the candidate place name data.

[0138] Regarding step S1041 , in this step, the language type of each place name is identified, wherein different transliteration and translation rules can be set for different language types.

[0139] Regarding step S1042, in this step, a corresponding translation rule is selected according to the language type (such as English) and the target language type (such as Chinese) of the recognized place name.

[0140] Among them, the target translation rules can be determined based on the established translation rule library. The translation rule library is used to store the multilingual transcription rule tables of various countries and the transliteration rule tables of various languages ​​required in the translation process of overseas place names.

[0141] Regarding step S1043, in this step, the place name data is split into two parts: proper noun and common noun, and then processed separately.

[0142] The proper noun is the part of a place name that is unique to it, typically referring to a specific geographic location. This part is usually translated phonetically, which means converting the pronunciation of the source language into its written form in the target language according to the pronunciation rules of the target language.

[0143] Common name part: refers to the common part of the place name, which usually describes geographical features, such as "river", "mountain", "city", etc. This part is generally translated directly, that is, its meaning is directly translated.

[0144] Regarding step S1044, in this step, after the translation is completed, the word order of the place name is adjusted to conform to the grammatical structure of the target language. For example, some languages ​​may say proper nouns first and then common nouns, while other languages ​​do the opposite.

[0145] After adjusting the word order, all processed place name data are combined into a candidate place name dataset.

[0146] For example, please refer to Figure 3 , Figure 3 A schematic diagram of the process of translating and writing place name data provided by this application, such as Figure 3 As shown, continuing with the above example, each place name to be translated in the place name dataset Name3 (the third place name dataset) is extracted, and the language type (such as Russian, German, English, etc.) of the place name is determined based on features such as text expression, etymology, and spelling rules; secondly, the component analysis of the place names is performed with reference to the translation rule library and other materials, and the translation is performed with reference to the corresponding translation guidelines. Among them, the "proper noun" part of the place name is first segmented into place name syllables and then transliterated, and the "common noun" part of the place name is translated into a common noun; then, the word order of the translated "proper noun" and "common noun" is adjusted. For example, some foreign common nouns are placed in front, and the Chinese translation should adjust the "common noun" to the back; finally, the translated place name translation result Name3 candidate place name dataset is output.

[0147] Regarding step S105, in this step, the spatial compliance processing includes checking the geometric position of the place name to ensure that it complies with spatial specifications, such as whether it is located in the correct administrative area and whether there is a logical conflict with other spatial elements.

[0148] The edge integrity processing includes ensuring that the place name data at the boundaries of different areas are seamlessly connected to avoid data overlap or breakage problems.

[0149] In one embodiment provided in the present application, the step of sequentially performing spatial compliance processing and edge integrity processing on the candidate place name dataset to obtain the target place name dataset includes:

[0150] S1051. For each candidate place name data in the candidate place name data set, verify the spatial topological relationship of the candidate place name data according to the corresponding core vector element data, and perform anomaly correction and anomaly record on the verification result.

[0151] S1052: Obtain the edge reference data corresponding to the candidate place name dataset.

[0152] S1053. Based on the edge connection reference data and a preset edge connection processing strategy, the edge connection place name data in the candidate place name dataset that has undergone spatial compliance processing is processed to obtain a target place name dataset.

[0153] In step S1051, each candidate place name in the candidate place name dataset is subjected to a spatial compliance check, i.e., the spatial topological relationship between these place names and their associated core vector feature data is verified. This process ensures that the geographic location and other spatial features of the candidate place names are correct.

[0154] Among them, if the topological relationship is found to be abnormal (for example, the location of a place name is not within the legal administrative area, or conflicts with the surrounding geographical boundaries), it needs to be corrected automatically or manually, and these abnormalities need to be recorded for subsequent processing and analysis.

[0155] With respect to step S1052 , corresponding edge reference data is obtained based on the geographic boundaries in the candidate place name dataset.

[0156] Border reference data is generally used to describe how place name data and adjacent areas are connected or matched.

[0157] Regarding step S1053, the principles followed by the preset edge connection processing strategy include: no duplication or omission of edge place name data, and maintaining consistency of the same elements.

[0158] In this way, after the edge connection process, the final result is a target place name dataset with no boundary conflicts and complete boundaries.

[0159] As an example, the process of place name collaborative processing (spatial compliance processing and edge integrity processing) is explained through the following content.

[0160] The translated Name3 place name data results are used as auxiliary reference data sets with basic geographic information 4D products, high-precision remote sensing images, map publications of various countries, etc. in the corresponding scope to conduct spatial topology checks (spatial compliance processing) and edge verification (edge ​​integrity processing) to ensure the rationality of the location relationship of the data results and the consistency of the names.

[0161] The spatial topology check includes verifying the spatial topology relationship between the translated Name3 place name data and the core vector element DLG results. Any DLG data anomalies found will be recorded in the "DLGBJ" field, and any Name3 place name data anomalies found will be recorded in the "DMBJ" field. The specific requirements are as follows:

[0162] 1) Within 1 km of each other, river place names and rivers in the DLG with the same Chinese or foreign name are considered to be the same river. Railways and major highways are treated the same as rivers.

[0163] 2) If the river place name point is no more than 200m away from the corresponding linear river, or is located within a planar river, the Chinese and foreign names should be unified.

[0164] 3) The place names of bays and straits should be located outside the coastline or on a surface water system.

[0165] 4) Lake place names should be located within the corresponding surface water system, and the Chinese names should be unified.

[0166] 5) The distance between the railway station place name and the railway is no more than 200m.

[0167] 6) The distance between road place names and the corresponding roads should not exceed 100m, and the Chinese names should be unified.

[0168] 7) The distance between railway place names and the corresponding railway should not exceed 100m, and the Chinese names should be unified.

[0169] 8) Railway branch lines, railway turnouts, railway sidings and other place names should be no more than 200m away from the main railway, and their Chinese names should be unified.

[0170] 9) The port terminal location should be no more than 200m away from the river, or no more than 1km away from the coastline on land.

[0171] 10) The distance between the waterway place-name point and the linear river is no more than 100m, or it is located within the planar river, or outside the coastline.

[0172] 11) Reservoir place names should be located within a surface water system.

[0173] 12) Correct unreasonable water drop point locations.

[0174] 13) First-level administrative center place names and second-level administrative center place names should be located in the city’s prosperous areas or residential areas.

[0175] 14) Airport place names should be located within 100 meters of the airport, and those collected incorrectly should be deleted.

[0176] 15) The dam place-name points should be located within 50 meters of the dam, and those collected incorrectly should be deleted.

[0177] 16) Mining area place names should be located within 200 meters of the mining area.

[0178] 17) Administrative place names at the third level and below should be located within 200 meters of the building complex.

[0179] Edge connection verification is to check and process the data of the same geographic entity between the place name data sets of different production units. The principles followed are as follows: there is no duplication or omission in the edge connection place name data, the same elements maintain consistency, and the uniqueness of the place names of countries and political regions across borders is guaranteed.

[0180] In this way, this application proposes a method for multi-source place name matching by obtaining the boundaries of the country's lowest-level administrative divisions as basic units, and associating and matching multi-source place name elements within the lowest administrative unit within the task area one by one, thereby improving the matching efficiency of multi-source place names and reducing the error rate of multi-source place name matching.

[0181] An improved edit distance algorithm (I_LevenshteinDistance) based on the lowest administrative unit is proposed. By constructing a place name common name synonym replacement rule base GSRI, the influencing factors such as place name abbreviations, abbreviations, and synonyms are excluded in the algorithm matching judgment, and the place name similarity threshold is set to improve the accuracy of place name matching fusion.

[0182] A multilingual translation method based on transliteration table is proposed, which mainly includes the construction of translation rule library, language and component analysis, syllable segmentation, word order adjustment, etc., realizing independent multi-source and multi-lingual Chinese-foreign translation.

[0183] The collaborative processing and verification of place name data and other reference materials has been added, which has improved the rationality and consistency of the spatial location matching relationship between place name data results and other basic geographic information data products.

[0184] Based on the same inventive concept, a processing device corresponding to the processing method is also provided in the embodiment of the present application. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned processing method in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0185] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a device for processing multi-source place name data provided by an embodiment of the present application. Figure 4 As shown in , the processing device 400 includes:

[0186] An acquisition module 410 is used to acquire an initial place name dataset, an incremental place name dataset, and an administrative division dataset within the mission area;

[0187] A preprocessing module 420 is used to preprocess the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet the general format requirements;

[0188] A fusion module 430 is configured to fuse the place name data belonging to the same basic matching fusion unit in the first and second place name datasets, using the lowest administrative unit level as a basic matching fusion unit, to obtain a fused third place name dataset; wherein the lowest administrative unit level is determined based on the administrative division dataset;

[0189] A translation module 440 is configured to perform translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type;

[0190] The collaborative processing module 450 is used to perform spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain a target place name dataset.

[0191] Optionally, the preprocessing includes at least one of the following processing methods: data format conversion, spatial reference system unification, structure normalization processing, data cleaning, place name classification mapping, field mapping, romanization transcription and attribute encoding.

[0192] Optionally, when the fusion module 430 is used to fuse the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset using the lowest administrative unit level as the basic matching fusion unit, the fusion module 430 is used to:

[0193] For the place name data in the first place name data set and the place name data in the second place name data set under the same basic matching fusion unit, determining whether the category codes of the two place name data are consistent, and determining whether the foreign name attribute values ​​of the two place name data are consistent;

[0194] If both are consistent, the two place name data are fused to form the place name data in the third place name data set.

[0195] Optionally, if the category codes of the two place name data are consistent but the foreign name attribute values ​​of the two place name data are inconsistent, when the fusion module 430 is used to fuse the place name data belonging to the same basic matching fusion unit in the first place name dataset and the second place name dataset, the fusion module 430 is used to:

[0196] According to the place name common name synonym comparison rule base, identify whether two place name data are synonymous variants;

[0197] If yes, the two place name data are merged;

[0198] If not, the edit distance algorithm is used to determine the place name similarity between the two place name data;

[0199] If the place name similarity between two place name data is greater than the similarity threshold, the two place name data will be fused.

[0200] Optionally, the calculation formula for the place name similarity is:

[0201] S=1-E / L

[0202] Among them, S is the place name similarity, E is the Levenshtein edit distance between two place name data, and L is the maximum length of the foreign name string of the two place name data.

[0203] Optionally, when the translation module 440 is used to perform translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type, the translation module 440 is used to:

[0204] For each place name data in the third place name data set, identifying the language type of the place name data;

[0205] Determine target translation rules based on the language type of the place name data and the target language type;

[0206] According to the target transliteration rule, firstly the proper noun part in the place name data is segmented into place name syllables and transliterated, and then the common noun part in the place name data is translated into common name.

[0207] The word order of the translated proper nouns and common nouns is adjusted to obtain candidate place name data of the target language type, and a candidate place name dataset is formed based on all the candidate place name data.

[0208] Optionally, when the collaborative processing module 450 is used to sequentially perform spatial compliance processing and edge integrity processing on the candidate place name dataset to obtain the target place name dataset, the collaborative processing module 450 is used to:

[0209] For each candidate place name data in the candidate place name dataset, the spatial topological relationship of the candidate place name data is verified according to the corresponding core vector element data, and the abnormality correction and abnormality record of the verification result are performed;

[0210] Obtain the edge reference data corresponding to the candidate place name dataset;

[0211] According to the edge reference data and a preset edge processing strategy, the edge place name data in the candidate place name dataset after spatial compliance processing is processed to obtain a target place name dataset.

[0212] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in FIG, the electronic device 500 includes a processor 510, a memory 520 and a bus 530.

[0213] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figure 1 as well as Figure 3 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0214] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 3 The specific implementation of the steps in the method embodiment shown can be found in the method embodiment and will not be repeated here.

[0215] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0216] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. There may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some communication interface, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0217] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0218] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0219] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0220] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The scope of protection of the present application is not limited thereto. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-mentioned embodiments within the technical scope disclosed in the present application, or perform equivalent replacements for some of the technical features thereof. These modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for processing multi-source place name data, characterized in that: The processing method comprises: Obtain the initial place name dataset, incremental place name dataset, and administrative division dataset within the mission area; the administrative division dataset is used to describe the boundaries of administrative regions at all levels. The elements in the administrative division dataset record the attribute fields corresponding to the name, administrative unit level, geographic location range, and official language of each province / city / county / town in each country in the world; the administrative division dataset also includes the administrative unit name, and the administrative unit level defines the level of each administrative region, which is used to identify the hierarchical relationship between regions. preprocessing the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet general format requirements; Taking the lowest administrative unit level as a basic matching and fusion unit, and fusing the place name data belonging to the same basic matching and fusion unit in the first and second place name datasets based on the category codes and foreign name attribute values ​​in the place name data, to obtain a fused third place name dataset; wherein the lowest administrative unit level is determined based on the administrative division dataset; performing translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type; Performing spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain a target place name dataset; The step of performing translation processing on the place name data in the third place name dataset based on a preset translation rule to obtain a candidate place name dataset of the target language type includes: For each place name data in the third place name data set, identifying the language type of the place name data; Determine target translation rules based on the language type of the place name data and the target language type; According to the target transliteration rule, firstly the proper noun part in the place name data is segmented into place name syllables and transliterated, and then the common noun part in the place name data is translated into common name. The word order of the translated proper nouns and common nouns is adjusted to obtain candidate place name data of the target language type, and a candidate place name dataset is formed based on all the candidate place name data.

2. The processing method according to claim 1, characterized in that The preprocessing includes at least one of the following processing methods: data format conversion, spatial reference system unification, structure normalization processing, data cleaning, place name classification mapping, field mapping, romanization transcription and attribute encoding.

3. The processing method according to claim 1, characterized in that The method of taking the lowest administrative unit level as the basic matching and fusion unit and fusing the place name data belonging to the same basic matching and fusion unit in the first place name dataset and the second place name dataset comprises: For the place name data in the first place name data set and the place name data in the second place name data set under the same basic matching fusion unit, determining whether the category codes of the two place name data are consistent, and determining whether the foreign name attribute values ​​of the two place name data are consistent; If both are consistent, the two place name data are fused to form the place name data in the third place name data set.

4. The processing method according to claim 3, characterized in that If the category codes of the two place name data are consistent but the foreign name attribute values ​​of the two place name data are inconsistent, fusing the place name data belonging to the same basic matching fusion unit in the first place name data set and the second place name data set includes: According to the place name common name synonym comparison rule base, identify whether two place name data are synonymous variants; If yes, the two place name data are merged; If not, the edit distance algorithm is used to determine the place name similarity between the two place name data; If the place name similarity between two place name data is greater than the similarity threshold, the two place name data will be fused.

5. The processing method according to claim 4, characterized in that: The calculation formula for the place name similarity is: Among them, S is the place name similarity, E is the Levenshtein edit distance between two place name data, and L is the maximum length of the foreign name string of the two place name data.

6. The processing method according to claim 1, characterized in that The step of sequentially performing spatial compliance processing and edge integrity processing on the candidate place name dataset to obtain a target place name dataset includes: For each candidate place name data in the candidate place name dataset, the spatial topological relationship of the candidate place name data is verified according to the corresponding core vector element data, and the abnormality correction and abnormality record of the verification result are performed; Obtain the edge reference data corresponding to the candidate place name dataset; According to the edge reference data and a preset edge processing strategy, the edge place name data in the candidate place name dataset after spatial compliance processing is processed to obtain a target place name dataset.

7. A device for processing multi-source place name data, characterized in that: The processing device comprises: An acquisition module is used to obtain an initial place name dataset, an incremental place name dataset, and an administrative division dataset within the mission area; the administrative division dataset is used to describe the boundaries of administrative regions at all levels; the elements in the administrative division dataset record the attribute fields corresponding to the name, administrative unit level, geographic spatial location range, and official language of each province / city / county / town in each country in the world; the administrative division dataset also includes the administrative unit name, and the administrative unit level defines the level of each administrative region, which is used to identify the hierarchical relationship between regions; A preprocessing module, configured to preprocess the initial place name dataset and the incremental place name dataset to obtain a first place name dataset and a second place name dataset that meet general format requirements; a fusion module, configured to fuse the place name data belonging to the same basic matching fusion unit in the first and second place name datasets based on the category codes and foreign name attribute values ​​in the place name data, taking the lowest administrative unit level as a basic matching fusion unit, to obtain a fused third place name dataset; wherein the lowest administrative unit level is determined based on the administrative division dataset; a translation module, configured to perform translation processing on the place name data in the third place name dataset based on preset translation rules to obtain a candidate place name dataset of the target language type; A collaborative processing module is used to perform spatial compliance processing and edge integrity processing on the candidate place name dataset in sequence to obtain a target place name dataset; When the translation module is used to perform translation processing on the place name data in the third place name dataset based on the preset translation rules to obtain the candidate place name dataset of the target language type, the translation module is used to: For each place name data in the third place name data set, identifying the language type of the place name data; Determine target translation rules based on the language type of the place name data and the target language type; According to the target transliteration rule, firstly the proper noun part in the place name data is segmented into place name syllables and transliterated, and then the common noun part in the place name data is translated into common name. The word order of the translated proper nouns and common nouns is adjusted to obtain candidate place name data of the target language type, and a candidate place name dataset is formed based on all the candidate place name data.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the processing method according to any one of claims 1 to 6 are executed.

Images