Place name and address cleaning and matching method and system based on confidence calculation

Abstract

The application discloses a place name and address cleaning and matching method and system based on confidence calculation, and specifically comprises the following: a to-be-cleaned data input interface, a to-be-cleaned data preprocessing module, a hierarchical standard address library, a word segmentation dictionary library, a high-frequency word management module, a cleaning engine with secondary matching, and a confidence calculation module. The application constructs a place name and address intelligent cleaning and matching system containing secondary matching, hot updating of a standard address library and intervention of high-frequency words, overcomes the static defect of a traditional hierarchical model in address cleaning and matching, and realizes precision improvement, efficiency breakthrough and resource optimization.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and more specifically, relates to a method and system for cleaning and matching place names and addresses based on confidence calculation. Background Technology

[0002] Place names and addresses are structured or unstructured texts describing the spatial location and attributes of geographic entities. They typically include key elements such as administrative region names (province / city / district / county), road names, house numbers, compound / unit names, building numbers, and geographic coordinates. Their core function is to transform spatial locations described in natural language into standardized, resolvable address information, providing fundamental support for geospatial information services.

[0003] With the acceleration of digitalization and intelligentization in my country, place name and address data has become a crucial infrastructure for scenarios such as smart city management, emergency disaster relief command, logistics route optimization, and "one-stop" government services. For example, in smart city construction, the consistency and timeliness of place names and addresses are key to information sharing and interaction between different systems and departments; in government emergency management, the accuracy of address information directly affects the efficiency of emergency response and the rationality of resource allocation; and in the logistics and distribution field, the accuracy of address resolution directly determines the timeliness of last-mile delivery and the calculation of logistics costs.

[0004] However, the application of place name and address data in actual business operations faces numerous challenges. First, the data collection channels are diverse and heterogeneous, encompassing structured databases from government departments (such as the public security standard address database), enterprise-built address databases (which may contain points of interest, aliases, and other non-standard addresses), and internet crowdsourced information (including colloquial descriptions). Data from different sources exhibit significant differences in naming standardization (e.g., the phenomenon of "Renmin Road" and "Renmin Street" being synonyms), granularity consistency (address element missing issues), and spatiotemporal dynamics (place name changes due to urban renewal). Second, new place names and addresses emerging during emergencies require immediate updates to the standard address database. Traditional data entry methods are lagging and cannot meet the business needs of high dynamism, multi-source integration, and real-time responsiveness.

[0005] The current state of technologies related to place name and address cleaning and matching is as follows:

[0006] 1) Static Hierarchical Model. This model parses address elements level by level using a pre-defined geographic entity hierarchy (e.g., country / province / city / district / street). Utilizing pre-defined dictionaries of administrative divisions, compound units, and regular expression rules, it segments the address text according to hierarchical priority and matches it level by level, verifying the logical consistency of the hierarchy. This method performs stably with highly standardized structured addresses, but when faced with addresses lacking hierarchical levels (such as the Jinhe Building in Nantong City), the mechanical reliance on a fixed order leads to a surge in matching failure rates. Furthermore, this mainstream technology employs a single-round matching paradigm, directly outputting the parsed result through a single round of rule matching without a secondary matching verification mechanism, making it difficult to match fine-grained addresses and achieve precise spatial positioning.

[0007] 2) Statistical Model Enhancement for Hierarchical Correction. This technique introduces statistical learning models (such as inverse document frequency, word vector similarity, and conditional random fields) into the hierarchical matching framework, optimizing the matching results based on probability weights. For example, by using a statistical model to perform sequence labeling on address text and identifying hierarchical tags such as "province / city / district," the results are then verified using a pre-defined hierarchical tree. However, this method relies on the static features of a standard address database. When place name address data is updated, address resolution will fail because new addresses are not promptly included in the standard address database.

[0008] 3) Introducing high-frequency word matching optimization technology. Introducing high-frequency words can enhance the recognition rate of corresponding address elements. For example, both Chongchuan District and Tongzhou District have a "Renmin Road". By introducing the high-frequency word "Chongchuan District Renmin Road", "Chongchuan District Renmin Road" will be matched first, and "Tongzhou District Renmin Road" will not be matched, reducing address ambiguity and improving matching accuracy. In existing technologies, the scope of high-frequency words is limited to a single level (e.g., high-frequency words are only associated with the road level, not the corresponding street information). The problem of broken cross-level associations is significant. When multiple cross-level high-frequency words exist, the inability to link with the associated levels will lead to mismatches of place name address data.

[0009] The existing technology currently has the following problems:

[0010] 1) Low matching success rate: Based on the results of a single round of parsing, the remaining information in the text is not fully utilized, resulting in a low success rate of fine-grained address matching and insufficient spatial positioning accuracy;

[0011] 2) Rigid matching pattern: The matching algorithm relies on a fixed address level, and its performance is poor when resolving skipped addresses in missing levels;

[0012] 3) Imperfect data update mechanism: When the standard address database is updated, the cleaning and matching service needs to be stopped, resulting in poor flexibility.

[0013] 4) Single confidence assessment factor: The matching confidence calculation relies on fixed hierarchical weights and does not consider the impact of spatiotemporal factors such as urban-rural differences, administrative division mergers, and historical place names on the confidence score;

[0014] 5) Insufficient semantic relevance of high-frequency words: High-frequency words only act on a single address level, lacking cross-level semantic linkage, which easily leads to ambiguity. Summary of the Invention

[0015] Purpose of the invention: The purpose of this invention is to address the shortcomings of the existing technology and provide a place name and address cleaning and matching method and system based on confidence calculation. When the input place name and address text to be cleaned is received, the system and method return standardized and spatialized address information to the user. Through methods such as secondary matching, hot updating of the standard address database, and intervention of high-frequency words, it provides efficient and accurate place name and address cleaning and matching services.

[0016] Technical solution: The place name address cleaning and matching method based on confidence calculation described in this invention includes the following steps:

[0017] S1. Input of data to be cleaned: Construct a data input interface, accept user request information, parse the address text to be cleaned from the input address file, and record the input metadata information;

[0018] S2. Preprocessing of data to be cleaned: Preprocessing the input data to be cleaned.

[0019] S3. Hierarchical Standard Address Database: Based on the standard address database of authoritative departments, it integrates multi-source information from alias / historical place name databases and expresses the logical relationship between address elements through a multi-level model;

[0020] S4. Hot update of word segmentation dictionary: This step is triggered after the hierarchical standard address library is hot updated in step S3. The new address entries are obtained and the Chinese word segmentation engine is used to segment the words according to the current dictionary. The word segmentation results are pushed to the database administrator within the system. The system automatically identifies text information with questionable word segmentation and marks the new addresses and adds them to the dictionary to achieve hot update.

[0021] S5. High-frequency word management: When the user enters an ambiguous address, the cleaning engine uses the corresponding high-frequency word information to prioritize matching high-frequency results.

[0022] S6. Initial matching: Perform initial matching on the data to be cleaned after preprocessing in step S2.

[0023] S7. Initial matching result evaluation: Evaluate the initial matching result of step S6 and decide whether to proceed to the second matching process.

[0024] S8. Secondary matching: Match the remaining unmatched address text after the initial matching.

[0025] S9. Confidence Calculation: After the matching process is completed, the matching confidence is calculated to reflect the reliability of the matching results.

[0026] S10, Standardized Address Output: Outputs standardized addresses that conform to a multi-level structure.

[0027] In some implementations, step S2 specifically includes: first, converting the address text into a string that meets the preset hierarchical requirements and is easy to segment based on regular expressions; then, using a text deduplication algorithm to delete redundant character information; and finally, using a rule-based description mapping table to achieve semantic normalization.

[0028] In some implementations, step S3 specifically includes: expressing the logical relationship between address elements through a seven-level model to meet the differentiated expression of addresses in urban and rural areas; organizing address entries using a tree-like hierarchical structure; and supporting hot updates of the hierarchical standard address database without stopping the cleaning service, while simultaneously triggering hot updates of the word segmentation dictionary database, in accordance with a regular periodic update + emergency event triggering update strategy.

[0029] In some implementations, step S5, high-frequency word management, specifically includes a high-frequency word database and a high-frequency word management tool. The high-frequency word database is automatically generated based on the standard addresses of high-frequency matches in the user's recent clean matching results, and is periodically updated synchronously with the hierarchical standard address database. It includes high-frequency word text, corresponding address hierarchy, high-frequency word coefficient, timestamp, and activation information.

[0030] In some implementations, step S9 confidence calculation specifically includes: calculating the matching confidence based on spatiotemporal differences, and the calculation method is: confidence = hierarchy coefficient × historical address coefficient × high-frequency word coefficient.

[0031] On the other hand, the present invention also discloses a place name and address cleaning and matching system based on confidence calculation, comprising:

[0032] Data input interface to be cleaned: Construct a data input interface that supports text, API calls and file input, and has built-in metadata information annotation function;

[0033] Data preprocessing module: performs preprocessing operations on the input data to be cleaned;

[0034] Hierarchical Standard Address Database: Based on the standard address database of authoritative departments, it integrates multi-source information from alias / historical place name databases and expresses the logical relationship between address elements through a multi-level model;

[0035] Word segmentation dictionary: used to collect, define and accurately label information at each level of place name address, assisting the Chinese word segmentation engine to correctly segment the address to be cleaned into hierarchical address text, and providing accurate address input for the cleaning engine;

[0036] High-frequency word management module: It uses a linear transformation function to map the coefficients of high-frequency words to a preset information interval, and updates it synchronously with the changes in the hierarchical standard address library. It supports manual addition, deletion, modification and query processing of high-frequency words, realizing full life cycle management of high-frequency words;

[0037] A cleaning engine with secondary matching: It performs hierarchical matching between the hierarchical segmented text output by the word segmentation dictionary and the hierarchical standard address library, and combines the high-frequency word coefficients provided by the high-frequency word management module to increase the probability of prioritizing matching addresses that users are interested in.

[0038] Confidence calculation module: This module is responsible for determining the secondary matching result and outputting the optimal matching result. It evaluates the initial matching result and then decides whether to trigger a secondary matching.

[0039] Standardized address output interface: Outputs standardized addresses that conform to a multi-level structure.

[0040] In some implementations, the data preprocessing module specifically includes: performing preprocessing operations, including regularization and text deduplication, on the data to be cleaned input into the data input interface, performing preliminary corrections on common non-standard address information, and removing some data noise.

[0041] In some implementations, the hierarchical standard address library expresses the logical relationship between address elements through a seven-level model, meets the differentiated expression of addresses in urban and rural areas, organizes address entries using a tree-like hierarchical structure, and supports hot updates of the hierarchical standard address library without stopping the cleaning service, while simultaneously triggering hot updates of the word segmentation dictionary library, in accordance with a regular periodic update + emergency event triggering update strategy.

[0042] In some implementations, the high-frequency word management module specifically includes a high-frequency word database and a high-frequency word management tool. The high-frequency word entries and coefficients in the high-frequency word database are automatically generated by summarizing the matching frequency of standard addresses in the user's recent cleaning and matching results. A linear transformation function is used to map the high-frequency word coefficients to a preset information interval [0.6, 0.9], and the database is updated synchronously with changes in the standard address database.

[0043] In some implementations, the confidence calculation module specifically includes: calculating the matching confidence based on spatiotemporal differences, and the calculation method is: confidence = hierarchical coefficient × historical address coefficient × high-frequency word coefficient.

[0044] Beneficial Effects: This invention constructs an intelligent place name and address cleaning and matching system that includes secondary matching, hot updating of the standard address database, and intervention of high-frequency words. It overcomes the static defects of traditional hierarchical models in address cleaning and matching, and achieves improved accuracy, efficiency breakthroughs, and resource optimization. Compared with the prior art, the beneficial effects of this application are as follows:

[0045] (1) Improved address matching performance

[0046] Traditional static hierarchical models have a high failure rate in matching hierarchical skip addresses. The cleaning method in this invention no longer simply relies on the fixed hierarchical order of the hierarchical model, but can automatically fill in the missing address levels based on fine-grained matching results, thereby improving the matching completeness.

[0047] Furthermore, since fine-grained addresses are often located at the end of Chinese address text, the cleaning engine with secondary matching proposed in this invention can deeply mine address text information compared to traditional single-round matching, has address error correction function, and can improve the spatial positioning capability of place names and addresses.

[0048] (2) The hot update mechanism of the standard address library improves response efficiency.

[0049] The hierarchical standard address library hot update mechanism proposed in this invention solves the drawback of needing to pause services when updating traditional static models. It supports automatic word segmentation and parsing of newly added addresses, and can instantly add and modify standard address entries in emergency scenarios, improving service flexibility while ensuring system stability.

[0050] (3) Optimize the address matching evaluation system by considering the confidence calculation method of spatiotemporal differences.

[0051] The confidence calculation method proposed in this invention comprehensively considers the differences in urban and rural address structures, uses a differentiation coefficient to calculate the confidence of urban-rural address matching, and superimposes the influence of historical place names and high-frequency words. This method is more effective than existing research that relies solely on hierarchical consistency to obtain confidence.

[0052] (4) The high-frequency word diffusion mechanism reduces the probability of address ambiguity.

[0053] The high-frequency word diffusion mechanism of this invention can realize the cross-level transmission of the influence of high-frequency words. For example, when a high-frequency word of a certain community is identified, the address weight of its administrative division will be increased at the same time, which can reduce the ambiguity and spatial positioning errors caused by the same address. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a matching method according to an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of a matching system according to an embodiment of the present invention. Detailed Implementation

[0056] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] In the description of this invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "inner", "outer", etc., indicate the orientation or positional relationship shown, and are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0058] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0059] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0060] Example 1

[0061] like Figure 1 As shown, a place name address cleaning and matching method based on confidence calculation includes the following steps:

[0062] S1. Input of Data to be Cleaned: A web data input interface is constructed to accept user HTTP requests. It then parses the input address text and address list file to obtain the address text to be cleaned, recording metadata information such as the source of the input data and timestamps. An in-memory database and message queue are used to ensure stable system operation under high concurrency, providing standardized data input for subsequent processing stages.

[0063] S2. Preprocessing of data to be cleaned: Preprocessing the input data to be cleaned. The preprocessing module integrates regular expression filtering, text deduplication, and rule-based description mapping algorithms. Specifically, it includes: first, converting the address text into a string that meets the preset hierarchical requirements and is easy to segment based on regular expressions; then, using a text deduplication algorithm to remove redundant character information; and finally, using a rule-based description mapping table (such as "Building 3" → "Building 3") to achieve semantic normalization, which serves as the input data for the initial matching.

[0064] S3. Hierarchical Standard Address Database: This database includes standard address data, historical place names, and alias address data. Based on standard address data collected from authoritative government departments, standard addresses are defined as addresses expressed in a seven-level continuous manner. The seven levels are: province, city, district / county, street / town, street address, unit / community / village, and building / group number. This allows for accurate representation of addresses in both urban and rural areas (urban standard addresses typically follow the structure "province / city / district / street / street address / unit / community / building number", such as "Jiangsu Province / Nantong City / Chongchuan District / Wenfeng Street / Hongqiao Road No. 82 / Chengnan New Village / Building 100"; rural standard addresses typically follow the structure "province / city / county / town / village / village / group number", such as "Jiangsu Province / Nantong City / Qidong City / Wangbao Town / Zhengping Village / Group 2 No. 11").

[0065] The standard address is segmented using a Chinese word segmentation engine and a thematic word segmentation dictionary. Then, spatial encoding technology is used to automatically assign a unique code of fixed length to each element at each level and attach the corresponding spatial coordinates. The address entries are organized in a tree-like hierarchical structure to form a standard address library containing the code, standard address text, and spatial location.

[0066] Furthermore, by constructing a mapping relationship between address aliases / historical place names and standard addresses (e.g., "Tongda" → "Nantong University"), correct matching of non-standard addresses such as aliases and former names can be achieved. This hierarchical standard address library supports periodic automatic updates or manual addition and removal of address entries via hot updates, enabling rapid response to new address cleaning needs in urban renewal scenarios without service interruption.

[0067] S4. Hot update of word segmentation dictionary: This step is triggered after the hierarchical standard address library is hot updated in step S3. New address entries are obtained and word segmentation is performed by the Chinese word segmentation engine according to the current dictionary. The word segmentation results are pushed to the database administrator within the system. Text information with questionable word segmentation is automatically identified. New addresses are marked and added to the word segmentation dictionary through human-computer interaction to achieve hot update.

[0068] For example, if a new address is added, “Yi’an Garden, No. 19 Hongqiao Road, Xinchengqiao Street, Chongchuan District”, the original dictionary does not contain “Yi’an Garden”. The word segmentation result is “Chongchuan District / Xinchengqiao Street / No. 19 Hongqiao Road / Yi’an / Garden”. After the system identifies this, it provides the database administrator with the word segmentation questionable address “Yi’an / Garden”. After manual processing and integration, it is merged into “Yi’an Garden” and then classified into the unit / community / village address level for use by the Chinese word segmentation engine.

[0069] S5. High-Frequency Word Management: High-frequency word management includes a high-frequency word database and management tools. The high-frequency word database is automatically generated based on the high-frequency matching standard addresses in the user's recent cleansing and matching results, and is regularly updated in sync with the standard address database. It covers information such as high-frequency word text, corresponding address level, high-frequency word coefficient, timestamp, and activation status. When a user enters an address with an unclear administrative region, the cleansing engine prioritizes matching high-frequency results based on the corresponding high-frequency word information.

[0070] For example, when a user enters the vague address "People's Park", it may correspond to People's Park in different cities. When the high frequency coefficient of "People's Park in Chongchuan District, Nantong City" in the high frequency word library is higher than others, the cleaning engine is provided with standard address results within the administrative region that the user is more interested in, so as to avoid matching unpopular locations.

[0071] S6. Initial Matching: First, using a Chinese word segmentation engine and word segmentation dictionary, the addresses to be cleaned after preprocessing in step S2 are accurately segmented hierarchically. For example, after segmenting "Jiangsu Province, Nantong City, Chongchuan District, Xinchengqiao Street, Chengshan Road No. 78, Jinhe Building", the result is "Jiangsu Province / Nantong City / Chongchuan District / Xinchengqiao Street / Chengshan Road No. 78 / Jinhe Building". Then, each segmented address is matched with the tree-like hierarchy in the standard address database, progressing from coarse-grained to fine-grained. When a segmented address completely matches the standard address, the match is considered successful, and all parent address information for the current address level is automatically completed.

[0072] For example, in the address phrase "Nantong City / Chengshan Road No. 78 / Jinhe Building", the first address segment "Nantong City" successfully matches "Nantong City" in the city-level address hierarchy of the standard address database. Subsequent matching will be limited to the administrative area of ​​Nantong City, and the coarse-grained provincial address hierarchy "Jiangsu Province" will be automatically completed. The second address segment "Chengshan Road No. 78" is then matched, finding the corresponding street address within Nantong City. Subsequent matching will be limited to the area of ​​Chengshan Road No. 78, and the district / county to which this street address belongs, "Chongchuan District," and the street, "Xinchengqiao Street," will be automatically completed. Finally, the third address segment "Jinhe Building" is matched with the standard address. After a successful match, the matching results from all levels are concatenated to form the standard address "Jinhe Building, No. 78 Chengshan Road, Xinchengqiao Street, Chongchuan District, Nantong City, Jiangsu Province".

[0073] When the high-frequency word coefficients from step S5 are received during matching, for example, if the user inputs "Chengnan Xincun", the high-frequency word coefficients of "Chongchuan District Chengnan Xincun" in the high-frequency word library are 0.8 and "Hai'an City Chengnan Xincun" are 0.6. This proves that according to the matching records of the cleaning engine, the user is more inclined to match, so the matching will be prioritized to "Chongchuan District Chengnan Xincun" with the higher coefficient. At the same time, the high-frequency word coefficients of addresses within "Chongchuan District" in the current matching round of the user will be temporarily increased. When the administrative region of other addresses entered by the user is unclear, the matching will also be prioritized to the standard addresses within the administrative region of "Chongchuan District", so as to realize the cross-level transmission of the influence of high-frequency words. The high-frequency word coefficients will also be output to step S9 to calculate the matching confidence.

[0074] After the initial matching is completed, the matching result is output to step S9 to calculate the confidence level and determine whether a second matching is needed.

[0075] S7. Initial Matching Result Evaluation: To improve address cleaning and matching efficiency, the initial matching results need to be evaluated to determine whether to proceed to the secondary matching process. The rules are as follows:

[0076] When all address levels are successfully matched and there are no unmatched address texts, the matching is considered complete, the level coefficient is set to 1, and the result is output to step S9 without secondary matching. When the address is matched to the building / village group level, no secondary matching is performed regardless of whether there are unmatched address texts. For example, if the input is "Chongchuan District / Chengnan New Village / Building 180 / Room 202", the standard address "No. 82 Hongqiao Road, Wenfeng Street, Chongchuan District, Nantong City, Jiangsu Province, Building 180, Chengnan New Village" is matched. The matching has reached the building level, which meets the spatial positioning requirements. Although there is an unmatched address text "Room 202", the level coefficient is set to 1, the result is output to step S9, and no secondary matching is performed.

[0077] If the initial match fails to reach the building / village group level and there is unmatched address text, the matched address level is retained and the level parameter is provided to step S9. For example, if the input address "Nantong City / Chengshan Road No. 87 / Jinhe Building" contains an incorrect road address, the match terminates because "Jinhe Building" is not found under the "Chengshan Road No. 87" branch (it is actually located at "Chengshan Road No. 78"). The initial match result is "Nantong City, Chongchuan District, Nantong City, Jiangsu Province, Xinchengqiao Street, Chengshan Road No. 87". Since the match only reaches the road address level and there is unmatched address text "Jinhe Building", the remaining text is output to step S9 for a second match.

[0078] S8. Secondary Matching: Fine-grained addresses in Chinese addresses are often located in the latter half of the address text. The secondary matching function can fully explore the remaining unmatched address information. Taking the incorrect address "Nantong City / Chengshan Road No. 87 / Jinhe Building" as an example, the initial matching result is "No. 87 Chengshan Road, Xinchengqiao Street, Chongchuan District, Nantong City, Jiangsu Province," and the remaining unmatched information is "Jinhe Building." The secondary matching directly inputs the unmatched address for cleaning, and can match it to the correct standard address "No. 78 Chengshan Road, Xinchengqiao Street, Chongchuan District, Nantong City, Jiangsu Province," which can automatically correct some incorrect addresses. After the secondary matching is completed, the hierarchical coefficient is output to step S9 and compared with the hierarchical coefficient of the initial matching result.

[0079] In addition, since individual building numbers / village group numbers (such as "Building 25" or "Group 5, No. 14") are usually not unique, it is not possible to directly deduce their superior unit / community / village. For example, if you enter an address with an incorrect building number: "Building 300, Chengnan New Village, Chongchuan District" (this building number does not exist in this community), the initial matching result will be "No. 82, Hongqiao Road, Wenfeng Street, Chongchuan District, Nantong City, Jiangsu Province, Chengnan New Village". The unmatched text used for the second matching is "Building 300". The matching engine will control the building number level to not skip the unit / community and directly match the text to avoid mismatching other communities.

[0080] S9. Confidence Calculation: After the matching process is completed, the matching confidence score is calculated to reflect the reliability of the matching results. The matching engine comprehensively considers the differences in standard address structures between urban and rural areas, the impact of historical place names on address validity, and the potential address ambiguity caused by addresses with the same name in different locations, and calculates the matching confidence score based on spatiotemporal differences. The calculation method is: Confidence Score = Hierarchical Coefficient × Historical Address Coefficient × High-Frequency Word Coefficient, and the calculation rules for each coefficient are as follows:

[0081] The level coefficient is determined by the finest-grained address level of the matching result. Based on the address spatial positioning capability, different calculation methods are used for urban and rural address level coefficients. When the finest-grained address level of the matching result is at the province, city, district / county, or street / town level, spatial positioning is not possible, and the match is considered a failure, with the level coefficient set to 0. When the finest-grained address level of the matching result is at the building / village group number, precise address positioning is possible, and the level coefficient is set to 1. In cases where only coarse positioning is possible, differentiated values ​​are assigned based on the urban / rural attribute of the address. When the finest-grained address level of an urban address is at the street address or unit / community level, the level coefficient is set to 0.7 and 0.8 respectively; when the finest-grained address level of a rural address is at the village level, the level coefficient is set to 0.6.

[0082] The default value of the historical address coefficient is 1. When the matching result is a historical place name, the time difference between the time when the historical place name disappeared and the time when the user queryed is calculated. The historical address coefficient is mapped to the preset interval [0.6, 1) using a linear function. The lower the value, the worse the timeliness of the matching result, thus achieving matching confidence suppression.

[0083] The high-frequency word coefficient is set to 1 by default. When the high-frequency word coefficient from step S5 is received during the matching process, the system uses this value to suppress the matching confidence and triggers a system prompt, suggesting that the user supplement the administrative division information to improve accuracy, because there may be ambiguity of "same name in different places".

[0084] When the confidence level is calculated according to the above formula, if there is only a single matching result, the confidence level and the matching result are directly output to step S10; if there is a second matching, the two matching results are compared. If the first matching and the second matching results belong to the same district or county-level administrative region, the result with higher confidence level is selected and output to step S10. Otherwise, the first matching result is still output to step S10 to reduce the probability of false matching.

[0085] For example, if the address to be cleaned is "No. 83 Hongqiao Road, Chongchuan District, Chengnan New Village" (incorrect address), the initial matching result is "No. 83 Hongqiao Road, Wenfeng Street, Chongchuan District, Nantong City, Jiangsu Province", while the second matching result is "No. 124 Changjiang Middle Road, Haian Street, Haian City, Nantong City, Jiangsu Province". Although the granularity of the second matching result reaches the unit community level, which is better than the initial matching result, the two matches do not belong to the same district or county administrative area. In this case, the output should still be based on the initial matching result to avoid misjudgment.

[0086] S10. Standardized Address Output: Outputs standardized addresses conforming to a seven-level structure, supporting JSON and Excel file formats. The output results include complete address information, hierarchical address information, spatial coordinates, matching confidence, high-frequency word influence parameters, and other information, providing data support for subsequent analysis.

[0087] Example 2

[0088] like Figure 2 As shown, a place name and address cleaning and matching system based on confidence calculation includes:

[0089] ① Data input interface to be cleaned: Build a web data input interface that supports text, API calls and file (CSV / XLS) input, and has built-in metadata annotation function (records source, system timestamp and other information) to provide a standardized data entry point for the subsequent cleaning process, and directly connect to ② Data preprocessing module to be cleaned.

[0090] ② Data preprocessing module: Performs preprocessing operations, including regularization and text deduplication, on the data to be cleaned input into the data input interface ①. Performs preliminary correction on common non-standard address information, removes some data noise, and outputs the preprocessed address to be cleaned to the word segmentation dictionary library ④ (address and place name topic).

[0091] ③ Hierarchical Standard Address Database: As the data foundation of the place name and address cleaning and matching system, it is based on the standard address database of authoritative departments, and integrates multi-source information such as alias database and historical place name data. It expresses the logical relationship between address elements through a seven-level model (the seven levels are: province, city, district / county, street / town, road address, unit / community / village, building number / village group number). It can meet the differentiated expression of addresses in urban and rural areas. It adopts a tree-like hierarchical structure to organize address entries. According to the update strategy of regular periodic update + emergency event trigger, it supports hot update of this hierarchical standard address database without stopping the cleaning service, and at the same time triggers hot update of ④ word segmentation dictionary database.

[0092] ④ Word segmentation dictionary: This is a database specifically built for place name and address text parsing tasks. It is used to collect, define, and accurately label information at each level of place names and addresses. It assists the Chinese word segmentation engine in correctly segmenting the addresses to be cleaned from the preprocessing module ② into hierarchical address text, providing accurate address input for the cleaning engine ⑥ which includes secondary matching. For example, it can segment "Nantong City Chongchuan District Jinhe Building" into "Nantong City / Chongchuan District / Jinhe Building". When the hierarchical standard address library ③ adds address entries due to hot updates, it automatically triggers the hot update of the word segmentation dictionary library ④, adding dictionary entries without restarting the Chinese word segmentation engine, ensuring the correct segmentation of the addresses to be cleaned.

[0093] ⑤ High-frequency word management module: This module includes a high-frequency word database and a high-frequency word management tool. The high-frequency word entries and coefficients in the high-frequency word database are automatically generated based on the matching frequency of standard addresses in the user's recent cleaning and matching results. A linear transformation function is used to map the high-frequency word coefficients to a preset confidence interval [0.6, 0.9], and the database is updated synchronously with changes in the hierarchical standard address database ③. It supports manual processing of high-frequency words, such as adding, deleting, modifying, and querying, to achieve full lifecycle management of high-frequency words. When the user's input address does not contain administrative region information, which may lead to mismatch due to ambiguity, the high-frequency word coefficients are pushed to the cleaning engine ⑥ which contains secondary matching.

[0094] ⑥ A cleaning engine with secondary matching: The hierarchical segmented text output from the segmentation dictionary in ④ is matched step-by-step with the hierarchical standard address library in ③. Combined with the high-frequency word coefficients provided by the high-frequency word management module in ⑤, the probability of prioritizing matching addresses that users are more interested in is increased. The hierarchical matching results are output to the confidence calculation module in ⑦. If the confidence calculation module in ⑦ determines that secondary matching is still necessary, the remaining address text after the initial matching is again segmented hierarchically and matched step-by-step with the hierarchical standard address library in ③. The results of the secondary matching are also output to the confidence calculation module in ⑦.

[0095] ⑦ Confidence Calculation Module: The confidence calculation module is one of the core modules of the system, mainly responsible for the key functions of secondary matching judgment and outputting the optimal matching result. This module evaluates the initial matching result to determine whether to trigger secondary matching: if the initial matching has met the precise location requirements (the finest granular address level of the matching result reaches the building / village group number) and there is no unmatched text, then the confidence score is directly calculated and the result is output to the ⑧ standardized address output interface; if the matching result does not meet the precise location requirements and there is unmatched text, then the remaining address text is fed back to the ⑥ cleaning engine containing secondary matching for secondary matching. Finally, by comparing the confidence values ​​of the two matches and the administrative region, if the two matching results are within the same district / county-level administrative region, the result with the higher confidence score is selected and output to the ⑧ standardized address output interface; otherwise, the initial matching result is still output.

[0096] The confidence level is calculated as follows: Confidence level = Hierarchical coefficient × Historical address coefficient × High-frequency word coefficient.

[0097] The level coefficient is determined by the finest-grained address level of the matching result. A fixed level coefficient is assigned to each level based on the urban / rural category of the address, as follows: When the finest-grained address level of the matching result is at the province, city, district / county, or street / town level, the matching is considered a failure because spatial positioning is not possible, and the level coefficient is set to 0. When the finest-grained address level of the matching result is at the building / village group number, precise address positioning is possible, and the level coefficient is set to 1. In cases where only rough positioning is possible, differentiated values ​​are assigned based on the urban / rural attribute of the address. When the finest-grained address level of an urban address is at the street address or unit / community level, the level coefficient is set to 0.7 and 0.8 respectively; when the finest-grained address level of a rural address is at the village level, the level coefficient is set to 0.6.

[0098] The default value of the historical address coefficient is 1. When the matching result is a historical place name, the time difference between the time when the historical place name disappeared and the time when the user queryed is calculated. The historical address coefficient is mapped to the preset interval [0.6, 1) using a linear function. The lower the value, the worse the timeliness of the matching result, thus achieving matching confidence suppression.

[0099] The high-frequency word coefficient is set to 1 by default. When the high-frequency word coefficient of the word segmentation dictionary is received during the matching process, the system will use this value to suppress the matching confidence and trigger a system prompt, which suggests that the user supplement the administrative division information to improve accuracy, because there may be ambiguity of "same name in different places".

[0100] ⑧ Standardized Address Output Interface: Based on the results of the cleaning engine (including secondary matching) in ⑥, the final multi-level standardized address is generated. It supports JSON / XLS file output and includes information such as matching confidence, whether it is a historical place name, whether the administrative division is questionable, and processing timestamp.

[0101] This invention constructs an intelligent place name and address cleaning and matching system that includes secondary matching, hot updating of the standard address database, and intervention of high-frequency words. It overcomes the static defects of traditional hierarchical models in address cleaning and matching, and achieves improved accuracy, efficiency breakthroughs, and resource optimization.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A place name address cleaning matching method based on confidence calculation, characterized in that: Includes the following steps: S1. Input of data to be cleaned: Construct a data input interface, accept user request information, parse the address text to be cleaned from the input address file, and record the input metadata information; S2. Preprocessing of data to be cleaned: Preprocessing the input data to be cleaned. S3. Hierarchical Standard Address Database: Based on the standard address database of authoritative departments, it integrates multi-source information from alias / historical place name databases and expresses the logical relationship between address elements through a multi-level model; S4. Hot update of word segmentation dictionary: This step is triggered after the hierarchical standard address library is hot updated in step S3. The new address entries are obtained and the Chinese word segmentation engine is used to segment the words according to the current dictionary. The word segmentation results are pushed to the database administrator within the system. The system automatically identifies text information with questionable word segmentation and marks the new addresses and adds them to the dictionary library through human-computer interaction to achieve hot update. S5. High-frequency word management: When the user enters an ambiguous address, the cleaning engine uses the corresponding high-frequency word information to prioritize matching high-frequency results. S6. Initial matching: Perform initial matching on the data to be cleaned after preprocessing in step S2. S7. Initial matching result evaluation: Evaluate the initial matching result of step S6 and decide whether to proceed to the secondary matching process. S8. Secondary matching: Match the remaining unmatched address text after the initial matching. S9. Confidence Calculation: After the matching process is completed, the matching confidence is calculated to reflect the reliability of the matching results. Match confidence is calculated based on spatiotemporal differences. The calculation method is: Confidence = Hierarchical Coefficient × Historical Address Coefficient × High-Frequency Word Coefficient. The calculation rules for each coefficient are as follows: The level coefficient is determined by the finest-grained address level of the matching result. Based on the address space positioning capability, a different calculation method is adopted for the address level coefficient of urban and rural areas. The default value of the historical address coefficient is 1. When the matching result is a historical place name, the time difference between the time when the historical place name disappeared and the time when the user queryed is calculated, and the historical address coefficient is mapped to the preset interval [0.6, 1) using a linear function. The high-frequency word coefficient is set to 1 by default. When the high-frequency word coefficient from step S5 is received during the matching process, the system uses this value to suppress the matching confidence and trigger a system prompt because of the possibility of "same name in different places". When the confidence level is calculated according to the above formula, if there is only a single matching result, the confidence level and the matching result are directly output to step S10; if there is a second matching, the two matching results are compared. If the first matching result and the second matching result belong to the same district or county-level administrative region, the result with higher confidence level is selected and output to step S10. Otherwise, the first matching result is still output to step S10 to reduce the probability of false matching. S10, Standardized Address Output: Outputs standardized addresses that conform to a multi-level structure.

2. The confidence score-based place name address cleaning matching method according to claim 1, characterized in that: Step S2 specifically includes: first, converting the address text into a string that meets the preset hierarchical requirements and is easy to segment based on regular expressions; then, using a text deduplication algorithm to remove redundant character information; and finally, using a rule-based description mapping table to achieve semantic normalization. 3.The method of claim 1, wherein the method further comprises: Step S3 specifically includes: expressing the logical relationship between address elements through a seven-level model to meet the differentiated expression of addresses in urban and rural areas; organizing address entries using a tree-like hierarchical structure; and supporting hot updates of the hierarchical standard address library without stopping the cleaning service, while simultaneously triggering hot updates of the word segmentation dictionary library, in accordance with a regular periodic update + emergency event triggering update strategy.

4. The confidence score based place name address cleansing matching method of claim 1, wherein: Step S5, high-frequency word management, specifically includes a high-frequency word database and a high-frequency word management tool. The high-frequency word database is automatically generated based on the standard addresses of high-frequency matches in the user's recent clean matching results, and is regularly updated synchronously with the hierarchical standard address database. It includes high-frequency word text, corresponding address hierarchy, high-frequency word coefficient, timestamp, and activation information.

5. A place name address cleansing matching system based on confidence calculation, characterized in that: include: Data input interface to be cleaned: Construct a data input interface that supports text, API calls and file input, and has built-in metadata information annotation function; Data preprocessing module: performs preprocessing operations on the input data to be cleaned; Hierarchical Standard Address Database: Based on the standard address database of authoritative departments, it integrates multi-source information from alias / historical place name databases and expresses the logical relationship between address elements through a multi-level model; Word segmentation dictionary: used to collect, define and accurately label information at each level of place name address, assisting the Chinese word segmentation engine to correctly segment the address to be cleaned into hierarchical address text, and providing accurate address input for the cleaning engine; High-frequency word management module: It uses a linear transformation function to map the coefficients of high-frequency words to a preset information interval, and updates it synchronously with the changes in the hierarchical standard address library. It supports manual addition, deletion, modification and query processing of high-frequency words, realizing full life cycle management of high-frequency words; A cleaning engine with secondary matching: It performs hierarchical matching between the hierarchical segmented text output by the word segmentation dictionary and the hierarchical standard address library, and combines the high-frequency word coefficients provided by the high-frequency word management module to increase the probability of prioritizing matching addresses that users are interested in. Confidence calculation module: This module is responsible for determining the secondary matching result and outputting the optimal matching result. It evaluates the initial matching result and then decides whether to trigger a secondary matching. The confidence calculation module specifically includes: calculating the matching confidence based on spatiotemporal differences. The calculation method is: Confidence = Hierarchical coefficient × Historical address coefficient × High-frequency word coefficient, and the calculation rules for each coefficient are as follows: The level coefficient is determined by the finest-grained address level of the matching result. Based on the address space positioning capability, a different calculation method is adopted for the address level coefficient of urban and rural areas. The default value of the historical address coefficient is 1. When the matching result is a historical place name, the time difference between the time when the historical place name disappeared and the time when the user queryed is calculated, and the historical address coefficient is mapped to the preset interval [0.6, 1) using a linear function. The high-frequency word coefficient is set to 1 by default. When the high-frequency word coefficient from step S5 is received during the matching process, the system uses this value to suppress the matching confidence and trigger a system prompt because of the possibility of "same name in different locations". Standardized address output interface: Outputs standardized addresses that conform to a multi-level structure.

6. The confidence score based place name address cleansing matching system of claim 5, wherein: The preprocessing module for the data to be cleaned specifically includes: performing preprocessing operations, including regularization and text deduplication, on the data to be cleaned input into the data input interface; performing preliminary correction on common non-standard address information; and removing some data noise.

7. The confidence score based place name address cleansing matching system of claim 5, wherein: The hierarchical standard address library expresses the logical relationship between address elements through a seven-level model, meeting the differentiated expression of addresses in urban and rural areas. It organizes address entries using a tree-like hierarchical structure and follows a regular periodic update plus event-triggered update strategy. It supports hot updates of the hierarchical standard address library without stopping the cleaning service, and simultaneously triggers hot updates of the word segmentation dictionary library.

8. The confidence score based place name address cleansing matching system of claim 5, wherein: The high-frequency word management module specifically includes a high-frequency word database and a high-frequency word management tool. The high-frequency word entries and coefficients in the high-frequency word database are automatically generated based on the matching frequency of standard addresses in the user's recent cleaning and matching results. A linear transformation function is used to map the high-frequency word coefficients to a preset confidence interval [0.6, 0.9], and the database is updated synchronously with changes in the standard address database.

Images