A method for obtaining standardized property addresses

Abstract

The present invention relates to a method for obtaining a standardized real estate address, comprising: step S1, collecting existing real estate address data, and constructing a real estate dictionary database according to the existing real estate address data; step S2, randomly obtaining a real estate address, and cutting the real estate address according to nine address levels; step S3, judging whether the cut real estate address is consistent with the community address in the real estate dictionary database, if consistent, proceeding to step S4; if inconsistent, proceeding to step S5; step S4, obtaining the basic information of the community corresponding to the cut real estate address, and judging whether it is consistent with the basic information of the community in the real estate dictionary database; step S5, performing word segmentation and marking on the cut real estate address, and obtaining the marked real estate address; step S6, matching the marked real estate address with the community address in the real estate dictionary database, and finding the final standardized real estate address.

Description

Technical Field

[0001] The present invention relates to the field of real estate, and more particularly to a method for obtaining a standardized real estate address. Background Art

[0002] An address is a natural language string describing spatial coordinates, while a Chinese address is a descriptive data. Due to the complexity of Chinese, existing technologies have many problems with real estate address matching: 1) Address data lacks key fields and can only identify recognizable parts. For example, Room 8B, Building 3, Meilan District, Hangzhou City, Zhejiang Province, can only identify administrative regions; 2) Address data is ambiguous and needs to be corrected through big data. For example, Room 33-102, Building 2, Chang'an Lido, Xuhui District, Chongqing, can be identified as Chang'an Lido in Jiangbei District, Chongqing; 3) Address data contains dirty data and needs to be cleaned. For example, Room 1801, Building 15, Shui'an Huating, West Block of Hongwei Hotel, Huaxing Village, Fucheng Town, Yancheng City, the "West Block of Hongwei Hotel" in this address is dirty data. However, there is currently no method that can directly convert random real estate addresses into standardized real estate addresses. Summary of the invention

[0003] In order to solve the above problems in the prior art, the present invention provides a method for obtaining a standardized real estate address, which can standardize a random real estate address.

[0004] A method for obtaining a standardized property address, comprising:

[0005] Step S1, collecting existing real estate address data, and building a real estate dictionary database based on the existing real estate address data, wherein the real estate dictionary database stores the community address and basic community information;

[0006] Step S2, randomly obtain a property address, and divide the property address into nine address levels: province, city, district / county, township / street, road, community name, building name, unit, and room number;

[0007] Step S3, determining whether the cut real estate address is consistent with the community address in the real estate dictionary database, if consistent, proceeding to step S4; if inconsistent, proceeding to step S5;

[0008] Step S4, obtaining the basic information of the community corresponding to the cut real estate address, and judging whether it is consistent with the basic information of the community in the real estate dictionary database, if not consistent, replacing and updating the basic information of the community in the real estate dictionary database;

[0009] Step S5, performing word segmentation and labeling on the segmented real estate address to obtain the labeled real estate address;

[0010] Step S6, matching the marked property address with the community address in the property dictionary database, finding the community address most similar to the marked property address in the property dictionary database as the final standardized property address.

[0011] Furthermore, the step S1 of constructing a building dictionary database includes:

[0012] Step S11, cleaning the existing real estate address data and filtering out abnormal community information;

[0013] Step S12, cutting the cleaned real estate address data into nine address levels: province, city, district / county, township / street, road, community name, building name, unit, and room number, and classifying the real estate address data by city;

[0014] Step S13, performing word segmentation processing on the community names in all the real estate address data in each city, and extracting keywords of the community names and TF-IDF values ​​corresponding to the keywords of the community names;

[0015] Step S14, obtaining administrative area information, road information and community basic information in the cleaned real estate address data, and calculating the similarity score of any two communities in all communities in each city according to the administrative area information, the road information, the community basic information, the keywords of the community name and the TF-IDF value corresponding to the keywords of the community name;

[0016] Step S15, if the similarity score of the two cells is greater than or equal to 8 points, the two cells are determined to be the same cell, and step S16 is performed; if the similarity score of the two cells is less than 8 points and greater than 6 points, the two cells are determined to be suspected to be the same cell, and step S17 is performed; if the similarity score of the two cells is less than or equal to 6 points, the two cells are determined to be different cells, and step S18 is performed;

[0017] Step S16, selecting the cell name of one of the cells as the primary cell name, the cell name of the other cell as the secondary cell name, and fusing other information of the two cells except the cell names;

[0018] Step S17, sorting the similarity scores from high to low to verify whether the two cells are the same cell, if so, proceed to step S16; if not, proceed to step S18;

[0019] Step S18, marking the two communities as different unique communities, and storing the property address data and community basic information of the two communities in the real estate dictionary database respectively.

[0020] Furthermore, the step S13 includes:

[0021] Step S131, using the Jieba word segmentation database to segment the community names in all the real estate address data in each city, counting the frequency of each word in the community name in all community names, and screening high-frequency words to filter out interference words or invalid words;

[0022] Step S132, calculating the TF-IDF value of each word segment of each community name, sorting the TF-IDF values, and taking the word segment with the largest TF-IDF value as the keyword of the corresponding community name.

[0023] Further, the step S6 comprises:

[0024] Step S61, determining whether the administrative region of the marked property address is the same as the administrative region of the community address in the real estate dictionary database, if they are the same, the region score of the community address in the real estate dictionary database is recorded as 1 point; if they are not the same, the region score is recorded as 0 point;

[0025] Step S62, calculating the basic score of the residential area address in the real estate dictionary database according to the road name, road number and residential area name of the marked real estate address;

[0026] Step S63, calculating the similarity score of the community address in the real estate dictionary database according to the community name of the marked real estate address;

[0027] Step S64, adding the area score, the basic score and the similarity score to obtain a matching score, and finding the community address with the highest matching score in the real estate dictionary database as the final standardized real estate address.

[0028] Furthermore, the step S63 includes:

[0029] Step S631, performing text segmentation on the community name of the marked real estate address and the community name of the community address in the real estate dictionary database;

[0030] Step S632, counting the number of occurrences of each word in the community name of the marked real estate address and the number of occurrences of each word in the community name of the community address in the real estate dictionary database, and obtaining two columns of word frequency vectors;

[0031] Step S633, substitute the two columns of word frequency vectors into the cosine similarity calculation formula to obtain the cosine value as the similarity score of the community address in the real estate dictionary database.

[0032] The present invention constructs a real estate dictionary database, segments and annotates the real estate address, and then matches it with the community address stored in the real estate dictionary database, so that the random real estate address can be directly converted into a standardized real estate address. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of a method for obtaining a standardized property address according to the present invention. DETAILED DESCRIPTION

[0034] The preferred embodiments of the present invention are given below in conjunction with the accompanying drawings and described in detail.

[0035] like Figure 1 As shown, the method for obtaining a standardized real estate address provided by the present invention comprises the following steps:

[0036] Step S1, collect existing real estate address data, and build a real estate dictionary database based on the existing real estate address data, and the real estate dictionary database stores the community address and basic community information. The community address includes province, city, district / county, township / street, road, community name, building name, unit, room number information, and the community basic information includes property type, construction age, greening rate, completion time, greening rate, volume ratio, parking space, number of households and other information.

[0037] Building a real estate dictionary database includes the following steps:

[0038] Step S11, clean the existing real estate address data and filter out abnormal community information to ensure the accuracy of the real estate dictionary database.

[0039] Step S12, the cleaned real estate address data is cut according to nine address levels: province, city, district / county, township / street, road, community name, building name, unit, and room number, and the real estate address data is classified according to the city.

[0040] Step S13, segmenting the community names in all the real estate address data in each city, and extracting the keywords of the community names and the TF-IDF values ​​corresponding to the community name keywords.

[0041] Step S131, use the Jieba word segmentation library to segment the community names in all real estate address data in each city, count the frequency of each word in the community name in all community names, and filter out high-frequency words to filter out interference words or invalid words. It should be noted that the frequency of each word is related to the amount of data processed at a time, and can be set according to actual conditions.

[0042] Step S132, calculate the TF-IDF value of each word segment of each community name, sort the TF-IDF values, and use the word segment with the largest TF-IDF value as the keyword of the corresponding community name. To facilitate recording and querying, the keywords of the community name are deduplicated and stored in a corpus with the corresponding TF-IDF value. The corpus is continuously updated and iterated in data fusion.

[0043] Step S14, obtain the administrative area information, road information and community basic information in the cleaned real estate address data, and calculate the similarity score of any two communities in all communities in each city based on the administrative area information, road information, community basic information, community name keywords and the TF-IDF value corresponding to the community name keywords.

[0044] The similarity calculation rules are shown in the following table:

[0045] Table 1 Similarity calculation rules

[0046]

[0047] If the community name keywords of two communities are the same, the similarity score calculation formula is:

[0048] X1+X2+X6+X8+X9+X10+X11+X12+X13-X4-X5

[0049] If the community name keywords of two communities are different, the similarity score calculation formula is:

[0050] X1+X6+X8+X9+X10+X11+X12+X13 (X2 and X6 take the maximum value)

[0051] Step S15, if the similarity score of the two cells is greater than or equal to 8 points, the two cells are determined to be the same cell, and step S16 is performed; if the similarity score of the two cells is less than 8 points and greater than 6 points, the two cells are determined to be suspected to be the same cell, and step S17 is performed; if the similarity score of the two cells is less than or equal to 6 points, the two cells are determined to be different cells, and step S18 is performed.

[0052] Step S16, select the name of one of the cells as the main cell name, the name of the other cell as the sub-cell name, and merge other information of the two cells except the cell names, the other information except the cell names includes administrative areas, roads and basic cell information.

[0053] For administrative regions, if the region names are the same from a single source or multiple sources, they are directly stored in the real estate dictionary database after standardization. If the same field has multiple sources and the values ​​are not unique: when the number of data sources with the same value is greater than or equal to 2, they are stored in the real estate dictionary database first; if the values ​​of each source are different, the correct region name is selected after manual verification and stored in the real estate dictionary database. If the region names are different but they do belong to the same region, one of the region names is selected as the main name and the other region is selected as the region alias.

[0054] For the basic information of the community, if it is from a single source or the basic fields from multiple sources are the same, it will be stored in the building dictionary database after standardization. If it is from multiple sources for the same field and the value is not unique: if the number of data sources with the same value is greater than or equal to 2, it will be stored in the building dictionary database first; if the values ​​from each source are different, the correct information will be selected after manual verification and stored in the building dictionary database; if the values ​​from each source are different and the correct information cannot be obtained, it will not be stored in the building dictionary database

[0055] Step S17, sort the similarity scores from high to low to verify whether the two cells are the same cell. If so, proceed to step S16; if not, proceed to step S18. It should be noted that if it is verified to be the same cell, the same cell is identified, and the highest similarity score of all the same cells is counted. The highest score can be used to batch update the similarity score of the same cell identified by the machine using a Python script.

[0056] Step S18, mark the two communities as different unique communities, and store the property address data and community basic information of the two communities in the real estate dictionary database. In addition, for two different communities, communities with high similarity scores can be regularly sampled to verify whether there are identical communities that have not been discovered. If so, the community identification needs to be corrected.

[0057] In the process of constructing a building dictionary database, the present invention effectively identifies the same community, similar community and unique community by introducing a calculation method of community similarity, greatly reducing the workload of manual intervention. In addition, in the calculation of community similarity, the TF-IDF algorithm and keyword library are introduced, and the importance of each word in the community name after word segmentation in the entire building name table is fully considered, and keywords and non-keywords in the community name are distinguished in this way. In addition, when the community is merged, in addition to comparing the community names, the basic information of the community is also taken into account, so that the basis for judging whether the community needs to be merged is more sufficient.

[0058] Step S2, randomly obtain a property address, and segment the property address according to nine address levels: province, city, district / county, township / street, road, community name, building name, unit, and room number. For example, 2-2-301, Changhe Watermark City, No. 2242, Binsheng Road, Binjiang District, Hangzhou City, Hangzhou Province, after segmentation, it becomes: Hangzhou Province | Hangzhou City | Binjiang District | Binsheng Road | No. 2242 | Changhe Watermark City | 2 | 2 | 301.

[0059] The corresponding address explanations of the nine levels are shown in the following table:

[0060] Table 2 Address levels and address explanations

[0061]

[0062] Step S3, determine whether the cut real estate address is consistent with the community address in the real estate dictionary database. If it is consistent, proceed to step S4; if it is inconsistent, proceed to step S5.

[0063] Step S4, obtain the basic information of the community corresponding to the real estate address after cutting, and determine whether it is consistent with the basic information of the community in the real estate dictionary database. If it is inconsistent, replace and update the basic information of the community in the real estate dictionary database. For example, the greening rate of a certain real estate stored in the real estate dictionary database is 45%, but the greening rate of the real estate obtained later is 40%. After manual verification, the greening rate of 40% is correct information, then the greening rate of 45% in the real estate dictionary database needs to be replaced and updated to 40%.

[0064] Step S5, segment and annotate the cut real estate address according to nine address levels to obtain the annotated real estate address. In this embodiment, the conditional random field (CRF) algorithm is used for annotation. The algorithm can perform word segmentation and part-of-speech tagging through context, which can reduce the dependence on dictionaries and rules, identify ambiguous words and unregistered words, and has high speed and efficiency. After the above address, No. 2242, Binsheng Road, Binjiang District, Hangzhou City, Hangzhou Province, is segmented and annotated, the address is obtained: Hangzhou Province | Hangzhou City | Binjiang District | Binsheng Road | No. 2242 | Changhe Watermark City | Building 2 | Unit 2 | Room 301, that is:

[0065] Hangzhou Province|Hangzhou City|Binjiang District|Binsheng Road|No. 2242|Changhe Watermark City|2|2|301, corresponding to:

[0066] Province|City|District|Road Name|Road Number|Community Name|Building|Unit|Room Number

[0067] Step S6, matching the marked property address with the community address in the property dictionary database, and finding the community address that is most similar to the marked property address in the property dictionary database as the final standardized property address.

[0068] Step S61, determine whether the administrative region of the marked property address is the same as the administrative region of the community address in the real estate dictionary database. If they are the same, the region score of the community address in the real estate dictionary database is recorded as 1 point; if they are not the same, the region score is recorded as 0. The same administrative region means the same province, the same city, the same district / county, and the same township / street.

[0069] Step S62, calculate the basic score of the community address in the real estate dictionary database according to the road name, road number and community name of the marked real estate address. The calculation method of the basic score is as follows:

[0070] Table 3 Basic score calculation method

[0071] Fields Judgment method Score Road name + road number + community name same 0.9 Road name + community name same 0.8 Road name + road number (including aliases) same 0.7 Community name (or alias) Include 0.6

[0072] Step S63, based on the community name of the marked real estate address, calculate the similarity score of the community address in the real estate dictionary database. In this embodiment, the text similarity algorithm is used to calculate the similarity score. It includes:

[0073] Step S631, performing text segmentation on the community name of the marked real estate address and the community name of the community address in the real estate dictionary database.

[0074] Step S632, count the number of occurrences of each word in the community name of the marked real estate address and the number of occurrences of each word in the community name of the community address in the real estate dictionary database, and obtain two columns of word frequency vectors.

[0075] Step S633, substitute the two columns of word frequency vectors into the cosine similarity calculation formula to obtain the cosine value, which is the similarity score of the community address in the real estate dictionary database. Cosine similarity is also called cosine similarity, which evaluates the similarity of two vectors by calculating the cosine value of the angle between them. Cosine similarity plots the vectors into the vector space according to the coordinate values, and uses the cosine value of the angle between the two vectors in the vector space as a measure of the difference between the two individuals. The closer the cosine value is to 1, the closer the angle is to 0 degrees, that is, the more similar the two vectors are, and vice versa, the closer it is to 0, the lower the similarity between the two vectors.

[0076] Step S64, add the area score, basic score and similarity score to obtain the matching score, find the community address with the highest matching score in the real estate dictionary database, and the community address is the final standardized real estate address.

[0077] For example, the above address - Hangzhou Province | Hangzhou City | Binjiang District | Binsheng Road | No. 2242 | Changhe Watermark City | Building 2 | Unit 2 | Room 301, is matched with the residential area address in the real estate dictionary database - Watermark City, No. 2242 Binsheng Road, Binjiang District, Hangzhou City, Zhejiang Province. The regional score is 0, the basic score is 0.9, the similarity score is 0.8, and the final matching score is 1.7. The original address - 2-2-301, Changhe Watermark City, No. 2242 Binsheng Road, Binjiang District, Hangzhou City, Hangzhou Province, is standardized to Room 301, Building 2, Unit 2, Watermark City, No. 2242 Bin, Binjiang District, Hangzhou City, Zhejiang Province.

[0078] The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. The above embodiment of the present invention can also be modified in various ways. That is, all simple, equivalent changes and modifications made according to the claims and the description of the present invention fall within the scope of protection of the claims of the present invention. The contents not described in detail in the present invention are all conventional technical contents.

Claims

1. A method for obtaining a standardized real estate address, characterized in that: include: Step S1, collecting existing real estate address data, and building a real estate dictionary database based on the existing real estate address data, wherein the real estate dictionary database stores the community address and basic community information; Step S2, randomly obtain a property address, and divide the property address into nine address levels: province, city, district / county, township / street, road, community name, building name, unit, and room number; Step S3, determining whether the cut real estate address is consistent with the community address in the real estate dictionary database, if consistent, proceeding to step S4; if inconsistent, proceeding to step S5; Step S4, obtaining the basic information of the community corresponding to the cut real estate address, and judging whether it is consistent with the basic information of the community in the real estate dictionary database, if not consistent, replacing and updating the basic information of the community in the real estate dictionary database; Step S5, performing word segmentation and labeling on the segmented real estate address to obtain the labeled real estate address; Step S6, matching the marked property address with the community address in the property dictionary database, finding the community address most similar to the marked property address in the property dictionary database as the final standardized property address; The step S1 of constructing a building dictionary database includes: Step S11, cleaning the existing real estate address data and filtering out abnormal community information; Step S12, cutting the cleaned real estate address data into nine address levels: province, city, district / county, township / street, road, community name, building name, unit, and room number, and classifying the real estate address data by city; Step S13, performing word segmentation processing on the community names in all the real estate address data in each city, and extracting keywords of the community names and TF-IDF values ​​corresponding to the keywords of the community names; Step S14, obtaining administrative area information, road information and community basic information in the cleaned real estate address data, and calculating the similarity score of any two communities in all communities in each city according to the administrative area information, the road information, the community basic information, the keywords of the community name and the TF-IDF value corresponding to the keywords of the community name; Step S15, if the similarity score of the two cells is greater than or equal to 8 points, the two cells are determined to be the same cell, and step S16 is performed; if the similarity score of the two cells is less than 8 points and greater than 6 points, the two cells are determined to be suspected to be the same cell, and step S17 is performed; if the similarity score of the two cells is less than or equal to 6 points, the two cells are determined to be different cells, and step S18 is performed; Step S16, selecting the cell name of one of the cells as the primary cell name, the cell name of the other cell as the secondary cell name, and fusing other information of the two cells except the cell names; Step S17, sorting the similarity scores from high to low to verify whether the two cells are the same cell, if so, proceed to step S16; if not, proceed to step S18; Step S18, marking the two communities as different unique communities, and storing the property address data and community basic information of the two communities in the real estate dictionary database respectively.

2. The method for obtaining a standardized real estate address according to claim 1, characterized in that: The step S13 comprises: Step S131, using the Jieba word segmentation database to segment the community names in all the real estate address data in each city, counting the frequency of each word in the community name in all community names, and screening high-frequency words to filter out interference words or invalid words; Step S132, calculating the TF-IDF value of each word segment of each community name, sorting the TF-IDF values, and taking the word segment with the largest TF-IDF value as the keyword of the corresponding community name.

3. The method for obtaining a standardized real estate address according to claim 1, characterized in that: The step S6 comprises: Step S61, determining whether the administrative region of the marked property address is the same as the administrative region of the community address in the real estate dictionary database, if they are the same, the region score of the community address in the real estate dictionary database is recorded as 1 point; if they are not the same, the region score is recorded as 0 point; Step S62, calculating the basic score of the residential area address in the real estate dictionary database according to the road name, road number and residential area name of the marked real estate address; Step S63, calculating the similarity score of the community address in the real estate dictionary database according to the community name of the marked real estate address; Step S64, adding the area score, the basic score and the similarity score to obtain a matching score, and finding the community address with the highest matching score in the real estate dictionary database as the final standardized real estate address.

4. The method for obtaining a standardized real estate address according to claim 3, characterized in that: The step S63 comprises: Step S631, performing text segmentation on the community name of the marked real estate address and the community name of the community address in the real estate dictionary database; Step S632, counting the number of occurrences of each word in the community name of the marked real estate address and the number of occurrences of each word in the community name of the community address in the real estate dictionary database, and obtaining two columns of word frequency vectors; Step S633, substitute the two columns of word frequency vectors into the cosine similarity calculation formula to obtain the cosine value as the similarity score of the community address in the real estate dictionary database.

Images