Method, system and device for deduplicating Chinese addresses based on status bits

Abstract

The present application discloses a method, system and device for deduplicating Chinese addresses based on status bits, and the method specifically includes the following steps: obtaining original address data; obtaining the first three levels of addresses and suffix address strings in the original address data in combination with national administrative division data; traversing the original address data using HashSet, and setting the status bits corresponding to the original address data; checking the status bits, and correcting the status bits; and removing the repeated first three levels of addresses and / or suffix addresses according to the corrected status bits. Through the present application, it is ensured that the repeated characters of the address string itself are found, and the address string is deduplicated.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more specifically, to a method, system and device for deduplicating Chinese addresses based on status bits. Background Art

[0002] With the development of Chinese search engines and data mining technology, efficient Chinese address duplicate detection technology has also developed rapidly and has received extensive attention from industry and academia. Compared with English addresses, Chinese addresses have more varied writing styles and more complex semantics, making Chinese address duplicate detection more challenging.

[0003] Generally speaking, the business scenarios that need to be handled for Chinese addresses include filling in express delivery addresses, bank creditor inspections, and personal information inquiries by public security, procuratorial and judicial departments. Especially in the express delivery industry, the number of express parcels has been growing exponentially in recent years based on the development of the logistics industry. The accuracy of express delivery has posed a challenge to the accuracy of addresses. The various existing data deduplication methods mainly focus on processing the judgment of text data similarity, the judgment of data dependency, the judgment of data abbreviations, and other issues. There are still the following problems in the deduplication of Chinese addresses:

[0004] 1. In filling in the address, the system only selects and fills in the first three levels of addresses such as provinces, cities, districts and counties. It is even possible that all addresses are filled in manually and then scanned by machines to obtain the final address. The character recognition system cannot guarantee completely accurate recognition of Chinese characters, so it is inevitable that addresses may be lost, some addresses may be repeated, resulting in information redundancy, scanning errors, etc.

[0005] 2. The duplication judgment for multiple addresses is mainly aimed at the situation where the same address is expressed in multiple ways. It is not possible to remove redundant information or standardize a single address itself.

[0006] 3. Except for the first three levels of address, such as province, city, district and county, the expression of other address strings is relatively flexible and cannot be uniformly deduplicated. Summary of the invention

[0007] The embodiment of the present application provides a Chinese address deduplication method based on status bits, so as to at least solve the problem of subjective factors in the related art.

[0008] The present invention provides a method for deduplicating Chinese addresses based on status bits, the method comprising the following steps:

[0009] Data acquisition steps: obtain original address data;

[0010] Address acquisition step: combining national administrative division data to acquire the first three levels of addresses and suffix address strings in the original address data;

[0011] Traversal step: traversing the original address data using HashSet, and setting a status bit corresponding to the original address data;

[0012] Checking step: checking the status bit and correcting the status bit;

[0013] Repeat removal step: removing the repeated first three level addresses and / or suffix addresses according to the corrected status bits.

[0014] As a further improvement of the present invention, the first three levels of addresses in the address acquisition step include affiliations of provinces, cities, districts and counties.

[0015] As a further improvement of the present invention, the status bit is divided into a repeated status bit and a non-repeated status bit.

[0016] As a further improvement of the present invention, the checking step specifically includes the following steps:

[0017] Status bit correction step: correcting the status bit;

[0018] Suffix checking step: performing a repeated string suffix check on the status bit.

[0019] As a further improvement of the present invention, the state bit correction step specifically includes the following steps:

[0020] The first judgment step: judging whether the repeated status bits are continuous or equal to two bits;

[0021] The first correction step: if yes, the repeated state bit remains unchanged; if no, the repeated state bit is corrected to the non-repeated state bit.

[0022] As a further improvement of the present invention, the suffix checking step specifically includes the following steps:

[0023] Setting steps: Set the suffix set;

[0024] The second judgment step: judging whether the repeated address marked with the repeated status bit in the original address data matches the suffix set;

[0025] The second correction step: if yes, correct the status bit of the repeated address to the non-repeated status bit; if no, the status bit of the repeated address remains unchanged.

[0026] Based on the same inventive concept, the present invention also discloses a Chinese address deduplication system based on status bits, based on the Chinese address deduplication method based on status bits disclosed in any one of the inventions.

[0027] The Chinese address deduplication system based on status bit includes:

[0028] Data acquisition module, acquiring original address data;

[0029] An address acquisition module, combining national administrative division data to acquire the first three levels of addresses and suffix address strings in the original address data;

[0030] A traversal module, using HashSet to traverse the original address data, and set a status bit corresponding to the original address data;

[0031] A checking module checks the status bit and corrects the status bit;

[0032] The duplicate removal module removes the duplicate first three level addresses and / or suffix addresses according to the corrected status bits.

[0033] As a further improvement of the present invention, the first three levels of addresses in the address acquisition module include affiliations of provinces, cities, districts and counties.

[0034] As a further improvement of the present invention, the inspection module specifically includes:

[0035] A status bit correction unit, which corrects the status bit;

[0036] The suffix checking unit performs a repeated string suffix check on the status bit.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a Chinese address deduplication method based on status bits when executing the computer program.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. A Chinese address deduplication method based on status bits is proposed. The method removes duplicate Chinese addresses based on status bits. By using HashSet, a Java built-in data structure, a status array with the same length as the address string is defined and set. The duplicate status is set for the specific index position, thereby obtaining the duplicate addresses in the address string.

[0040] 2. Ensure that repeated characters in the address string are found, and the address string is standardized or de-redundant, so as to provide relatively complete and accurate address information for downstream functions of the address service and can be applied to public security, banking, logistics and distribution, etc., to improve business efficiency and reduce costs;

[0041] 3. It can represent effective address information at the lowest cost, reducing the cost of business practice and communication;

[0042] 4. Address string deduplication reduces information storage costs, improves the accuracy and uniqueness of customer service in specific business scenarios (express delivery industry, banks and insurance companies, etc.), facilitates clustering and mining of multiple pieces of information in the same prefix address, and provides a foundation for downstream data business development;

[0043] 5. The status bit can be used to effectively remove the duplication of province, city, district and county level addresses (Top3 level addresses), the duplication of Top3 level addresses in the remaining addresses other than the Top3 level addresses, and the partial duplication of the remaining addresses themselves, so as to obtain more accurate addresses.

[0044] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 It is an overall flow chart of a Chinese address deduplication method based on status bits provided by an embodiment of the present invention;

[0047] Figure 2 It is a schematic diagram of the main flow chart disclosed in this embodiment;

[0048] Figure 3 yes Figure 1 The overall flow chart of step S4 disclosed;

[0049] Figure 4 yes Figure 3 The overall flow chart of step S41 disclosed;

[0050] Figure 5 yes Figure 3 The overall flow chart of step S42 disclosed;

[0051] Figure 6 This is a structural framework diagram of a Chinese address deduplication system based on status bits provided in this embodiment;

[0052] Figure 7 is a framework diagram of a computer device according to an embodiment of the present invention.

[0053] In the above picture:

[0054] 1. Data acquisition module; 2. Address acquisition module; 3. Traversal module; 4. Check module; 5. Duplicate removal module; 41. Status bit correction unit; 411. First judgment unit; 412. First correction unit; 42. Suffix check unit; 421. Setting unit; 422. Second judgment unit; 423. Second correction unit; 80. Bus; 81. Processor; 82. Memory; 83. Communication interface. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0056] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0057] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0058] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects, and do not represent a specific ordering of the objects.

[0059] The present invention is described in detail below in conjunction with the various embodiments shown in the accompanying drawings, but it should be noted that these embodiments are not limitations of the present invention, and any equivalent transformations or substitutions in functions, methods, or structures made by ordinary technicians in the field based on these embodiments are all within the scope of protection of the present invention.

[0060] Before elaborating on each embodiment of the present invention in detail, the core inventive concept of the present invention is summarized and elaborated in detail through the following several embodiments.

[0061] The present invention can perform Chinese address deduplication based on the status bit, effectively remove duplicate address data, and accurately obtain valid address information.

[0062] Embodiment 1:

[0063] Reference Figures 1 to 5 As shown, this example discloses a specific implementation method of a Chinese address deduplication method based on status bits (hereinafter referred to as "method").

[0064] Specifically, the overall idea of ​​the method is first introduced. The method is based on HashSet, a Java built-in data structure, and uses the non-repeatability of HashSet to sequentially scan the address string, then add the characters that are not in the created HashSet, and create a status bit array with the same length as the address string. If the character in the address string already exists in the HashSet, the status bit under the corresponding index is set to a duplicate status bit, otherwise it is set to a non-duplicate status bit.

[0065] Specifically, after scanning, the status bit array and repetition status of the address string are obtained, and then the status bits are self-corrected according to the relationship of the repeated status bits. The two methods of status bit self-correction and status bit suffix check based on address aliases are used to ensure that the repeated characters of the address string itself are found, and the address string is normalized or de-redundant.

[0066] Specifically, refer to Figure 1 and Figure 2 As shown, the method disclosed in this embodiment mainly includes the following steps:

[0067] Step S1, obtaining original address data.

[0068] Then, step S2 is executed to obtain the first three levels of addresses and the suffix address character string in the original address data in combination with the national administrative division data.

[0069] Specifically, in some of the embodiments, the top three levels of addresses (Top3 level addresses) are obtained in combination with the national administrative division data. The Top3 level addresses include the affiliations of provinces, cities, districts and counties. The Top3 level addresses are then matched from the address data to obtain the Top3 level addresses and the remaining address strings (suffix address strings) in the original address data except the Top3 level addresses.

[0070] Specifically, in some of the embodiments, there are four main address repetition modes, as follows:

[0071] 1. South Gate 2 of Zhejiang Art Museum, No. 138, Nanshan Road, Xihu District, Hangzhou, Zhejiang Province;

[0072] 2. 402, Unit 4, Building 2, Tianducheng Jazz Garden, Yuhang District, Hangzhou City, Zhejiang Province;

[0073] 3. No. B8-1, Sizhou International Decoration City, Sizhou Town, Sizhou County, Suzhou City, Anhui Province;

[0074] 4. 13-1301, Vanke Yungu Apartment, No. 300 Dunyu Road, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province Yungu Apartment, No. 300 Dunyu Road, Sandun Town, Zhejiang Province.

[0075] Among them, the first address has a Top3 address in addition to the province, city, district, and county level addresses, but it cannot be deduplicated; the second address has a Top3 address in addition to the Top3 address, and needs to be deduplicated; the third address also has a Top3 address duplication that needs to be deduplicated; the fourth address has a Top3 address duplication and a suffix address duplication. The four incorrect addresses are all duplications in different situations, some of which need to be deduplicated and some need to be retained, some are Top3 address duplications and some are duplications in the following addresses.

[0076] Then, step S3 is executed to traverse the original address data using HashSet, and set the status bit corresponding to the original address data.

[0077] Specifically, in some embodiments, the status bit is divided into two types: a repeating status bit and a non-repeating status bit. In this embodiment, the repeating status bit is marked as 1, and the non-repeating status bit is marked as 0, but the present invention is not limited thereto.

[0078] Specifically, in some of the embodiments, the original address string is traversed, and the status bit under the corresponding position index is set using HashSet and the status bit array. The initial status bit defaults to 0, and if there is a repetition after traversal, it is 1.

[0079] Then, step S4 is executed to check the status bit and correct the status bit.

[0080] Specifically, refer to Figure 3 As shown, step S4 specifically includes the following steps:

[0081] S41, correcting the status bit;

[0082] S42, performing a repeated string suffix check on the status bit.

[0083] Specifically, in some of the embodiments, after using HashSet to traverse the address string and set the status bit array and the corresponding string status bit, not all repeated characters need to be deduplicated, so the status bit will be corrected. The status bit correction algorithm and repeated string suffix check are used to check and correct the status bit to further ensure the accuracy of the status bit identification.

[0084] Specifically, for the first address among the four address duplication methods mentioned above, if deduplication is performed only based on the mark of the initial state bit, the result is as follows:

[0085]

[0086] The first line represents the original address string, and the second line represents the status bit of the original address string based on the non-duplicate state obtained by HashSet, where 0 represents non-duplicate and 1 represents duplicate. The last line represents the non-duplicate address string, but it is obvious that "Zhejiang" cannot be deleted from the string, as it is a valid part of a complete noun expression. In this way, address deduplication will be inaccurate, so the status bit needs to be checked and corrected.

[0087] Specifically, refer to Figure 4 As shown, step S41 specifically includes the following steps:

[0088] S411, determining whether the repeated status bits are continuous or equal to two bits;

[0089] S412: If yes, the repeat status bit remains unchanged; if no, the repeat status bit is corrected to the non-repeat status bit.

[0090] Specifically, in some embodiments, when the repeated status bit exceeds or includes 2 bits continuously, it can be judged as a valid status bit. Otherwise, the status bit is changed from 1 to 0, and the status bit is self-corrected. For example, if the status bit corresponding to a character in the status array is 1, and the status bits of the two adjacent characters on the left and right are 0, and the status bit marked as 1 does not exceed or equal to two bits continuously, then the status bit is automatically set to 0 after the status bit correction. The correction process is as follows:

[0091]

[0092] Specifically, refer to Figure 5 As shown, step S42 specifically includes the following steps:

[0093] S421, set suffix set;

[0094] S422, determining whether the repeated address marked with the repeated status bit in the original address data matches the suffix set;

[0095] S423, if yes, correct the status bit of the repeated address to the non-repeated status bit, if no, the status bit of the repeated address remains unchanged.

[0096] Specifically, in some of the embodiments, a repeated substring suffix check is performed based on the address alias. For example, in the first address of the four address duplication methods mentioned above, the field "Zhejiang" is an alias of "Zhejiang Province" and needs to be checked. When the "Zhejiang" character index position is marked as "1" in this duplication state, "Zhejiang" is followed by "Art Gallery". Therefore, a suffix set is set, such as "Art Gallery, Swimming Pool, Children's Palace" and other fields. Once the match is completed, the duplication state is set to "0" for correction, because although the "Zhejiang" field is repeated with the Top3 level address, it has real significance and cannot be deleted.

[0097] Then, step S5 is executed to remove the repeated first three level addresses and / or suffix addresses according to the corrected status bits.

[0098] Specifically, in some of the embodiments, after correction and inspection, according to the setting of the status bit, the corresponding index position character with the status bit of 0 is output to obtain the deduplicated address data. For example, the second address of the four address duplication methods mentioned above, "Yuhang District, Hangzhou City, Zhejiang Province" is a repeated invalid information, which needs to be deduplicated. First, the original address string is traversed and added to the HashSet. Once a duplication is found, the corresponding index position in the status bit array (initial value is 0) is set to 1, and then the status bit is used to obtain the deduplicated address string. The process is as follows:

[0099]

[0100] A Chinese address deduplication method based on status bits disclosed in an embodiment of the present application removes duplicates in Chinese addresses based on status bits, uses a Java built-in data structure such as HashSet, defines and sets a status array with the same length as the address string, sets a duplicate status for a specific index position, and thereby obtains duplicate addresses in the address string; ensures that duplicate characters in the address string itself are found, normalizes or removes redundancy in the address string, provides relatively complete and accurate address information for downstream functions of address services, and can be applied to public security, banking, logistics and distribution services, thereby improving business efficiency and reducing costs; can represent effective address information at a minimum cost, thereby reducing the cost of business practice and communication; address string deduplication reduces information storage costs, improves the accuracy and uniqueness of customer service in specific business scenarios (express delivery industry, banking and insurance companies, etc.), is conducive to clustering and mining of a number of information in the same prefix address, and provides a basis for downstream data business development; can effectively remove duplicates of province, city, district and county level addresses (Top3 level addresses), duplicates of Top3 level addresses in the remaining addresses other than the Top3 level addresses, and partial duplicates in the remaining addresses themselves, thereby obtaining more accurate addresses.

[0101] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent.

[0102] Embodiment 2:

[0103] In combination with the Chinese address deduplication method based on status bit disclosed in the first embodiment, this embodiment discloses a specific implementation example of a Chinese address deduplication system based on status bit (hereinafter referred to as “system”).

[0104] Reference Figure 6 As shown, the system comprises:

[0105] Data acquisition module 1, acquiring original address data;

[0106] Address acquisition module 2, combining national administrative division data to acquire the first three levels of addresses and suffix address strings in the original address data;

[0107] A traversal module 3 uses a HashSet to traverse the original address data and set a status bit corresponding to the original address data;

[0108] A checking module 4 checks the status bit and corrects the status bit;

[0109] The duplicate removal module 5 removes the duplicate first three level addresses and / or suffix addresses according to the corrected status bits.

[0110] Specifically, in some of the embodiments, the first three levels of addresses in the address acquisition module 2 include the affiliations of provinces, cities, districts and counties.

[0111] Specifically, in some of the embodiments, the status bit is divided into a repeated status bit and a non-repeated status bit.

[0112] Specifically, in some embodiments, the inspection module 4 specifically includes:

[0113] A status bit correction unit 41, which corrects the status bit;

[0114] The suffix checking unit 42 performs a repeated string suffix check on the status bit.

[0115] Specifically, in some embodiments, the status bit correction unit 41 specifically includes:

[0116] A first judging unit 411 is configured to judge whether the repeated status bits are continuous or equal to two bits;

[0117] The first correction unit 412 keeps the repeated state bit unchanged if yes, and corrects the repeated state bit to the non-repeated state bit if no.

[0118] Specifically, in some embodiments, the suffix checking unit 42 specifically includes:

[0119] Setting unit 421, setting a suffix set;

[0120] A second determination unit 422 determines whether the repeated address marked with the repeated status bit in the original address data matches the suffix set;

[0121] The second correcting unit 423 corrects the status bit of the repeated address to the non-repeated status bit if yes, and if no, the status bit of the repeated address remains unchanged.

[0122] For the technical solutions of the remaining identical parts of the Chinese address deduplication system based on status bits disclosed in this embodiment and the Chinese address deduplication method based on status bits disclosed in the first embodiment, please refer to the description in the first embodiment, which will not be repeated here.

[0123] Embodiment three:

[0124] Combination Figure 7 As shown, this embodiment discloses a specific implementation of a computer device. The computer device may include a processor 81 and a memory 82 storing computer program instructions.

[0125] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0126] Among them, the memory 82 may include a large-capacity memory for data or instructions. By way of example and not limitation, the memory 82 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 82 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 82 may be inside or outside the data processing device. In a specific embodiment, the memory 82 is a non-volatile memory. In a specific embodiment, the memory 82 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable ProgrammableRead-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable ProgrammableRead-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0127] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0128] The processor 81 implements any one of the Chinese address deduplication methods based on status bits in the above embodiments by reading and executing computer program instructions stored in the memory 82 .

[0129] In some of these embodiments, the computer device may further include a communication interface 83 and a bus 80. Figure 7 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0130] The communication interface 83 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication port 83 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0131] The bus 80 includes hardware, software or both, and couples the components of the computer device to each other. The bus 80 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 80 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 80 may include one or more buses. Although embodiments of the present application describe and illustrate a particular bus, the present application contemplates any suitable bus or interconnect.

[0132] The computer device can realize Chinese address deduplication based on the status bit, thereby realizing the combination Figure 1 Describe the method.

[0133] In addition, in combination with the Chinese address deduplication method based on status bits in the above embodiments, the present application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the Chinese address deduplication methods based on status bits in the above embodiments is implemented.

[0134] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] In summary, the beneficial effect of the present invention is that a Chinese address deduplication method based on status bits is proposed, which removes the duplication in the Chinese address based on the status bits, uses the Java built-in data structure HashSet, defines and sets a status array with the same length as the address string, sets the duplication state for the specific index position, and thus obtains the duplicate address in the address string; ensures that the repeated characters of the address string itself are found, the address string is standardized or de-redundant, and relatively complete and accurate address information is provided for the downstream functions of the address service and can be applied to public security, banking, logistics and distribution and other businesses, thereby improving business efficiency and reducing costs; it can represent effective address information at the lowest cost, reducing the cost of business practice and communication; address string deduplication reduces the information storage cost, improves the accuracy and uniqueness of customer service in specific business scenarios (express delivery industry, bank insurance company, etc.), is conducive to clustering and mining of many information in the same prefix address, and provides a basis for the development of downstream data business; the duplication of provincial, municipal, district and county level addresses (Top3 level addresses), the duplication of Top3 level addresses in the remaining addresses other than the Top3 level addresses, and the partial duplication of the remaining addresses themselves can be effectively removed through the status bits, thereby obtaining more accurate addresses.

[0136] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for deduplicating Chinese addresses based on status bits, characterized in that: The steps include: Data acquisition steps: obtain original address data; Address acquisition step: acquiring the first three levels of addresses and suffix addresses in the original address data in combination with the national administrative division data; Traversal step: traversing the original address data using HashSet, and setting a status bit corresponding to the original address data; Checking step: checking the status bit and correcting the status bit, wherein the status bit is divided into a repeated status bit and a non-repeated status bit; Repeat removal step: removing the repeated first three level addresses and / or suffix addresses according to the corrected status bits; The checking step specifically includes the following steps: Status bit correction step: correcting the status bit; Suffix checking step: performing a repeated string suffix check on the status bit; The state bit correction step specifically includes the following steps: The first judgment step: judging whether the repeated status bits are continuous or equal to two bits; The first correction step: if yes, the repeated state bit remains unchanged; if no, the repeated state bit is corrected to the non-repeated state bit; The suffix checking step specifically includes the following steps: Setting steps: Set the suffix set; The second judgment step: judging whether the repeated address marked with the repeated status bit in the original address data matches the suffix set; The second correction step: if yes, correct the status bit of the repeated address to the non-repeated status bit; if no, the status bit of the repeated address remains unchanged.

2. The Chinese address deduplication method based on status bit as claimed in claim 1, characterized in that: The first three levels of addresses in the address acquisition step include the affiliation of provinces, cities, districts and counties.

3. A Chinese address deduplication system based on status bits, characterized in that: include: Data acquisition module, acquiring original address data; An address acquisition module, combining national administrative division data to acquire the first three levels of addresses and suffix address strings in the original address data; A traversal module, using HashSet to traverse the original address data, and set a status bit corresponding to the original address data; A checking module checks the state bit and corrects the state bit, wherein the state bit is divided into a repeated state bit and a non-repeated state bit; A duplicate removal module, removing duplicate first three level addresses and / or suffix addresses according to the corrected status bits; Wherein, the inspection module specifically includes: A status bit correction unit, which corrects the status bit; A suffix checking unit, performing a repeated string suffix check on the status bit; Wherein, the status bit correction unit specifically includes: A first judging unit: judging whether the repeated status bits are continuous or equal to two bits; A first correction unit: if yes, the repeated state bit remains unchanged; if no, the repeated state bit is corrected to the non-repeated state bit; The suffix checking unit specifically includes the following steps: Setting unit: setting suffix set; A second judgment unit: judging whether the repeated address marked with the repeated status bit in the original address data matches the suffix set; The second correction unit: if yes, correct the status bit of the repeated address to the non-repeated status bit; if no, the status bit of the repeated address remains unchanged.

4. The Chinese address deduplication system based on status bit as claimed in claim 3, characterized in that: The first three levels of addresses in the address acquisition module include the affiliation of provinces, cities, districts and counties.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the Chinese address deduplication method based on status bits as described in any one of claims 1 to 2 is implemented.

Images