A method, apparatus, device, medium and product for address data compression

Abstract

Embodiments of the present application provide a method, device, equipment, medium and product for address data compression. The method relates to the field of address data compression, and comprises performing word segmentation and splitting processing on each address in an initial address database to obtain a segmented word set corresponding to each address, wherein the initial address database comprises a plurality of addresses, and each segmented word set comprises at least one segmented word; performing deduplication processing on the segmented words in the segmented word sets corresponding to all addresses in the initial address database to obtain a segmented address word table corresponding to the initial address database, the segmented address word table comprising all deduplicated segmented words, and the segmented words in the segmented address word table being different from each other; determining an identifier of each segmented word in the segmented address word table, the identifiers of different segmented words being different; and replacing the segmented words in the segmented word set corresponding to each address using the identifiers of the segmented words to obtain a compressed address database. The method of the embodiments of the present application can compress the data volume of an address database.

Description

Technical Field

[0001] This application relates to the field of data compression, and more specifically, to a method, apparatus, device, medium, and product for address data compression. Background Technology

[0002] Address data forms the digital backbone of urban operations, supporting precise delivery of goods and services, real-time navigation guidance, scientific layout of commercial outlets, and rational allocation of public resources. Through the digital description of geospatial space, it efficiently connects people with services and demand with supply, driving efficiency optimization and intelligent upgrading from daily life to macro-level decision-making. It is an indispensable infrastructure for modern socio-economic activities.

[0003] The current address database is experiencing exponential growth, with complete address texts covering provinces, cities, districts, counties, streets, and even house numbers accumulating to massive levels. This not only consumes a large amount of storage resources but also significantly reduces the efficiency of address matching, querying, and analysis, especially in scenarios such as real-time positioning, logistics planning, and spatial data analysis, where system response latency is becoming increasingly prominent.

[0004] Therefore, how to provide a technology to compress the data volume of the address database has become a problem that needs to be solved. Summary of the Invention

[0005] The purpose of one embodiment of this application is to provide a method, apparatus, device, medium and product for address data compression. The technical solution of the embodiment of this application can realize the compression of the data volume of the address database.

[0006] In a first aspect, embodiments of this application provide a method for address data compression, comprising: performing word segmentation on each address in an initial address database to obtain a segmented word set corresponding to each address, wherein the initial address database includes multiple addresses, and each segmented word set includes at least one segmented word; performing deduplication on the segmented words in the segmented word sets corresponding to all addresses in the initial address database to obtain a segmented address word table corresponding to the initial address database, wherein the segmented address word table includes all deduplicated segmented words, and the segmented words in the segmented address word table are all different; determining the identifier of each segmented word in the segmented address word table, wherein different segmented words have different identifiers; and replacing the segmented words in the segmented word set corresponding to each address with the identifiers of each segmented word to obtain a compressed address database.

[0007] This application embodiment obtains segmented words by splitting address data, and uses the identifiers of the segmented words to replace the original address text. Since the identifier data of the segmented words is smaller than the text data, the address data is compressed.

[0008] In one embodiment, determining the identifier of each segment word in the segmented address word table includes: using the sequence number of each segment word in the segmented address word table to determine the identifier of each segment word.

[0009] In one embodiment, the method further includes: determining a set of segmented words to be searched corresponding to the address to be searched, the set of segmented words to be searched including at least one segmented word to be searched; replacing the segmented words in the set of segmented words to be searched with the identifiers of each segmented word to obtain the transcoded address corresponding to the address to be searched; and matching the transcoded address with the compressed address database to obtain at least one address to be searched.

[0010] In one embodiment, the method further includes: performing extended encoding on each segment word to obtain segment word groups corresponding to each segment word, each segment word group including one segment word and multiple extended words; and establishing an association relationship between the segment word groups and the corresponding segment words.

[0011] In one implementation, determining the set of searchable segment words corresponding to the address to be searched includes: performing word segmentation processing on the address to be searched to obtain at least one search segment word; performing extended encoding on the at least one search segment word to obtain search segment word groups corresponding to each search segment word, each search segment word group including one search segment word and multiple search extended words; matching the search segment word groups with the segment word groups, and determining the set of searchable segment words corresponding to the address to be searched based on the association relationship.

[0012] Secondly, embodiments of this application provide an address data compression apparatus, comprising: a splitting unit, configured to perform word segmentation processing on each address in an initial address database to obtain a segmented word set corresponding to each address, wherein the initial address database includes multiple addresses, and each segmented word set includes at least one segmented word; a deduplication unit, configured to perform deduplication processing on the segmented words in the segmented word sets corresponding to all addresses in the initial address database to obtain a segmented address word table corresponding to the initial address database, wherein the segmented address word table includes all deduplicated segmented words, and the segmented words in the segmented address word table are all different; a determining unit, configured to determine the identifier of each segmented word in the segmented address word table; and a replacement unit, configured to replace the segmented words in the segmented word set corresponding to each address using the identifier of each segmented word to obtain a compressed address database.

[0013] Thirdly, one embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the methods described in the first aspect and any embodiment of the first aspect.

[0014] Fourthly, one embodiment of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, can implement the method as described in the first aspect and any embodiment of the first aspect.

[0015] Fifthly, one embodiment of this application provides a computer program product, the computer program product including a computer program, wherein the computer program, when executed by a processor, can implement the method as described in the first aspect and any embodiment of the first aspect. Attached Figure Description

[0016] To more clearly illustrate the technical solution of one embodiment of this application, the accompanying drawings used in one embodiment of this application will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of an address data compression method provided for one embodiment of this application; Figure 2 A schematic diagram of an address data compression apparatus provided for one embodiment of this application; Figure 3 This is a schematic diagram of an electronic device provided for one embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0019] The current address database is experiencing exponential growth, with complete address texts covering provinces, cities, districts, counties, streets, and even house numbers accumulating to massive levels. This not only consumes a large amount of storage resources but also significantly reduces the efficiency of address matching, querying, and analysis, especially in scenarios such as real-time positioning, logistics planning, and spatial data analysis, where system response latency is becoming increasingly prominent.

[0020] To standardize management, the industry typically uses hierarchical standards to classify addresses, such as China's 11-level address standard, which refines addresses from provincial administrative divisions down to specific room numbers. However, this complete address text data is quite large, requiring significant data storage resources and time-consuming operations such as address retrieval on such a massive dataset. Therefore, providing a technology to compress the data volume of address databases has become a problem that needs to be solved.

[0021] In view of the above problems, this application provides a method for address data compression, which involves splitting the address data into segment words and using the identifiers of the segment words to replace the original address text. Since the identifier data of the segment words is smaller than the text data, the address data compression is thus achieved.

[0022] The following explanation, for ease of understanding and illustration, is provided as an example rather than a limitation to illustrate the method of compressing addresses in this application.

[0023] The following is in conjunction with the appendix Figure 1 An example of an embodiment of this application illustrates a method for address data compression. Figure 1 The method shown can be executed by an address storage server or a third-party device for address compression, but the embodiments of this application are not limited thereto.

[0024] like Figure 1 The methods shown include: 110. Perform word segmentation on each address in the initial address database to obtain the segmented word set corresponding to each address.

[0025] The initial address database includes multiple addresses, and each segment word set includes at least one segment word.

[0026] It should be understood that in this embodiment, the initial address database is an uncompressed address database. The initial database has a relatively large amount of data. The initial database can refer to a conventional address database, such as a standard address database. For example, it can be an address database based on China's 11-level address standard, which is an address database that refines addresses from provincial administrative divisions down to specific room numbers.

[0027] For example, for each address record in the standard address database, a word segmentation tool is used to perform structured splitting, resulting in a set of ordered address segment words. For example, the address “AA Province BB City CC District DD Street EE No. FF Unit GG Floor HH Room” can be split into a segment word set: {“AA Province”, “BB City”, “CC District”, “DD Street”, “EE No.”, “FF Unit”, “GG Floor”, “HH Room”}. It should be understood that the word segmentation tool in this application embodiment can be: rule-based segmentation logic based on address keywords such as “province”, “city”, “district”, “street”, “road”, “number”, and “unit”; or it can be based on a professional address word segmentation model, such as a sequence labeling model based on Conditional Random Field (CRF) or Bidirectional Long Short-Term Memory Network (BiLSTM-CRF); or it can be based on the address parsing capabilities provided by Large Language Model (LLM).

[0028] 120. The segment words in the segment word set corresponding to all addresses in the initial address database are deduplicated to obtain the segment address word table corresponding to the initial address database.

[0029] The segmented address word list includes all segmented words after deduplication, and the segmented words in the segmented address word list are all different.

[0030] Specifically, all address segment terms generated in step 110 are deduplicated to form a unique "segment address term list" with a significantly reduced size. This embodiment of the application transforms the processing task of tens of millions of address records into the processing of segment term lists of millions or even hundreds of thousands, greatly improving the efficiency of subsequent compression processing.

[0031] 130, Determine the identifier of each segment word in the segment address word table.

[0032] The different markers for different segments are used.

[0033] It should be understood that in the embodiments of this application, the identifier of the segment word corresponds one-to-one with the segment word, and the identifier of the segment word can uniquely represent the corresponding segment word. It should be understood that the identifier of the segment word in the embodiments of this application can also be called the encoding of the segment word, as long as it can uniquely represent the segment word and the data volume is less than the segment word. The specific name is not limited in the embodiments of this application.

[0034] Since the addresses in the initial address database are strings, in order to compress the addresses, this application proposes to convert the segmented word strings into identifiers, and replace the segmented words with identifiers, thereby achieving address compression.

[0035] It should be understood that the identification of segment words in the embodiments of this application can be determined in a variety of ways, as long as the identification data of segment words is less than the data of segment words and can uniquely represent segment words.

[0036] Optionally, as an embodiment, determining the identifier of each segment word in the segmented address word table includes: using the sequence number of each segment word in the segmented address word table to determine the identifier of each segment word.

[0037] Specifically, the segmented address word table can be sorted, and the sorting sequence number of the segmented words can be used as the identifier of the segmented words. For example, in the embodiments of this application, the segmented words can be randomly sorted, or they can be sorted in descending order of the frequency of occurrence of the segmented words in the initial address database. In this way, the identifiers of the earlier segmented words are smaller, and replacing the segmented words with identifiers can further reduce the overall data volume of the database.

[0038] 140. Replace the segment words in the segment word set corresponding to each address with the identifiers of each segment word to obtain the compressed address library.

[0039] Specifically, in this embodiment of the application, the segmentation word identifier is used instead of the segmentation word. Since the data volume of the segmentation word identifier is smaller than that of the segmentation word itself, address data compression is achieved in this way.

[0040] Through the above methods, the embodiments of this application can achieve address data compression. Furthermore, based on the compressed address data, the embodiments of this application can further improve applications based on this address data, for example, improve address search based on this address database. This will be further explained below with specific examples.

[0041] Optionally, as an embodiment, the method of this application embodiment may further include: Determine the set of segmented terms to be searched corresponding to the address to be searched, wherein the set of segmented terms to be searched includes at least one segmented term to be searched.

[0042] The segment words in the set of segment words to be searched are replaced using the identifiers of each segment word to obtain the transcoded address corresponding to the address to be searched.

[0043] The transcoded address is matched with the compressed address database to obtain at least one address to be searched.

[0044] Specifically, in this embodiment of the application, the address to be searched is converted into a form consistent with the address in the compressed address database described above. In this way, the compressed address database can be used to search for addresses.

[0045] Optionally, as an embodiment, the method of the embodiments of the present application may further include: performing extended encoding on each of the segmented words to obtain segmented word groups corresponding to the segmented words, where each segmented word group includes a segmented word and multiple extended words; establishing an association relationship between the segmented word group and the corresponding segmented word, that is, the words in the segmented word group can all be corresponding to this segmented word. Subsequently, when performing a search, if any word in the segmented word group is matched, the searched word can be corresponding to this segmented word, that is, the identifier of this segmented word can be used to represent the searched word.

[0046] Specifically, the embodiments of the present application first obtain a compressed address library. In order to improve the address search based on this compressed address library and enhance the generalization ability of this address library. The embodiments of the present application first perform extended encoding on the segmented words in this address library. Each segmented word is corresponding to the segmented word itself and the extended words of this segmented word.

[0047] For example, for each word in the "segmented address word list", generate encoding representations in multiple dimensions to capture its different fuzzy features. These encodings may include but are not limited to: Original text code: that is, the segmented word itself, such as "Daoli District", for exact matching.

[0048] Pinyin code: Generate the voiceless full pinyin of the segmented word, such as "daoliqu", for matching homophonic words or pinyin input errors.

[0049] Initials of pinyin code (phonetic shape code): Generate the initials abbreviation of the pinyin of the segmented word, such as "dlq", for matching cases where the pronunciation is similar but the character shape is different.

[0050] Character shape code: Disassemble the Chinese character into components through a Chinese character splitting tool, and optionally剔除 the radicals and generate the main structure code. For example, "滨" can generate "宾", for matching similar character shape errors.

[0051] Stroke code: Convert the Chinese character into a stroke sequence through a font file or a stroke data set. For example, the stroke code of the character "未" is "horizontal, horizontal, vertical, left-falling stroke, right-falling stroke", for deeper comparison of character shape similarity.

[0052] Numeric code: Extract Arabic numerals or Chinese numerals from the segmented word. For example, extract "216" from "No. 216" and extract "6" from "Unit 6", for fast matching of the numeric part.

[0053] Optionally, as an embodiment, the determining of the set of segmented words to be searched corresponding to the address to be searched includes: Performing word segmentation and splitting processing on the address to be searched to obtain at least one search segmented word; The at least one search segment term is extended and encoded to obtain search segment term groups corresponding to each search segment term, and each search segment term group includes one search segment term and multiple search extended terms; The search segment words are matched with the segment words, and the set of search segment words corresponding to the address to be searched is determined according to the association relationship.

[0054] For example, the address to be searched can be an address entered by the user. Since user-entered address text is often highly arbitrary, ambiguous, and non-standard, this embodiment improves the accuracy of address search by splitting the address to be searched and expanding the resulting search segments. This improves search accuracy. Then, the search segment phrases are matched with corresponding segment phrases in the database's segment phrase table to find the corresponding segment phrases, thus determining the set of search segment phrases. This method maps inaccurate user-entered segment phrases to accurate segment address phrases in the database's segment address phrase table. Furthermore, based on the association relationship, the identifiers corresponding to the search segment phrases are determined. Finally, the identifiers of the search segment phrases are combined and searched in the compressed address database to find one or several recommended addresses with high similarity. This ultimately improves the accuracy of the address search.

[0055] Please refer to Figure 2 , Figure 2 A block diagram of an address data compression apparatus provided in one embodiment of this application is shown. Figure 2 The device 200 shown can be an execution end for address data compression. It should be understood that the device 200 corresponds to the execution end in the above method embodiment and can perform the various steps involved in the above method embodiment. The specific functions of the device 200 can be found in the description above. To avoid repetition, detailed descriptions are omitted here.

[0056] Figure 2 The illustrated device 200 includes at least one software function module that can be stored in a memory or embedded in the device in the form of software or firmware. Figure 2The apparatus 200 shown includes: a splitting unit 210, used to perform word segmentation processing on each address in the initial address database to obtain a segmented word set corresponding to each address, wherein the initial address database includes multiple addresses, and each segmented word set includes at least one segmented word; a deduplication unit 220, used to perform deduplication processing on the segmented words in the segmented word sets corresponding to all addresses in the initial address database to obtain a segmented address word table corresponding to the initial address database, wherein the segmented address word table includes all deduplicated segmented words, and the segmented words in the segmented address word table are all different; a determining unit 230, used to determine the identifier of each segmented word in the segmented address word table; and a replacement unit 240, used to replace the segmented words in the segmented word set corresponding to each address using the identifier of each segmented word to obtain a compressed address database.

[0057] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0058] like Figure 3 As shown, one embodiment of this application provides an electronic device 300, which includes a memory 310, a processor 320, and a computer program stored in the memory 310 and executable on the processor 320. When the processor 320 reads the program from the memory 310 via a bus 330 and executes the program, it can achieve the above-described functionality. Figure 1 The method. Optional, Figure 3 The device shown may also include a transceiver that can be used to send and / or receive data streams.

[0059] Processor 320 can process digital signals and may include various computing architectures. For example, it may be a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements multiple instruction set combinations. In some examples, processor 320 may be a microprocessor.

[0060] The memory 310 can be used to store instructions executed by the processor 320 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all of the functions of one or more modules described in the embodiments of this application. The processor 320 of this disclosure embodiment can be used to execute the instructions in the memory 310 to implement the above-described methods. The memory 310 includes dynamic random access memory, static random access memory, flash memory, optical memory, or other memories well known to those skilled in the art.

[0061] One embodiment of this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the methods described in the above embodiments.

[0062] An embodiment of this application also provides a computer program product, which includes a computer program, wherein the computer program, when executed by a processor, can implement the methods provided in the above embodiments.

[0063] It should be noted that the processor in the embodiments of the present invention (e.g., Figure 3 The processor in the above method embodiments can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by software instructions. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The software module can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0064] It can be understood that the memory in the embodiments of the present invention (e.g., Figure 3The memory in the memory can be volatile or non-volatile, or it can include both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0065] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0066] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0067] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, apparatuses, or units, or they may be electrical, mechanical, or other forms of connection.

[0068] In summary, the above description is merely a preferred embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for address data compression, characterized in that, include: Each address in the initial address database is segmented and split into words to obtain a segmented word set corresponding to each address. The initial address database includes multiple addresses, and each segmented word set includes at least one segmented word. The segment words in the segment word set corresponding to all addresses in the initial address database are deduplicated to obtain the segment address word table corresponding to the initial address database. The segment address word table includes all deduplicated segment words, and the segment words in the segment address word table are all different. Determine the identifier of each segment word in the segmented address word list, wherein different segment words have different identifiers; The segment words in the segment word set corresponding to each address are replaced using the identifiers of each segment word to obtain the compressed address library.

2. The method according to claim 1, characterized in that, Determining the identifier of each segment word in the segmented address word table includes: The identifier of each segment word is determined by its sequence number in the segment address word table.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Determine the set of segmented terms to be searched corresponding to the address to be searched, wherein the set of segmented terms to be searched includes at least one segmented term to be searched; The segment words in the set of segment words to be searched are replaced using the identifiers of each segment word to obtain the transcoded address corresponding to the address to be searched. The transcoded address is matched with the compressed address database to obtain at least one address to be searched.

4. The method according to claim 3, characterized in that, The method further includes: Each segment word is extended and encoded to obtain a segment word group corresponding to each segment word, and each segment word group includes a segment word and multiple extended words; Establish an association between the segmented word groups and their corresponding segmented words.

5. The method according to claim 4, characterized in that, The determination of the set of segmented words corresponding to the address to be searched includes: The address to be searched is segmented into words to obtain at least one search segment word; The at least one search segment term is extended and encoded to obtain search segment term groups corresponding to each search segment term, and each search segment term group includes one search segment term and multiple search extended terms; The search segment words are matched with the segment words, and the set of search segment words corresponding to the address to be searched is determined according to the association relationship.

6. An apparatus for address data compression, characterized in that, include: The splitting unit is used to perform word segmentation processing on each address in the initial address database to obtain a segmented word set corresponding to each address. The initial address database includes multiple addresses, and each segmented word set includes at least one segmented word. The deduplication unit is used to deduplicate the segment words in the segment word set corresponding to all addresses in the initial address database to obtain the segment address word table corresponding to the initial address database. The segment address word table includes all deduplicated segment words, and the segment words in the segment address word table are all different. A determining unit is used to determine the identifier of each segment word in the segmented address word list; The replacement unit is used to replace the segment words in the segment word set corresponding to each address with the identifiers of each segment word to obtain the compressed address library.

7. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program is executed by the processor to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, performs the method as described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1-5.

Images