A method, device, storage medium and server for address search based on deep learning

Abstract

The embodiments of the present application disclose an address search method, device, storage medium and server based on deep learning, which relate to the field of logistics information processing. The method first obtains data such as text address, spatial coordinates, surrounding POI and street view pictures from a multimodal address database. The text address is decomposed into different levels and converted into corresponding vectors, and then spliced ​​into a text address vector. The spatial coordinates, POI and street view pictures are also converted into vectors respectively, and each vector is weighted and summed to generate a multimodal vector. At the same time, the text address entity query knowledge base obtains semantic enhancement information and encodes it into a vector, which is spliced ​​with the multimodal vector to form a final address vector and stored in the address vector library. After the user enters the query text address, a query vector is generated and candidate address vectors with high similarity are retrieved in the library. The detailed information of the candidate address is displayed in the interactive interface, which improves the accuracy and richness of the address description and realizes an efficient address query function.

Description

Technical Field

[0001] The present application relates to the field of logistics information processing, and in particular to an address search method, device, storage medium and server based on deep learning. Background Art

[0002] In today's digital age, with the rapid development of e-commerce, logistics, and geographic information systems (GIS), the accurate processing and efficient utilization of address information has become increasingly important. As key information for locating and describing geographic locations, the accuracy and completeness of addresses directly impact the efficiency and quality of related businesses. However, traditional text-based address search and processing methods have gradually exposed numerous technical issues in the face of increasingly complex and diverse address data, making them unable to meet the demands of modern applications.

[0003] Traditional text addresses are typically presented in natural language. Different people may use different vocabulary, grammatical structures, and expressions to describe the same address. Cities develop and change rapidly, with new roads and buildings constantly emerging, and existing address information may change accordingly. Updating traditional address databases often relies on manual collection and entry, which is not only inefficient but also prone to delayed data updates. Traditional text addresses exist in unstructured natural language and lack a unified standard or format. Address information from different regions and sources varies significantly in terms of expression and field order, making it difficult for computers to directly process and analyze them effectively. Traditional text address searches typically use string matching methods, which compare each character to determine the similarity between addresses. This method is computationally complex and inefficient when processing large amounts of address data. Traditional address databases primarily focus on the textual information of addresses, ignoring the spatial topological relationships between them. Understanding the spatial relationships between addresses is crucial in many practical applications, such as route planning and regional analysis. However, traditional text address search systems cannot directly provide spatial topology information between addresses, which requires additional complex data processing and calculations when performing related spatial analysis, increasing the complexity and computational cost of the system.

[0004] In summary, traditional text address search and processing methods have many technical problems in semantic understanding, data update, structured processing, matching efficiency, and spatial relationship utilization, and are unable to meet the needs of modern applications for accurate and efficient processing of address information. Summary of the Invention

[0005] The embodiments of the present application provide a method, apparatus, storage medium, and terminal device for address search based on deep learning, which can solve the problem of inaccurate and limited text address search results in the prior art. The technical solution is as follows:

[0006] In a first aspect, an embodiment of the present application provides an address search method based on deep learning, the method comprising:

[0007] Acquire various multimodal data in a multimodal address database, wherein the multimodal data includes: a text address, spatial coordinates, surrounding POIs, and street view images;

[0008] Converting the text address into a text address vector, converting the spatial coordinates into a spatial vector, converting the surrounding POI into a POI vector, and converting the street view image into a street view image vector;

[0009] Performing a weighted summation on the text address vector, the spatial vector, the POI vector, and the street view image vector to generate a multimodal vector;

[0010] Dividing the text address into multiple entities, using the entities to query corresponding semantic enhancement information in a knowledge base, and encoding the queried semantic enhancement information to generate a semantic enhancement vector;

[0011] concatenating the semantic enhancement vector and the multimodal vector to generate a final address vector, and writing the final address vector into an address vector library;

[0012] A query is performed in the address vector library according to query information input by the user on the interactive interface, and the query result is displayed on the interactive interface.

[0013] In a second aspect, an embodiment of the present application provides an address search device based on deep learning, the device comprising:

[0014] An acquisition unit, configured to acquire various multimodal data from a multimodal address database, wherein the multimodal data includes: a text address, spatial coordinates, surrounding POIs, and street view images;

[0015] a conversion unit, configured to convert the text address into a text address vector, convert the spatial coordinates into a spatial vector, convert the surrounding POI into a POI vector, and convert the street view image into a street view image vector;

[0016] a generating unit configured to perform a weighted summation of the text address vector, the spatial vector, the POI vector, and the street view image vector to generate a multimodal vector; divide the text address into multiple entities, use the entities to query corresponding semantic enhancement information in a knowledge base, encode the queried semantic enhancement information to generate a semantic enhancement vector; concatenate the semantic enhancement vector and the multimodal vector to generate a final address vector, and write the final address vector into an address vector library;

[0017] The query unit is used to query the address vector library according to the query information input by the user on the interactive interface, and display the query result on the interactive interface.

[0018] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0019] In a fourth aspect, an embodiment of the present application provides a server, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0020] The beneficial effects of the technical solutions provided by some embodiments of the present application include at least:

[0021] By integrating multimodal data such as text addresses, spatial coordinates, nearby POIs, and street view images, address information can be represented more comprehensively and accurately, improving the richness and precision of address descriptions. Converting multimodal data into vector form facilitates mathematical operations and similarity comparisons, providing an efficient data foundation for subsequent address queries and matching. By dividing text addresses into multiple entities and querying the knowledge base for corresponding semantically enhanced information, the semantic representation of the address is further enriched, improving the accuracy and intelligence of address matching. A query text address vector is generated based on the query text address entered by the user, and the address vector library is quickly searched for the candidate address vector with the highest similarity, enabling efficient address query functionality. The text address, spatial coordinates, nearby POIs, and street view images corresponding to the queried candidate address vectors are displayed on the interactive interface, providing users with intuitive and comprehensive query results, facilitating selection and judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 This is a schematic diagram of the network architecture provided by this application;

[0024] Figure 2 This is a flowchart of the deep learning-based address search method provided by this application;

[0025] Figure 3 This is a flowchart of the application's dynamic weight adjustment based on context information;

[0026] Figure 4 This is a structural diagram of an address search device based on deep learning provided by this application;

[0027] Figure 5 is a schematic diagram of the computer storage medium provided by this application;

[0028] Figure 6 This is a structural diagram of a server provided by this application. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

[0030] It should be noted that the deep learning-based address search method provided in this application is generally executed by a server, and accordingly, the deep learning-based address search device is generally set in the server.

[0031] Figure 1 An exemplary network architecture that can be applied to the deep learning-based address search method or deep learning-based address search device of the present application is shown.

[0032] like Figure 1 As shown, the network architecture may include: a terminal device 501 and a server 600. The terminal device 501 and the server 600 may communicate via a network, which serves as a medium for providing communication links between the aforementioned units. The network may include various types of wired or wireless communication links, such as optical fiber, twisted pair, or coaxial cable, and wireless communication links such as Bluetooth, Wi-Fi, or microwave.

[0033] The server 600 deploys a vector address library, the terminal device 501 displays an interactive interface, the user enters a query text in the interactive interface, and an address search is performed in the vector address library based on the entered query text.

[0034] The core concept of this application is:

[0035] 1. Multimodal Address Vector Search: This approach combines address text with other modal data (such as map coordinates, POI information, and images) to construct a multimodal address vector. By integrating text, spatial, and visual information, the accuracy and richness of address searches are improved. For example, when a user enters an address, the system not only matches the text vector but also combines map coordinates and surrounding POI information to provide more accurate search results. This approach is particularly suitable for complex scenarios, such as fuzzy address matching or cross-region searches.

[0036] 2. Dynamic Context-Aware Search: This approach leverages contextual information (such as the user's previous search history, location, and time) to dynamically adjust the address vector search strategy. By incorporating a context-aware model, the system understands the user's real-time needs and provides personalized search results. For example, when a user searches for "City A train station," the system prioritizes recommendations for nearby transportation hubs or related services based on the user's current location and historical preferences. This approach enhances search intelligence and user experience.

[0037] 3. Hierarchical Address Vector Search: Address vectors are broken down into hierarchical levels (e.g., country, province, city, street, etc.) to construct a hierarchical vector representation. During searches, high-level vectors (e.g., city) are matched first, and then gradually refined to lower levels (e.g., street). This approach significantly reduces the search space and improves search efficiency, making it particularly suitable for large-scale address databases. Furthermore, hierarchical search supports fuzzy matching, returning relevant results even for incomplete input addresses.

[0038] 4. Semantically Enhanced Address Vector Search: This approach enhances the semantic representation of address vectors by incorporating external knowledge bases (such as geographic information databases and administrative division databases). For example, by integrating related information such as "District A" with "Village A" and "University A," a more semantically rich vector is generated. During searches, the system not only matches literal information but also understands the implicit semantics of the address, providing more relevant search results. This approach is particularly suitable for complex queries, such as "addresses near the Science and Technology Park."

[0039] 5. Real-time incremental learning and updates: A real-time incremental learning mechanism is introduced into the address vector search system to dynamically update the address vector representation. For example, when new address data is added or old address information is changed, the system can adjust the vector model in real time to ensure the timeliness and accuracy of search results. This approach is suitable for scenarios where address data frequently changes, such as in industries such as logistics and food delivery, and can effectively address the challenges brought about by address updates.

[0040] This technical solution is not only applicable to general address searches (such as map navigation and real estate inquiries), but can also be extended to scenarios such as logistics distribution, O2O services (such as food delivery and taxi-hailing), and smart cities. It can meet the business needs of different fields through customized knowledge bases and weight configurations.

[0041] It should be noted that the terminal device 501 and the server 600 can be either hardware or software. When the terminal device 501 and the server 600 are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the terminal device 501 and the server 600 are software, they can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or software module, without specific limitation herein.

[0042] Various communication client applications can be installed on the terminal device of this application, such as: video recording applications, video playback applications, voice interaction applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0043] A terminal device can be either hardware or software. If the terminal device is hardware, it can be any terminal device with a display screen, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. If the terminal device is software, it can be installed in the terminal devices listed above. It can be implemented as multiple software programs or software modules (for example, to provide distributed services) or as a single software program or software module, without specific limitation here.

[0044] If the terminal device is hardware, it may also be equipped with a display device and a camera. The display device can be any device capable of displaying a video, and the camera is used to capture video streams. For example, the display device may be a cathode ray tube display (CR), a light-emitting diode display (LED), an electronic ink screen, a liquid crystal display (LCD), or a plasma display panel (PDP). Users can use the display device on the terminal device to view text, images, videos, and other information.

[0045] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the embodiment is only illustrative. Any number of terminal devices, networks, and servers may be used as needed.

[0046] The following will be combined with the Figure 2 , the address search method based on deep learning provided by the embodiment of the present application is described in detail. Among them, the address search device based on deep learning in the embodiment of the present application can be Figure 1 The terminal device shown.

[0047] See Figure 2 , provides a flowchart of an address search method based on deep learning for an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps:

[0048] S201. Acquire various multimodal data in a multimodal address database.

[0049] The server retrieves data from a multimodal address database using a database interface (such as JDBC, ODBC, or a RESTful API), supporting local deployment or cloud storage (such as MySQL, MongoDB, or AWS S3). During data retrieval, the server dynamically generates query statements based on pre-set criteria (such as data update time and data integrity), for example, filtering records updated within the past 30 days or excluding records with empty key fields.

[0050] Text addresses: Checks whether they contain garbled characters (such as non-Chinese and English characters), special symbols (such as !@#$%^&*), or exceed a threshold (such as 200 characters). If abnormal, replace the illegal characters using regular expressions or mark them as invalid.

[0051] Spatial coordinates: Verify that longitude and latitude are within a reasonable range (longitude [-180,180], latitude [-90,90]) and unify the coordinate system (e.g., converting the GCJ-02 Martian coordinate system to WGS84).

[0052] Nearby POIs: Check whether the POI list is empty. If so, fill it with a default value (such as "Unknown Landmark") or mark it as missing. If the POI name contains uncommon words, split it using a word segmentation tool (such as IK Analyzer) and then complete it with the associated knowledge base.

[0053] Street View images: Verify whether the image file exists (for example, check whether the file path is valid) and whether the file format is supported (such as JPEG, PNG). If the image file is missing, replace it with a default placeholder image (such as a gray background image).

[0054] For example:

[0055] The server retrieves a record from the database:

[0056] Text address: "No. 1, Jiacun Street, Jia District, Jia City @#￥%" (including garbled characters)

[0057] Space coordinates: 116.3184, 39.9642 (WGS84 format, no conversion required)

[0058] Nearby POIs: [] (empty list)

[0059] Street view image: changan_zhongcun_001.jpg (file exists)

[0060] Cleaning results:

[0061] The text address is corrected to "No. 1, Jiacun Street, Jia District, Jia City";

[0062] The POI list is filled with the default value "Unknown Landmark";

[0063] The image file verification passed and the original path was retained.

[0064] S202: Convert the text address into a text address vector, convert the spatial coordinates into a spatial vector, convert the surrounding POIs into a POI vector, and convert the street view image into a street view image vector.

[0065] The server converts each modal data into a high-dimensional vector and needs to select an encoding model that is suitable for the scenario:

[0066] Text address vector: Use pre-trained language models (such as BERT and RoBERTa) to extract semantic features and output a 768-dimensional vector.

[0067] Spatial vector: Normalize the longitude and latitude to the range [0, 1] (for example, using Min-Max scaling), or use GeoHash encoding to generate a binary hash value (for example, wx4g09c6).

[0068] POI vector: After the POI list is segmented, word vectors are generated using TF-IDF or Word2Vec, and then aggregated (such as average pooling) into a fixed-dimensional vector (such as 300 dimensions).

[0069] Street view image vector: Use a convolutional neural network (such as ResNet-50 and EfficientNet) to extract global features and output a 2048-dimensional vector.

[0070] For example:

[0071] The text address "No. 1000, Jiamingzhu (building name), Ring Road, Jiaxin District, Jia City" is encoded into a 768-dimensional vector by BERT;

[0072] The spatial coordinate (121.4737, 31.2304) is normalized to [0.674, 0.348];

[0073] The POI list ["A City Center Building"] generates two 300-dimensional vectors through Word2Vec, and the average is used to obtain a 300-dimensional POI vector;

[0074] The street view image Abc_001.jpg is extracted into a 2048-dimensional vector by ResNet-50.

[0075] S203 : Perform a weighted summation on the text address vector, the spatial vector, the POI vector, and the street view image vector to generate a multimodal vector.

[0076] The server performs a weighted summation of the vectors for each modality. Weights can be assigned based on service priorities (e.g., 0.4 for text, 0.3 for space, 0.2 for POIs, and 0.1 for images). Alternatively, the server can train a classification model (e.g., SVM or XGBoost) to infer the contribution of each modality to the target task (e.g., address matching). After weighting, the vectors are L2-normalized to ensure consistent distribution across modalities.

[0077] For example:

[0078] Assuming the weights are text 0.4, space 0.3, POI 0.2, and image 0.1, then:

[0079] The multimodal vector = 0.4 × text vector + 0.3 × space vector + 0.2 × POI vector + 0.1 × image vector. After normalization, the final vector is 512 dimensions.

[0080] S204: Divide the text address into multiple entities, use the entities to query corresponding semantic enhancement information in the knowledge base, and encode the query semantic enhancement information to generate a semantic enhancement vector.

[0081] The server splits the text address into entities (such as "City A" and "New District A") and queries the related information through a knowledge base (such as OpenStreetMap):

[0082] Use NER models (such as Stanford NER and LTP) to identify entity types (such as cities, regions, and landmarks); query the knowledge base to obtain entity attributes (such as the area and population of "New District A") or the relationship between nearby subway stations), and convert the completed semantic information into vectors through graph neural networks (such as GAT and GCN) or knowledge embedding models (such as TransE and RotatE).

[0083] For example:

[0084] The entity "New District A, City A" is associated with the knowledge base information: "Population: 5.68 million" and "GDP: 1.6 trillion yuan." The semantic enhancement vector is encoded into a 256-dimensional vector through GAT.

[0085] S205 , concatenating the semantic enhancement vector and the multimodal vector to generate a final address vector, and writing the final address vector into an address vector library.

[0086] The server concatenates the semantically enhanced vector with the multimodal vector (e.g., directly concatenating or fusing via an attention mechanism) to generate the final address vector. When writing the vector to the vector library, you must select an index type: FlatIP indexing (inner product calculation) for exact matches; Approximate Nearest Neighbor (ANN) indexing: IVF Flat or HNSW indexing to accelerate large-scale queries.

[0087] For example:

[0088] The concatenated vector has a dimension of 512 (multimodal) + 256 (semantic enhancement) = 768. The vector is stored in the Faiss vector library with an index type of IVF1024 and Flat.

[0089] S206: Query the address vector library according to the query information input by the user in the interactive interface, and display the query result on the interactive interface.

[0090] The server receives a user query (such as the text "near the Oriental Building" or an image), converts it into a vector, and then searches the vector library. For text queries, BERT generates vectors; for image queries, ResNet-50 extracts feature vectors. Candidate results are sorted using inner product or cosine similarity. Results are filtered based on business rules (such as spatial distance threshold and POI relevance). Results are sorted in descending order of similarity, displaying the text address, POI tag, thumbnail, and distance information.

[0091] For example:

[0092] The user enters the text "Ping'anli" (the specific name of a small area in a certain place). The server generates a query vector and finds the top three addresses with the highest similarity in the vector database.

[0093] The interface displays: "No. 1000, Ping'anli Ring Road, Jiaxin District, Jia City (similarity 0.95, distance 500 meters)" and "Jia City Center Building (similarity 0.90, distance 800 meters)", etc., with street view preview and navigation link.

[0094] In some possible embodiments of the present application, converting the text address into a text address vector includes:

[0095] Decomposing the text address into a country address, a province / municipality address, a city / district address, and a street / house number address;

[0096] According to the pre-trained PromptCSE-Base model, addresses at each level are converted into national address vectors, province / municipality address vectors, city / district vectors, and street address vectors, and the address vectors at each level are concatenated to obtain text address vectors.

[0097] Specifically, the server selects an address resolution tool based on the data source and accuracy requirements. The address resolution tool can be:

[0098] Regular expressions: Applicable to addresses with standardized formats (such as the "province-city-region-street" structure).

[0099] NLP models: such as the BERT-based address parsing model, which can process unstructured addresses.

[0100] Third-party API: such as a map's address resolution API, which supports complex address splitting.

[0101] The server determines the country by keyword matching (such as "country A") or geocoding.

[0102] Province / Municipality Address: Identify the name of the provincial administrative region (such as "Province A" and "City A").

[0103] City / District Address: Extract the city or district name (such as "Chang'an District").

[0104] Street / house number address: Extract the remaining part as the street and house number (for example, "No. 103, Tiyu West Road").

[0105] It should be noted that if a level cannot be resolved (e.g., a missing province in an address), a default value (e.g., "Unknown Province") may be filled in or marked as missing data. Contextual analysis or manual intervention are performed for ambiguous addresses (e.g., "Zhongshan Road" may exist in multiple cities).

[0106] The server loads the pre-trained PromptCSE-Base model from the model repository, which may be stored in the local file system or cloud object storage. The decomposed address layers (country, province, city, street) are input into the model separately, and the model generates corresponding vectors through a multi-layer Transformer encoder. The vector dimension may be 768 or 1024, depending on the model structure. The vectors of the four layers are concatenated in a fixed order (such as country first and street last) to form a complete text address vector. The concatenation method may be direct connection or weighted connection (such as country vector weight 0.2, street vector weight 0.4). The concatenated vector is normalized (such as L2 normalization) to ensure the consistency of vector length. Metadata (such as vector source, generation time) can be attached when storing the vector.

[0107] For example, input "No. 103, Tiyuxi Road, District A, City A":

[0108] Country vector: [0.1, 0.3, ..., -0.2] (country A);

[0109] Province vector: [0.2, -0.1, ..., 0.4] (Province A);

[0110] City vector: [-0.3, 0.5, ..., 0.1] (Area A);

[0111] Street vector: [0.4, -0.2, ..., 0.3] (No. 103, Tiyu West Road);

[0112] The dimension of the text address vector after splicing is the sum of the lengths of four partial vectors (e.g., 768×4 = 3072).

[0113] In some embodiments of the present application, the process of converting the surrounding POIs into connected POIs includes: respectively converting each POI included in the surrounding POIs into a corresponding vector, and taking the average value of the converted vectors to obtain a POI vector.

[0114] Specifically, the server de-duplicates and standardizes the POI list (e.g., merging "KenKen" and "肯肯"), and filters out invalid POIs (e.g., "unknown", "other"). Each POI name is converted into a vector using a word embedding model (e.g., Word2Vec, FastText) or a pre-trained language model (e.g., BERT). The arithmetic mean of all POI vectors is taken to obtain a POI vector. If the POI list is empty, a default vector (e.g., a vector of all zeros) is used or this step is skipped. The POI vector is smoothed (e.g., weighted average, assigning weights according to the importance of the POI).

[0115] For example: POI list ["City A", "Square A", "East Station A"]:

[0116] Vector of "City A": [0.5, -0.2, ..., 0.1];

[0117] Vector of "Square A": [0.3, 0.4, ..., -0.1];

[0118] Vector of "East Station A": [-0.1, 0.6, ..., 0.2];

[0119] POI vector after averaging: [0.23, 0.27, ..., 0.07].

[0120] In some possible embodiments of the present application, the query information is a query text address;

[0121] Among them, the query in the address vector library according to the query information input by the user in the interaction interface, and the display of the query result on the interaction interface includes:

[0122] Obtain the query text address input by the user on the interaction interface;

[0123] Decompose the query text address into a national address, a provincial / municipal address, a city / district address, and a street / house number address according to a preset rule;

[0124] Call the pre-trained PromptCSE-Base model to convert the decomposed addresses of each level into corresponding address vectors;

[0125] The address vectors of each level are concatenated to obtain the text address vector;

[0126] A multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector;

[0127] Dividing the query text address into multiple entities, using each entity to query corresponding semantic enhancement information in a knowledge base, and encoding the semantic enhancement information to generate a semantic enhancement vector;

[0128] The generated semantic enhancement vector and the multimodal vector obtained by weighted summation are concatenated to obtain a query text address vector corresponding to the query text address;

[0129] Querying the address vector library for several candidate address vectors with the highest similarity according to the query text address vector;

[0130] The text addresses, spatial coordinates, surrounding POIs and street view pictures corresponding to the plurality of candidate address vectors are displayed on the interactive interface.

[0131] Specifically, the server generates a query text address vector according to the query text address input by the user on the interactive interface.

[0132] Among them, the query address entered by the user is cleaned (such as removing punctuation and unifying upper and lower case), and it is verified whether the query is empty or has an abnormal format.

[0133] Use the same PromptCSE-Base model as above to generate a vector representation of the query text. The vector dimension is consistent with the final address vector generated above. The query vector is normalized or standardized to ensure comparability with the vectors in the address vector library.

[0134] In some embodiments of the present application, the process of generating a query text address vector includes:

[0135] Get the query text address entered by the user on the interactive interface;

[0136] Decomposing the query text address into country address, province / municipality address, city / district address, and street / house number address according to preset rules;

[0137] Call the pre-trained PromptCSE-Base model to convert the decomposed addresses of each level into corresponding address vectors;

[0138] The address vectors of each level are concatenated to obtain the text address vector;

[0139] A multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector;

[0140] Dividing the query text address into multiple entities, using each entity to query corresponding semantic enhancement information in a knowledge base, and encoding the semantic enhancement information to generate a semantic enhancement vector;

[0141] The generated semantic enhancement vector and the multimodal vector obtained by weighted summation are concatenated to obtain a query text address vector corresponding to the query text address.

[0142] Among them, the server receives the text address entered by the user (for example, "No. 88, Jianguo Road, District A, City A") through an interactive interface (such as a web page, mobile APP or API interface).

[0143] Preprocess the input text, including:

[0144] Format standardization: unify upper and lower case, remove redundant spaces or special symbols (for example, "Chang'an City" and "Chang'an" are considered equivalent).

[0145] Anomaly detection: Checks whether the input is empty or contains illegal characters. If so, an error message is returned.

[0146] The server decomposes the query text address into the following levels based on preset address hierarchy rules (such as administrative division code table and geographic dictionary):

[0147] Country level: Extract country names (such as "Country A").

[0148] Province / Municipality Level: Extract the provincial administrative region name (such as "City A").

[0149] City / District Level: Extract the city-level or district-level administrative district name (such as "District A").

[0150] Street / house number level: Extract street names and house numbers (e.g., "88 Jianguo Road").

[0151] Fuzzy matching support: If the user input is incomplete (for example, only "Jianguo Road, District A"), the missing level will be supplemented based on the context or historical records (for example, the default city is "Chang'an").

[0152] The server calls the pre-trained PromptCSE-Base model (a text encoding model based on contrastive learning) to convert the decomposed addresses at each level into vector representations:

[0153] Each address level is input into the model as independent text (e.g., "Country A," "City A," "District A," "88 Jianguo Road"). The model generates semantic vectors for each level (e.g., 128- or 256-dimensional), each capturing the semantic features of the corresponding level. The server concatenates the address vectors at each level into a single textual address vector in a preset order. The concatenation rule can be, for example, country vector + province vector + city vector + street vector. Vectors can be separated by special delimiters (e.g., an all-zero vector) or by direct concatenation. The concatenated vectors are L2-normalized to ensure consistent vector modulo lengths and prevent numerical discrepancies from affecting subsequent calculations.

[0154] The server performs a weighted summation of the concatenated text address vector and the following default vector to generate a multimodal vector:

[0155] Default spatial vector: An embedding vector representing the geographic coordinates (such as longitude and latitude) of an address, generated through GeoHash or a deep learning model. The geographic coordinates can be manually entered by the user or pre-stored fixed coordinates. For example, the fixed coordinates are empty.

[0156] Default POI vector: a semantic vector representing points of interest (such as shopping malls and schools) near the address. Points of interest can be selected by the user from the POI database or pre-stored fixed POIs. For example, POI is empty.

[0157] Default street view image vector: represents the visual feature vector of the street view images around the address. It is obtained by encoding the street view images using a CNN model (such as ResNet). The surrounding street view images can be input by the user or left blank.

[0158] The weight of each vector can be adjusted through experiments (e.g., text vector weight 0.6, spatial vector weight 0.2, POI vector weight 0.1, street view vector weight 0.1) to balance the influence of different modalities.

[0159] The server divides the query text address into multiple entities (such as "City A", "District A", and "No. 88 Jianguo Road") and uses a knowledge base (such as a geographic knowledge graph) to query the semantic enhancement information of each entity:

[0160] Information types include entity categories (e.g., "administrative district," "street"), aliases (e.g., "District A" is also known as "Dong'an"), and related entities (e.g., subway stations near "Jianguo Road"). Semantic enhancement information for each entity is encoded to generate a semantic enhancement vector. This can be done using a pre-trained language model (e.g., BERT) to encode semantic text or by mapping rules to a predefined semantic vector space. The semantic enhancement vectors for all entities are concatenated or weighted summed to generate a global semantic enhancement vector.

[0161] The server concatenates the generated semantically enhanced vector with the multimodal vector to obtain the final query text address vector. This can be directly added to the end of the multimodal vector or dynamically integrated through a gating mechanism. The final vector is normalized to ensure a consistent numerical range.

[0162] The server converts the text address entered by the user into a vector representation containing multimodal information and semantically enhanced features, providing high-dimensional, highly semantic input features for subsequent address matching, significantly improving the accuracy and efficiency of address resolution.

[0163] In some embodiments of the present application, the weights of the default spatial vector, the default POI vector, the default street view image vector, and the concatenated text address vector are dynamically adjusted according to context information, see Figure 3 As shown, a flow chart of a method for dynamically adjusting weights based on context information provided in an embodiment of the present application is as follows:

[0164] S301. When a user inputs a query text address, user context information is collected. The context information includes user search history, address location, and time.

[0165] S302: Convert the context information into a context vector.

[0166] S303: Input the context vector into a pre-trained context-aware model to calculate the weights of a default space vector, a default POI vector, a default street view image vector, and a concatenated text address vector.

[0167] The server extracts the user's most recent N (e.g., most recent 10) address search records from the user behavior log, including the text address searched, the clicked address results, the search time, etc. The historical search records are denoised (e.g., removing duplicate or invalid records), and key features (e.g., search frequency, search result preferences) are extracted. The user's current geographic location (e.g., longitude and latitude coordinates) is obtained through the user's device's GPS, IP positioning, or Wi-Fi positioning. If the user does not authorize positioning permissions, commonly used addresses in the history (e.g., home address, work address) are used as a substitute. The timestamp of the user's query is recorded (accurate to the second), and time features are extracted (e.g., weekday / weekend, daytime / nighttime, holidays, etc.).

[0168] For text addresses in historical search records, the same text encoding model as in S210 (e.g., PromptCSE-Base) is used to generate an address vector. Search result preferences (e.g., the types of POIs frequently clicked by users) are one-hot encoded or embedded to generate a preference vector. The user's geographic location (latitude and longitude) is converted into a GeoHash code (e.g., an 8-bit string) and a spatial vector is generated using a pre-trained GeoHash embedding model. Alternatively, the latitude and longitude coordinates can be directly used as input and mapped to a low-dimensional vector space using a fully connected layer (Dense Layer). The timestamp is split into features such as hour, day of the week, and month, and a time vector is generated using an embedding layer. Special time markers, such as holidays, are one-hot encoded and concatenated with the time vector.

[0169] The server concatenates the historical search record vector, address location vector, and time vector into a single context vector:

[0170] Concatenation rule: For example, the historical search vector (256 dimensions) + the address location vector (128 dimensions) + the time vector (64 dimensions) = the context vector (448 dimensions). If the dimensions of the vectors are inconsistent, dimension mapping or zero padding can be performed through a fully connected layer.

[0171] The server inputs the context vector into a pre-trained context-aware model (such as a lightweight Transformer-based model or a graph neural network). The context vector is used as the model input, and the model captures the correlation between context features through self-attention or graph convolution.

[0172] The model outputs the following weight parameters:

[0173] Default spatial vector weight: used to adjust the contribution of geographic coordinate vectors in the multimodal vector (e.g. 0.2).

[0174] Default POI vector weight: used to adjust the contribution of the point of interest vector in the multimodal vector (such as 0.15).

[0175] Default street view image vector weight: used to adjust the contribution of the street view image vector in the multimodal vector (e.g. 0.1).

[0176] Concatenated text address vector weight: used to adjust the contribution of the text vector in the multimodal vector (such as 0.55).

[0177] Weight constraint: The sum of all weights is 1, and normalization is performed through the Softmax layer.

[0178] The server converts user context (historical searches, location, and time) into vectors and uses a context-aware model to dynamically weight the multimodal vectors, ultimately generating a context-aware query text address vector. This process significantly improves the accuracy and personalization of address resolution, especially in complex scenarios (such as users frequently searching for addresses in the same area or within a specific time period).

[0179] The server calculates the cosine similarity or Euclidean distance between the query vector and all vectors in the address vector database. This similarity calculation may be parallelized to improve efficiency. The top N candidate vectors with the highest similarity (e.g., top-5) are returned. A threshold (e.g., similarity > 0.8) may be set for selection. Candidate vectors are sorted from highest to lowest similarity.

[0180] S304 , performing weighted summation on the default spatial vector, the default POI vector, the default street view image vector, and the concatenated text address vector according to the calculated weights to obtain a multimodal vector.

[0181] Furthermore, the server supports hierarchical search, the specific process includes:

[0182] If the query vector has been marked with level information when it is generated (such as the vectors are spliced ​​by level in S203), the sub-vectors of each level are directly split.

[0183] If the query vector is a whole vector, it is dynamically split using a model or rules (such as using an attention mechanism to extract city-level features). Example: When querying "City A", the server splits its vector into:

[0184] City-level vector: corresponds to the semantic representation of "City A".

[0185] Street-level vector: corresponds to the semantic representation of "Zone A Sports West Road".

[0186] High-level vector matching (city level): The server first uses the city-level vector to perform a coarse-grained search within the address vector database, filtering out all candidate addresses containing that city vector. It then calculates the cosine similarity between the query city vector and the city vectors in the candidate addresses, ignoring all other levels. Candidate addresses with a similarity above a preset threshold (e.g., 0.8) are retained, initially narrowing the search scope. This step quickly filters out over 90% of irrelevant addresses, significantly reducing subsequent computational effort.

[0187] For the candidate addresses after high-level matching, the server further uses the street-level vector for refined search: calculate the similarity between the query street vector and the street vectors in the candidate addresses, and combine the similarity at the city level for weighting (such as city weight 0.4 and street weight 0.6). If the street-level information is incomplete (such as the query is "A District" but the street is not specified), then reduce the street weight and increase the weight at the city or POI level. Example: When querying "City A", first match the addresses within "A District", and then refine to the addresses near "Sports West Road".

[0188] For the candidate addresses after middle-level matching, the server further combines the house number or POI vector for exact matching: If the query contains a POI (such as "City A Shopping Center"), then calculate the similarity between the query POI vector and the POI vectors in the candidate addresses. Use a string similarity algorithm (such as Levenshtein distance) for fuzzy matching of the house number, tolerating partial character differences (such as "No. 103" and "No. 105" may be matched). Weight and sum the similarity scores at each level to generate the final similarity ranking.

[0189] The server sorts the candidate addresses in descending order according to the comprehensive similarity and returns the Top-N results (such as Top-10).

[0190] Among them, retrieve the corresponding original data (text address, coordinates, POI, street view) from the database according to the candidate vector, and may retrieve associated information (such as user reviews, business hours) at the same time. Format the information into a list, map marker or card form, which may include address details, picture thumbnails, navigation buttons, etc. On the interactive interface displayed on the terminal device, the user is allowed to click on the candidate address to view more details (such as panoramic view, surrounding facilities), and the interactive interface also provides a feedback mechanism (such as "This address is incorrect"). After reviewing the feedback information, update the multi-modal address database and address vector database.

[0191] In some embodiments of the present application, the server can report address deviations, new addresses or temporary change information through the crowdsourcing client to update the multi-modal address database in real time. The specific process includes:

[0192] The server receives data correction instructions from the crowdsourcing client (such as manual annotators or user feedback tools) through the crowdsourcing task platform or API interface. The instruction content includes the following information:

[0193] Address ID to be corrected: Uniquely identifies the address to be modified (e.g., database primary key, address hash value). Correction Field: Clearly indicates the field to be modified (e.g., street name, house number, POI category, etc.). Corrected Value: Corrected value provided by the crowdsourced user (e.g., "88 Jianguo Road" to "90 Jianguo Road"). Correction Basis: Optional field that records the source of the correction (e.g., user feedback, field verification records, satellite image comparison, etc.).

[0194] The server first automatically verifies the correction instruction, including:

[0195] Check whether the instruction contains necessary fields and whether the field values ​​meet the format requirements (such as house numbers are numbers). Verify whether the corrected value is logically consistent with other fields of the address (such as the street it belongs to, the administrative division) (such as "No. 90 Jianguo Road" should be located in "District A" instead of "District A"). Check whether the correction instruction conflicts with other unsubmitted correction instructions in the database (such as two different correction requests received for the same address at the same time). Correction instructions that fail the automated review or have high risks (such as those involving administrative division changes) are submitted to the manual review queue. The reviewer verifies the address information before and after the correction through visual tools (such as the map annotation interface) and verifies it with reference to external data sources (such as government public data, third-party map APIs). Instructions that pass the review are marked as "confirmed", and instructions that fail the review are marked as "rejected" and the reason for rejection is returned (such as "the correction value does not match the satellite image").

[0196] For approved correction instructions, the server first locks the database record of the address to be modified to prevent concurrent modifications that could cause data inconsistencies. The server directly modifies the address's text fields (such as street name and house number) and simultaneously updates the address's hierarchical representation (such as the city, district, and street hierarchy). If the correction involves geographic location (e.g., a house number change resulting in a shift in longitude and latitude), the address's geographic coordinates are recalculated and the GeoHash encoding or spatial index is updated. Nearby POI associations are rematched (e.g., the bus stops near "90 Jianguo Road" may have changed). If the corrected address corresponds to a new street view image (e.g., a building's exterior has changed), a street view image collection task is triggered, and the street view image vector is updated. Historical street view images are archived to retain a record of changes for retrospective purposes. A semantically enhanced vector for the address is regenerated (e.g., by querying the knowledge base for new entity association information). The address's text vector is reencoded (e.g., by using the PromptCSE-Base model to generate a new vector). Recalculate the weighted multimodal vector based on the updated text vector, spatial vector, POI vector, and street view image vector (the weights continue to use the weights output by the most recent context-aware model, or trigger the weight update process).

[0197] Updates the address's associated data (such as address information in user favorites and historical searches). Cascade updates to other data dependent on the address (such as logistics delivery ranges and merchant service areas). Generates a version record for each modification to address data, including modification time, modification person, and modification details, supporting data rollback and auditing.

[0198] The server returns the correction results to the crowdsourcing client: If successful, it returns "Correction Successful" and the updated address details. If unsuccessful, it returns "Correction Failed" and the specific reason (e.g., "Administrative Division Conflict"). For critical corrections (e.g., address changes involving public services), a notification mechanism (e.g., email, SMS) is triggered to inform relevant stakeholders (e.g., businesses, government agencies).

[0199] The server can securely and efficiently respond to data correction requests from crowdsourcing clients, ensuring the accuracy and real-time nature of the multimodal address database. This process not only supports rapid response to human feedback but also ensures data quality through rigorous review and version control mechanisms, making it suitable for scenarios requiring high-precision geographic information.

[0200] In some embodiments of the present application, a full synchronization method is used to synchronize all multi-module data from the industry address database to the multi-modal address database;

[0201] When it is detected that data in the industry address database is updated, the updated data in the industry address database is synchronized to the multimodal address database in an incremental synchronization manner;

[0202] When it is detected that data update occurs in the multimodal address database, the address vector library is updated according to the updated data in the multimodal address database.

[0203] Among them, 1. Full synchronization stage.

[0204] When initializing the multimodal address database, or periodically (e.g., weekly), perform full data synchronization to ensure initial consistency between the database and the industry address database. The server connects to the industry address database through a preset API interface or data file transfer protocol (e.g., FTP, SFTP) to obtain its multi-module data (e.g., basic address information, administrative divisions, POI data, spatial coordinates, etc.). Extract all module data from the industry address database and perform format conversion (e.g., convert XML format to JSON format) to ensure compatibility with the table structure of the multimodal address database. Write the converted data in batches to the multimodal address database, overwriting the original data (or clearing it and reinserting it). For large data volume scenarios, adopt a batch loading strategy to avoid single operation timeouts or memory overflows. Record the start time, end time, amount of data processed, and synchronization results (success / failure) of the full synchronization to facilitate subsequent auditing and troubleshooting.

[0205] 2. Incremental synchronization phase.

[0206] The server detects data updates through a polling mechanism (e.g., every 5 minutes) or through change notifications provided by the industry address database (e.g., WebSocket push). If the industry address database supports a change log (e.g., an operation log table), the server directly queries the change log since the most recent synchronization. If not, the server identifies changes by comparing the current data with a snapshot of the data from the last synchronization (e.g., by comparing hash values). The server extracts the change data (including new, modified, and deleted records) and converts it into the format required by the multimodal address database.

[0207] Incremental data loading:

[0208] New data: directly inserted into the multimodal address database.

[0209] Modify data: Update the corresponding record based on the primary key or unique identifier.

[0210] Delete data: Delete the corresponding record from the multimodal address database (or mark it as "deleted") based on the primary key or unique identifier.

[0211] If the incremental data conflicts with unsynchronized local modifications in the multimodal address database (e.g., the same address is modified simultaneously by a crowdsourcing client and an industry address database), a conflict resolution process is triggered: the industry address database data (considered the authoritative data source) is prioritized. The conflict is logged, and the administrator is notified for manual intervention. The synchronization schedule is updated to record the latest timestamp or change ID of this incremental synchronization for use in the next synchronization.

[0212] 3. Address vector library update process.

[0213] When a data update is detected in the multimodal address database (including initial updates after full synchronization, updates after incremental synchronization, or direct modifications by crowdsourcing clients), the server captures the change records in the multimodal address database in real time through database triggers (such as INSERT / UPDATE / DELETE triggers) or change data capture (CDC) technology. For newly added or modified address records, the address vector is generated or updated according to the following steps:

[0214] Text vector generation: Call a pre-trained text encoding model (such as PromptCSE-Base) to convert the address text into a vector.

[0215] Spatial vector generation: The geographic coordinates (latitude and longitude) of the address are converted into GeoHash encoding and a spatial vector is generated through the embedding model.

[0216] POI and street view vector generation: Query the POI data and street view images associated with the address, and generate corresponding semantic vectors and visual vectors respectively.

[0217] Multimodal vector concatenation and weighting: Based on the weights output by the context-aware model, the above vectors are weighted and summed to generate a multimodal vector.

[0218] Semantic enhancement vector fusion: Combine the semantic enhancement information in the knowledge base to generate the final address vector.

[0219] The generated address vector is written to the address vector library and associated with the unique identifier (e.g., ID) of the corresponding address. For deleted address records, the corresponding vector is deleted from the address vector library (or marked as "invalid"). The address vector library is scanned regularly (e.g., once a day) to check for vectors with no associated addresses or invalid vectors, and these are cleaned up.

[0220] This embodiment combines full and incremental synchronization to allow the server to efficiently and reliably maintain consistency between the multimodal address database and the industry address database. At the same time, by updating the address vector database in real time, the accuracy of address matching and retrieval services is ensured.

[0221] Furthermore, the specific process of full synchronization and incremental synchronization includes:

[0222] The server divides the data into multiple logical partitions (such as "City A partition" or "POI data partition") based on the table structure of the industry address database (such as by administrative division, address type or data volume distribution). For large data volume tables (such as the address master table), hash partitioning (modulo the primary key hash value) or range partitioning (by ID range) is used to ensure that the data volume of each partition is balanced. The partitioning rules (such as partition fields, number of partitions) and partition boundaries (such as hash value range, ID range) are recorded in the metadata table for subsequent task scheduling and interruption recovery. Based on the xxjob distributed task scheduling framework, the server creates an independent synchronization task (Job) for each data partition, and configures the following parameters in the task:

[0223] Data source configuration: connection information of the industry address library (such as database URL, user name, password).

[0224] Target configuration: connection information of the multimodal address database and the target table corresponding to the partition.

[0225] Synchronization range: the data boundary of the current partition (such as hash value range, ID range).

[0226] The xxjob framework dynamically allocates tasks based on the number of partitions and server resources, supporting parallel execution (for example, launching 10 partition tasks simultaneously). Task dependencies are managed (for example, subsequent tasks can only be started after the predecessor task has completed). During execution, each synchronization task periodically (for example, every 1,000 records) reports progress information to the xxjob monitoring center, including: processed data volume, total data volume, processing speed (records / second), and the location of the data currently being processed (such as the ID or hash value of the last record). The server displays the progress bar, status (running / completed / failed), and log information for each partition task through the xxjob web management interface. If a task is interrupted due to network failure, server downtime, or timeout, the xxjob framework automatically marks the task as "failed" and records the data location at the time of the interruption (such as the ID of the last successfully processed record). When the task is restarted, the server reads the interrupted data location from the metadata table and configures the synchronization task to start at that location, skipping the processed data to avoid duplicate synchronization. For failed tasks, log analysis identifies the cause of the interruption (such as database connection timeout or data format error), triggering a retry or manual intervention.

[0227] The server deploys the canal client on the database server of the industry address database, disguises itself as a MySQL slave (for example, by simulating the MySQL replication protocol) and monitors the master's binlog in real time. The canal client parses the binlog for inserts, updates, and deletes, extracting the change data (such as row data, change timestamp, and operation type).

[0228] Canal encapsulates the change data into a message in JSON or Protobuf format, which contains the following fields:

[0229] Operation type: INSERT / UPDATE / DELETE. Data content: Complete row data after the change (for UPDATE operations, both old and new data are included). Metadata: Table name, primary key value, change timestamp.

[0230] The server creates a separate Kafka topic (such as address_db_changes) for incremental synchronization and configures multiple partitions (such as 8 partitions) to support high throughput.

[0231] Canal acts as a producer to send change messages to the topic and selects partitions based on the table name or primary key hash value to ensure that change messages with the same address are sent to the same partition.

[0232] The server deploys a Kafka consumer group in the multimodal address database synchronization service. The consumer group subscribes to the address_db_changes topic and is configured with the following parameters: Multiple consumer instances are started based on the number of Kafka partitions to enable parallel processing. Kafka's consumer group mechanism automatically manages offsets to ensure that messages are not lost or processed repeatedly.

[0233] The consumer parses the change message and performs the following operations based on the operation type:

[0234] INSERT: Insert new data into the multimodal address database.

[0235] UPDATE: Updates the corresponding record in the multimodal address database based on the primary key.

[0236] DELETE: Delete the corresponding record in the multimodal address database based on the primary key (or mark it as "deleted").

[0237] For critical operations (such as batch updates), a transaction mechanism is used to ensure data consistency. Messages that fail synchronization are recorded in a dead letter queue (DLQ) and trigger an alarm, supporting manual retries.

[0238] This embodiment combines the full synchronization partitioning and breakpoint resumption mechanism of xxjob with the incremental synchronization real-time capture and push of canal+kafka. The server can efficiently and reliably maintain the consistency of the multimodal address database and the industry address database.

[0239] Beneficial effects of the embodiments of the present application:

[0240] By integrating multimodal data such as text addresses, spatial coordinates, nearby POIs, and street view images, address information can be represented more comprehensively and accurately, improving the richness and precision of address descriptions. Converting multimodal data into vector form facilitates mathematical operations and similarity comparisons, providing an efficient data foundation for subsequent address queries and matching. By dividing text addresses into multiple entities and querying the knowledge base for corresponding semantically enhanced information, the semantic representation of the address is further enriched, improving the accuracy and intelligence of address matching. A query text address vector is generated based on the query text address entered by the user, and the address vector library is quickly searched for the candidate address vector with the highest similarity, enabling efficient address query functionality. The text address, spatial coordinates, nearby POIs, and street view images corresponding to the queried candidate address vectors are displayed on the interactive interface, providing users with intuitive and comprehensive query results, facilitating selection and judgment.

[0241] In one or more possible embodiments, the query process is optimized to increase the query speed.

[0242] The generating of the final address vector by concatenating the semantic enhancement vector and the multimodal vector, and writing the final address vector into the address vector library, comprises:

[0243] Calculating a hash value of the final address vector, and writing the final address vector to a target node among the deployed multiple nodes according to the hash value;

[0244] The step of searching the address vector library according to the query information input by the user on the interactive interface and displaying the query result on the interactive interface includes:

[0245] Determine whether the query task is a precise query task or a fuzzy query task based on the query information entered by the user on the interactive interface;

[0246] If it is a precise query task, query the cache server according to the precise query task;

[0247] If the query result is hit on the cache server, the query result returned by the cache server will be displayed on the interactive interface;

[0248] If there is no hit in the cache server, a hash value is calculated according to the query information, a target node is determined among the multiple nodes according to the calculated hash value, the precise query task is routed to the target node, a query result returned by the target node is received, and a final query result is generated on the interactive interface;

[0249] If it is a fuzzy query task, query the cache server according to the fuzzy query task;

[0250] If the query result is hit in the cache server, the query result returned by the cache server will be displayed on the interactive interface;

[0251] If the cache server does not hit, the fuzzy query task is broadcast to the multiple nodes, the query results returned by each node are aggregated to generate a final query result, and the final query result is displayed on the interactive interface.

[0252] Specifically, after generating the final address vector, the server uses a consistent hashing algorithm (such as the Ketama algorithm) to calculate the hash value of the vector. The hash space is divided into multiple virtual nodes, with each physical node responsible for managing a continuous section of the hash ring. Based on the calculated hash value, the server determines the target node where the vector should be stored and writes the vector to the vector library of that node. This design ensures uniform data distribution while minimizing data migration when nodes are dynamically added or removed.

[0253] For example, assume the hash space is 0 to 2^32-1, and servers A, B, and C are responsible for the regions [0, 2^31), [2^31, 32^30), and [3230, 2^32-1] on the hash ring, respectively. If the hash value of an address vector is 500,000,000, it falls into the region responsible for server B and is ultimately routed to server B for storage.

[0254] The server determines the query task type based on the query information entered by the user (such as whether it contains wildcards and whether an exact match is required). Exact queries require results that fully match the query criteria, while fuzzy queries allow for partial matches or results with a high degree of similarity. This determination logic can be implemented through regular expression matching or query syntax parsing.

[0255] For example, if a user enters "88 Jianguo Road, District A, City A", the server identifies it as a precise query task. If a user enters "* Road, District A, City A", the server identifies it as a fuzzy query task.

[0256] The server first checks a cache server (such as Redis) to see if there's a cached result for the exact query. If so, it returns the cached result. If not, it calculates the hash value of the query information, determines the target node using a consistent hashing algorithm, and routes the query task to that node. After the target node executes the query, it returns the result to the requesting server, which then writes the result to the cache and returns it to the user.

[0257] For example, a user queries the precise location of "88 Jianguo Road, District A, City A." The server doesn't find the query result in its cache, so it calculates its hash value and routes it to Server B. Server B retrieves the detailed information for the address in its vector library and returns it. The server then writes the result to its cache and displays it to the user.

[0258] The server first queries the cache server for cached results for the fuzzy query. If a match is found, it is returned directly; if not, the fuzzy query task is broadcast to all nodes. Each node performs a similarity search (e.g., using FAISS's IVFPQ index) within its vector library and returns the most similar results. The server aggregates the results returned by each node (e.g., sorting by similarity and taking the top N results) and displays the final result to the user.

[0259] For example, a user queries for the fuzzy location information for "* Road, District A, City A." The server finds no results in its cache and broadcasts the query to servers A, B, and C. Each node returns a list of addresses with high similarity. The server aggregates and sorts them by similarity, displaying results such as "88 Jianguo Road" and "90 Jianguo Road."

[0260] This embodiment uses a consistent hashing algorithm to achieve uniform data distribution and avoid overloading a single node. When adding or removing nodes, only a small amount of data needs to be migrated, ensuring high system availability. Precise queries prioritize cache hits, reducing computational overhead; fuzzy queries achieve efficient similarity searches through broadcasting and aggregation, balancing accuracy and performance. The system can dynamically add or remove nodes, and the hash ring automatically adjusts without interrupting service to adapt to changes in business scale. This dual mechanism of caching and node querying ensures consistent and real-time query results, improving the user experience.

[0261] To sum up, the technical solution of this application improves the accuracy and richness of address descriptions through technical means such as multimodal data fusion, vectorized processing, semantic enhancement, efficient query and result visualization, realizes efficient address query function, and provides users with a more convenient and intelligent address service experience.

[0262] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0263] See Figure 4 , which shows a schematic diagram of the structure of a deep learning-based address search device provided by an exemplary embodiment of the present application, hereinafter referred to as device 4. Device 4 can be implemented as all or part of a server through software, hardware, or a combination of both. Device 4 includes: an acquisition unit 401, a conversion unit 402, a generation unit 403, and a query unit 404.

[0264] An acquisition unit, configured to acquire various multimodal data from a multimodal address database, wherein the multimodal data includes: a text address, spatial coordinates, surrounding POIs, and street view images;

[0265] a conversion unit, configured to convert the text address into a text address vector, convert the spatial coordinates into a spatial vector, convert the surrounding POI into a POI vector, and convert the street view image into a street view image vector;

[0266] a generating unit configured to perform a weighted summation of the text address vector, the spatial vector, the POI vector, and the street view image vector to generate a multimodal vector; divide the text address into multiple entities, use the entities to query corresponding semantic enhancement information in a knowledge base, encode the queried semantic enhancement information to generate a semantic enhancement vector; concatenate the semantic enhancement vector and the multimodal vector to generate a final address vector, and write the final address vector into an address vector library;

[0267] The query unit is used to query the address vector library according to the query information input by the user on the interactive interface, and display the query result on the interactive interface.

[0268] In one or more possible embodiments, converting the text address into a text address vector includes:

[0269] Decomposing the text address into a country address, a province / municipality address, a city / district address, and a street / house number address;

[0270] According to the pre-trained PromptCSE-Base model, addresses at each level are converted into national address vectors, province / municipality address vectors, city / district vectors, and street address vectors, and the address vectors at each level are concatenated to obtain text address vectors.

[0271] In one or more possible embodiments, the query information is a query text address;

[0272] The querying in the address vector library according to the query information input by the user on the interactive interface and displaying the query result on the interactive interface include:

[0273] Get the query text address entered by the user on the interactive interface;

[0274] Decomposing the query text address into country address, province / municipality address, city / district address, and street / house number address according to preset rules;

[0275] Call the pre-trained PromptCSE-Base model to convert the decomposed addresses of each level into corresponding address vectors;

[0276] The address vectors of each level are concatenated to obtain the text address vector;

[0277] A multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector;

[0278] Dividing the query text address into multiple entities, using each entity to query corresponding semantic enhancement information in a knowledge base, and encoding the semantic enhancement information to generate a semantic enhancement vector;

[0279] The generated semantic enhancement vector and the multimodal vector obtained by weighted summation are concatenated to obtain a query text address vector corresponding to the query text address;

[0280] Querying the address vector library for several candidate address vectors with the highest similarity according to the query text address vector;

[0281] The text addresses, spatial coordinates, surrounding POIs and street view pictures corresponding to the plurality of candidate address vectors are displayed on the interactive interface.

[0282] In one or more possible embodiments, performing weighted summation on the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector to obtain the multimodal vector includes:

[0283] When a user enters a query text address, the user's context information is collected, and the context information includes: user's historical search records, address location and time;

[0284] Converting the context information into a context vector;

[0285] Input the context vector into a pre-trained context-aware model to calculate the weights of a default spatial vector, a default POI vector, a default street view image vector, and a concatenated text address vector;

[0286] According to the calculated weights, the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector are weightedly summed to obtain a multimodal vector.

[0287] In one or more possible embodiments, before converting addresses at each level into a country address vector, a province / municipality address vector, a city / district vector, and a street address vector according to the pre-trained PromptCSE-Base model, the process further includes:

[0288] Clean, segment and standardize addresses at each level.

[0289] In one or more possible embodiments, it further includes:

[0290] The correction unit is used to receive a data correction instruction from a crowdsourcing client, and after the data correction instruction is reviewed and approved, modify the corresponding multimodal data in the multimodal address database based on the data correction instruction.

[0291] In one or more possible embodiments, it further includes:

[0292] A synchronization unit, configured to synchronize all multi-module data in the industry address database to the multi-modal address database in a full synchronization manner;

[0293] When it is detected that data in the industry address database is updated, the updated data in the industry address database is synchronized to the multimodal address database in an incremental synchronization manner;

[0294] When it is detected that data update occurs in the multimodal address database, the address vector library is updated according to the updated data in the multimodal address database.

[0295] In one or more possible embodiments, during the full synchronization process, the industry address database is divided into multiple data partitions based on the xxjob technology, a synchronization task is created for each data partition, and the progress information of each synchronization task is monitored in real time; when the synchronization task is interrupted, the interrupted data location is obtained, and the synchronization operation is re-executed based on the interrupted data location;

[0296] During the incremental synchronization process, the canal middleware is used to detect when new, modified, and deleted operations occur in the industry address database, and the updated data is pushed to Kafka, and then Kafka is used to synchronize the updated data to the multimodal address database.

[0297] In one or more possible embodiments, the generating a final address vector by concatenating the semantic enhancement vector and the multimodal vector, and writing the final address vector into an address vector library, includes:

[0298] Calculating a hash value of the final address vector, and writing the final address vector to a target node among the deployed multiple nodes according to the hash value;

[0299] The step of searching the address vector library according to the query information input by the user on the interactive interface and displaying the query result on the interactive interface includes:

[0300] Determine whether the query task is a precise query task or a fuzzy query task based on the query information entered by the user on the interactive interface;

[0301] If it is a precise query task, query the cache server according to the precise query task;

[0302] If the query result is hit on the cache server, the query result returned by the cache server will be displayed on the interactive interface;

[0303] If there is no hit in the cache server, a hash value is calculated according to the query information, a target node is determined among the multiple nodes according to the calculated hash value, the precise query task is routed to the target node, a query result returned by the target node is received, and a final query result is generated on the interactive interface;

[0304] If it is a fuzzy query task, query the cache server according to the fuzzy query task;

[0305] If the query result is hit in the cache server, the query result returned by the cache server will be displayed on the interactive interface;

[0306] If the cache server does not hit, the fuzzy query task is broadcast to the multiple nodes, the query results returned by each node are aggregated to generate a final query result, and the final query result is displayed on the interactive interface.

[0307] It should be noted that the device 4 provided in the above embodiment, when executing the deep learning-based address search method, only uses the division of the above-mentioned functional modules as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the above-mentioned functions. In addition, the deep learning-based address search device provided in the above embodiment and the deep learning-based address search method embodiment are based on the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0308] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0309] See also Figure 5 FIG. 1 is a schematic diagram of a computer storage medium provided in an embodiment of the present application, wherein the computer storage medium can store multiple instructions (i.e. Figure 5 The computer program shown in FIG. 1 is suitable for being loaded and executed by the processor as described above. Figure 2 The method steps of the embodiment shown, the specific execution process can be found in Figure 2 The detailed description of the illustrated embodiment will not be repeated here.

[0310] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the deep learning-based address search method described in the above embodiments.

[0311] See Figure 6 , provides a structural diagram of a server according to an embodiment of the present application. Figure 6 As shown, the server 600 may include: at least one processor 601 , at least one network interface 604 , a user interface 603 , a memory 605 , and at least one communication bus 602 .

[0312] The communication bus 602 is used to implement the connection and communication between these components.

[0313] The user interface 603 may include input units such as a mouse and a keyboard.

[0314] The network interface 604 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0315] The processor 601 may include one or more processing cores. The processor 601 utilizes various interfaces and circuits to connect various components within the server 600. It executes instructions, programs, code sets, or instruction sets stored in the memory 605, and accesses data stored in the memory 605 to perform various server 600 functions and process data. Optionally, the processor 601 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 601 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may also be implemented as a separate chip, rather than integrated into the processor 601.

[0316] Among them, the memory 605 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 605 includes a non-transitory computer-readable storage medium. The memory 605 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 605 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 605 may also be optionally at least one storage device located away from the aforementioned processor 601. As Figure 6 As shown, the memory 605 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program.

[0317] exist Figure 6In the server 600 shown in FIG. 1 , the user interface 603 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 601 can be used to call the application stored in the memory 605 and specifically execute the following operations: Figure 2 The specific process can be referred to the method shown in Figure 2 As shown, no further details are given here.

[0318] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0319] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An address search method based on deep learning, characterized in that: include: Acquire various multimodal data in a multimodal address database, wherein the multimodal data includes: a text address, spatial coordinates, surrounding POIs, and street view images; Converting the text address into a text address vector, converting the spatial coordinates into a spatial vector, converting the surrounding POI into a POI vector, and converting the street view image into a street view image vector; Performing a weighted summation on the text address vector, the spatial vector, the POI vector, and the street view image vector to generate a multimodal vector; Dividing the text address into multiple entities, using the entities to query corresponding semantic enhancement information in a knowledge base, and encoding the queried semantic enhancement information to generate a semantic enhancement vector; concatenating the semantic enhancement vector and the multimodal vector to generate a final address vector, and writing the final address vector into an address vector library; Get the query text address entered by the user on the interactive interface; Decomposing the query text address into country address, province / municipality address, city / district address, and street / house number address according to preset rules; Call the pre-trained PromptCSE-Base model to convert the decomposed addresses of each level into corresponding address vectors; The address vectors of each level are concatenated to obtain a concatenated text address vector; A multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector; Dividing the query text address into multiple entities, using each entity to query corresponding semantic enhancement information in a knowledge base, and encoding the semantic enhancement information to generate a semantic enhancement vector; The generated semantic enhancement vector and the multimodal vector obtained by weighted summation are concatenated to obtain a query text address vector corresponding to the query text address; Querying the address vector library for several candidate address vectors with the highest similarity according to the query text address vector; Displaying text addresses, spatial coordinates, surrounding POIs, and street view images corresponding to the plurality of candidate address vectors on the interactive interface; The multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector, including: When a user enters a query text address, the user's context information is collected, and the context information includes: user's historical search records, address location and time; Converting the context information into a context vector; Input the context vector into a pre-trained context-aware model to calculate the weights of a default spatial vector, a default POI vector, a default street view image vector, and a concatenated text address vector; According to the calculated weights, the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector are weightedly summed to obtain a multimodal vector; The conversion method of the text address vector is the same as the conversion method of the query text address vector.

2. The method according to claim 1, characterized in that Also includes: A data correction instruction from a crowdsourcing client is received, and after the data correction instruction is reviewed and approved, corresponding multimodal data in the multimodal address database is modified based on the data correction instruction.

3. The method according to claim 2, characterized in that Also includes: Synchronize all multi-module data from the industry address database to the multi-modal address database using a full synchronization method; When it is detected that data in the industry address database is updated, the updated data in the industry address database is synchronized to the multimodal address database in an incremental synchronization manner; When it is detected that data update occurs in the multimodal address database, the address vector library is updated according to the updated data in the multimodal address database.

4. The method according to claim 3, characterized in that The generating a final address vector by splicing the semantic enhancement vector and the multimodal vector, and writing the final address vector into an address vector library, comprises: Calculating a hash value of the final address vector, and writing the final address vector into target nodes among the deployed multiple nodes according to the hash value; The step of searching the address vector library according to the query information input by the user on the interactive interface and displaying the query result on the interactive interface includes: Determine whether the query task is a precise query task or a fuzzy query task based on the query information entered by the user on the interactive interface; If it is a precise query task, query the cache server according to the precise query task; If the query result is hit on the cache server, the query result returned by the cache server will be displayed on the interactive interface; If there is no hit in the cache server, a hash value is calculated according to the query information, a target node is determined among the multiple nodes according to the calculated hash value, the precise query task is routed to the target node, a query result returned by the target node is received, and a final query result is generated on the interactive interface; If it is a fuzzy query task, query the cache server according to the fuzzy query task; If the query result is hit in the cache server, the query result returned by the cache server will be displayed on the interactive interface; If the cache server does not hit, the fuzzy query task is broadcast to the multiple nodes, the query results returned by each node are aggregated to generate a final query result, and the final query result is displayed on the interactive interface.

5. An address search device based on deep learning, characterized in that: include: An acquisition unit, configured to acquire various multimodal data from a multimodal address database, wherein the multimodal data includes: a text address, spatial coordinates, surrounding POIs, and street view images; a conversion unit, configured to convert the text address into a text address vector, convert the spatial coordinates into a spatial vector, convert the surrounding POI into a POI vector, and convert the street view image into a street view image vector; a generating unit configured to perform a weighted summation of the text address vector, the spatial vector, the POI vector, and the street view image vector to generate a multimodal vector; divide the text address into multiple entities, use the entities to query corresponding semantic enhancement information in a knowledge base, encode the queried semantic enhancement information to generate a semantic enhancement vector; concatenate the semantic enhancement vector and the multimodal vector to generate a final address vector, and write the final address vector into an address vector library; A query unit, used to obtain the query text address entered by the user on the interactive interface; Decomposing the query text address into country address, province / municipality address, city / district address, and street / house number address according to preset rules; Call the pre-trained PromptCSE-Base model to convert the decomposed addresses of each level into corresponding address vectors; The address vectors of each level are concatenated to obtain the text address vector; A multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector; Dividing the query text address into multiple entities, using each entity to query corresponding semantic enhancement information in a knowledge base, and encoding the semantic enhancement information to generate a semantic enhancement vector; The generated semantic enhancement vector and the multimodal vector obtained by weighted summation are concatenated to obtain a query text address vector corresponding to the query text address; Querying the address vector library for several candidate address vectors with the highest similarity according to the query text address vector; Displaying text addresses, spatial coordinates, surrounding POIs, and street view images corresponding to the plurality of candidate address vectors on the interactive interface; The multimodal vector is obtained by weighted summing the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector, including: When a user enters a query text address, the user's context information is collected, and the context information includes: user's historical search records, address location and time; Converting the context information into a context vector; Input the context vector into a pre-trained context-aware model to calculate the weights of a default spatial vector, a default POI vector, a default street view image vector, and a concatenated text address vector; According to the calculated weights, the default spatial vector, the default POI vector, the default street view image vector, and the spliced ​​text address vector are weightedly summed to obtain a multimodal vector.

6. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps of any one of claims 1 to 4.

7. A server, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps of any one of claims 1 to 4.

Images