A geographical entity multi-modal storage and retrieval method and system based on a four-layer progressive structure

Abstract

This invention discloses a method and system for multimodal storage and retrieval of geographic entities, comprising: a four-layer progressive storage structure divided according to semantic abstraction levels, with each layer vertically connected by a globally unique entity identifier, and each layer capable of independent field expansion or data updates without affecting the query availability of other layers; a six-modal fusion unified vector generation module, which generates a unified dense vector by projecting and fusioning six modalities (visual, symbolic, geometric, contextual, attribute, and knowledge) through a network; a unified query engine for seven types of queries, which automatically selects the optimal storage layer according to the query type, supporting spatial, attribute, semantic, vector, hybrid, relational, and natural language queries; a human-computer collaborative interface based on open standard protocols; and a relational graph module. This invention achieves comprehensive, multimodal, progressive storage and intelligent retrieval of geographic entities from geometry to semantics.

Description

Technical Field

[0001] This invention belongs to the field of interdisciplinary technology of geographic information systems and artificial intelligence, specifically relating to a method and system for multimodal storage and retrieval of geographic entities based on a four-layer progressive structure. Background Technology

[0002] With the rapid development of remote sensing technology, IoT sensor networks, and large language modeling technology, the sources of geographic information data are becoming increasingly diversified. Data types have expanded from traditional vector graphics and raster images to include text descriptions, sensor readings, knowledge graphs, and deep learning vector representations. Geographic entities, as the core data units of geographic information systems (GIS), are directly affected by their storage and retrieval methods, impacting the efficiency and intelligence level of spatial data management. Under embodied intelligence and large model-driven task orchestration, the static storage architecture of traditional GIS leads to severe semantic fragmentation and inaccurate multimodal features—large language models cannot directly understand the flattened layer-attribute table structure of traditional GIS, visual language models struggle to parse map symbol parameters used only for rendering, and multimodal features, lacking a unified fusion framework, cannot support the agent's cognition and manipulation of the physical world. Existing technical solutions have the following shortcomings in handling multimodal storage and retrieval of geographic entities.

[0003] In terms of storage architecture, existing technologies lack a progressive storage architecture divided according to semantic abstraction levels. They typically organize data storage around processing stages, with each layer operating on a serial processing basis. This lack of a vertically integrated mechanism to maintain consistent referencing of the same entity across layers makes independent expansion and asynchronous updates of data at each layer impossible. When updating information at a specific level of an entity, the entire processing pipeline often needs to be re-executed, making single-layer incremental updates difficult. In schemes combining knowledge graphs and vector matching, the storage structure is organized around modality types rather than hierarchically abstracting around geographic entities. Relationships between different modalities are achieved through knowledge graph edge relationships, lacking a progressive storage hierarchy centered on entities. Specifically, CN115064054A discloses a method for constructing spatial identity encoding for geographic entities. This method designs spatial identity encoding as a combination of location code, category code, time code, and sequence code to provide a unique identifier for geographic entities. While this technology solves the entity identification problem, it does not involve a progressive storage architecture divided according to semantic abstraction levels. Its encoding only serves identity recognition and does not achieve independent expansion and asynchronous updates of data at each layer. CN114490902A discloses a multi-dimensional spatial adaptive partitioning and encoding method for two-dimensional geographic entities. It automatically determines the spatial partitioning method and generates spatial location codes based on the spatial type of the geographic entity. This technology solves the spatial partitioning and encoding problem, but its encoding object is spatial location rather than the entity's semantic level, and it does not construct a progressive storage structure from geometry to semantics. CN119379820A discloses a spatiotemporal encoding method for geographic entities in a real-scene 3D system. It divides a multi-level grid based on the GeoSOT model and generates spatiotemporal codes. This technology solves the spatiotemporal encoding problem, but its encoding focuses on spatiotemporal positioning and does not involve the hierarchical storage and independent updating of the entity's structured attributes, knowledge text, and vector representations. CN120031121A discloses a method for generating geographic entity knowledge graphs that considers spatiotemporal features. It establishes a hierarchical structure of "concept layer + entity layer + relation layer" and incorporates spatial identity coding. However, this hierarchical structure serves only the knowledge graph generation process; the three layers are intermediate products and cannot be used as the native organizational dimensions of the data. Each node is fixed after the generation process is completed, making it impossible to independently expand fields or asynchronously update data in one layer without affecting other layers in subsequent use. CN121458895A discloses a geographic entity intelligent recognition and reconstruction system based on multi-source surveying and mapping data. Its "cross-modal feature coding and fusion module" fuses heterogeneous information such as laser point clouds, oblique images, and multispectral images into a unified coding vector. However, the scope of this fusion is limited to surveying and mapping sensor data and does not incorporate higher-level semantic information such as traditional map cartographic symbols and business knowledge description text generated by large language model reasoning into the unified vector fusion framework.CN119938938A discloses a method for storing maritime knowledge graph data based on spatiotemporal grids. This method encodes data using a spatiotemporal coding standard and then stores it in an index database. However, it employs a traditional "encoding → index → ​​database" chain, failing to address the multi-layered semantic organization within entities, and data at each layer cannot be updated independently and asynchronously. CN121351831A discloses a geospatial entity vectorization method based on multimodal fusion and metric learning. This method extracts multimodal features from the text and coordinate information of geospatial entities. However, its multimodal fusion only covers text and coordinate modalities, lacking a mechanism for uniformly encoding and fusing more heterogeneous modalities such as visual features, map symbol semantics, and geographic context.

[0004] In terms of multimodal fusion, existing technical solutions lack the ability to incorporate higher-level semantic information, such as geographic entities and their traditional cartographic symbols, as well as business knowledge description text generated by large language models, into a unified fusion framework. Specifically, CN121580127A discloses a geographic entity information processing method that uses a geographic semantic feature encoder, a distance feature encoder, and a geographic context feature encoder to encode geographic entities in multiple dimensions, and outputs entity matching results through a classification head network. Although this technology involves multi-dimensional feature encoding and fusion, its encoding object is the matching relationship between entities, and it does not construct a progressive storage architecture centered on geographic entities. Furthermore, its fusion network serves a single matching task and does not support the unified fusion of high-level semantic modalities such as symbol style and business knowledge text. CN119128610A discloses a dual-tower entity recognition method that fuses multimodal features. It generates dense feature vectors for images and dense feature vectors for text separately, and then fuses them. This technique handles document-level multimodal entity recognition, and its image features are derived from general visual perception. It does not address the specific symbolic style, geometric location, and geographic context of geographic entities, nor does it solve the multimodal fusion problem of geographic entities in a progressive storage architecture. US 12,182,125 discloses a system and method for training embedding maps to improve retrieval enhancement generation. It maps different embedding spaces to a unified space by training and transforming the data structure. This technique addresses the fundamental problem of cross-modal vector space alignment but does not address the progressive extraction and fusion mechanism of the six heterogeneous modalities (visual, symbolic, geometric, contextual, attribute, and knowledge) specific to geographic entities. In the field of geospatial embedding learning, the Google S2Vec paper proposed a geospatial cell embedding learning method based on masked autoencoders. It divides the Earth into S2 cells and learns grid-level embeddings. However, it deals with macroscopic regional grid cells rather than independent geographic entities. Grid cells lack independent identity and multimodal attributes, and cannot handle heterogeneous information such as entity-level visual features, symbolic semantics, and structured attributes. Its conclusion explicitly lists "extending beyond the grid" as a direction to be addressed. Space2Vec, by simulating the multi-scale periodic representation mechanism of grid cells, uses sine functions of different frequencies to encode the original geographic coordinates, generating representations that capture absolute location and spatial relationships. However, it deals with spatial locations rather than geographic entities and does not involve multimodal fusion.In the field of multimodal mask autoencoders, MultiMAE extends MAE to three modalities: RGB images, depth maps, and semantic segmentation masks. PiMAE aligns the mask regions of 3D point clouds and 2D images. MoCA designs cross-masking strategies for multimodal physiological signals from wearable devices. CIG-MAE performs cross-modal reconstruction on the amplitude and phase of WiFi signals. However, the above-mentioned multimodal MAEs process multi-sensor data collected in the same scene or at the same time, which has a natural spatiotemporal alignment relationship. The masking strategy is executed across modalities at the same spatial location, rather than across heterogeneous semantic spaces at the entity level. In the field of geographic entity representation learning, GeoNN learns spatially perceptual embeddings of geographic entities based on graph neural networks, GeoHG learns region embeddings based on heterogeneous graphs fused with satellite imagery and POI information, SpaBERT provides a general representation of geographic entities based on named neighbor entities, and RegionEncoder jointly learns region representations from satellite imagery, POIs, pedestrian flow, and spatial maps. However, all of these works employ graph neural network message passing, contrastive learning alignment, or pre-trained language model paradigms, and none adopt the masked autoencoder paradigm of randomly masking entity modalities and reconstructing using neighbor information. Hex2Vec uses H3 spatial indexes to statistically analyze OpenStreetMap labels within a region and trains it using a skip-gram model, selecting directly adjacent neighbors in the H3 grid as its context region. GeoVex is also based on H3 indexes and OSM labels, but uses a hexagonal convolutional autoencoder to generate embeddings, which can better consider contextual information from neighboring H3 regions. However, Hex2Vec and GeoVex still process H3 grid cells (grid level) rather than geographic entities with independent identities, and cannot handle the six heterogeneous modal information of a single geographic entity.

[0005] Regarding query engines, existing technologies support the combination of spatial queries and vector retrieval, but the supported query types are incomplete. Some solutions only support joint queries of spatial filtering and vector similarity, failing to perform semantic matching and relation traversal within the same query, and lacking unified processing capabilities for attribute queries, semantic queries, relation queries, and even natural language queries. Furthermore, existing query engines mostly use fixed rules for query routing, failing to automatically select the optimal storage layer based on the semantic organization characteristics of the storage architecture. Specifically, CN119129722A (Changhe Data Intelligence) discloses a method for constructing a knowledge graph based on a large language model and vector library. It utilizes the RoBERTa model for semantic parsing and constructs a fused knowledge graph, further transforming the knowledge graph into a low-dimensional dense vector representation and creating an inverted index library. This technology achieves the combination of knowledge graph and vector, but its vector representation serves the entity linking task of the knowledge graph, failing to collaboratively build a unified query engine with spatial and attribute indexes, and also lacking a routing distribution mechanism that automatically selects the optimal storage layer based on the query type. CN202410606946 discloses a distributed retrieval method for multimodal data based on knowledge graphs and vector matching. It embeds multimodal data into vectors and integrates vector matching with knowledge graph retrieval through feature fusion and alignment. This technology achieves the integration of graph retrieval and vector matching, but its integration is a simple superposition at the retrieval result level. It does not achieve unified routing and distribution and result aggregation for spatial queries, attribute queries, semantic queries, vector retrieval, relation queries and natural language queries at the storage architecture level.

[0006] In terms of human-machine collaboration, recent years have seen the emergence of solutions that encapsulate Geographic Information System (GIS) tools into standard interfaces, enabling large language models to call GIS capabilities. However, these tools are loosely coupled with the underlying storage, failing to achieve deep coupling with the multi-layered semantic abstraction storage architecture. This prevents the optimization of tool execution efficiency by leveraging the semantic organization characteristics of the storage architecture. More importantly, existing technologies lack a standardized tool mounting mechanism for AI agents, failing to support Skill chain orchestration and dynamic execution plan generation based on Model Context Protocols (MCPs). This severely limits the task orchestration capabilities of large models in GIS scenarios.

[0007] In summary, the common shortcomings of existing technologies can be summarized as follows: a lack of a storage architecture centered on geographic entities, progressively organized according to semantic abstraction levels, and with each layer being independently scalable; a lack of a complete multimodal vector representation integrating traditional map symbols and business knowledge text; a lack of a unified query engine that automatically selects the optimal storage layer; and a lack of a human-machine collaborative interface deeply coupled with the progressive storage architecture and supporting the orchestration of AI agent skill chains. These shortcomings prevent geographic information systems from achieving intelligent storage and generation of semantic maps in embodied intelligence and large-model-driven mapping scenarios. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention aims to provide a method and system for multimodal storage and retrieval of geographic entities based on a four-layer progressive structure. Existing technologies lack a progressive storage architecture divided according to semantic abstraction levels, failing to achieve a unified multi-layered representation from geometric graphics to business semantics centered on geographic entities. Each layer cannot be independently expanded or asynchronously updated. Existing technologies have limited multimodal vector fusion dimensions and do not incorporate map symbol parameters and business knowledge text into a unified fusion framework. Existing query engines lack the ability to automatically select the optimal storage layer. The human-computer collaboration interface and storage layer in existing technologies are loosely coupled, failing to support Skill chain orchestration for AI agents. To solve these problems, this invention proposes a storage and retrieval scheme centered on geographic entities, progressively organized according to semantic abstraction levels, with unified multimodal fusion representation, unified processing of seven types of queries, and integration of a human-computer collaboration interface based on a model context protocol. This invention provides system-level architectural support for comprehensive, multimodal, and progressive storage and retrieval of geographic entities from geometry to semantics.

[0009] To achieve the above-mentioned objectives, the present invention provides the following technical solution: I. Four-level progressive storage structure This invention provides a four-layer progressive storage structure based on semantic abstraction levels for multi-level storage and management of geographic entities. The four-layer progressive storage structure includes the following four data layers: The Vector layer (precise spatiotemporal anchoring layer) stores the geometric data, coordinate reference system information, acquisition timestamps, and accuracy metrics of geographic entities, enabling precise anchoring of geographic entities in the spatiotemporal domain. The geometric data includes at least one of vector geometry and raster mosaic indexes. The coordinate reference system information supports dynamic conversion between various geodetic coordinate reference systems and projected coordinate reference systems. The accuracy metrics include at least one of acquisition accuracy, cartographic accuracy, and semantic accuracy.

[0010] The Object layer (structured attribute and symbolic representation layer) stores structured attribute information and traditional map symbol description parameters for geographic entities. The structured attribute information includes feature type and sensor attributes, employing an extensible structural design to support dynamic expansion of attribute fields based on the geographic entity's domain. The traditional map symbol description parameters support symbolic rendering and style resolution, including at least one of the following: shape identifiers for point symbols, style parameters for line symbols, and fill parameters for area symbols. The Object layer further maps the visual symbol parameters of geographic entities to a semantic space through a symbol encoding module, transforming the symbol parameters from static descriptions used only for visual rendering into machine-computable dense vectors that support cross-modal retrieval.

[0011] The Knowledge layer (business knowledge description layer) stores natural language description information, summary information, and semantic relationship triples between entities generated based on a large language model, enabling the semantic expression of geographic entities within the business knowledge domain. The natural language description information is generated by the large language model based on structured attributes of the Object layer, geometric features of the Vector layer, and external domain knowledge. The semantic relationship triples include a subject entity, a relationship type, and an object entity; the relationship type covers at least one of spatial relationships, semantic relationships, and business relationships.

[0012] The Embedding layer (multimodal semantic vector layer) is used to store a unified dense vector representation generated by fusing multimodal information. The unified dense vector is generated by fusing visual feature vectors, symbolic style vectors, geometric location vectors, geographic context vectors, structured attribute vectors, and knowledge semantic vectors.

[0013] Each geographic entity is assigned a globally unique entity identifier. The Vector, Object, Knowledge, and Embedding layers are vertically connected through these globally unique entity identifiers, ensuring that data records in each layer point to the same geographic entity. Crucially, each layer can independently expand fields or update data without affecting the query availability of other layers—corrections to the positioning accuracy of the Vector layer do not affect the knowledge text already generated in the Knowledge layer; upgrades or manual corrections to the large language model in the Knowledge layer can overwrite old knowledge without rewriting geometric records; and updates to sensor data in the Object layer do not trigger synchronous changes in the Vector and Knowledge layers.

[0014] II. Six-modal fusion unified vector generation This invention uses visual feature vectors, symbolic style vectors, geometric location vectors, geographic context vectors, structured attribute vectors, and knowledge semantic vectors as six input modalities to generate a unified dense vector representation. The generation process of the unified dense vector is as follows: First, layer normalization is performed on the six modal vectors to eliminate dimensional differences and distribution shifts among them.

[0015] Next, each modal vector after layer normalization is projected onto a preset intermediate dimension through an independent linear transformation matrix. The preset intermediate dimension is configured differently according to the information content of each modality—modalities with more information (such as knowledge semantic vectors) are configured with larger intermediate dimensions so that the modality can obtain a more sufficient representation space in the fusion; modalities with less information (such as geometric position vectors) are configured with smaller intermediate dimensions to avoid redundant calculations.

[0016] Then, the projected modal vectors are concatenated according to a preset concatenation order to generate a multimodal concatenated vector. The multimodal concatenated vector is input into a multilayer perceptron, processed through at least one layer of nonlinear transformation and activation function, and mapped to the target dimension via the output layer to generate the unified dense vector representation.

[0017] Furthermore, this invention also includes a cold-start vector generation mechanism. When training data is insufficient, L2 normalization is performed on each modality vector separately, and then the vectors are directly concatenated to generate an initial concatenated vector. Principal component analysis is then performed on the initial concatenated vector to reduce its dimensionality to the target dimension, generating a cold-start unified vector. Once the accumulated training data reaches a preset threshold, the cold-start unified vector is switched to a fusion unified vector generated by a multilayer perceptron, achieving a gradual upgrade.

[0018] III. A unified query engine for seven types of queries This invention provides a method for constructing a unified query engine with seven types of queries. The unified query engine, applied to the aforementioned four-layer progressive storage structure, comprises: The query parsing function receives user-input query requests, performs syntax parsing and semantic understanding on the requests, identifies the query type, and extracts query conditions. The query types include seven categories: spatial query, attribute query, semantic query, vector retrieval, hybrid query, relational query, and natural language query.

[0019] Routing and distribution is a key innovation of this invention. Based on the query type identified through query parsing, the optimal storage layer corresponding to the four-layer progressive storage structure is automatically selected, and the query request is routed to that storage layer for retrieval operations—spatial queries are routed to the Vector layer for geometric operations, attribute queries to the Object layer for structured field matching, semantic queries to the Knowledge layer for natural language understanding and semantic relation matching, and vector retrieval to the Embedding layer for similarity calculation. Conditions for mixed queries are decomposed into multiple sub-constraints and then distributed in parallel to multiple storage layers for independent retrieval. Relational queries perform graph traversal and inference operations in the relation graph module. Natural language queries are parsed into structured queries by the large language model and then distributed to the corresponding storage layer.

[0020] The results are aggregated, and the query results returned by each storage layer are merged, sorted, and deduplicated. The multi-source results are then associated with the corresponding geographic entities through the globally unique entity identifier.

[0021] IV. Human-Machine Collaboration Interface Based on Open Standard Protocols This invention provides a method for constructing a human-computer collaboration interface based on an open standard protocol. The human-computer collaboration interface is deeply coupled with the aforementioned four-layer progressive storage structure, constituting the following functional units: The tools are encapsulated, integrating semantic querying, spatial analysis, entity maintenance, relation graph traversal, and multimodal fusion capabilities into functional tools conforming to open standard protocols. Each functional tool has standardized input parameter definitions and output format specifications. These functional tools include at least: a semantic query tool, a spatial analysis tool, an entity maintenance tool, a relation exploration tool, and a vectorized retrieval tool.

[0022] The AI ​​Agent orchestration receives natural language intent expressions from human users, transforms these expressions into a sequence of tool calls through intent understanding, and coordinates the execution order and data flow of multiple functional tools, forming a closed-loop interactive process from human intent understanding to tool invocation, result presentation, and feedback correction. This functional unit supports single-tool invocation mode and multi-tool combined invocation mode. In multi-tool combined invocation mode, a directed acyclic graph execution plan is constructed based on the data dependencies between tools, scheduling the execution of each functional tool according to the topological sorting order, and using the output of the preceding tool as the input parameter for the subsequent tool.

[0023] Context management maintains dialogue history, intermediate calculation results, and entity reference status in a closed-loop interaction process, supporting cross-round referencing resolution and context-aware query optimization.

[0024] V. Relationship Graph Construction Methods This invention also provides a method for constructing a relationship graph. The method includes: constructing a dynamic association network between entities, where edges in the dynamic association network represent spatial, semantic, or business relationships between entities, and edges have directionality, type labels, and weight attributes. The weight attributes are dynamically updated based on at least one of the following factors: relationship confidence, relationship timeliness, and user interaction frequency. The dynamic association network is stored in a relationship graph module, and entities are associated through globally unique entity identifiers and a four-layer progressive storage structure. The relationship graph module supports graph query, graph traversal, and graph reasoning operations. The graph reasoning operation supports path-pattern-based reasoning for discovering indirect relationships between entities. The data sources for the relationship graph include: direct import from semantic relationship triples stored in the Knowledge layer; automatic generation of spatial relationships from geometric data in the Vector layer using spatial analysis algorithms; generation of semantic relationships from structured attributes in the Object layer through attribute similarity calculations; and dynamic generation of business relationships through user interaction behavior.

[0025] VI. Storage and Retrieval System The present invention also provides a geographic entity multimodal storage and retrieval system based on a four-layer progressive structure, including a processor and a storage medium, wherein the storage medium stores computer program instructions, and when the processor executes the computer program instructions, it implements the storage and retrieval method described above.

[0026] The system consists of the following system-level modules: The data storage module, built upon a relational database combined with spatial database extensions and discrete global grid extensions, is used to implement physical table storage for the Vector, Object, and Knowledge layers. Collaborative queries among these three layers are supported by establishing spatial indexes on geometric fields, GIN indexes on attribute fields, and full-text search indexes on knowledge text fields. The Vector layer stores geometric data, coordinate reference system information, collection timestamps, and accuracy metrics of geographic entities; the Object layer stores structured attribute information and traditional map symbol description parameters; and the Knowledge layer stores natural language description information, summary information, and semantic relation triples generated based on a large language model.

[0027] The vector generation module asynchronously extracts multiple modal feature vectors from the data at each layer, generates a unified vector for the embedding layer through multi-head fusion network inference, writes it into a vector table, and establishes a vector index. The multiple modal feature vectors include visual feature vectors, symbolic style vectors, geometric location vectors, geographic context vectors, structured attribute vectors, and knowledge semantic vectors.

[0028] The knowledge enhancement module is used to call the locally deployed large language model, assemble entity information and spatial context according to the preset prompt template, generate business knowledge description text and relation triples, and after performing pattern validation, confidence filtering and predicate whitelist interception on the output, write the valid results into the knowledge description table and relation table of the Knowledge layer.

[0029] The query engine module, which is the unified query engine for the seven types of queries mentioned above, provides a unified query interface and supports spatial range queries, structured attribute filtering queries, full-text semantic queries, vector similarity retrieval, and a hybrid query mode of first spatial filtering and then vector sorting. It can automatically select the optimal storage layer in the four-layer progressive storage structure to perform retrieval according to the query type, and supports calling the relationship graph module to perform graph traversal and reasoning operations when performing relation queries.

[0030] The human-machine collaboration interface module, namely the aforementioned human-machine collaboration interface based on the Model Context Protocol (MCP), integrates a standardized toolset based on the MCP, supporting the AI ​​Agent to mount and invoke the Skill chain of VOK storage layer capabilities in real time. This human-machine collaboration interface module exposes four calling interfaces to the AI ​​agent: semantic search tools, spatial analysis tools, entity editing tools, and symbol recommendation tools. It automatically constructs an execution plan based on the query intent and dynamically orchestrates the spatial analysis Skill, outputting a comprehensive view including geographic entities, spatial relationships, and visualized symbols. Upon receiving user natural language commands, the large language model parses them into a corresponding tool call sequence, and optimizes the execution efficiency of the tools based on the semantic organization characteristics of the progressive storage architecture, realizing a closed-loop workflow for the AI ​​agent from natural language to geographic entity operations.

[0031] The relationship graph module, deployed in the graph database, associates entities through the globally unique entity identifier and the four-layer progressive storage structure. The module stores a dynamic network of relationships between entities. Edges in this network represent spatial, semantic, or business relationships between entities, and each edge has directionality, type label, and weight attributes. The weight attributes are dynamically updated based on at least one of the following factors: relationship confidence, relationship timeliness, and user interaction frequency. The module supports graph querying, graph traversal, and path-pattern-based graph reasoning operations to discover indirect relationships between entities. Beneficial effects

[0032] Compared with the prior art, the present invention has the following beneficial effects: First, this invention achieves comprehensive and progressive storage of geographic entities, from geometric figures to business semantics. The four-layer progressive storage structure proposed in this invention, divided according to semantic abstraction levels, is vertically connected through globally unique entity identifiers. This ensures consistent referencing of the same entity across layers while enabling independent expansion and asynchronous updates of each layer. Compared to storage organization methods centered on modality type or processing stage, this progressive storage structure better aligns with the natural abstraction levels of geographic entities from geometry to semantics.

[0033] Second, it achieves a unified vector representation of six modalities, in which the intermediate dimension of each modality is configured differently according to its information content, so that the modality with a larger amount of information—such as knowledge semantic vectors—obtains a more sufficient representation space, which significantly improves the machine computability of geographic entities.

[0034] Third, it achieves unified processing for seven types of queries. The unified query engine of this invention can automatically select the optimal storage layer corresponding to the four-layer progressive storage structure based on the query type, avoiding redundant overhead of cross-layer queries and significantly improving retrieval efficiency.

[0035] Fourth, it achieves deep coupling between the human-machine collaboration interface and the four-layer progressive storage architecture. Unlike the loose coupling design between general tools and the storage layer, the human-machine collaboration interface of this invention is deeply coupled with the four-layer progressive storage architecture, which can intelligently select the optimal tool combination and execution strategy according to the semantic organization characteristics of the four-layer architecture.

[0036] Fifth, it enables efficient construction and reasoning of dynamic relationship networks between entities. The relationship graph construction method of this invention automatically generates spatial, semantic, and business relationships between entities from multi-source data, supporting cross-entity relationship reasoning and knowledge discovery. The edge weight attributes in the dynamic relationship network can be dynamically updated based on multiple factors.

[0037] Sixth, it achieves independent expansion and asynchronous update capabilities for a four-layer progressive storage structure. Each layer can independently perform data updates and storage expansion without synchronously modifying the storage structure of other layers. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the overall architecture of the four-layer progressive storage and retrieval system of the present invention; Figure 2 This is a schematic diagram of the hierarchical relationship and vertical connection mechanism of the four-layer progressive storage structure of the present invention. Figure 3 This is a schematic diagram of the six-modal fusion unified vector generation process of the present invention; Figure 4 This is a schematic diagram of the unified query engine workflow for the seven types of queries of the present invention; Figure 5 This is a schematic diagram of the cold start vector generation mechanism and training model switching process of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0040] Example 1: Overall Deployment of a Four-Layer Progressive Storage Structure like Figure 1 and Figure 2 As shown, this embodiment describes the overall deployment scheme of a geographic entity multimodal storage and retrieval system based on a four-layer progressive structure. The system is deployed in a hardware environment consisting of a computing node cluster, a distributed storage cluster, and network infrastructure.

[0041] The four-layer progressive storage structure is physically deployed in a distributed storage cluster, wherein: the Vector layer is deployed in a distributed database that supports spatial indexing, storing geometric data, coordinate reference system information, collection timestamps, and accuracy indicators of geographic entities; the Object layer is deployed in a document database that supports dynamic structures, storing structured attribute information and traditional map symbol description parameters; the Knowledge layer is deployed in a combined architecture of graph database and full-text search engine, storing natural language descriptions, summaries, and semantic relation triples; and the Embedding layer is deployed in a dedicated vector database, storing a unified dense vector representation generated by six-modal fusion.

[0042] When a geographic entity is entered into the system, it is assigned a globally unique entity identifier by a globally unique identifier generator. Each data record in the Vector layer, Object layer, Knowledge layer, and Embedding layer carries the globally unique entity identifier, thereby achieving vertical integration across the four layers of storage.

[0043] Taking a bridge geographic entity as an example, its storage method in the four-layer progressive storage structure is as follows: In the Vector layer, the centerline geometry, coordinate reference system identifier, acquisition timestamp, and positioning accuracy index of the bridge are stored; in the Object layer, the structured attributes of the bridge—including feature type and sensor attributes—as well as traditional map symbol description parameters are stored; in the Knowledge layer, the natural language description, summary information, and semantic relationship triples with other entities generated by the large language model are stored; in the Embedding layer, the unified dense vector representation generated by six-modal fusion is stored.

[0044] Through the above four-layer progressive storage structure, the same bridge entity achieves comprehensive and multimodal storage expression at four semantic levels: geometry, attributes, knowledge, and vectors. Moreover, each layer can be updated independently without affecting each other. The correction of positioning accuracy in the Vector layer does not affect the knowledge text already generated in the Knowledge layer. Upgrading or manually correcting the large language model in the Knowledge layer can overwrite old knowledge without rewriting geometric records. The update of sensor data in the Object layer does not trigger synchronous changes in the Vector and Knowledge layers.

[0045] Example 2: Six-modal fusion unified vector generation like Figure 3 As shown, this embodiment describes in detail the generation process of the six-modal fusion unified vector.

[0046] Step S201: Feature extraction and vector representation for each modality. Visual feature vectors are extracted from image data using a visual encoder; symbolic style vectors are generated by parsing symbolic description parameters stored in the Object layer using a symbolic encoder, and then mapping them through an embedding layer and a metric learning network; geometric location vectors are generated by extracting features from geometric shapes in the Vector layer using a geometric encoder; geographic context vectors are generated by retrieving neighboring entities within a preset spatial range centered on the target entity using a context encoder, and then statistically analyzing the type distribution, spatial density, and relationships of neighboring entities; structured attribute vectors are generated by encoding structured attribute information in the Object layer using an attribute encoder; and knowledge semantic vectors are generated by a pre-trained language model using a semantic encoder with natural language descriptions and summary information from the Knowledge layer as input.

[0047] Step S202: Layer Normalization and Linear Transformation Projection. Layer normalization is performed on the six modal vectors generated in step S201 to eliminate dimensional differences and distribution shifts between the modal vectors. Each layer-normalized modal vector is then projected onto a preset intermediate dimension using an independent linear transformation matrix. The preset intermediate dimension is configured differently based on the information content of each modality—modalities with higher information content (such as knowledge semantic vectors) are configured with larger intermediate dimensions to provide a more comprehensive representation space during fusion; modalities with lower information content (such as geometric position vectors) are configured with smaller intermediate dimensions to avoid redundant computation.

[0048] Step S203: Vector Concatenation and Nonlinear Fusion. The projected modal vectors are concatenated according to a preset concatenation order to generate a multimodal concatenated vector. This multimodal concatenated vector is then input into a multilayer perceptron for nonlinear fusion. The multilayer perceptron contains several hidden layers, each employing a nonlinear activation function, and the output layer dimension is equal to the target dimension. The multilayer perceptron is trained using fused training data. The fused training data consists of geographic entity samples with complete six-modal vector annotations. Each sample contains a complete six-tuple including a visual feature vector, a symbolic style vector, a geometric location vector, a geographic context vector, a structured attribute vector, and a knowledge semantic vector. The loss function during training is a weighted combination of contrastive loss and reconstruction loss.

[0049] Step S204: Unified Dense Vector Storage. The unified dense vector generated in step S203 is stored in the Embedding layer, using the globally unique entity identifier as the index key, supporting efficient retrieval based on vector similarity.

[0050] In one specific implementation of this embodiment, the multilayer perceptron fusion network adopts a 3-layer fully connected network structure, with the number of neurons in each hidden layer decreasing progressively (e.g., 1024 in the first layer, 512 in the second layer, and 256 in the third layer). A ReLU activation function is used to introduce nonlinearity, enabling the fusion network to learn complex interactions between modalities. The preset intermediate dimension is 256 dimensions, determined by the number of heterogeneous modal vectors (6 types in this embodiment) and the statistical characteristics (mean and variance) of the original dimensions of each modality, allowing modalities with higher information density to obtain a more sufficient representation space. The above network architecture parameters are merely examples; in practical applications, they can be adapted and adjusted according to the geographic entity type, the number of modalities, and computing resources.

[0051] Example 3: A unified query engine for seven types of queries like Figure 4 As shown, this embodiment details the workflow of a unified query engine for seven types of queries. The unified query engine is deployed on a query coordination node within a compute node cluster.

[0052] Spatial query processing flow: Query parsing: Receives spatial query conditions, parses them, and extracts the spatial relationship type and reference geometry. Routing and distribution: After identifying the query type as a spatial query, it automatically selects the Vector layer as the optimal storage layer and routes the query request to the Vector layer to perform geometric operations.

[0053] The attribute query processing flow is as follows: Query parsing: Receive attribute query conditions, parse them, and extract attribute field names, comparison operators, and attribute values. Routing and distribution: Automatically select the Object layer as the optimal storage layer and route the query request to the Object layer to perform structured field matching.

[0054] Semantic query processing flow: Query parsing receives semantic query conditions, parses them, and extracts semantic keywords or semantic vector representations. Routing and distribution automatically selects the Knowledge layer as the optimal storage layer, and the Knowledge layer's full-text search engine performs natural language matching.

[0055] Vector retrieval processing flow: Query parsing extracts the globally unique entity identifier of the reference entity or the target vector provided by the user. Route distribution automatically selects the embedding layer as the optimal storage layer, and performs an approximate nearest neighbor search within the embedding layer.

[0056] The processing flow for hybrid queries is as follows: Query parsing decomposes the hybrid query conditions into multiple sub-constraints. Routing and distribution distributes each sub-constraint in parallel to the corresponding multiple optimal storage layers for independent retrieval. Result aggregation performs intersection or union operations on the candidate entity sets of each layer using the globally unique entity identifier, and sorts the merged candidate entities according to a preset sorting strategy.

[0057] The relationship query processing flow is as follows: Query parsing extracts the globally unique entity identifier and relationship type constraints of the target entity. Routing and distribution routes the relationship query to the relationship graph module for graph traversal operations.

[0058] The natural language query processing flow is as follows: Query parsing submits the natural language query statement to the large language model for intent recognition and entity extraction. The large language model transforms the recognition results into structured query parameters, and routing and distribution distribute the query to the corresponding storage layer for retrieval based on these parameters. The retrieval results are summarized and presented in natural language form by the large language model.

[0059] Example 4: Human-Machine Collaboration Interface Based on Open Standard Protocols This embodiment describes in detail the implementation scheme of a human-machine collaboration interface based on an open standard protocol. The human-machine collaboration interface is deployed in the application service layer of a computing node cluster.

[0060] Tool encapsulation implementation: The tool encapsulation encapsulates the core capabilities of the four-layer progressive storage structure into standard functional tools. Each functional tool has standardized input parameter definitions and output format specifications. The functional tools include at least: semantic query tools, spatial analysis tools (supporting spatial operators such as buffers, overlays, and line-of-sight), entity maintenance tools, relation exploration tools, and vectorized retrieval tools.

[0061] Implementation of AI Agent Orchestration: AI Agent orchestration receives natural language intent expressions from human users and transforms them into a sequence of tool invocations through intent understanding. In single-tool invocation mode, the corresponding tool is directly invoked; in multi-tool combined invocation mode, a directed acyclic graph execution plan is constructed based on the data dependencies between tools, and execution is scheduled according to the topological sorting order, with the output of the preceding tool used as the input parameter for the subsequent tool.

[0062] Taking urban planning applications as an example, when a user inputs "Please find all buildings within a kilometer radius of a certain bridge and analyze their spatial relationship with the bridge," the AI ​​Agent orchestrates and constructs the following execution plan: First, it calls the spatial analysis tool to perform buffer operations using the globally unique entity identifier of the bridge entity as input to obtain the spatial range geometry; second, it calls the semantic query tool to retrieve a set of candidate buildings using the feature type "building" as the filter condition; third, it calls the spatial analysis tool to perform an intersection operation between the spatial range of the first step and the candidate set of the second step to obtain the set of buildings within the range; fourth, it calls the relationship exploration tool to explore the spatial relationship paths between the bridge and the buildings; and fifth, it summarizes and presents the analysis results.

[0063] Implementation of context management: Context management maintains the dialogue history, intermediate calculation results and entity reference status in the closed-loop interaction process, and supports cross-round referencing resolution and context-aware query optimization.

[0064] Example 5: Cold Start Vector Generation Mechanism and Relationship Graph Construction like Figure 5 As shown, this embodiment describes the implementation scheme of the cold start vector generation mechanism. The implementation scheme of the relationship graph construction method is as follows.

[0065] Cold-start vector generation mechanism: In the initial stage of system deployment or when new types of geographic entities are added to the database, due to insufficient training data, the six-modal fusion unified vector generation module enables a cold-start vector generation mechanism: L2 normalization is performed on each modality vector to eliminate modulus differences between modalities; the normalized modality vectors are directly concatenated according to a preset concatenation order to generate an initial concatenated vector; principal component analysis is performed on the initial concatenated vector to reduce the vector dimension to the target dimension; the dimensionality-reduced vector is stored as the cold-start unified vector in the embedding layer. When the accumulated training data reaches a preset threshold, the system switches from cold-start mode to training model mode, employing a gradual replacement strategy during the switching process.

[0066] Relationship Graph Construction Method: The construction of the relationship graph includes the following steps: First, import relationship data directly from the semantic relationship triples stored in the Knowledge layer, including the subject entity identifier, relationship type, and object entity identifier; second, automatically generate spatial relationships from the geometric data in the Vector layer using spatial analysis algorithms, such as determining spatial associations between entities through spatial operations like spatial intersection, containment, and proximity; third, generate semantic relationships from the structured attributes in the Object layer through attribute similarity calculation, such as establishing a "same type" relationship between entities with the same feature type; finally, dynamically generate business relationships based on user interaction behavior, such as establishing business associations between two entities frequently associated by a user in a query. The constructed dynamic association network is stored in the relationship graph module. Nodes represent geographic entities, edges represent the associations between entities, and edges have directionality, type labels, and weight attributes. The weight attributes are dynamically updated based on at least one of the following factors: relationship confidence, relationship timeliness, and user interaction frequency. The relational graph module supports graph query, graph traversal, and graph reasoning operations. For example, if there is "entity A manages entity B" and "entity B contains entity C", then "entity A indirectly manages entity C" can be discovered through path pattern reasoning.

Claims

1. A method for multimodal storage of geographic entities based on a four-layer progressive structure, characterized in that, Includes the following steps: A four-layer progressive storage structure, divided according to semantic abstraction levels, is constructed for multi-level storage and management of geographic entities. The four-layer progressive storage structure includes: The Vector layer is used to store the geometric data, coordinate reference system information, collection timestamps and accuracy indicators of geographic entities, so as to achieve accurate anchoring of geographic entities in the spatiotemporal domain. The Object layer is used to store structured attribute information of geographic entities and traditional map symbol description parameters. The structured attribute information includes feature type and sensor attribute, and the traditional map symbol description parameters are used to support symbolic rendering and style parsing. The Knowledge layer is used to store natural language description information, summary information, and semantic relationship triples between entities generated based on the large language model, so as to realize the semantic expression of geographic entities in the business knowledge domain. The Embedding layer is used to store a unified dense vector representation generated by fusing multimodal information. The unified dense vector is generated by fusing visual feature vectors, symbolic style vectors, geometric location vectors, geographic context vectors, structured attribute vectors, and knowledge semantic vectors. Each geographic entity is assigned a globally unique entity identifier. The Vector layer, Object layer, Knowledge layer and Embedding layer are vertically connected through the globally unique entity identifier, so that the data records of each layer point to the same geographic entity. Each layer can independently perform field expansion or data update without affecting the query availability of other layers. The visual feature vector, symbol style vector, geometric location vector, geographic context vector, structured attribute vector, and knowledge semantic vector are used as six input modalities. They are projected to a preset intermediate dimension through layer normalization and linear transformation, and then vector concatenation is performed. Finally, nonlinear fusion is performed through a multilayer perceptron to generate the unified dense vector representation.

2. A unified query engine with seven types of queries, characterized in that, The unified query engine is applied to the four-layer progressive storage structure as described in claim 1, and the unified query engine comprises: The query parsing function receives query requests input by the user, performs syntax parsing and semantic understanding on the query requests, identifies the query type, and extracts query conditions. The query types include at least one of spatial query, attribute query, semantic query, vector retrieval, hybrid query, relational query, and natural language query. The routing and distribution process automatically selects the optimal storage layer corresponding to the four-layer progressive storage structure based on the query type identified by the query parsing. The query request is then routed to that storage layer to perform the retrieval operation. Specifically: spatial queries are routed to the Vector layer to perform geometric operations; attribute queries are routed to the Object layer to perform structured field matching; semantic queries are routed to the Knowledge layer to perform natural language understanding and semantic relationship matching; vector retrieval is routed to the Embedding layer to perform similarity calculation; mixed queries are performed in joint retrieval across multiple storage layers; relation queries are performed in the relation graph module to perform graph traversal and reasoning operations; and natural language queries are parsed into structured queries by the large language model and then distributed to the corresponding storage layer. The results are aggregated, and the query results returned by each storage layer are merged, sorted, and deduplicated. The multi-source results are then associated with the corresponding geographic entities through the globally unique entity identifier.

3. A human-machine collaborative interface based on an open standard protocol, characterized in that, The human-machine collaboration interface is deeply coupled with the four-layer progressive storage structure as described in claim 1, and the human-machine collaboration interface comprises the following functional units: The tool encapsulation functional unit is used to encapsulate semantic query, spatial analysis, entity maintenance, relation graph traversal and multimodal fusion capabilities into functional tools that conform to open standard protocols. Each functional tool has standardized input parameter definitions and output format specifications. The AI ​​Agent orchestration unit is used to receive the natural language intent expression of human users, and transform the natural language intent expression into a tool call sequence through intent understanding. It coordinates the execution order and data flow of multiple functional tools to form a closed-loop interactive process from human intent understanding to tool call to result presentation to feedback correction. The context management function unit is used to maintain the dialogue history, intermediate calculation results and entity reference status in the closed-loop interaction process, and supports cross-round referencing resolution and context-aware query optimization.

4. The storage method according to claim 1, characterized in that, The geometric data stored in the Vector layer includes at least one of vector geometry and raster mosaic index. The coordinate reference system information supports dynamic conversion between multiple geodetic coordinate reference systems and projected coordinate reference systems. The accuracy indicators include at least one of acquisition accuracy, mapping accuracy, and semantic accuracy.

5. The storage method according to claim 1, characterized in that, The structured attribute information in the Object layer adopts an extensible structure design, which supports dynamic expansion of attribute fields based on the different domains to which geographic entities belong; the traditional map symbol description parameters include at least one of the following: shape identifiers for point symbols, style parameters for line symbols, and fill parameters for area symbols.

6. The storage method according to claim 1, characterized in that, The natural language description information in the Knowledge layer is generated by the large language model based on the structured attributes of the Object layer, the geometric features of the Vector layer, and external domain knowledge; the semantic relation triple includes a subject entity, a relation type, and an object entity, and the relation type covers at least one of spatial relations, semantic relations, and business relations.

7. The storage method according to claim 1, characterized in that, The generation process of the six-modal fusion unified vector includes the following steps: Layer normalization is performed on the visual feature vector, symbol style vector, geometric location vector, geographic context vector, structured attribute vector, and knowledge semantic vector, respectively. Each modal vector after layer normalization is projected onto a preset intermediate dimension through an independent linear transformation matrix. The preset intermediate dimension is configured differently according to the information content of each modality. The projected modal vectors are concatenated according to a preset concatenation order to generate a multimodal concatenation vector; The multimodal concatenated vector is input into a multilayer perceptron, processed through at least one layer of nonlinear transformation and activation function, and mapped to the target dimension via the output layer to generate the unified dense vector representation.

8. The storage method according to claim 1 or 7, characterized in that, It also includes a cold-start vector generation mechanism, which generates the unified dense vector representation when training data is insufficient, using the following steps: L2 normalization is performed on the visual feature vector, symbol style vector, geometric location vector, geographic context vector, structured attribute vector, and knowledge semantic vector, respectively. The L2-normalized modal vectors are directly concatenated to generate the initial concatenation vector; Principal component analysis is performed on the initial spliced ​​vector to reduce the vector dimension to the target dimension, generating a cold-start unified vector. Once the accumulated training data reaches a preset threshold, the cold start unified vector is switched to a fusion unified vector generated by the multilayer perceptron.

9. The storage method according to claim 1, characterized in that, It also includes a relation graph construction method, which includes: constructing a dynamic association network between entities, wherein the edges in the dynamic association network represent spatial relationships, semantic relationships or business relationships between entities; storing the dynamic association network in a relation graph module, and associating entities with the four-layer progressive storage structure through the globally unique entity identifier; the relation graph module supports graph query, graph traversal and graph reasoning operations.

10. The unified query engine according to claim 2, characterized in that, The combined retrieval process of the hybrid query includes: Decompose the mixed query conditions into at least two of the following: spatial constraints, attribute constraints, semantic constraints, and vector similarity constraints; Each constraint condition is distributed in parallel to the corresponding storage layer to perform independent retrieval, thereby obtaining the candidate entity set of each layer; Based on the globally unique entity identifier, perform intersection or union operations on the candidate entity sets of each layer; The merged candidate entities are weighted and sorted according to a preset sorting strategy to generate the final query results.

11. The unified query engine according to claim 2, characterized in that, The natural language query processing procedure includes: The large language model performs intent recognition and entity extraction on the natural language query statements input by the user; The identified query intent is mapped to the corresponding query type, and the extracted entities and query conditions are transformed into structured query parameters. The query will be routed to the corresponding storage layer for retrieval based on the structured query parameters. The search results are summarized and presented in natural language form using the large language model.

12. The human-machine collaborative interface according to claim 3, characterized in that, The AI ​​Agent orchestration function unit supports single-tool invocation mode and multi-tool combined invocation mode. In the multi-tool combined invocation mode, a directed acyclic graph execution plan is constructed based on the data dependencies between tools, the execution of each functional tool is scheduled according to the topological sorting order, and the output of the preceding tool is used as the input parameter of the subsequent tool.

13. The human-machine collaborative interface according to claim 3, characterized in that, The functional tools include at least: a semantic query tool for retrieving geographic entities based on natural language descriptions; a spatial analysis tool for performing spatial relationship calculations and geometric analysis operations; an entity maintenance tool for performing CRUD operations on geographic entities in a four-layer progressive storage structure; a relationship exploration tool for traversing the association paths between entities in the relationship graph module; and a vectorized retrieval tool for retrieving similar geographic entities based on a unified dense vector.

14. The relationship graph construction method according to claim 9, characterized in that, The edges in the dynamic association network have directionality, type labels, and weight attributes. The weight attributes are dynamically updated based on at least one of the following factors: relationship confidence, relationship timeliness, and user interaction frequency. The relationship graph module also supports path-pattern-based reasoning operations for discovering indirect relationships between entities.

15. A geographic entity multimodal storage and retrieval system based on a four-layer progressive structure, characterized in that, The system includes a processor and a storage medium, the storage medium storing computer program instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 14.

Images