A method and system for sharing and exchanging urban information data based on a CIM platform

Abstract

The application provides a city information data sharing and exchanging method and system based on a CIM platform. The method comprises the following steps: loading a city information domain ontology graph and registering a data source; constructing a unified semantic data model based on the ontology graph; calculating a mapping score of a source data field and a target field in the unified semantic data model by using a multi-index weighted mapping model, and performing semantic mapping and unit normalization conversion; calculating a comprehensive quality score according to a multi-dimensional quality score model, and fusing multi-source data by using a weighted fusion conflict resolution model; and performing matching and distribution according to the subscription conditions of a data consumer. The application reduces the interface complexity from a square level to a linear level by constructing a unified semantic intermediate layer, and realizes semantic lossless sharing and exchanging of multi-source heterogeneous city data.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and system for sharing and exchanging urban information data based on a CIM platform. Background Technology

[0002] With the advancement of smart city construction, the data sources involved in urban management are becoming increasingly diversified, including spatial data from Geographic Information Systems (GIS), Building Information Modeling (BIM) data, real-time data from IoT sensors, and government business management data. Currently, data exchange between different departments and systems typically employs a point-to-point interface approach, where each pair of systems exchanging data develops a dedicated data conversion interface. As the number of systems increases, the number of required conversion interfaces grows exponentially, leading to a sharp rise in development and maintenance costs. Furthermore, the point-to-point approach only handles structural conversions at the data format level, failing to establish a unified semantic model. Inconsistencies in attribute field names, coding systems, and units of measurement for the same urban entity across different systems result in semantic distortion of the exchanged data, hindering cross-system semantic-level data fusion. Summary of the Invention

[0003] To address the shortcomings of existing technologies in multi-source heterogeneous urban data exchange, such as the quadratic increase in the number of interfaces with the number of systems, semantic distortion due to the lack of a unified semantic model for cross-system data exchange, and the inability of point-to-point format conversion to meet the needs of multi-source data conflict resolution, this application provides a method and system for sharing and exchanging urban information data based on a CIM platform. This method constructs a unified semantic data model as a semantic intermediate layer for multi-source data. Each data source is connected to this intermediate layer via a standardized adapter to complete semantic mapping and conflict resolution before being distributed according to a publish-subscribe mechanism. This reduces interface complexity from quadratic to linear, while simultaneously achieving semantically lossless cross-system data sharing and exchange. Specifically, this application provides the following technical solutions: Firstly, this application provides a method for sharing and exchanging urban information data based on a CIM platform, including: Load the urban information domain ontology graph and register multiple data sources, generate a data source descriptor for each data source and instantiate the corresponding standardized adapter; A unified semantic data model is constructed based on the domain ontology graph. The unified semantic data model includes multiple urban information entity classes and their standardized attribute sets extracted from the domain ontology graph. The source data of each data source is obtained through the standardized adapter. A multi-index weighted mapping model is used to calculate the mapping score between each source data field and the candidate target field in the unified semantic data model. The source data fields with mapping scores exceeding the mapping confidence threshold are mapped to the corresponding target fields, and the mapped data is subjected to unit normalization transformation. Multidimensional quality verification is performed on the mapped data. The comprehensive quality score of each data record is calculated according to the multidimensional quality scoring model. Data records whose quality scores reach the preset threshold are entered into conflict resolution. The weighted fusion conflict resolution model is used to fuse the multi-source data of the same entity to obtain the fused dataset. Based on the subscription conditions of the data consumer, condition matching is performed on the fused dataset to generate a distribution task, and the matched data is distributed according to the output format and protocol specified by the data consumer.

[0004] Optionally, the step of constructing a unified semantic data model based on the domain ontology graph includes: starting from a predefined seed set of core urban information concepts, traversing along hierarchical relationships in the domain ontology graph, collecting candidate entity classes and their attribute nodes, generating a standardized attribute set for each candidate entity class containing field names, data types, measurement units, and encoding rules; and constructing a spatial semantic joint index based on the spatial geometric information and semantic tags of the candidate entity classes.

[0005] Optionally, the spatial semantic composite index includes a first-level spatial range index and a second-level semantic label inverted index. When performing a composite query, the spatial candidate set is first obtained through the first-level spatial range index, and then the spatial candidate set is semantically filtered through the second-level semantic label inverted index to obtain the query result.

[0006] Optionally, the mapping score of the multi-index weighted mapping model is obtained by calculating the name similarity index, type compatibility index, and ontology semantic distance index for the source data field and the candidate target field respectively, and then summing the three indices according to preset weights to obtain the mapping score.

[0007] Optionally, the ontology semantic distance metric is obtained by: calculating the nearest common ancestor depth of the concept node to which the source data field belongs and the concept node to which the candidate target field belongs in the domain ontology graph, and calculating the ratio of the nearest common ancestor depth to the sum of the depths of the two concept nodes.

[0008] Optionally, the unit normalization transformation includes: performing a linear transformation on the mapped source data value according to a predefined unit transformation matrix, wherein each entry in the unit transformation matrix defines a scaling factor and offset from the source units to the target units.

[0009] Optionally, the comprehensive quality score of the multidimensional quality scoring model is obtained by calculating the sub-scores of the four dimensions: completeness score, format compliance score, scope reasonableness score, and consistency score, and then summing the sub-scores of the four dimensions according to preset dimension weights to obtain the comprehensive quality score.

[0010] Optionally, the weighted fusion conflict resolution model fuses multi-source data of the same entity, including: for numerical attributes, using the product of the static credibility weight of each data source and the data freshness decay factor as the fusion weight, performing a weighted average of the attribute values ​​provided by multiple data sources to obtain the fused attribute value; for string attributes, selecting the attribute value provided by the data source with the largest fusion weight as the fused attribute value.

[0011] Optionally, the data freshness decay factor is an exponential decay function with the data update time difference as the independent variable.

[0012] Optionally, performing condition matching on the fused dataset based on the subscription conditions of the data consumer includes: compiling the subscription conditions into a Boolean expression tree, wherein the leaf nodes of the Boolean expression tree are atomic filtering conditions, and the atomic filtering conditions include at least one of entity type filtering, attribute value filtering, spatial range filtering, and time window filtering, and the internal nodes are logical operators; and performing condition matching by traversing the Boolean expression tree for each record in the fused dataset.

[0013] Optionally, the execution order of the distributed tasks is determined by dynamically calculated task priority values. The task priority values ​​are calculated based on the task's base priority, urgency flag, and current queue depth of tasks to be executed. The urgency flag increases the task priority value by a preset amplification factor, and the queue depth decreases the task priority value by a preset attenuation factor.

[0014] Optionally, it also includes an incremental synchronization step: for each data source, the latest data is retrieved in each synchronization cycle; for each record, a content hash value is calculated; the content hash value is compared with the hash snapshot saved in the last synchronization; records with inconsistent hash values ​​are marked as changed records; differential encoding is used on the changed records, and only the changed fields and their new values ​​are extracted to generate incremental data packets; the incremental data packets are distributed after the semantic mapping, the quality verification and conflict resolution.

[0015] Optionally, the incremental data packet generated by the differential encoding includes a record primary key, a change type identifier, a change timestamp, and a set of change fields, wherein the change type identifier includes addition, modification, and deletion.

[0016] Optionally, it also includes security control steps: performing permission verification on each distribution request based on data classification and grading and a role permission matrix, wherein the role permission matrix defines the set of accessible fields and the highest data grading for each consumer role for each entity type; and performing dynamic desensitization processing on sensitive fields that exceed the permission level of the consumer role.

[0017] Optionally, the dynamic desensitization process selects a desensitization strategy based on the data type of the sensitive field. The desensitization strategy includes performing partial masking on identifier fields, performing precision generalization on address fields, and performing random permutation on name fields.

[0018] Optionally, it also includes: recording the entire operation log in an append-only manner, calculating the chain hash once for every preset number of logs and writing it to the checkpoint for post-event integrity verification.

[0019] Secondly, this application provides a city information data sharing and exchange system based on a CIM platform, including: The initialization module is used to load the ontology graph of the urban information domain, register the data source, and instantiate the standardized adapter; The semantic modeling module is used to construct a unified semantic data model based on the domain ontology graph; The mapping engine module is used to calculate the mapping score between the source data fields and the target fields in the unified semantic data model using a multi-index weighted mapping model, and to perform semantic mapping and unit normalization transformation. The quality verification module is used to perform multi-dimensional quality verification and weighted fusion conflict resolution on the mapped data; The exchange scheduling module is used to perform matching based on the subscription conditions of the data consumer and distribute the data to the data consumer.

[0020] Optionally, it also includes: an incremental synchronization module, used to detect data changes through content hash comparison and generate differentially encoded incremental data packets; and a security control module, used to perform permission verification and dynamic de-identification processing based on the role permission matrix.

[0021] The technical solution provided in this application has the following beneficial effects: By constructing a unified semantic data model as a semantic intermediate layer for multi-source heterogeneous urban data, each data source only needs to access the intermediate layer through a standardized adapter, reducing the number of interfaces required for data exchange from a quadratic growth with the number of systems to a linear growth.

[0022] By employing a multi-index weighted mapping model to automatically calculate mapping scores between source data fields and target fields in the unified semantic data model, and filtering mapping results based on mapping confidence thresholds, automatic semantic alignment of cross-system data fields is achieved.

[0023] A multidimensional quality scoring model was used to assess data quality from four dimensions: completeness, format compliance, scope reasonableness, and consistency. A weighted fusion conflict resolution model was used to fuse conflicting data from multiple sources based on the credibility and freshness of the data source, thus ensuring the uniqueness and credibility of the exchanged data.

[0024] On-demand data distribution is achieved by compiling the subscription conditions of data consumers into a Boolean expression tree and performing condition matching. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the urban information data sharing and exchange method provided in the embodiments of this application.

[0026] Figure 2 This is a schematic diagram of a star-shaped data exchange topology provided in an embodiment of this application.

[0027] Figure 3 This is a schematic diagram of the ontology graph loading and data source registration sub-process provided in an embodiment of this application.

[0028] Figure 4 A schematic diagram of the sub-process for constructing a unified semantic data model provided in this application embodiment.

[0029] Figure 5 This is a schematic diagram of the semantic mapping subprocess provided in an embodiment of this application.

[0030] Figure 6 This is a schematic diagram of the data quality verification and conflict resolution sub-process provided in the embodiments of this application.

[0031] Figure 7 This is a schematic diagram of the subscription matching and priority distribution sub-process provided in the embodiments of this application.

[0032] Figure 8 This is a schematic diagram of the incremental synchronization subprocess provided in an embodiment of this application.

[0033] Figure 9 This is a schematic diagram of the unified semantic data model entity attribute set and data source descriptor data structure provided in the embodiments of this application.

[0034] Figure 10 This is a schematic diagram of the module architecture of the urban information data sharing and exchange system provided in the embodiments of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0037] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0038] This embodiment provides a method for sharing and exchanging urban information data based on a CIM platform. For example... Figure 2 As shown, this method adopts a star-shaped data exchange topology. Each data source connects to a unified semantic data model intermediate layer via standardized adapters, forming a star-shaped topology with this intermediate layer at its core. In a specific implementation, this method constructs a unified semantic data model as a semantic intermediate layer for urban multi-source heterogeneous data. Each data source is connected to this intermediate layer via standardized adapters to complete semantic mapping and conflict resolution, and then distributed according to a publish-subscribe mechanism. This reduces the interface complexity of data exchange from a quadratic increase with the number of systems to a linear increase. This method solves the technical problems in existing technologies, such as the need to develop numerous point-to-point conversion interfaces for multi-source urban data exchange, cross-system data semantic inconsistencies, and the difficulty in resolving multi-source data conflicts. It achieves the beneficial effects of reducing interface development and maintenance costs, realizing semantically lossless cross-system data exchange, and ensuring the uniqueness and reliability of fused data.

[0039] like Figure 1 As shown, this method includes the following steps: S000: Load the urban information domain ontology graph and register multiple data sources; generate a data source descriptor for each data source and instantiate the corresponding standardized adapter. like Figure 3 As shown, this phase is executed when the CIM platform is deployed for the first time or when a new data source is connected.

[0040] S010. Perform parsing on the domain ontology file to construct a directed graph structure containing concept hierarchy indexes and attribute association indexes.

[0041] The system reads the urban information domain ontology file and parses it into a directed graph structure in memory. The domain ontology file is encoded in OWL format and contains standard terminology definitions and semantic relationships across four major domains: geographic information, building information modeling, IoT sensing, and e-government. The ontology graph is represented as follows: ,in It is a collection of nodes, where each node represents an ontology concept class or instance; It is a set of directed edges, representing the semantic relationships between concepts; This is a function for labeling node types, mapping each node to one of the following: class, instance, or data type. This is a function for labeling edge types, mapping each edge to a type such as hierarchical relationship, attribute relationship, or association relationship.

[0042] During loading, a two-level index is constructed to support efficient subsequent queries. The first level is a concept-level index, which builds a tree structure based on hierarchical relationships to support fast traversal of hierarchical concepts. The second level is an attribute association index, which builds a hash table based on attribute relationships and value domain relationships to support fast lookup of all attributes and attribute value domain types from any concept node. The concept-level index is constructed using depth-first traversal, with a time complexity of O(log n). The attribute association index is implemented using a hash table, with an insertion time complexity of O(n log n). .

[0043] For example, loading a city information ontology file containing 4826 concept nodes and 14580 relation edges, with a file size of approximately 2.1MB, takes approximately 7 seconds (including the construction of two-level indexes) on a server equipped with an 8-core CPU and 32GB of memory. The concept hierarchy tree in the index has a depth of 6 levels, and the attribute association hash table contains approximately 8200 mapping rules. The entire loading process meets the constraint of not exceeding 30 seconds.

[0044] S020. Generate a structured data source descriptor for each data source to be connected and write it to the registry.

[0045] The system iterates through the data source configuration list, generating a structured data source descriptor for each data source to be connected. The data source descriptor is a structured object containing five required fields: the source identifier field contains a globally unique identifier, encoded using a three-part format of department code, system code, and sequence number; the data type field identifies the data domain category to which the data source belongs; the data schema field describes the data source's field list, primary key definition, and spatial geometry fields; the interface protocol field describes the data source's communication protocol type, connection address, and authentication method; and the update characteristics field describes the data source's update mode and update frequency.

[0046] For example, a geographic information data source is registered, with the source identifier "GHZRZYJ-DYDK-001", representing the first data interface of the Planning and Natural Resources Bureau's regional land parcel system. The data source is of geographic information type, and the data schema includes five fields: an integer object identifier field, a string-type parcel number field, a string-type parcel name field, a floating-point parcel area field (in square meters), and a geometric spatial shape field. The interface protocol is HTTP, and the authentication method is token authentication. The update mode is batch update, and the update frequency is [missing information]. The frequency is Hz (once per hour), with an average of 500 records per batch. The registration process for a single data source takes approximately 0.5 seconds, meeting the constraint of not exceeding 5 seconds.

[0047] S030. Instantiate a standardized adapter of the corresponding type according to the data source descriptor and perform connectivity detection.

[0048] Based on the data type and interface protocol fields in each registered data source descriptor, a matching template is selected from the pre-built adapter template library, and a standardized adapter object of the corresponding type is instantiated. After instantiation, a connectivity probe test is performed on each adapter to verify the availability of the communication link between the adapter and the data source.

[0049] For example, for a data source with a data type of Geographic Information and an interface protocol of HTTP, an HTTP Geographic Information Adapter is instantiated. During connectivity testing, a test request is sent to the data source endpoint, and a response with status code 200 is received, indicating a successful connection. For a data source with a data type of Building Information Modeling (BIM) and an interface protocol of File Sharing, a File BIM Adapter is instantiated. During connectivity testing, an attempt is made to access the shared directory and confirm readability. For a data source with a data type of Internet of Things (IoT) and an interface protocol of MQTT, an MQTT IoT Adapter is instantiated. During connectivity testing, a test topic is subscribed to and a connection confirmation message is received.

[0050] S040. Read the global exchange parameters from the configuration file and verify the validity of the value range.

[0051] Read the global exchange parameters from the configuration file and validate their value range. The global exchange parameters include the mapping confidence threshold. Quality dimension weight vector Synchronization frequency Mapping weight vector Urgency amplification factor Queue decay coefficient and the freshness decay rate constant .

[0052] For example, configuration , , Hz (meaning an incremental synchronization check is performed every 10 seconds). , , , After validating the validity of the value range of each parameter, write it into the global configuration parameter table. The aforementioned The object is stored in the process's global context for subsequent steps to access.

[0053] S100. Construct a unified semantic data model based on the domain ontology graph. The unified semantic data model includes multiple urban information entity classes and their standardized attribute sets extracted from the domain ontology graph.

[0054] like Figure 4 As shown, this stage is based on the ontology map loaded by S000. Construct a unified semantic data model for urban information using industry standard reference documents. For example... Figure 9 As shown, the entity attribute set structure of the unified semantic data model includes various city information entity classes and their field definitions.

[0055] S110. Starting from the predefined seed set of core urban information concepts, traverse along the hierarchical relationship in the domain ontology graph, collect candidate entity classes and their attribute nodes, and generate a standardized attribute set for each candidate entity class.

[0056] Define the seed set of core concepts for urban information It contains 24 seed concepts, covering 8 in the geographic information field (buildings, roads, plots, pipelines, bridges, green spaces, water bodies, and administrative divisions), 6 in the building information modeling field (floors, rooms, components, equipment, air conditioning systems, and electrical systems), 5 in the Internet of Things field (environmental sensors, security equipment, traffic detectors, energy consumption meters, and alarm events), and 5 in the government affairs field (population information, enterprise information, planning approvals, urban management events, and emergency resources).

[0057] Starting from the seed set, in The algorithm expands downwards along the hierarchical relationship edges using a breadth-first traversal to a maximum of two levels, collecting all sub-concepts as candidate entities. The selection of two levels for the expansion depth is based on the following: Level 1 only obtains direct subclasses (e.g., from "building" to "residential building"), resulting in insufficient coverage; Level 3 introduces overly fine-grained implementation-level concepts (e.g., "brick-concrete structure residential building"), leading to model redundancy. Level 2 strikes a balance between coverage and accuracy. The candidate entity set is defined as follows: .

[0058] For each candidate entity, collect all its attribute nodes along the attribute relation edges to form an attribute candidate set. For each attribute node, the data type is obtained from its value domain relationship, and the unit of measurement and encoding rules are extracted from the annotation tags. Finally, a standardized attribute set containing field names, data types, units of measurement, and encoding rules is generated for each seed concept (as the top-level entity class). Expanded sub-concepts, as attribute variants of the seed concept, do not become separate top-level entity classes to maintain the simplicity of the attribute set library. The final number of entity classes... .

[0059] For example, triple extraction is performed on the seed concept "building" to obtain the entity. This entity contains 12 standard attribute fields: string-type building identifier (primary key, required), string-type building name (optional), enumerated building type (values ​​can be one of four: residential, commercial, industrial, and public, required), integer number of floors (unit: floors, optional), floating-point type building height (unit: meters, optional), floating-point type building area (unit: square meters, required), integer-type construction year (unit: years, optional), enumerated structure type (values ​​can be one of four: frame, shear wall, brick-concrete, and steel structure, optional), string-type address (required), polygon-type spatial geometry (required), floating-point type altitude (unit: meters, optional), and string-type ownership unit (optional). This entity contains 12 attribute fields, satisfying the constraint that each entity type must have at least 10 standard attribute fields.

[0060] After extracting all 24 seed concepts, a unified semantic data model is obtained. Includes entity attribute set library Each entity type contains 12 to 18 standard attribute fields. Generate a versioned snapshot, including the version number, creation timestamp, and entity class directory.

[0061] S120. Based on the spatial geometric information and semantic labels of the candidate entity classes, construct a spatial semantic joint index.

[0062] Construct a spatial semantic joint index for entities in the unified semantic data model. The index employs a two-level structure. The first level is a spatial range index, indexing the minimum bounding rectangle of all entities. It supports spatial range operations such as window queries and nearest neighbor queries, with a query time complexity of O(log n). The spatial range index has a page size of 4KB, with each page accommodating approximately 100 minimum bounding rectangle entries, adapting to the operating system's page caching mechanism. For entities in the tens of thousands, the spatial range index has approximately 3 to 4 levels, with no more than 4 input / output operations per query.

[0063] The second level is a semantic tag inverted index, which builds an inverted list for each entity's type tag, subject domain tag, and attribute keywords, supporting fast filtering based on semantic conditions. This inverted index is implemented using a hash table, with a query time complexity of O(log n). .

[0064] When performing a compound query, the spatial candidate set is first obtained through a window query of the first-level spatial range index. Then, semantic filtering is performed on the spatial candidate set using the second-level semantic tag inverted index, and the intersection of the two result sets is the final query result. .

[0065] For example, in a scenario where 10,000 entity instances are initially loaded (the first full import from 3 data sources), the spatial extent index construction takes approximately 0.5 seconds, with a tree height of 3 levels; the inverted index construction covers 24 entity type tags and 8 subject domain tags, totaling 32 inverted lists, and takes approximately 0.1 seconds. Executing a compound query "all buildings within the latitude and longitude range [113.9, 22.5, 114.1, 22.6]", the spatial extent index window query returns 523 candidate entities, and the inverted index, filtered by the type tag "buildings", yields 312 results. The total query time is approximately 15 milliseconds, meeting the response time constraint of no more than 100 milliseconds.

[0066] During the parsing of the OWL format ontology file, the parser traverses the three types of statements in the ontology file—class declarations, attribute declarations, and relation declarations—in depth-first order. For class declaration statements, it extracts three elements: class identifier, class label, and parent class reference, creating nodes and establishing hierarchical relationship edges in the directed graph. For attribute declaration statements, it extracts four elements: attribute identifier, attribute label, class reference, and value type, creating attribute nodes and establishing attribute relationship edges in the directed graph. For relation declaration statements, it extracts three elements: subject class, object class, and relation type, establishing association relationship edges. The parser processes each statement in approximately 0.5 milliseconds, and the total parsing time for 4826 concept nodes and 14580 relation edges is approximately [missing information - likely a timeframe or timeframe]. Milliseconds. Including the overhead of index building, the overall loading time is kept under 30 seconds.

[0067] The source identifier encoding rules during the data source registration process are further explained. The first segment of the three-segment encoding is the department code, using the initials of the full name of the department in Chinese Pinyin, with a length not exceeding 8 characters; the second segment is the system code, using an abbreviation of the system function, with a length not exceeding 6 characters; the third segment is the serial number, using a three-digit code, incrementing from 001. The three segments are connected by hyphens. This encoding rule ensures that each data source interface has a globally unique identifier within the same CIM platform deployment scope. When the total number of registered data sources exceeds 999, the third segment automatically expands to a four-digit code.

[0068] The adapter template library pre-defines four basic adapter types. The HTTP adapter communicates with the data source via the Hypertext Transfer Protocol (HTTP), supporting both GET and POST request methods, and is suitable for Geographic Information Systems (GIS) and Government Administration Systems (GES) that provide RESTful APIs. The file adapter accesses data files in shared directories via file system protocols, supporting both file-by-file and batch read modes, and is suitable for Building Information Modeling (BIM) systems that output data in file format. The MQTT adapter subscribes to message topics published by the data source via the Message Queuing Telemetry Transport Protocol (MQTT), supporting three transmission guarantees from Quality of Service (QoS) levels 0 to 2, and is suitable for IoT sensor platforms that push data in real time. The database adapter directly queries the relational database of the data source via a Structured Query Language (SCL) interface, supporting both paginated queries and cursor traversal, and is suitable for databases in government business systems. Each adapter implements a unified adapter abstract interface, including four standard methods: connection, mode retrieval, data retrieval, and close.

[0069] The process of generating entity attribute sets will be further explained. Attribute candidate sets. Each attribute node contains five metadata fields: field name (an English underscore naming format identifier extracted from the attribute label), data type (the basic data type derived from the value range type, including seven types: string, integer, floating-point, boolean, date / time, enumeration, and geometric), unit of measurement (extracted from the annotation label; fields without physical dimensions are marked as dimensionless), encoding rules (extracted from the data constraint label, such as the list of allowed values ​​for enumeration types, the regular expression pattern for string types, and the value range for numeric types), and whether it is required (derived from the cardinality constraint; fields with a minimum cardinality greater than 0 are required).

[0070] After standardization, the entity attribute set requires three verifications: first, primary key uniqueness verification, confirming that each entity attribute set has exactly one field marked as the primary key; second, geometric field existence verification, confirming that entities with spatial attributes (in the geographic information and building information modeling domains) contain at least one geometric field; and third, enumeration value validity verification, confirming that the list of allowed values ​​for enumeration fields is not empty and contains no duplicate values. Entities that fail verification will be marked as "incomplete attribute set" and skipped in subsequent mapping processes.

[0071] The design decision to use a two-level index instead of a single composite index is based on the following analysis: a single spatial index is inefficient when processing purely semantic queries (such as "find all entities of all alarm event types"); a single inverted index cannot utilize spatial locality when processing spatial range queries. The two-level structure allows both types of queries to leverage the advantages of their respective indexes: spatial range queries directly use the first-level index, semantic queries directly use the second-level index, and composite queries are achieved through the intersection of the result sets of the two-level indexes.

[0072] When inserting a new entity, the spatial range index first calculates the entity's minimum bounding rectangle, then selects the leaf node that minimizes the increase in tree node area after insertion. When the number of entries in a leaf node exceeds the page capacity (100 minimum bounding rectangles), a node splitting operation is performed, using a two-stage splitting strategy to distribute the entries into two new nodes. The goal of this splitting strategy is to minimize the total bounding rectangle area of ​​the two new nodes, thereby improving pruning efficiency during subsequent queries.

[0073] The inverted index maintains a mapping table from tags to a list of entity identifiers. For a newly inserted entity, its type tag, subject domain tag, and attribute keywords are extracted, and the entity identifier is appended to the end of each corresponding inverted list. The inverted lists are arranged in the order of entity identifier insertion, and queries return the intersection of this list and the spatial candidate set. When an entity is deleted or updated, the corresponding entry in the inverted list must be maintained synchronously.

[0074] S200. Obtain source data from each data source through the standardized adapter, calculate the mapping score between each source data field and the candidate target field in the unified semantic data model using a multi-index weighted mapping model, map source data fields with mapping scores exceeding the mapping confidence threshold to the corresponding target fields, and perform unit normalization transformation on the mapped data.

[0075] like Figure 5 As shown, this stage transforms the raw data from various data sources into a unified semantic data model format through standardized adapters and a semantic mapping engine.

[0076] S210. Connect to the data source through the standardized adapter to obtain the data pattern definition and sample data of the source data.

[0077] Each standardized adapter connects to its corresponding data source, obtaining the data schema definition (including a list of field names, field data types, and constraint information) and 10 sample data entries. The data schema definition is automatically extracted from the data source by calling the adapter's schema retrieval interface, requiring no manual configuration. The sample data is used to assist in determining the actual data characteristics of the fields during subsequent mapping processes.

[0078] S220. Perform name normalization processing on each source data field.

[0079] The acquired source data field names undergo name standardization. This name standardization is achieved through a Chinese-English thesaurus containing approximately 2000 mapping rules, converting Chinese abbreviations, English abbreviations, and non-standard names in the source field names into a unified English underscore naming format.

[0080] Exemplarily, the source data field name "DKMJ" is mapped to the Chinese "地块面积" (land parcel area) through the thesaurus lookup and then standardized to the English "parcel_area". The source data field name "OBJECTID" is directly mapped to "object_id" through the thesaurus. For non-standard field names that do not exist in the thesaurus (such as "ZZRQ"), the original name is retained after standardization and naturally enters the queue for manual confirmation due to low name similarity in subsequent mapping calculations.

[0081] S230. Use a multi-index weighted mapping model to calculate the mapping score.

[0082] For each standardized source data field and all candidate target fields in the unified semantic data model calculate the mapping score. The mapping score is obtained by weighted summation of three independent indicators: the name similarity index, the type compatibility index, and the ontology semantic distance index, according to preset weights: The name similarity index is calculated using the normalized edit distance. Calculate the edit distance between the standardized source field name and the target field name, and divide it by the maximum value of the lengths of the two strings, and then take the complement value: where is the edit distance function, is the synonym standardization function, is the string length. When the two standardized field names are exactly the same, .

[0083] The type compatibility index is determined based on a predefined type compatibility matrix. A perfect match (such as integer and integer) has a score of 1.0; a compatible type (such as integer and floating point) has a score of 0.8; a convertible type (such as string and integer, which requires parsing) has a score of 0.5; an incompatible type (such as geometric and string) has a score of 0.0.

[0084] The ontology semantic distance index calculates the semantic similarity between the concept node to which the source field belongs and the concept node to which the target field belongs in the ontology graph . The semantic similarity is obtained by calculating the ratio of the depth of the nearest common ancestor of the two concept nodes to the sum of the depths of the two concept nodes: where and are respectively [[ID= fifty]]and The ontology concept node to which it belongs. The nearest common ancestor node, This represents the depth of the node in the concept hierarchy tree. When two concepts are identical... The greater the conceptual distance, the smaller the value.

[0085] The preset weights of the three indicators are: . The value range is [0.2, 0.6], and it can be increased to 0.6 when the naming convention of the data source fields is consistent. The value range is [0.1, 0.3]. The value range is [0.2, 0.6].

[0086] For example, the field "DKMJ" (normalized to "parcel_area") of the geographic information data source "GHZRZYJ-DYDK-001" is compared with the parcel entity in the unified semantic data model. The mapping score is calculated for the candidate target field "parcel_area". Name similarity: After standardization, the two field names are exactly the same. , Type compatibility: Both the source and target fields are floating-point types, resulting in an exact match. Ontology semantic distance: Since both the source and target fields belong to the concept of "land parcel," the lowest common ancestor is the "land parcel" itself. (The path is: root node, city elements, land use, land parcel). Comprehensive mapping score .

[0087] As a counterexample, the mapping score between the same field "DKMJ" and the candidate target field "parcel_id": , (Floating-point and string types are convertible). (Both belong to the concept of "land parcel"). Although it exceeds the threshold, its score is lower than 1.0 for "parcel_area", so it will not be selected as the optimal mapping.

[0088] S240. Map the source data fields whose mapping scores exceed the mapping confidence threshold to the corresponding target fields, and perform unit normalization transformation on the mapped data.

[0089] For each source data field Select the target with the highest mapping score from all candidate target fields. As the mapping target. If the highest mapping score is not lower than the mapping confidence threshold. If the value is within the threshold, the mapping relationship will be automatically confirmed; if it is below the threshold, the field will be marked as "awaiting manual confirmation" and added to the manual review queue.

[0090] After the mapping is confirmed, a linear transformation is performed on the mapped source data values ​​according to a predefined unit transformation matrix. Each entry in the unit transformation matrix defines a scaling factor from the source units to the target units. and offset The conversion formula is: When the source unit and the target unit are the same (Identity transformation). If the source unit is not in the transformation matrix, mark the field as "Unit unknown" and transfer it to the manual confirmation queue.

[0091] For example, the source unit of the field "DKMJ" is "mu" (acres), while the standard unit of "parcel_area" in the Unified Semantic Data Model is "square meters". Looking up the transformation matrix yields... For sample values mu, the conversion result is Square meters. Other commonly used conversion rules include: kilometers to meters ( ), Fahrenheit to Celsius ( ), pounds per square inch to Pascal ( ), feet to meters ( ).

[0092] After all fields are mapped, a mapping rule table is output. For example, all five fields of the "GHZRZYJ-DYDK-001" data source were automatically mapped successfully (each mapping score was no less than 0.7), with no items requiring manual confirmation. The total mapping processing time was approximately 1.2 seconds (5 fields multiplied by 24 entity types multiplied by an average of 15 attributes equals 1800 pairs of calculations, each pair taking approximately 0.6 milliseconds), meeting the constraint of not exceeding 10 seconds.

[0093] The working process of the semantic mapping engine is further explained. The input to the mapping calculation is a list of source fields and a list of target fields for all entity attribute sets in the unified semantic data model. The engine adopts a two-stage strategy: the first stage is the candidate generation stage, in which, for each source field, candidate target fields with name prefix matching or edit distances not exceeding 3 are retrieved from the target field list based on its standardized name, generating a set of candidate pairs; the second stage is the precision scoring stage, in which a weighted mapping score based on three metrics is calculated for each pair in the candidate pair set. This two-stage strategy reduces computational complexity from... Reduce to ,in The average number of candidates for each source field (typically less than 10).

[0094] The edit distance function mentioned in the name similarity metric is specifically defined as: given two strings and , Define as Convert to The minimum number of single-character editing operations required, including insertion, deletion, and replacement, is calculated using a dynamic programming algorithm, with a time complexity of O(n log n). The space complexity is For field names with an average length of 15 characters, a single pair of calculations takes approximately 0.01 milliseconds.

[0095] The type compatibility matrix is ​​designed based on the natural conversion relationships between data types. A perfect match (score 1.0) applies to mappings between the same type. Compatible types (score 0.8) include: integer and floating-point (integers can be losslessly converted to floating-point numbers, but the reverse may truncate the decimal part), and date / time and string (dates can be serialized to ISO8601 strings and can be reverse-parsed). Convertible types (score 0.5) include: string and numeric (attempt to parse the string content; conversion occurs on success, and an exception is marked on failure), and boolean and integer (0 corresponds to a false value, 1 to a true value). Incompatible types (score 0.0) include: geometric and non-geometric types (spatial data structures cannot be converted to scalar values), and enumeration and numeric types (the semantic domains are completely different).

[0096] In the process of calculating the ontology semantic distance index, when two concept nodes do not have a nearest common ancestor (i.e., they belong to different ontology subtrees), Setting it to 0.0 indicates that the semantics are completely unrelated. The depth of the root node is defined as 1. The lowest common ancestor (LCA) is calculated using a two-pointer uptracing method: traversing simultaneously from two concept nodes towards the root node, maintaining their respective ancestor sets, the node that first appears in both ancestor sets is the LCA. The time complexity of this two-pointer uptracing method is O(log n). ,in The depth of the concept hierarchy tree (in this system) ).

[0097] The rules for handling mapping conflicts are further elaborated. When a source field has multiple candidate target fields with the same mapping score, the following priority is applied: First, the candidate whose data type exactly matches the source field (type compatibility is 1.0) is selected; if there are still ties, the candidate with the closest ontology semantic distance (i.e., the candidate with the closest semantic distance) is selected. If the values ​​are still tied, the candidate with the smallest edit distance between the target field name and the source field name is selected. Ties remaining after three levels of priority filtering are extremely rare in practice (probability less than 0.1%). In this case, the candidate with the smallest lexicographical order of the field identifier is automatically selected, and this automatic decision is recorded in the mapping log.

[0098] The unit conversion matrix consists of 58 entries, covering 11 dimensional categories: length (7), area (5), volume (4), mass (4), temperature (3), time (5), angle (3), velocity (4), pressure (5), coordinates (4), and currency (4), as well as 10 special cross-dimensional conversion rules (e.g., from kilometers per hour to meters per second). The conversion matrix is ​​loaded into memory from the configuration file during system initialization and stored as a hash table with keys consisting of "source unit - target unit" string pairs. The lookup time complexity is O(n log n). .

[0099] Further explanation of the completeness statistics of the mapping results. After completing the mapping of all fields, the mapping engine outputs a mapping statistics report, including the following metrics: number of successfully mapped fields (number of fields with mapping scores not lower than the threshold), number of fields awaiting manual confirmation (number of fields with mapping scores lower than the threshold), automatic mapping rate (percentage of successfully mapped fields out of the total number of fields), and average mapping score. In typical scenarios of urban information data sources, the automatic mapping rate usually reaches over 90%, and manual confirmation mainly focuses on legacy system fields with non-standard naming.

[0100] S300. Perform multidimensional quality verification on the mapped data, calculate the comprehensive quality score of each data record according to the multidimensional quality scoring model, and enter the conflict resolution for data records whose quality scores reach the preset gate threshold. Use the weighted fusion conflict resolution model to fuse the multi-source data of the same entity to obtain the fused dataset.

[0101] like Figure 6 As shown, this stage performs quality gating and conflict resolution on the data after S200 semantic mapping.

[0102] S310. Perform multidimensional quality verification on the mapped data and calculate the comprehensive quality score based on the multidimensional quality scoring model.

[0103] For each data record that has undergone semantic mapping, perform quality checks in four dimensions in sequence, and then sum the sub-scores of the four dimensions according to the preset dimension weights to obtain the comprehensive quality score.

[0104] The integrity score The fill rate of required fields is measured and calculated as the actual number of non-empty required fields. The total number of required fields for this entity type The ratio: When all required fields are not empty .

[0105] The format compliance score The system measures whether the values ​​of each field conform to the data type and encoding rules defined by the unified semantic data model, and calculates the number of fields that pass the format validation. The total number of fields that actually have values ​​in the record The ratio. Format validation includes whether numeric fields are valid numbers, whether date fields conform to the ISO 8601 standard, whether enumeration field values ​​are within a predefined enumeration set, and whether geometric fields are valid spatial data formats.

[0106] The scope of the reasonableness score To determine whether a numeric field is within a physically reasonable range, the number of numeric fields whose values ​​fall within a preset range is calculated. With all numeric fields The ratio. The scope is defined to include buildings with heights between 0 and 1000 meters and plot areas between 0 and... Between square meters and temperatures between -50 and 60 degrees Celsius. When the record has no numeric field. .

[0107] The consistency score This measures the degree of attribute consistency among multiple sources of data for the same entity, and is calculated as follows: ,in The number of fields with conflicting attribute values. This represents the total number of overlapping fields across multiple data sources. The conflict determination rules are: numeric fields with a deviation exceeding 5% are considered conflicting; string fields that are not completely equal after standardization are also considered conflicting. This applies when there is only a single data source. .

[0108] The formula for calculating the overall quality score is: in . and Give it the highest weight because missing required fields and multi-source conflicts have the most direct impact on downstream usage. Quality gating threshold. Records with values ​​below this threshold will not be included in the data exchange pool and will be marked as "pending repair".

[0109] For example, for the building entity "BLD-2024-00123", Source 1 (geographic information data source, credibility weight) Data quality checks were performed: all 6 required fields are not empty. All six fields with values ​​are in compliant format. All three numerical fields (building height 45.2 meters, building area 12,500 square meters, number of floors 10) are within a reasonable range. Compared with Source 2 (Building Information Modeling data source), there are contradictions in the two fields of building name and spatial geometry type among the six overlapping fields. Overall rating If the threshold value exceeds 0.6, the conflict resolution process begins.

[0110] S320. A weighted fusion conflict resolution model is used to fuse multi-source data of the same entity.

[0111] For data records whose quality scores reach the gating threshold, when the same entity has records from multiple data sources, conflict resolution is performed to obtain a unique fused attribute value. Identification of the same entity is achieved through primary key matching or nearest neighbor matching with a spatial distance of less than 5 meters.

[0112] For numerical attributes, a weighted average fusion strategy is used. The fusion weight for each data source is determined by a static confidence weight. Data freshness decay factor The product is determined as follows: The freshness decay factor is an exponential decay function with the data update time difference as the independent variable: in This is the difference between the current time and the last update time of this data source (in days). The decay rate constant ( The aforementioned This causes the weight of the data to decay to half of its original value after about 69 days. The value range is [0.001, 0.1]. For real-time IoT data, it can be set to 0.1 (approximately 7 days half-life), and for long-term effective planning data, it can be set to 0.001 (approximately 693 days half-life).

[0113] For string attributes, the highest weight priority strategy is used to select... The attribute values ​​provided by the largest data source are used as the fusion result.

[0114] For example, a weighted fusion is performed on the building height attribute of building "BLD-2024-00123". The current time is May 17, 2026. Source 1 (Geographic Information): rice, Updated on May 10, 2026. sky, Source 2 (Building Information Modeling): rice, Updated on May 15, 2026. sky, Fusion calculation: The numerator is The denominator is , rice.

[0115] Perform string fusion on the building name attribute: Source 1 Source 2 Source 1 has a higher weight, and the fusion result is "City Exhibition Center".

[0116] After the conflict is resolved, all attribute values ​​of the same entity are unique, satisfying the constraint that multiple values ​​cannot coexist.

[0117] The evaluation methods for each dimension of the quality score are explained in more detail. In the calculation of the integrity score, the definition of required fields is derived from the cardinality constraint of the entity's attribute set in the unified semantic data model. Taking a building entity as an example, 6 of its 12 standard attribute fields are marked as required (building identifier, building type, building area, address, spatial geometry, and building name), and the remaining 6 are optional. Integrity checks are performed only on required fields; null values ​​in optional fields do not affect the integrity score. When one of the required fields of a record is empty, .

[0118] The format compliance scoring rules are categorized by field data type. Numeric fields are checked to ensure their values ​​can be parsed as valid integers or floating-point numbers; invalid examples include strings containing alphanumeric characters. Date and time fields are checked to ensure their values ​​conform to the ISO 8601 standard format (i.e., "year-month-day T hour:minute:second Z" pattern or its valid variations); invalid examples include "2026 / 13 / 01" (month is out of range). Enumeration fields are checked to ensure their values ​​belong to a predefined enumeration set; invalid examples include a building type field with the value "temporary" while the predefined enumeration set only includes residential, commercial, industrial, and public types. Geometric fields are checked to ensure their values ​​are expressed in a valid spatial data format; invalid examples include polygons with unclosed coordinate point sequences.

[0119] The range for the scope reasonableness score is defined based on the physical reasonable range and industry standards in the field of urban information. The reasonable range for building height is set at 0 to 1000 meters, covering the entire range from single-story buildings to super high-rise buildings; heights exceeding 1000 meters are considered data anomalies. The reasonable range for plot area is set at 0 to... The maximum allowable temperature range is 100 square meters (square kilometers). Values ​​exceeding this range are considered abnormal at the urban area level. The acceptable temperature range is set at -50 to -60 degrees Celsius, covering extreme temperature records for all climate zones in China. The acceptable population density range is set at 0 to 100,000 people per square kilometer, covering areas from sparsely populated regions to extremely dense urban areas.

[0120] In determining inconsistencies in consistency scoring, the 5% threshold for numerical field deviation is chosen based on the following considerations: Common measurement errors in urban information data typically range from 2% to 3% (e.g., measurement discrepancies between Geographic Information Systems (GIS) and Building Information Models (BIM) regarding the same building height). Therefore, a 5% threshold can tolerate normal measurement errors while capturing genuine data conflicts (e.g., data source A reports a building height of 45 meters while data source B reports 55 meters, a deviation of 22.2%, far exceeding the 5% threshold). For scenarios requiring high precision (e.g., pipeline coordinates), this threshold can be lowered to 1%.

[0121] Mathematical property analysis of the weighted fusion conflict resolution model. This model is essentially a weighted arithmetic average, with weights calculated as the product of static credibility and dynamic freshness. When the credibility and freshness of all data sources are identical, It degenerates into a simple arithmetic mean. When there is only one data source... (The fusion result equals the unique source value). The freshness decay factor. It satisfies the monotonically decreasing property: The larger (the older the data). The smaller the value (the lower the weight), the half-life of this decay function is... The data's freshness weight decays to half of its initial value after approximately 69 days.

[0122] The boundary conditions for conflict resolution are discussed. When a certain data source... When the value is extremely high (e.g., no update for more than 365 days), The contribution of this data source to the fusion result is almost negligible. In extreme cases (such as...), ), This data source is automatically excluded from the fusion computation. This property allows the system to handle scenarios where the data source is offline or obsolete naturally, without requiring manual intervention to delete expired data sources.

[0123] Engineering implementation details for verification processing throughput. Quality verification for each record includes sub-score calculations across four dimensions and a weighted summation operation. The total verification time for a single record is approximately 0.8 milliseconds (0.1 milliseconds for integrity verification, 0.3 milliseconds for format verification, 0.1 milliseconds for range verification, 0.2 milliseconds for consistency verification, and 0.1 milliseconds for weighted summation). In single-threaded mode, approximately 1250 records can be processed per second, meeting a throughput constraint of at least 1000 records per second. The system supports multi-threaded parallel verification; each additional worker thread linearly increases the throughput by approximately 900 records per second (actual value considering thread synchronization overhead).

[0124] S400. Perform condition matching on the fused dataset according to the subscription conditions of the data consumer, generate a distribution task, and distribute the matched data according to the output format and protocol specified by the data consumer.

[0125] like Figure 7 As shown, this stage involves distributing the merged dataset to data consumers on demand via a publish-subscribe mechanism.

[0126] S410. Compile the subscription conditions of the data consumer into a Boolean expression tree and perform condition matching.

[0127] Data consumers declare their required filtering conditions through a subscription interface. These filtering conditions are compiled into a Boolean expression tree, where leaf nodes are atomic filtering conditions and internal nodes are logical operators (including AND, OR, and NOT). The atomic filtering conditions include four types: entity type filtering (determining whether the entity type of a record belongs to a specified set), attribute value filtering (determining whether a record's attribute value satisfies a numerical comparison condition), spatial range filtering (determining whether the spatial geometry of a record falls within a specified rectangular area), and time window filtering (determining whether the record's update time is within a specified time interval).

[0128] The matching process employs a short-circuit evaluation strategy to optimize performance: it first performs the entity type filtering with the lowest computational cost (hash lookup, time complexity...). Then perform attribute value filtering (numerical comparison, time complexity) Finally, spatial range filtering is performed (through the spatial semantic joint index). Spatial range index lookup, time complexity ).

[0129] For example, the subscription criteria for the planning bureau's building monitoring system are "the entity type is a building, the building height is greater than 30 meters, and the spatial range is within the latitude and longitude [113.9, 22.5, 114.1, 22.6]". This criterion is compiled into a Boolean expression tree: the root node is an AND operation, and the three leaf nodes are type filtering (building), attribute value filtering (height greater than 30), and spatial range filtering, respectively. Matching is performed on three records in the fused dataset: record R1 (building, height 46.00 meters, spatial range matched) passes all conditions; record R2 (building, height 28.5 meters) is short-circuited and eliminated during the attribute value filtering stage; record R3 (alarm event type) is short-circuited and eliminated during the type filtering stage.

[0130] S420. Determine the execution order of the distributed tasks based on the dynamically calculated task priority value and distribute the data to the data consumer.

[0131] Calculate dynamic task priority values ​​for the generated distribution tasks, and then distribute them sequentially according to priority from high to low. The task priority values ​​are based on the task's base priority. Urgency level marker and the current depth of the pending task queue calculate: The The value is specified by the subscription configuration, ranging from [1, 10], with a default value of 5. Emergency management subscriptions can be set to 10, while daily statistics subscriptions can be set to 2. The value can be 0 or 1; it is 1 when the data contains alarm events, and 0 otherwise. This increases the priority of urgent tasks by three times. This results in a lower priority for new tasks as the queue deepens. Priority decays to .

[0132] During distribution, the publishing engine serializes the data according to the output format specified by the consumer (JSON, XML, or GeoJSON, etc.) and transmits it through the communication protocol specified by the consumer (HTTP push, MQTT publish, or WebSocket push, etc.). After distribution is completed, a distribution receipt is recorded, including the distribution time, data volume, and receipt confirmation status.

[0133] For example, two distribution tasks from different consumers compete for execution. In the subscription matching of the planning bureau's building supervision system, record R1 generates task T1 through condition matching (record R1 is sent to the planning bureau, ), In the Emergency Management Bureau's independent subscription matching (subscription condition is "entity type is alarm event"), it is recorded that R3 generates task T2 (alarm event R3 is sent to the Emergency Management Bureau) through the subscription condition matching. ), The execution order is T2 (25.71) over T1 (5.0), with emergency alarms sent first. End-to-end latency verification: approximately 50 milliseconds for condition matching, approximately 5 milliseconds for priority sorting, approximately 100 milliseconds for format conversion, and approximately 200 milliseconds for network push, totaling approximately 355 milliseconds, which meets the constraint of not exceeding 2 seconds.

[0134] The compilation process of the Boolean expression tree is further explained. The subscription conditions submitted by the consumer are represented in JSON format. The compiler recursively parses the JSON structure into node objects of the Boolean expression tree. When logical operator keywords ("AND", "OR", or "NOT") appear in the JSON, an internal node is created; otherwise, a leaf node is created. The leaf node is instantiated into the corresponding atomic filter object based on the type of the filtering condition: the entity type filter maintains a set of allowed types (hash set), the attribute value filter maintains a triplet of field name, comparison operator, and threshold, the spatial range filter maintains a latitude and longitude range rectangle, and the time window filter maintains a start and end time pair. The compiled Boolean expression tree is cached in memory and recompiled only when the subscription configuration changes.

[0135] The execution order optimization of the short-circuit evaluation strategy is based on the computational cost analysis of each atomic filtering condition. Entity type filtering has the lowest cost because it only requires one hash set lookup to determine whether a record matches, with a time complexity of O(log n). The average processing time is approximately 0.01 milliseconds. Attribute value filtering is the next most expensive, as it requires extracting the specified field value from the record and performing a numerical comparison or string matching, resulting in a time complexity of O(n). The average processing time is approximately 0.05 milliseconds. Spatial range filtering has the highest overhead because it requires determining whether the recorded geometric objects intersect with the specified rectangle. If spatial indexing is used, the time complexity is O(n log n). The average time taken is approximately 1 millisecond. The overhead of time window filtering is between that of attribute value and spatial range, with an average time taken of approximately 0.02 milliseconds.

[0136] The data structure of the distribution receipt is described below. Each distribution receipt contains the following fields: distribution task identifier (a globally unique UUID format identifier), distribution timestamp (server time in ISO8601 format), target consumer identifier (subscription identifier), data volume (number of bytes and records sent), output format (the actual serialization format name used), protocol type (the actual transport protocol name used), and reception acknowledgment status (success, timeout, or rejection). The timeout threshold for reception acknowledgment is set to 5 seconds. If the consumer does not return an acknowledgment response within 5 seconds, the distribution task is marked as timed out and enters the retry queue, with a maximum of 3 retries.

[0137] The working process of the distribution format converter is described. The format converter serializes the internal data of the Uniform Semantic Data Model (USM) into the corresponding output format specified by the consumer. For JSON output, each entity record is serialized into a JSON object, with attribute fields using the USM field names as keys and attribute values ​​as values, and geometry fields output as coordinate arrays. For GeoJSON output, each entity record is serialized into a Feature object, with geometry information placed in the geometry attribute and other attributes placed in a properties object. For XML output, each entity record is serialized into an XML element named after the entity type, with attribute fields as child elements.

[0138] S500, incremental synchronization.

[0139] like Figure 8 As shown, after the initial full data exchange is completed, data changes from each data source are continuously monitored.

[0140] S510: For each data source, retrieve the latest data in each synchronization cycle and detect changes by comparing content hashes.

[0141] Change the detection probe in each synchronization cycle ( Retrieve the latest data from the data source within seconds. Calculate the content hash value for each record. The hash snapshot saved with the last synchronization Compare each record individually. Records with inconsistent hash values ​​are marked as "modified"; records that currently exist but were not in the last snapshot are marked as "added"; records that existed in the last snapshot but are not currently exist are marked as "deleted".

[0142] The hash algorithm used is SHA-256, outputting 256 bits (32 bytes), and the hash calculation for a single record takes approximately 2 microseconds. The synchronization period is set to the default value. Hz (meaning it executes once every 10 seconds). IoT data sources can be configured to execute every 1 second (Hz). (Hz), the government batch data source can be configured to 1 hour. The storage space for hash snapshots is 32 bytes per record, and a snapshot of 100,000 records is approximately 3.2MB.

[0143] For example, in a scheduled synchronization scenario of a geographic information data source, the last synchronization contained 500 land parcel records. This synchronization retrieves 502 records: 498 records with the same hash as the last (no change), 2 records with different hashes (modified), and 2 new records (added). There were no missing records in the last snapshot (no deletions). The change detection latency is approximately 80 milliseconds, meeting the constraint of no more than 500 milliseconds.

[0144] S520. Use differential encoding to generate incremental data packets for change records.

[0145] For records marked as changed, differential encoding is used to extract only the changed fields and their new values ​​to generate an incremental data packet. The incremental data packet structure includes the record primary key, a change type identifier (add, modify, or delete), a change timestamp, and a set of changed fields. For modify changes, the old and new values ​​are compared field by field, and only the changed fields and their new values ​​are included. For add changes, the entire record is included. For delete changes, only the record primary key is included.

[0146] Incremental compression ratio When only 2 fields of a record with 15 fields change, The incremental data packets are distributed via S400 after undergoing S200 semantic mapping and S300 quality verification and conflict resolution. Change logs are recorded in an append-only manner, with a retention period of no less than 30 days, and historical traceability is supported.

[0147] For example, the old value of the modified record is a plot area of ​​10133.4 square meters and a plot name of "Plot A". The new value is a plot area of ​​10250.0 square meters and a plot name of "Plot A". Only the plot area field changes, and the generated incremental data packet only contains the primary key "DYDK-123", the change type "Modified", the timestamp, and the set of changed fields (plot area of ​​10250.0). The complete record is approximately 200 bytes, and the incremental data packet is approximately 50 bytes. This satisfies the constraint that the incremental synchronization bandwidth saving rate is not less than 70%.

[0148] S600, Safety Management Procedures.

[0149] S610: Perform permission verification on distribution requests based on data classification and hierarchical structure and role-permission matrix.

[0150] The security gateway performs a three-step verification for each distribution request. The first step is identity authentication, verifying the signature and validity period of the consumer's identity token. The second step is permission verification, querying the set of accessible fields and the highest data level for the target entity type by the consumer's role based on the role-permission matrix. The role-permission matrix... The access permissions for each consumer role to each entity type are defined.

[0151] The data is categorized into four levels: Level 1 is public data (basic urban geographic information); Level 2 is internal data (building details and planning approval information); Level 3 is sensitive data (demographic and business operation data); and Level 4 is confidential data (personal identification information and security deployment information). Access control is implemented using a hash table double-key lookup, with a time complexity of O(log n). The delay is approximately 2 milliseconds, which meets the constraint of no more than 10 milliseconds.

[0152] S620: Perform dynamic desensitization processing on sensitive fields that exceed the consumer's role permission level.

[0153] The third step is data anonymization, where sensitive fields exceeding the consumer's access level are processed according to anonymization strategies. These strategies include three types: partial masking for identification fields (ID card numbers and mobile phone numbers), retaining the first 3 and last 4 digits and replacing the rest with asterisks; precision generalization for address fields, reducing detailed addresses to district-level precision; and random permutation for name fields, replacing real names with random names. The anonymized data retains its analytical usability.

[0154] The entire operation log is appended to the audit log. Each log entry includes a timestamp, operator, operation type, data identifier, and result. A chain hash is calculated and written to a checkpoint every 100 log entries to support post-event integrity verification. The chain hash ensures that the logs are immutable.

[0155] For example, a public query role requests to view information about building "BLD-2024-00123". The access permission matrix shows that the public query role's accessible fields for building types are building name and spatial geometry, with the highest access level being Level 1. The requested field, building area, belongs to Level 2 internal data, and due to insufficient permissions, this field is filtered. The building name and spatial geometry are both Level 1 public data, so no data masking is performed, and their original values ​​are returned. The audit log records information such as the operator, requested fields, authorized fields, and denied fields.

[0156] System Implementation Examples like Figure 10 As shown, this embodiment also provides a city information data sharing and exchange system based on the CIM platform. The functional modules of the system can be implemented by the processor executing computer program instructions stored in the memory.

[0157] The system includes an initialization module, a semantic modeling module, a mapping engine module, a quality verification module, and an exchange scheduling module. The initialization module loads the urban information domain ontology graph, registers data sources and generates data source descriptors, instantiates standardized adapters and performs connectivity detection, and loads global exchange parameters. The semantic modeling module constructs a unified semantic data model based on the domain ontology graph, including extracting entity classes and their standardized attribute sets from a seed concept set, and constructing a spatial semantic joint index. The mapping engine module calculates mapping scores between source data fields and target fields in the unified semantic data model using a multi-index weighted mapping model, performs semantic mapping, and performs unit normalization transformation based on a unit transformation matrix. The quality verification module performs four-dimensional quality verification on the mapped data, calculates a comprehensive quality score, and uses a weighted fusion conflict resolution model to fuse multi-source data of the same entity. The exchange scheduling module compiles the subscription conditions of data consumers into Boolean expression trees for condition matching, calculates distribution task priorities, and distributes data according to specified formats and protocols through a publishing engine.

[0158] Optionally, the system further includes an incremental synchronization module and a security control module. The incremental synchronization module detects data changes through content hash comparison during each synchronization cycle and generates incremental data packets using differential encoding. These incremental data packets are processed by the mapping engine module and the quality verification module before being distributed by the exchange scheduling module. The security control module performs permission verification on distribution requests based on a role-based permission matrix, performs dynamic de-identification processing on sensitive fields, and records end-to-end operation audit logs in an append-only manner.

[0159] The hash chain integrity verification mechanism is further explained below. In addition to using independent hashes for comparing individual records, the change detection model maintains a hash chain structure for integrity verification of the entire dataset. The extended hash of each record in the hash chain contains not only its own content but also the extended hash value of the previous record, forming a chain-like dependency. The last value of the hash chain (the tail hash) serves as the integrity check code for the entire dataset; any tampering with an intermediate record will cause the tail hash to change. After each incremental synchronization, the tail hash is transmitted to the audit log system of the security control module for recording, serving as evidence of the overall integrity of that batch of datasets. This mechanism ensures that even if an attacker replaces the content and hash value of a single record, the tampering can be detected as long as the tail hash does not match the value recorded in the audit log.

[0160] Each record in the audit log is stored in JSON line format and includes eight required fields: timestamp, operator identifier, operation type (query, distribution, or configuration change), target data entity identifier, request field list, authorization field list, denial field list, and operation result (success, partial access, or denial). The log file is split by date, with one file per day, facilitating quick retrieval by date range.

[0161] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for sharing and exchanging urban information data based on a CIM platform, characterized in that, include: Load the urban information domain ontology graph and register multiple data sources, generate a data source descriptor for each data source and instantiate the corresponding standardized adapter; A unified semantic data model is constructed based on the domain ontology graph. The unified semantic data model includes multiple urban information entity classes and their standardized attribute sets extracted from the domain ontology graph. The source data of each data source is obtained through the standardized adapter. A multi-index weighted mapping model is used to calculate the mapping score between each source data field and the candidate target field in the unified semantic data model. The source data fields with mapping scores exceeding the mapping confidence threshold are mapped to the corresponding target fields, and the mapped data is subjected to unit normalization transformation. Multidimensional quality verification is performed on the mapped data. The comprehensive quality score of each data record is calculated according to the multidimensional quality scoring model. Data records whose quality scores reach the preset threshold are entered into conflict resolution. The weighted fusion conflict resolution model is used to fuse the multi-source data of the same entity to obtain the fused dataset. Based on the subscription conditions of the data consumer, condition matching is performed on the fused dataset to generate a distribution task, and the matched data is distributed according to the output format and protocol specified by the data consumer.

2. The method according to claim 1, characterized in that, The construction of a unified semantic data model based on the domain ontology graph includes: Starting from a predefined seed set of core urban information concepts, the system traverses along hierarchical relationships in the domain ontology graph to collect candidate entity classes and their attribute nodes, and generates a standardized attribute set for each candidate entity class, which includes field names, data types, measurement units, and encoding rules. Based on the spatial geometric information and semantic labels of the candidate entity classes, a spatial semantic joint index is constructed.

3. The method according to claim 2, characterized in that, The spatial semantic composite index includes a first-level spatial range index and a second-level semantic label inverted index. When performing a composite query, the spatial candidate set is first obtained through the first-level spatial range index, and then the second-level semantic label inverted index is used to perform semantic filtering on the spatial candidate set to obtain the query result.

4. The method according to claim 1, characterized in that, The mapping score of the multi-index weighted mapping model is obtained in the following way: For the source data field and the candidate target field, calculate the name similarity index, type compatibility index and ontology semantic distance index respectively. Then, sum the three indices according to the preset weights to obtain the mapping score.

5. The method according to claim 4, characterized in that, The ontology semantic distance metric is obtained by calculating the nearest common ancestor depth of the concept node to which the source data field belongs and the concept node to which the candidate target field belongs in the domain ontology graph, and calculating the ratio of the nearest common ancestor depth to the sum of the depths of the two concept nodes.

6. The method according to claim 4, characterized in that, The unit normalization transformation includes: performing a linear transformation on the mapped source data values ​​according to a predefined unit transformation matrix, wherein each entry in the unit transformation matrix defines a scaling factor and offset from the source units to the target units.

7. The method according to claim 1, characterized in that, The comprehensive quality score of the multidimensional quality scoring model is obtained through the following methods: The sub-scores for the four dimensions of completeness, format compliance, scope reasonableness, and consistency are calculated separately. The sub-scores of the four dimensions are then weighted and summed according to the preset dimension weights to obtain the comprehensive quality score.

8. The method according to claim 7, characterized in that, The weighted fusion conflict resolution model fuses multi-source data for the same entity, including: For numerical attributes, the product of the static credibility weight of each data source and the data freshness decay factor is used as the fusion weight. The attribute values ​​provided by multiple data sources are weighted and averaged to obtain the fused attribute value. For string-type attributes, the attribute value provided by the data source with the largest fusion weight is selected as the fusion attribute value.

9. The method according to claim 8, characterized in that, The data freshness decay factor is an exponential decay function with the data update time difference as the independent variable.

10. A city information data sharing and exchange system based on a CIM platform, characterized in that, include: The initialization module is used to load the ontology graph of the urban information domain, register the data source, and instantiate the standardized adapter; The semantic modeling module is used to construct a unified semantic data model based on the domain ontology graph; The mapping engine module is used to calculate the mapping score between the source data fields and the target fields in the unified semantic data model using a multi-index weighted mapping model, and to perform semantic mapping and unit normalization transformation. The quality verification module is used to perform multi-dimensional quality verification and weighted fusion conflict resolution on the mapped data; The exchange scheduling module is used to perform matching based on the subscription conditions of the data consumer and distribute the data to the data consumer.

Images