Smart city management platform integrating tri-modal data

Abstract

The invention discloses a smart city management platform for convergence and fusion of trimodal data, and relates to the technical field of urban management. The platform proposes a dynamic regional logic database method formed by convergence and fusion of urban trimodal data based on semantic logic, and the advantages are: a new unified semantic-based organization method for trimodal data is proposed, and a new unified model of structured data, text data, and visual data is constructed; the existing system fails to realize the construction of a unified model of the three modal data; the dynamic regional logic database method formed by convergence and fusion of trimodal data based on semantic logic can represent relational data, data association, and data blood relationship at the semantic level; compared with the single management relationship of the current system, the data semantic expression is more, which is convenient for retrieval and processing; the dynamic regional logic database is organized according to the logical relationship of the data, which reduces the demand for data storage and reduces the redundancy of the data.

Description

Technical Field

[0001] The present invention relates to the field of urban management technology, and more specifically, to a smart city management platform for converging and fusing tri-modal data. Background Art

[0002] Digital cities and smart services are the main directions of development of smart city management in my country. "Data aggregation, three-dimensional integration, security and reliability, smart services, and rapid response" are the goals of building a smart city management platform that aggregates and integrates tri-modal data.

[0003] There are three main modalities of urban data: structured data modality, visual modality (such as video, image) and text modality (official documents, files, signs). Urban structured data is stored in the database in the form of industry and community grids, and the latter two modalities are often mixed in unstructured data. Static and dynamic information such as urban personnel, vehicles, signs and ecology is stored in visual data files, and urban management policies and regulations are mainly in official documents and files.

[0004] Smart urban management platforms represented by manufacturers such as Zhejiang Dahua, Qingdao Zhuoer, and Beijing Tongchuang all rely on information technology support such as primary Internet of Things, mobile technology, and cloud computing to achieve a "one-picture" management of urban management personnel, work processes, work standards, work performance, and comprehensive evaluation. Most urban management platforms have built the following basic subsystems, or their functions are included in the following subsystems:

[0005] 1. Wireless collection subsystem, collecting and verifying information on urban management issues;

[0006] 2. Call acceptance subsystem, receive information on urban management issues, and generate case records;

[0007] 3. Collaborative work subsystem, providing task dispatch, task processing, task feedback, case verification and closure for the supervisory command center and professional departments;

[0008] 4. Supervise and command the subsystem, collect business handling information, comprehensive evaluation information, etc.;

[0009] 5. Basic data resource system to carry out spatial and digital management of urban components;

[0010] 6. Application maintenance subsystem: configure basic information of urban management business, management personnel information, and organizational structure;

[0011] 7. Data exchange subsystem: provides guarantee for information sharing between various systems.

[0012] These smart city management platforms mainly realize collaborative management of various departments through data sharing and exchange, and mainly adopt business collaborative linkage mode in important urban data source basic data resource systems, video surveillance subsystems, outdoor advertising management systems, storm drainage monitoring systems, garden management subsystems and other systems. They have not yet achieved high-quality, intelligent aggregation and integration of basic data and urban management data. Therefore, urban management based on this type of system has not yet formed intelligent urban management command and decision-making, nor has it formed a new pattern of integrated urban management and services. The main work flow is that decision makers manually form management decisions based on information prompts from the system (or platform) and urge relevant departments to work together to achieve the goal of urban management.

[0013] The main shortcomings of current technology are as follows:

[0014] The data aggregation of existing platforms is mainly the physical aggregation of data, that is, the data of various business systems are aggregated in the (big) data center, and statistical analysis is performed in the (big) data center to form a basis for decision support. Its disadvantage is that the logical aggregation of various types of data is not achieved, that is, the internal logical relationship between various types of data is not established during aggregation.

[0015] Urban data is mainly stored in various systems in three modes: structured data mode, visual mode (such as video, image) and text mode (official documents, files, logos). The existing platforms have not fully realized the cross-platform semantic fusion of tri-modal data, and have failed to realize the urban space definition and perception capabilities based on tri-modal data.

[0016] In the unified command of urban management, decision makers need to integrate the analysis results of various types of data, make decisions based on their personal knowledge and experience, and formulate management plans. It is difficult to achieve refined and accurate decision-making. The existing urban management platform cannot form a comprehensive and refined solution. Summary of the invention

[0017] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a smart city management platform that aggregates and integrates tri-modal data.

[0018] To achieve the above object, the present invention provides the following technical solutions:

[0019] A smart city management platform that aggregates and integrates tri-modal data, including a data aggregation module, a data semantic fusion module, and a smart city dynamic area management module;

[0020] The data aggregation module logically aggregates three types of spatiotemporal data, namely structured data, visual data and text data of various existing business systems to construct a regional logical database;

[0021] The data semantic fusion module obtains the relevant original business structured database according to each matter of regional city management, and uses the metadata in the original business system to build a mesh-shaped regional logical database based on regional city management data. This database becomes a regional logical database based on the structured data of each business system in the trimodal data;

[0022] The urban dynamic area smart management module is used to construct an urban dynamic area smart management architecture, which includes a basic layer, an urban management operation layer, and an application layer.

[0023] Furthermore, the region can be determined in three ways: (1) it can be designated according to administrative regions; (2) it can be circled on an electronic map; (3) the central location can be determined first, and then the scope of the region can be determined;

[0024] In the platform, according to the urban management matters, the relevant data of each matter and its related original business database are determined. According to the logical relationship of the relevant data, a tree-shaped data logic model of the data is constructed. The tree-shaped logic model is organized in layers and refined layer by layer to represent the city terminal data. Each tree-shaped data logic model corresponds to the data involved in the urban management matters in a region and its logical relationship. The tree-shaped data logic model of each matter of urban management in the region constitutes a data logic model forest of regional management matters. According to the data association of urban management matters, the node connection is established, and the root node of each tree is connected with a mesh model to form a regional logical database (RLDB).

[0025] Furthermore, RLDB allows nodes to have more than one parent node. RLDB removes the restrictions in the hierarchical model, allowing multiple nodes to have no parent nodes, allowing nodes to have multiple parent nodes, and allowing many-to-many connections between nodes. The three elements of RLDB are:

[0026] (1) Data structure: Use node names to describe entities layer by layer, use records in a relational database to represent entity data, and use attributes to describe data items;

[0027] (2) Data operation: hierarchical access, branch access, and hyperlink access can be used;

[0028] (3) Integrity constraint: Each node in RLDB is used to uniquely identify the set of data items recorded in that layer.

[0029] Furthermore, the key technologies in RLDB logic design are:

[0030] (1) Determine the data aggregation granularity and the division of granularity levels. The aggregation granularity is actually the granularity of each layer of nodes. The design principle is to use different granularities in different layers. A coarser granularity description is used at a higher level, and then refined layer by layer. The leaf node is the finest granularity data and is directly described by data item data.

[0031] (2) Determine the establishment of data logic relationships and data logic segmentation strategies. This is the key content of data aggregation from the original business system to the RLDB. First, according to the city management matters, formulate the data segmentation standards for the matters, mainly considering the following factors: data subject, data volume, actual situation of data analysis and processing, simplicity and ease of implementation, and granularity division strategy. Among them, the data subject is the basis for establishing the relationship between data logics. RLDB takes the content of the matter as the data subject; data volume is an important factor in data segmentation. According to the amount of data involved in the actual matter and the relationship between the data, determine the data logic segmentation strategy;

[0032] (3) Construct a logical model for RLDB aggregation and determine the data hierarchy and parent nodes based on the relationships between the data involved in the actual event.

[0033] Furthermore, the realization of data ontology semantic fusion is mainly carried out from the following three aspects:

[0034] (1) Mapping and transformation of attribute features based on urban management rule files: Content-based text modal data ontology semantics (OSoTMD) can be obtained through three-level processing at the vocabulary level, syntax level and text level. The processing of the vocabulary that constitutes the text at the vocabulary level includes part-of-speech recognition, proper noun recognition, word importance annotation, synonym annotation, and position relationship annotation; the processing of the text at the syntax level includes syntactic structure analysis, topic model construction, knowledge graph and word vector analysis; the processing of the text at the text level includes key information extraction, text content semantics extraction, prohibited content analysis, and sentiment analysis; thereby obtaining the text ontology semantics at the text level; therefore, OSoTMD can be composed of a four-tuple of key information (KI), text content semantics (TCS), prohibited content (PC), and sentiment tendency (ET), that is: OSoTMD = {KI, TCS, PC, ET};

[0035] The KI, TCS, PC and ET information in the tuple can be obtained through natural language processing related methods. The first urban management rule document is marked with its applicable scope and time span. Each urban management item has specific semantic constraints. A set of urban management item signal words (mapping attribute features in RLDB) is constructed. Based on this, specific semantic constraints of the text in the urban management document are detected. The key issue here is that the same signal word may be used as a signal word for different semantic relationship constraints. The signal word and the semantics are networked. For the problem of detecting multiple semantic constraints in the text of urban management documents, and the key issue of multiple semantic constraints indicated by the same signal word, a method based on building a dependency tree is adopted to define the dependency relationship of each item in the urban management area. The dependency tree (PDT) kernel function treats all the relevant city management rule file texts as a long string structure, and parses it through the NLP processor to obtain a dependency tree. The dependency tree is represented by a triple of <signal word, dependency tree, semantic word>. Therefore, the content of the city management file text rules is converted into a dependency tree set based on information words, thus constructing a dependency tree set of regional management rules based on city management rule files. Since signal words may be used as signal words for different semantic relationship constraints, the dependency tree set of regional management rules based on information words is also a tree-like mesh structure, which constitutes an isomorphism and mapping relationship with RLDB, and becomes a regional logical database based on city management rule files in trimodal data;

[0036] (2) Semantic attribute fusion of mapping transformation rules for urban management visual modality data: In the relevant business system, the external spatiotemporal data of visual modality and text modality are stored in the database system of the system in a structured manner. The semantics of these data ontology can be directly fused and calculated through E-R;

[0037] The content-based visual modality data ontology semantics (OSoVDC) can be described by a four-tuple, visual data file name (FNoVD), key frame (KF), key frame semantics (KFS), key frame number (KFN), OSoVDC = {FNoVD, KF, KFS, KFN};

[0038] The FNoVD, KF, KFS and KFN information in the tuple can be obtained through hierarchical visual information annotation and video or image processing; in the visual information annotation stage, the hierarchical visual information semantic annotation uses the external semantic description, global features, local features and content features of the visual information for comprehensive processing, and classifies the image into a specific scene according to the semantic features of the external structured description of the image to be annotated, and finds the scene semantic tree corresponding to the scene; in the visual information processing stage, under the guidance of the external features of the annotated visual information, the local visual information is segmented, and the content features of the target are extracted respectively, and its semantics based on the local content is annotated according to the urban management features and the semantic mapping table; the visual information processing unit walks in the scene semantic tree until a leaf node, and the set of semantic annotation words of all nodes that the visual information passes through in the scene semantic tree constitutes a scene-based visual information semantic annotation tree. For the scene-based visual information semantic annotation tree, the root node of the annotation tree is used as the vertex structure of the mesh database to constitute a regional visual information logical database, which constitutes an isomorphism and mapping relationship with RLDB, and becomes a visual information modal regional logical database in trimodal data;

[0039] (3) Attribute fusion based on the semantics of urban trimodal data ontology

[0040] The ontological semantics of the three modalities exist in each system database;

[0041] Ontology-based semantic data attribute similarity refers to the semantic similarity of concepts in the metadata description ontology;

[0042] The attributes of trimodal data ontology are divided into data type attributes and object type attributes. The semantic similarity calculation of attributes uses the overlapping degree of ontology attributes to measure the semantic similarity. It adopts a cross-ontology semantic similarity calculation method and comprehensively considers the position information and attribute information of concept words, including synonym sets, semantic neighbor nodes and feature attribute items.

[0043] Furthermore, the basic layer includes the urban infrastructure perception layer, the smart city management transmission layer, and the general system and business platform layer.

[0044] Furthermore, the urban infrastructure perception layer consists of three layers:

[0045] (1) Sensor equipment technology layer: including ubiquitous sensor networks, RFID, collection equipment, aggregation equipment, content security acquisition equipment, etc.;

[0046] (2) Perception technology layer: including perception modeling technology, dynamic perception technology, observation and navigation technology, trusted acquisition technology, etc.

[0047] (3) Perception system layer: including urban infrastructure perception system, spatial information perception system, urban three-dimensional perception network, environmental and disaster monitoring perception, vehicle-mounted perception system, smart city three-dimensional perception, etc.;

[0048] The transport layer for smart city management consists of three layers:

[0049] (1) Communication technology layer: mainly includes wireless broadband network, optical transmission network, electronic transmission network, etc.;

[0050] (2) Transmission control technology layer: mainly includes network service quality control technology, trusted, controllable and manageable network technology, network and information security technology, etc.

[0051] (3) Transmission control layer for smart city management: urban control products for smart city management, basic application technologies for smart city management, transmission control technologies dedicated to smart city management, etc.

[0052] The general system and business platform layer consists of two layers:

[0053] (1) General system layer: The general operating system layer consists of various operating systems that support the operation of computer systems, as well as various resource management management systems built on the operating systems, such as cloud platforms and big data platforms;

[0054] (2) Business platform layer: a business platform that supports various business management and business operations.

[0055] Furthermore, the urban management operation layer is divided into four layers:

[0056] (1) Basic data layer: business data of various business systems, monitoring data and urban management data of the public security system, urban management rules and documents of government departments, urban community grid data, and related data obtained from the Internet;

[0057] (2) Data operation layer: There are three-modal data annotation, data governance, data logic aggregation, data fusion, urban dynamic regional logic database construction and regional data activation based on the basic data layer. Regional data activation mainly refers to data description and cognition technology, data maintenance and management technology, data association growth and security technology;

[0058] (3) AI-assisted decision-making layer: data gateway with data security control, spatiotemporal data engine for building data spatiotemporal association, data rule mining, comparative analysis of data rules and urban management rules, so that data can be deeply penetrated across industries and systems, generate new energy efficiency, serve to build the urban management brain, form urban area smart management action plans, and make recommendations;

[0059] (4) Decision-making layer: Provide a decision-making support interactive interface, visualize relevant data, conduct human-computer interactive analysis of the regional action plan provided by the AI-assisted decision-making layer, form a preliminary regional action plan, and conduct sandbox simulations of the regional action plan.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] The present invention proposes a dynamic regional logical database method formed by the convergence and fusion of urban trimodal data based on semantic logic, and its advantages are: a new unified semantic-based organization method for trimodal data is proposed, and a new unified model of structured data, text data, and visual data is constructed; the existing system fails to realize the construction of a unified model of the three modal data; the dynamic regional logical database method formed by the convergence and fusion of trimodal data based on semantic logic can represent relational data, data association, and data blood relationship at the semantic level; compared with the single management relationship of the current system, the data semantic expression is more, which is convenient for retrieval and processing; the dynamic regional logical database is organized according to the logical relationship of the data, which reduces the demand for data storage and reduces data redundancy. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flowchart of the smart city management platform for the convergence and fusion of tri-modal data of the present invention;

[0063] Figure 2 This is a diagram of the urban dynamic regional smart management architecture. DETAILED DESCRIPTION

[0064] Example 1

[0065] Reference Figure 1-Figure 2 Smart city management (referred to as "smart urban management") makes full use of technologies such as the Internet of Things, cloud computing, big data, artificial intelligence, and the Internet, with the aim of effectively serving the people's livelihood. Through resource integration, means innovation, and function expansion, it creates asset management, public services, industry supervision, collaborative office, non-site law enforcement, law enforcement, integrity management, decision analysis, and comprehensive command as the core, innovates management methods, technical methods, and management models, realizes the full digitization and intelligence of urban management elements, urban management processes, and urban management decisions, promotes the improvement of urban management public service capabilities, and promotes the transformation of urban management work from passive to active, static to dynamic, extensive to refined, and disorderly to standardized.

[0066] The workflow of the smart city management platform for the convergence and fusion of trimodal data is shown in the figure below: Figure 1 , mainly building a new generation of smart city management platform from the following five aspects.

[0067] Data aggregation

[0068] Logically aggregate the three types of spatiotemporal data, namely structured data, visual data and text data from various existing business systems to build a regional logical database, where regions can be determined in three ways:

[0069] (1) Designated according to administrative region.

[0070] (2) The area can be circled on an electronic map.

[0071] (3) You can first determine the central location and then determine the regional scope.

[0072] In the platform, according to the urban management matters, the relevant data of each matter and its related original business database are determined. According to the logical relationship of the relevant data, a tree-shaped data logic model of the data is constructed. The tree-shaped logic model is organized in layers and refined layer by layer to represent the city terminal data. Each tree-shaped data logic model corresponds to the data involved in the urban management matters in a region and its logical relationship. The tree-shaped data logic model of each matter of urban management in the region constitutes a data logic model forest of regional management matters. According to the data association of urban management matters, the node connection is established, and the root node of each tree is connected with a mesh model to form a regional logical database (Regionalllogic database-RLDB).

[0073] RLDB allows nodes to have more than one parent node. This design can meet the correlation between data and multiple events. RLDB removes the restrictions in the hierarchical model, allowing multiple nodes to have no parent nodes, allowing nodes to have multiple parent nodes, and allowing many-to-many connections between nodes. The three elements of RLDB are:

[0074] (1) Data structure: Use node names to describe entities layer by layer, use records in the relational database to represent entity data, and use attributes to describe data items. This makes it easy to correspond the relational data of the original business system with the RLDB and build logical relationships.

[0075] (2) Data operation: hierarchical access, branch access and hyperlink access can be used, making node query and update more convenient.

[0076] (3) Integrity constraints: Each node in RLDB is used to uniquely identify the set of data items recorded in this layer, which can ensure that there is a one-to-many relationship between parent nodes and child nodes, and at the same time support constraints between parent records and child records.

[0077] RLDB has good performance and high access efficiency. Compared with the hierarchical model, the connection between nodes in RLDB is flexible, which can represent the complex connection between things and is more suitable for describing the objective world. Since the entities are represented by access paths, applications also need to specify the planned access path when accessing data.

[0078] In RLDB logic design, there are three key technologies:

[0079] (1) Determine the data aggregation granularity and granularity level division. The aggregation granularity is actually the granularity of each node layer. The design principle is to use different granularities in different layers. A coarser granularity description is used at a higher level, and then refined layer by layer. The leaf node is the finest granularity data, which is directly described by data item data. This design realizes the expression of hierarchical logical relationships from data subject to data content, and can also realize operations of different granularities, thereby improving the speed of complex queries.

[0080] (2) Determine the establishment of data logic relationships and data logic segmentation strategies. This is the key content of data aggregation from the original business system to the RLDB. First, according to the city management matters, formulate the data segmentation standards for the matters, mainly considering the following factors: data subject, data volume, actual situation of data analysis and processing, simplicity and granularity division strategy, etc. Among them, the data subject is the basis for establishing the relationship between data logics. RLDB takes the content of the matter as the data subject; data volume is an important factor in data segmentation. According to the amount of data involved in the actual matter and the relationship between the data, determine the data logic segmentation strategy.

[0081] (3) Construct a logical model for RLDB aggregation and determine the data hierarchy and parent nodes based on the relationships between the data involved in the actual event.

[0082] Data semantic fusion

[0083] According to each matter of regional urban management, the relevant original business structured database is obtained, and the metadata in the original business system is used to build a network-shaped regional logical database based on regional urban management data. This database becomes a regional logical database based on the structured data of each business system in the trimodal data.

[0084] Since RLDB aggregates existing business system data based on city management matters to form a logical model of data;

[0085] Therefore, data semantic fusion is a key technology for building a logical model of data. The commonly used method of data fusion is data ontology semantic fusion. The components of data ontology include: individuals (instances), classes, attributes, relationships, function terms, constraints (restrictions), rules, axioms, and events. Ontology contains four basic meanings: conceptual model, clarification, formalization, and sharing.

[0086] The realization of data ontology semantic fusion is mainly carried out from the following three aspects:

[0087] (1) Mapping and conversion based on the attribute characteristics of urban management rule files

[0088] Content-based ontology semantics of text modal data (OSoTMD) can be obtained through three-level processing at the lexical level, syntactic level and text level. At the lexical level, the processing of the words that make up the text includes part-of-speech recognition, proper noun recognition, word importance tagging, synonym tagging, and position relationship tagging; at the syntactic level, the processing of the text includes syntactic structure analysis, topic model construction, knowledge graph and word vector analysis; at the text level, the processing of the text includes key information extraction, text content semantics extraction, prohibited content analysis, and sentiment analysis; thus, the text ontology semantics is obtained at the text level. Therefore, OSoTMD can be composed of the four-tuple of key information (KI), text content semantics (TCS), prohibited content (PC), and sentiment tendency (ET), that is, OSoTMD = {KI, TCS, PC, ET}.

[0089] The KI, TCS, PC and ET information in the tuple can be obtained through natural language processing related methods. The first urban management rule document is annotated with its applicable scope and time span. Each urban management item has specific semantic constraints, and a set of urban management item signal words (mapping attribute features in RLDB) is constructed to detect specific semantic constraints of the text in the urban management document. The key issue here is that the same signal word may be used as a signal word for different semantic relationship constraints, and the signal word and semantics present a mesh posture. In order to detect multiple semantic constraints in the text of urban management documents, and the key issue of multiple semantic constraints indicated by the same signal word, a method based on dependency tree is adopted to define the dependency tree (PDT) kernel function of each item in the urban management area, and the relevant urban management rule file text is treated as a long string structure. After parsing through the NLP processor, a dependency tree is obtained. The dependency tree is represented by a triple of <signal word, dependency tree, semantic word>. Therefore, the content of the urban management file text rules is converted into a dependency tree set based on information words. In this way, a dependency tree set of regional management rules based on urban management rule files is constructed. Since signal words may be used as signal words for different semantic relationship constraints, the dependency tree set of regional management rules based on information words is also a tree-like mesh structure, which constitutes an isomorphism and mapping relationship with RLDB, and becomes a regional logical database based on urban management rule files in trimodal data.

[0090] (2) Semantic attribute fusion of mapping transformation rules for urban management visual modality data

[0091] In the relevant business system, the external spatiotemporal data of visual and textual modalities (such as time, location format, size, data generation method, etc.) are stored in the system's database system in a structured manner. These data ontology semantics may be directly fused and calculated through E-R.

[0092] The ontology semantics of visual data content (OSoVDC) can be described by a four-tuple: visual data file name (File name VisualData-FNoVD), key frames (Key Frames-KF), key frame semantics (Key Frame Semant i cs-KFS), key frame number (Key Frame Number-KFN)}, OSoVDC = {FNoVD, KF, KFS, KFN}.

[0093] The FNoVD, KF, KFS and KFN information in the tuple can be obtained through hierarchical visual information annotation and video or image processing. In the visual information annotation stage, hierarchical visual information semantic annotation uses the external semantic description, global features, local features and content features of visual information to comprehensively process, and classifies the image into a specific scene according to the external structured description semantics (time, space, type, etc.) features of the image to be annotated, and finds the scene semantic tree corresponding to the scene. In the visual information processing stage, under the guidance of the external features of the annotated visual information, the local visual information is segmented, and the content features of the target are extracted respectively. According to the urban management features and the semantic mapping table, its semantics based on the local content are annotated; the visual information processing unit wanders in the scene semantic tree until a leaf node. The set of semantic annotation words of all nodes passed by the visual information in the scene semantic tree constitutes a scene-based visual information semantic annotation tree. For the scene-based visual information semantic annotation tree, the root node of the annotation tree is used as the vertex structure of the mesh database to form a regional visual information logical database, which constitutes an isomorphism and mapping relationship with RLDB and becomes a visual information modal regional logical database in the trimodal data.

[0094] (3) Attribute fusion based on the semantics of urban trimodal data ontology

[0095] Urban management matters have very clear ontological semantics, and the ontological semantics of the three modalities exist in each system database. Since urban business systems are built independently, they all have construction standards, but there are many differences between the standards. The metadata ontological semantics are seriously heterogeneous. If we simply use manual rule sets to achieve data semantic fusion, the workload is very large. Using the three modal ontological semantic similarity calculation can reduce the workload of data semantic fusion.

[0096] Although the semantic heterogeneity of trimodal data is serious and there are many differences between the standards, in each business data system, it is a structured and unambiguous knowledge representation ontology of the business domain concepts, which is an abstract description of the business domain knowledge. The semantic data attribute similarity based on the ontology refers to the semantic similarity of the concepts in the metadata description ontology.

[0097] The ontology attributes of trimodal data are divided into data type attributes and object type attributes. The semantic similarity calculation of attributes is to measure the semantic similarity by using the overlap degree of ontology attributes. In order to more accurately calculate the similarity of ontology attributes, the cross-ontology semantic similarity calculation method is usually adopted, that is, the position information and attribute information of concept words are comprehensively considered, including synonym sets, semantic neighbor nodes and characteristic attribute items. If the synonym set of a concept and the adjacent concept set of an interpretation word are similar in lexical terms, then the similarity of the two concepts is relatively high. The relationship between upper and lower, part-whole, etc. can be fully utilized to construct a context vector to measure the information of the concept context. The cosine value of the context vector of the two concepts is taken to calculate the similarity of the ontology attributes. The hierarchical structure of the metadata ontology attributes is very clear, so the concept structure knowledge can be used as the attribute feature. The concept structure knowledge accounts for about 80% of the ontology knowledge. Adding this effective feature can improve the accuracy of the trimodal data ontology attribute similarity calculation.

[0098] Construction of the “brain” for smart management of urban dynamic areas

[0099] Urban management is based on the structured data and visual data of existing business systems, guided by urban management rules and documents, based on urban trimodal data, supported by new generation information technology such as the Internet of Things, cloud computing, big data and artificial intelligence, and using planning, organization, decision-making, and command mechanisms, through the interaction of government, market and society, to provide refined management and precise services for cities.

[0100] There are many differences in the basic attributes of urban areas. The refined management of cities is achieved through the refinement of urban area management. Therefore, building a city management brain is actually building a new model of urban regional management that can automatically adapt to various regions.

[0101] Three characteristics of the “brain” of urban dynamic regional smart management:

[0102] (1) Data security and trustworthiness based on data gateways: Data gateways enable data communication between different application systems, open up data paths between different communication protocols, and perform functions such as data analysis, storage, and forwarding. Based on data ownership and user permissions, they provide an open, reliable, secure, high-performance, and scalable integrated framework for business data interaction and secure and trustworthy management.

[0103] (2) Smart services based on refined management and precise services: Guided by social needs and based on the text of urban management rules, and based on urban community grid data, relevant business system data, public security data, and Internet data, a dynamic urban regional logical network database is constructed. By using spatiotemporal data engines, data statistics, blood relationship analysis, rule mining and other technologies, data empowerment is achieved, forming a dynamic regional management brain for the city, generating smart service work plans for refined management and precise services, and recommending them to decision makers.

[0104] (3) Rapid response based on fingertip command: Utilize data visualization, human-computer interaction, digital sandbox and other technologies to support regional management plan analysis and deduction, quickly form urban area management implementation plans, and realize fingertip command.

[0105] 1. Urban infrastructure is divided into three categories according to the nature of its services:

[0106] (1) Production infrastructure: including water supply, power supply, roads and transportation facilities, storage equipment, postal and telecommunications facilities, sewage disposal, greening and other environmental protection and disaster prevention facilities serving the production sector.

[0107] (2) Social infrastructure: refers to various institutions and facilities that serve residents, such as commerce and catering, service industries, financial and insurance institutions, housing and public utilities, public transportation, transportation and communication institutions, education and health institutions, cultural and sports facilities, etc.

[0108] (3) Institutional support agencies: such as public security, political and legal, and urban construction planning and management departments.

[0109] The level of infrastructure continues to improve with the development of economy and technology, with more types and more complete services.

[0110] Urban infrastructure perception relies on the Internet of Things technology to obtain holographic data of the city. Through various information sensors, radio frequency identification technology, global positioning systems, infrared sensors, laser scanners and other devices and technologies, it collects real-time data on any objects or processes that need to be monitored, connected, and interacted with, and collects various required information such as sound, light, heat, electricity, mechanics, chemistry, biology, and location. Perception refers to the perception and sensitivity to the attribute status of things and the way they change.

[0111] 2. Urban infrastructure perception mainly consists of three layers

[0112] (1) Sensor equipment technology layer: mainly includes ubiquitous sensor networks, RFID, collection equipment, aggregation equipment, content security acquisition equipment, etc.

[0113] (2) Perception technology layer: mainly includes perception modeling technology, dynamic perception technology, observation and navigation technology, trusted acquisition technology, etc.

[0114] (3) Perception system layer: mainly includes urban infrastructure perception system, spatial information perception system, urban three-dimensional perception network, environment and disaster monitoring perception, vehicle-mounted perception system, smart city three-dimensional perception, etc.

[0115] 3. Transmission for smart city management can be composed of three layers

[0116] Communication technology layer: mainly includes wireless broadband network, optical transmission network, electronic transmission network, etc.

[0117] Transmission control technology layer: mainly includes network service quality control technology, trusted, controllable and manageable network technology, network and information security technology, etc.

[0118] Transmission control layer for smart city management: urban control products for smart city management, basic application technologies for smart city management, transmission control technologies dedicated to smart city management, etc.

[0119] 4. The general system and business platform layer can be composed of two layers

[0120] (1) General system layer: The general operating system layer consists of various operating systems that support the operation of computer systems, as well as various resource management management systems built on the operating systems, such as cloud platforms and big data platforms.

[0121] (2) Business platform layer: a business platform that supports various business management and business operations.

[0122] The urban management operation layer can be divided into four layers:

[0123] (1) Basic data layer: mainly includes business data of various business systems, monitoring data and urban management data of the public security system, urban management rules and documents of government departments, urban community grid data, and related data obtained from the Internet (such as public opinion data, citizens' spontaneous activity behavior data, etc.).

[0124] (2) Data operation layer: mainly includes trimodal data annotation, data governance, data logic aggregation, data fusion, urban dynamic regional logic database construction and regional data activation based on the basic data layer. Regional data activation mainly refers to data description and cognition technology, data maintenance and management technology, data association growth and security technology.

[0125] (3) AI-assisted decision-making layer: mainly includes data gateways for data security control, spatiotemporal data engines for building data spatiotemporal associations, data rule mining, comparative analysis of data rules and urban management rules, and enabling data to penetrate deeply across industries and systems to generate new energy efficiency, serve to build the urban management brain, form urban area smart management action plans, and make recommendations.

[0126] (4) Decision-making layer: Provide decision-making support interactive interfaces, visualize relevant data, conduct human-computer interaction analysis on the regional action plans provided by the AI-assisted decision-making layer, form a preliminary regional action plan, and conduct sandbox simulations on the regional action plan to form a regional action execution plan with the goal of refined management. According to the execution plan, implement resource allocation and rapid action scheduling to achieve precise services.

[0127] Application Layer

[0128] According to the resource allocation and rapid action scheduling of the implementation plan, relevant business departments work together to implement comprehensive management of urban areas within the specified time and scope, and conduct comprehensive evaluation of the effectiveness of management.

[0129] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0130] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0131] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0136] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A smart city management platform that integrates tri-modal data, characterized by: It includes data aggregation module, data semantic fusion module, and urban dynamic area intelligent management module; The data aggregation module logically aggregates three types of spatiotemporal data, namely structured data, visual data and text data of various existing business systems to construct a regional logical database; The data semantic fusion module obtains the relevant original business structured database according to each matter of regional city management, and uses the metadata in the original business system to build a mesh-shaped regional logical database based on regional city management data. This database becomes a regional logical database based on the structured data of each business system in the trimodal data; The realization of data ontology semantic fusion is carried out from the following three aspects: (1) Mapping and transformation based on the attribute characteristics of urban management rule files: The content-based text modal data ontology semantics OSoTMD can be obtained through three-level processing at the lexical level, syntactic level and text level. OSoTMD includes key information KI, text content semantics TCS, prohibited content PC, and emotional tendency ET, that is: OSoTMD = {KI, TCS, PC, ET}; The KI, TCS, PC and ET information in the tuple are obtained through natural language processing related methods. The first urban management rule file is annotated with its applicable scope and time span. Each urban management item has specific semantic constraints. A set of urban management item signal words is constructed to detect the specific semantic constraints of the text in the urban management file. The same signal word can be used as a signal word for different semantic relationship constraints. The signal word and the semantics present a mesh posture. The method of building a dependency tree is adopted to define the dependency tree kernel function of each item in the urban management area. The relevant urban management rule file text is treated as a long string structure. The dependency tree is obtained by parsing it through the NLP processor. The dependency tree is represented by a triple of <signal word, dependency tree, semantic word>. The urban management file text rule content is converted into a dependency tree set based on information word to construct a dependency tree set of regional management rules based on urban management rule files. Since signal words can be used as signal words for different semantic relationship constraints, the dependency tree set of regional management rules based on information word is also a tree-like mesh structure, which constitutes an isomorphism and mapping relationship with RLDB. (2) Semantic attribute fusion of mapping transformation rules for urban management visual modality data; (3) Attribute fusion based on the semantics of urban trimodal data ontology; The urban dynamic area smart management module is used to construct an urban dynamic area smart management architecture, which includes a basic layer, an urban management operation layer, and an application layer.

2. The smart city management platform for tri-modal data convergence and fusion according to claim 1 is characterized in that: Regions can be determined in three ways: (1) by designating them according to administrative areas; (2) by circling them on electronic maps; (3) by first determining the central location and then determining the scope of the region; In the platform, according to urban management matters, the relevant data of each matter and its related original business database are determined. According to the logical relationship of the relevant data, a tree-type data logic model of the data is constructed. The tree-type data logic model is organized in layers and refined layer by layer to represent the city terminal data. Each tree-type data logic model corresponds to the data involved in urban management matters in a region and its logical relationship. The tree-type data logic model of each matter of urban management in the region constitutes a data logic model forest of regional management matters. According to the data association of urban management matters, node connections are established, and the root nodes of each tree are connected with a mesh model to form a regional logical database RLDB.

3. The smart city management platform for tri-modal data convergence and fusion according to claim 2 is characterized in that: RLDB allows a node to have more than one parent node. RLDB removes the restrictions in the hierarchical model, allowing multiple nodes to have no parent nodes, allowing nodes to have multiple parent nodes, and allowing many-to-many connections between nodes. RLDB has the following features: (1) Data structure: Use node names to describe entities layer by layer, use records in a relational database to represent entity data, and use attributes to describe data items; (2) Data operation: hierarchical access, branch access and hyperlink access can be used; (3) Integrity constraint: Each node in RLDB is used to uniquely identify the set of data items recorded in that layer.

4. The smart city management platform for tri-modal data convergence and fusion according to claim 3 is characterized in that: The following features are included in the RLDB logic design: (1) Determine the data aggregation granularity and the division of granularity levels. The aggregation granularity is actually the granularity of each layer of nodes. The design principle is to use different granularities in different layers. A coarser granularity description is used at the high level, and then it is refined layer by layer. The leaf node is the finest granularity data and is directly described by data item data. (2) Determine the establishment of data logic relationships and data logic segmentation strategies. This is the key content of data aggregation from the original business system to the RLDB. First, according to the city management matters, formulate the data segmentation standards for the matters, considering the following factors: data subject, data volume, actual situation of data analysis and processing, simplicity and granularity division strategy. Among them, the data subject is the basis for establishing the relationship between data logic. RLDB takes the content of the matter as the data subject; data volume is an important factor in data segmentation. According to the amount of data involved in the actual matter and the relationship between the data, determine the data logic segmentation strategy; (3) Construct a logical model for RLDB aggregation and determine the data hierarchy and parent nodes based on the relationship between the data involved in the actual event.

5. The smart city management platform for tri-modal data convergence and fusion according to claim 1 is characterized in that: The basic layer includes the urban infrastructure perception layer, the smart city management transmission layer, and the general system and business platform layer.

6. The smart city management platform for tri-modal data convergence and fusion according to claim 5 is characterized in that: The urban infrastructure perception layer consists of three layers: (1) Sensor equipment technology layer: including ubiquitous sensor network, RFID, collection equipment, aggregation equipment, and content security acquisition equipment; (2) Perception technology layer: including perception modeling technology, dynamic perception technology, observation and navigation technology, and trusted acquisition technology; (3) Perception system layer: including urban infrastructure perception system, spatial information perception system, urban three-dimensional perception network, environmental and disaster monitoring perception, vehicle-mounted perception system, and smart city three-dimensional perception system; The transport layer for smart city management consists of three layers: (1) Communication technology layer: including wireless broadband network, optical transmission network, and electronic transmission network; (2) Transmission control technology layer: including network service quality control technology, trusted, controllable and manageable network technology, and network and information security technology; (3) Transmission control layer for smart city management: urban control products for smart city management, basic application technologies for smart city management, and transmission control technologies dedicated to smart city management; The general system and business platform layer consists of two layers: (1) General system layer: The general operating system layer consists of various operating systems that support the operation of computer systems, as well as various resource management management systems built on the operating systems; (2) Business platform layer: a business platform that supports various business management and business operations.

7. The smart city management platform for tri-modal data convergence and fusion according to claim 1 is characterized in that: The urban management operation layer is divided into four layers: (1) Basic data layer: including business data of various business systems, monitoring data and urban management data of the public security system, urban management rule files of government departments, urban community grid data, and related data obtained from the Internet; (2) Data operation layer: including trimodal data annotation, data governance, data logic aggregation, data fusion, urban dynamic regional logic database construction and regional data activation based on the basic data layer. Regional data activation refers to data description and cognition technology, data maintenance and management technology, data association growth and security technology; (3) AI-assisted decision-making layer: including data gateways for data security control, spatiotemporal data engines for building data spatiotemporal associations, data rule mining, and comparative analysis of data rules and urban management rules, so that data can be deeply penetrated across industries and systems, generate new energy efficiency, serve to build the urban management brain, and form urban area smart management action plans; (4) Decision-making layer: Provide a decision-making support interactive interface, visualize relevant data, conduct human-computer interactive analysis and judgment on the regional action plan provided by the AI-assisted decision-making layer, form a preliminary regional action plan, and conduct sandbox simulations of the regional action plan.

Images