Urban planning decision-making method and system based on multi-modal remote sensing and knowledge graph

Abstract

The invention provides a multi-modal remote sensing and knowledge graph-based urban planning decision-making method and system, and the method comprises the steps: integrating multi-source heterogeneous data, achieving the feature alignment and fusion of an optical image and SAR data in a satellite remote sensing image through a deep learning technology, and generating an urban ground feature feature vector; associating the urban ground feature feature vector with an urban planning policy database, outputting a structured early warning report of an illegal construction early warning event set and a policy compliance label, and forming a dynamic policy constraint condition for subsequent multi-objective optimization; processing historical traffic flow data based on the dynamic graph model, and outputting a time-space distribution prediction result of future traffic conditions; and generating a Pareto optimal city planning scheme by combining multi-objective optimization with a spatial-temporal distribution prediction result of a future traffic condition. According to the method, high-precision urban surface feature classification, real-time violation extension early warning and traffic flow accurate prediction are realized through multi-modal remote sensing data fusion and a space-time knowledge graph technology, and multi-target optimization and digital twinborn verification are combined, so that the planning efficiency is improved, and extension applications such as carbon neutralization are supported.

Description

Technical Field

[0001] This application relates to the interdisciplinary field of artificial intelligence and smart cities, and specifically to an urban planning decision-making method and system based on multimodal remote sensing and knowledge graphs. Background Art

[0002] The report of the 20th National Congress of the Communist Party of China stated that high-quality development is the primary task in comprehensively building a modern socialist country. The concept of high-quality development places deeper demands on planning companies and even the planning industry. It not only requires planning companies to achieve refined, intelligent, and service-oriented urban construction, management, and operations, but also promotes their own scientific and technological innovation to improve planning capabilities. Leveraging information technology, we will build smart city planning systems to enhance the scientific nature and accuracy of urban planning. We will design and demonstrate planning schemes through methods such as digital models, reduce risks, and improve planning quality. With the continuous advancement of informatization and intelligence, the level of smart city construction continues to rise, but there are also issues such as a lack of clarity in the direction of smart city development.

[0003] In recent years, the integration of remote sensing technology and artificial intelligence has provided new technical approaches for smart city management. Existing technologies have already emerged, including land use classification based on remote sensing images, rule-based reasoning based on knowledge graphs, and multi-objective optimization algorithms. For example, some systems use optical satellite imagery to identify urban building outlines and verify construction compliance by integrating them with static knowledge bases. Other studies have employed genetic algorithms to optimize road network layouts or used time-series remote sensing data to monitor urban expansion trends. However, existing technologies still have significant practical limitations. In terms of data fusion, traditional remote sensing analysis systems often rely on single-modality data (such as optical or SAR imagery) and lack cross-modal collaboration mechanisms. For example, while optical imagery can clearly depict surface building forms, it cannot penetrate clouds or detect underground pipelines. SAR data, while capable of all-weather monitoring, lacks resolution and semantic understanding accuracy. Furthermore, existing methods fail to effectively integrate real-time dynamic data streams, such as those from transportation IoT sensors, resulting in blind spots in urban status perception.

[0004] In terms of knowledge modeling, current knowledge graph technologies mostly construct static relationship networks based on historical data, making it difficult to link real-time entity changes detected by remote sensing (such as illegal building expansions and road topology updates) with dynamic policy constraints (such as floor area ratio adjustments and environmental regulations). For example, one existing system uses a monthly updated land planning knowledge base, which is unable to respond promptly to sudden construction activities, causing planning decisions to lag behind urban development needs.

[0005] In terms of spatiotemporal prediction and decision-making optimization, traditional traffic models (such as ARIMA-based time-series forecasting) fail to fully consider the impact of road network topology on traffic flow, making it difficult to accurately simulate the chain reaction of events such as construction road closures. Mainstream multi-objective optimization algorithms (such as NSGA-II) lack dynamic weight adjustment mechanisms and three-dimensional visualization verification methods when faced with multi-objective conflicts such as land use efficiency, traffic carrying capacity, and carbon emission constraints, resulting in limited interpretability and practicality of planning schemes.

[0006] The above problems restrict urban planning, scientificity and operability. There is an urgent need for an intelligent decision-making system that can deeply integrate multi-source data, dynamically update domain knowledge and achieve multi-objective collaborative optimization. Summary of the Invention

[0007] This application provides an urban planning decision-making method and system based on multimodal remote sensing and knowledge graphs, which can solve the technical problems existing in the existing technology, such as insufficient multi-source data fusion, lagging knowledge updating, and lack of multi-objective collaborative optimization capabilities.

[0008] First, this application provides an urban planning decision-making method based on multimodal remote sensing and knowledge graph, including the following steps: Integrate multi-source heterogeneous data related to urban planning, use deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generate urban feature vectors; By associating urban feature vectors with the urban planning policy database, the system uses a temporal change detection algorithm to track dynamic information in real time, constructing an urban knowledge graph with spatiotemporal dimensions. Based on the urban knowledge graph, the system automatically matches entity state changes with policy rules, outputting a structured warning report containing a set of illegal construction warning events and policy compliance labels, and forming dynamic policy constraints for subsequent multi-objective optimization. Build a dynamic graph model of the urban road network based on real-time traffic data, process historical traffic flow data based on the dynamic graph model, and output the spatiotemporal distribution prediction results of future traffic conditions; Taking land use efficiency, traffic carrying capacity and carbon emission intensity as optimization objectives, combined with the spatiotemporal distribution prediction results of future traffic conditions, a Pareto optimal urban planning scheme is generated through a non-dominated sorting genetic algorithm under the constraints of dynamic policy conditions.

[0009] Furthermore, the integration includes multi-source heterogeneous data related to urban planning, and uses deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images to generate urban feature vectors, which specifically includes the following steps: Acquire multi-source heterogeneous data including satellite remote sensing images, traffic GPS data, and IoT sensors; Performing radiation correction and geometric correction processing on the acquired optical image to obtain a corrected optical image; Perform denoising and terrain correction on the SAR data to obtain corrected SAR data; Based on the corrected optical image and SAR data, a cross-modal Transformer deep learning model is constructed. The feature fusion weights of the optical image and SAR data are dynamically assigned through a multi-head attention mechanism to obtain cross-modal aligned optical image and SAR data. Perform time synchronization and spatial interpolation on GPS and IoT data to obtain processed traffic GPS data and IoT data; The processed optical images, SAR data, traffic GPS data and IoT sensor data are uniformly represented and the urban feature vector is output.

[0010] Furthermore, performing radiation correction and geometric correction processing on the acquired optical image to obtain the corrected optical image specifically includes the following steps: Input the original optical image data, convert the DN value into apparent reflectance through the radiometric calibration formula, eliminate the sensor response difference, and obtain the optical image data after eliminating the sensor response difference; For the optical image data after eliminating the sensor response differences, a polynomial correction model is used based on the DEM data to eliminate terrain distortion, and the image geometric correction is achieved through ground control points to obtain the geometrically corrected optical image data; The improved dark pixel method is used to perform atmospheric correction on the geometrically corrected optical image data to output an optical image with unified radiation and geometric characteristics.

[0011] Furthermore, based on the corrected optical image and SAR data, a cross-modal Transformer deep learning model is constructed, and the feature fusion weights of the optical image and SAR data are dynamically allocated through a multi-head attention mechanism. Specifically, the following steps are included: Input pre-processed optical images and SAR data, extract multi-scale features through the CNN backbone network, and generate optical feature tensors and SAR feature tensors respectively; The query matrix, key matrix, and value matrix are calculated in the cross-modal attention layer, and the feature fusion weights of the optical feature tensor and the SAR feature tensor are assigned by scaling the dot product attention scores.

[0012] Furthermore, the urban feature vectors are associated with the urban planning policy database, dynamic information is tracked in real time through a temporal change detection algorithm, and an urban knowledge graph with spatiotemporal dimensions is constructed. Based on the urban knowledge graph, entity state changes and policy rules are automatically matched, and a structured warning report with a set of illegal construction warning events and policy compliance labels is output to form dynamic policy constraints for subsequent multi-objective optimization. Specifically, the following steps are included: Perform spatiotemporal encoding on the urban feature vectors, and use a cross-modal matching algorithm based on the attention mechanism to calculate the semantic similarity between the encoded urban feature vectors and the planning control indicators in the policy database, and establish a feature-policy mapping relationship. Based on the feature-policy mapping relationship, the improved ST-DBSCAN spatiotemporal clustering algorithm is used to detect abnormal changes in key building parameters. When abnormal changes are detected, the change event marker is automatically triggered and a change trajectory map is generated. Using entities as nodes and spatial relationships as edges, each node is assigned a state attribute and a node state weight is set to construct a city knowledge graph. A city entity state change matrix marked with timestamps is used to update building expansions and road topology changes in real time, and output the city knowledge graph. The urban planning policy database is transformed into an executable set of logical rules and matched with the knowledge graph node status. When a rule violation is detected, a structured warning report with an output of illegal construction warning event sets and policy compliance labels is automatically generated to form dynamic policy constraints for subsequent multi-objective optimization.

[0013] Furthermore, the improved ST-DBSCAN spatiotemporal clustering algorithm is used to detect abnormal changes in key building parameters, specifically including the following steps: The building height temporal change matrix is ​​obtained, and density accessibility analysis is used to identify spatially continuous and temporally coherent change clusters in the building height temporal change matrix. When the average change rate within a cluster is greater than the change threshold, it is marked as an abnormal event.

[0014] Furthermore, the method of constructing a dynamic graph model of the urban road network based on real-time traffic data, processing historical traffic flow data based on the dynamic graph model, and outputting a spatiotemporal distribution prediction result of future traffic conditions specifically includes the following steps: Pre-process the real-time collected traffic data to generate a standardized traffic status matrix; Based on the topological structure of the urban road network, a dynamic graph model is constructed with road sections as nodes, connections between road sections as edges, real-time traffic volume, road width and slope as node attributes, and traffic capacity as edge weight. A graph neural network is used to iteratively update the node states in the constructed dynamic graph model to obtain an iteratively updated dynamic graph model; Based on the iteratively updated dynamic graph model and historical traffic flow time series data, GCN is used to predict node traffic flow changes, identify road sections with traffic exceeding the threshold as traffic hotspots, and output the spatiotemporal distribution prediction results of future traffic conditions.

[0015] Furthermore, the method of generating a Pareto optimal urban planning scheme by using a non-dominated sorting genetic algorithm with land use efficiency, traffic carrying capacity, and carbon emission intensity as optimization targets and combining the spatiotemporal distribution prediction results of future traffic conditions specifically includes the following steps: The land use efficiency is calculated based on the building density and underground space utilization rate characteristics in the urban land feature vector; Combined with the traffic hotspot prediction results, the traffic carrying capacity is calculated with road network saturation as the core indicator; Carbon emission intensity is calculated by combining the energy consumption characteristics in the urban feature vector and the traffic flow data in the spatiotemporal distribution prediction results. Taking land use efficiency, transportation carrying capacity and carbon emission intensity as optimization objectives, a Pareto optimal urban planning scheme is generated through a non-dominated sorting genetic algorithm based on constraints.

[0016] In a second aspect, the present application provides an urban planning decision-making system based on multimodal remote sensing and knowledge graphs, characterized by comprising: a multimodal data fusion module for integrating multi-source heterogeneous data related to urban planning, using deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generating urban feature vectors; A dynamic knowledge graph engine, in communication with the multimodal data fusion module, is used to associate urban feature vectors with the urban planning policy database, track dynamic information in real time through a temporal change detection algorithm, construct an urban knowledge graph with spatiotemporal dimensions, automatically match entity state changes with policy rules based on the urban knowledge graph, and output a structured warning report with a set of illegal construction warning events and policy compliance labels, thereby forming dynamic policy constraints for subsequent multi-objective optimization; A spatiotemporal graph neural network, communicating with the dynamic knowledge graph engine, constructs a dynamic graph model of the urban road network based on real-time traffic data, processes historical traffic flow data based on the dynamic graph model, and outputs spatiotemporal distribution prediction results of future traffic conditions; The multi-objective evolutionary decision-making module is communicatively connected to the multimodal data fusion module, the dynamic knowledge graph engine and the spatiotemporal graph neural network, and is used to generate a Pareto optimal urban planning scheme through a non-dominated sorting genetic algorithm under the constraints of dynamic policy constraints, taking land use efficiency, traffic carrying capacity and carbon emission intensity as optimization goals, and combining the spatiotemporal distribution prediction results of future traffic conditions.

[0017] Furthermore, the dynamic knowledge graph engine also includes: The feature and policy mapping relationship acquisition unit is used to perform spatiotemporal encoding processing on the urban feature feature vectors. It uses a cross-modal matching algorithm based on the attention mechanism to calculate the semantic similarity between the encoded urban feature feature vectors and the planning control indicators in the policy database, and establish a feature-policy mapping relationship. a change trajectory map acquisition module, which is in communication with the feature and policy mapping relationship acquisition unit and is used to detect abnormal changes in key building parameters based on the feature-policy mapping relationship using an improved ST-DBSCAN spatiotemporal clustering algorithm. When an abnormal change is detected, a change event marker is automatically triggered and a change trajectory map is generated; A knowledge graph acquisition unit is in communication with the change trajectory graph acquisition unit, and is used to construct a city knowledge graph with entities as nodes and spatial relationships as edges, attach state attributes to each node, and set node state weights, thereby constructing a city entity state change matrix marked by timestamps, updating building expansions and road topology changes in real time, and outputting a city knowledge graph; The constraint condition acquisition unit is in communication with the urban knowledge graph acquisition unit and is used to convert and process the urban planning policy database into an executable logical rule set and match it with the knowledge graph node status. When a rule violation is detected, it automatically generates and outputs a structured warning report with an illegal construction warning event set and a policy compliance label, forming dynamic policy constraints for subsequent multi-objective optimization.

[0018] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least: Deep learning technology enables adaptive feature alignment between optical images and SAR radar data, effectively integrating surface building outlines with underground pipeline network distribution information, significantly improving the accuracy of urban land feature classification. Construct a four-dimensional spatiotemporal knowledge graph to reflect changes in building heights in real time, and trigger early warnings in conjunction with land planning thresholds, significantly shortening the response time for identifying illegal expansions. The dynamic graph model is used to predict the traffic flow redistribution effect caused by road closures due to construction, and the prediction error is significantly reduced compared with the traditional model.

[0019] Generate Pareto optimal solutions through multi-objective optimization. Combined with digital twin technology, VR interactive verification is achieved, allowing planners to adjust building layouts and other plans in real time. By covering the entire process of "data perception-knowledge reasoning-decision generation", the planning scheme generation cycle is significantly shortened compared to traditional methods, and can be extended to emerging scenarios such as carbon neutral path planning and disaster resilience assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flowchart of the urban planning decision-making method based on multimodal remote sensing and knowledge graphs for this application; Figure 2 The spatiotemporal knowledge graph and multi-objective evolutionary decision-making diagram provided in the embodiments of this application; Figure 3 Schematic diagram of the cross-modal Transformer remote sensing large model provided in the embodiment of this application; Figure 4 This is the overall system architecture diagram of the urban planning decision-making system based on multimodal remote sensing and knowledge graph for this application; Figure 5 A flowchart of the multimodal data fusion module provided in the embodiment of the present application; DETAILED DESCRIPTION In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0022] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0023] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0024] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0025] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0026] First, as Figure 1 and Figure 2 As shown, this application provides an urban planning decision-making method based on multimodal remote sensing and knowledge graph, including the following steps: Step S1: Integrate multi-source heterogeneous data related to urban planning, use deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generate urban feature vectors; Step S2: Associating urban feature vectors with the urban planning policy database, tracking dynamic information in real time through a temporal change detection algorithm, and constructing an urban knowledge graph with spatiotemporal dimensions. Based on the urban knowledge graph, the entity state changes and policy rules are automatically matched. A structured warning report with a set of illegal construction warning events and policy compliance labels is output, forming dynamic policy constraints for subsequent multi-objective optimization. Step S3: constructing a dynamic graph model of the urban road network based on real-time traffic data, processing historical traffic flow data based on the dynamic graph model, and outputting the spatiotemporal distribution prediction results of future traffic conditions; Step S4: Taking land use efficiency, traffic carrying capacity and carbon emission intensity as optimization objectives, combined with the spatiotemporal distribution prediction results of future traffic conditions, a Pareto optimal urban planning scheme is generated through a non-dominated sorting genetic algorithm under the constraints of dynamic policy conditions.

[0027] In one embodiment, step S1: integrating multi-source heterogeneous data related to urban planning, using deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generating urban feature vectors, specifically includes the following steps: Step S11: Acquire multi-source heterogeneous data including satellite remote sensing images, traffic GPS data, and IoT sensor data. Satellite remote sensing images include optical images and SAR data. IoT sensor data is real-time environmental monitoring information collected by Internet of Things (IoT) devices, including urban operating parameters such as air quality, temperature and humidity, noise, and light. Specifically, optical images (such as Landsat-8 OLI, with a spatial resolution of 30 meters and a band covering visible light to near-infrared), SAR data (such as Sentinel-1, C-band, with a spatial resolution of 10 meters), traffic GPS tracks (sampling frequency of 1 Hz), and IoT sensor data such as air quality and noise sensors are collected at a sampling interval of 5 minutes. Step S12A: performing radiometric correction and geometric correction processing on the acquired optical image to eliminate cloud-blocked images and obtain a corrected optical image; Step S12B: performing denoising and terrain correction processing on the SAR data to obtain corrected SAR data; Step S12C: Based on the corrected optical image and SAR data, a cross-modal Transformer deep learning model is constructed, and feature fusion weights of the optical image and SAR data are dynamically assigned through a multi-head attention mechanism to obtain cross-modal aligned optical image and SAR data. Step S13: Perform temporal synchronization and spatial interpolation on the GPS and IoT data to obtain processed traffic GPS data and IoT data, ensuring that the multimodal data are aligned in the temporal and spatial dimensions; Step S14: Perform unified feature representation on the processed optical image, SAR data, traffic GPS data, and IoT sensor data, and output a city feature vector.

[0028] In one embodiment, step S12A: performing radiation correction and geometric correction processing on the acquired optical image to obtain the corrected optical image specifically includes the following steps: Input the original optical image data, convert the DN value into apparent reflectance through the radiometric calibration formula, eliminate the sensor response difference, and obtain the optical image data after eliminating the sensor response difference; For the optical image data after eliminating the sensor response differences, a polynomial correction model is used based on the DEM data to eliminate terrain distortion, and the image geometric correction is achieved through ground control points to obtain the geometrically corrected optical image data; The improved dark pixel method is used to perform atmospheric correction on the geometrically corrected optical image data to output an optical image with unified radiation and geometric characteristics.

[0029] In one embodiment, step S12C: constructing a cross-modal Transformer deep learning model based on the corrected optical image and SAR data, and dynamically allocating feature fusion weights of the optical image and SAR data through a multi-head attention mechanism, specifically includes the following steps: Input pre-processed optical images and SAR data, extract multi-scale features through the CNN backbone network, and generate optical feature tensors and SAR feature tensors respectively; The query matrix, key matrix, and value matrix are calculated in the cross-modal attention layer, and the feature fusion weights of the optical feature tensor and the SAR feature tensor are assigned by scaling the dot product attention scores.

[0030] In one embodiment, step S2: associating urban feature vectors with an urban planning policy database, tracking dynamic information in real time through a temporal change detection algorithm, constructing an urban knowledge graph with spatiotemporal dimensions, automatically matching entity state changes with policy rules based on the urban knowledge graph, outputting a structured warning report with a set of illegal construction warning events and policy compliance labels, and forming dynamic policy constraints for subsequent multi-objective optimization, specifically includes the following steps: Step S21: Perform spatiotemporal encoding on the urban feature feature vector, and use a cross-modal matching algorithm based on an attention mechanism to calculate the semantic similarity between the encoded urban feature feature vector and the planning control indicators in the policy database, and establish a feature-policy mapping relationship; Step S22: Based on the feature-policy mapping relationship, the improved ST-DBSCAN spatiotemporal clustering algorithm is used to detect abnormal changes in key building parameters. When abnormal changes are detected, the change event marker is automatically triggered and a change trajectory map is generated; Step S23: With entities as nodes and spatial relationships as edges, each node is assigned a state attribute and a node state weight is set to construct and output a city knowledge graph. Information such as building expansion and road topology changes is updated in real time through a city entity state change matrix marked with a timestamp. The node state weight update formula is:

[0031] Where, is the historical data attenuation factor, It is the entity status difference matrix of remote sensing monitoring on that day, and realizes minute-level urban entity status update; Step S24: Convert and process the urban planning policy database (such as land use restrictions and building height specifications) into an executable logical rule set and match it with the knowledge graph node status; at fixed intervals, call the remote sensing change detection algorithm to compare the building height and area change values ​​in the time series remote sensing image with the land planning permission threshold, triggering an early warning for illegal expansion. When a violation of the rules is detected, a structured early warning report with an output of an illegal construction warning event set and policy compliance labels is automatically generated to form dynamic policy constraints for subsequent multi-objective optimization.

[0032] In one embodiment, step S21: performing spatiotemporal encoding processing on the urban feature feature vector, using a cross-modal matching algorithm based on an attention mechanism, calculating the semantic similarity between the encoded urban feature feature vector and the planning control indicators in the policy database, and establishing a feature-policy mapping relationship, specifically includes the following steps: Performing spatiotemporal coding processing on the urban feature feature vector to obtain the spatiotemporal coded urban feature feature vector; Using attention-based mechanisms such as Figure 3 The cross-modal Transformer remote sensing large model shown is a cross-modal Transformer deep learning model. It inputs the urban feature vector after spatiotemporal coding, calculates the semantic similarity between the urban feature vector after coding and the planning control indicators in the policy database, and establishes a feature-policy mapping relationship.

[0033] In a more specific embodiment, the cross-modal Transformer remote sensing large model structure includes an encoder, an attention layer, and a decoder; In the encoder, optical images are processed through a convolutional neural network (CNN) to extract surface building outline features, SAR data is processed through a convolutional layer to extract underground pipeline distribution features, and GPS and IoT data are mapped to a low-dimensional feature space through a fully connected layer to obtain traffic flow patterns and environmental status characteristics (such as traffic density, air quality index, and other dynamic urban operation parameters). The attention layer dynamically assigns feature fusion weights. The optical image weight focuses on surface texture, the SAR data weight focuses on penetration information, and the GPS and IoT data weights are adjusted according to real-time performance. The calculation formula for GPS and IoT data weights is:

[0034] Where, is the attention weight after softmax normalization, is the dimension scaling factor of the feature vector and are the vectors of query and key respectively, is the feature dimension.

[0035] The decoder fuses multimodal features and outputs a unified urban feature vector with a dimension of 512.

[0036] In one embodiment, the step S22 detects abnormal changes in key building parameters using an improved ST-DBSCAN spatiotemporal clustering algorithm, specifically comprising the following steps: The building height temporal change matrix is ​​obtained, and density accessibility analysis is used to identify spatially continuous and temporally coherent change clusters in the building height temporal change matrix. When the average change rate within a cluster is greater than the change threshold, it is marked as an abnormal event.

[0037] In one embodiment, step S3: constructing a dynamic graph model of the urban road network based on real-time traffic data, processing historical traffic flow data based on the dynamic graph model, and outputting a spatiotemporal distribution prediction result of future traffic conditions, specifically includes the following steps: Step S31: pre-processing the real-time collected traffic data to generate a standardized traffic state matrix; Step S32: Based on the urban road network topology, a dynamic graph model is constructed with road sections as nodes, connections between road sections as edges, real-time traffic volume, road width and slope as node attributes, and traffic capacity as edge weights; Step S33: Use the graph neural network to iteratively update the node states in the constructed dynamic graph model to obtain the iteratively updated dynamic graph model; the dynamic propagation formula of traffic flow in the road network is:

[0038] Where, For road sections In the The feature vector of the layer, A collection of adjacent road segments used to predict the traffic redistribution effect caused by road closures. The model input is historical traffic flow time series data, and the output is a short-term traffic flow forecast for the next 6 hours and a long-term traffic flow forecast for the next 30 days. Step S34: Based on the iteratively updated dynamic graph model and historical traffic flow time series data, the GCN is used to predict the changes in node traffic flow, identify road sections with traffic exceeding the threshold as traffic hotspots, and output the spatiotemporal distribution prediction results of future traffic conditions.

[0039] In one embodiment, step S4: taking land use efficiency, traffic carrying capacity, and carbon emission intensity as optimization targets, and combining the spatiotemporal distribution prediction results of future traffic conditions, generating a Pareto optimal urban planning scheme using a non-dominated sorting genetic algorithm, specifically includes the following steps: Step S41: Calculating and obtaining land use efficiency based on building density and underground space utilization rate characteristics in the urban land feature feature vector;

[0040] Where, represents the base area of ​​the i-th building (extracted from the optical image of A1), Indicates the number of floors (obtained by dividing the inverted height of A1 SAR data by the standard floor height of 3 meters). Indicates the maximum building area allowed by the policy (obtained from A2's land planning permit database). When A2 detects that the actual building area exceeds the permitted value, the system automatically As the calculation upper limit, ensure that the indicator complies with policy constraints. For example, if the maximum building area allowed for a plot is 100,000 square meters and remote sensing detection shows 120,000 square meters, the efficiency value will be calculated based on 100,000 square meters. Step S42: combining the traffic hotspot prediction results and taking the road network saturation as the core indicator to calculate and obtain the traffic carrying capacity;

[0041] Where, E represents the set of road network edges (obtained from the GPS data and road network topology in step S1), is the predicted traffic volume of the i-th road (calculated by the graph convolutional network in step S3), The traffic capacity (calculated based on the road width and number of lanes provided in step S1): = × ×0.5, where is the number of lanes, is the road width). When the saturation of a road section / When it is >0.8, the indicator value will drop significantly.

[0042] Step S43: Calculate and obtain carbon emission intensity through the energy consumption characteristics in the urban feature feature vector and the vehicle flow data in the spatiotemporal distribution prediction result;

[0043] Where, is the floor area of ​​the j-th building (from step S1), is the energy consumption coefficient (determined by the building type identified by optical image recognition in step S1: 0.08 for glass curtain wall buildings and 0.05 for traditional buildings), Tk is the k-th type of vehicle flow (predicted in step S3), is the emission factor (obtained from the environmental policy database in step S2: 0.02 kg / km for electric vehicles and 0.15 kg / km for fuel vehicles), is the total area of ​​the planning area (obtained from the remote sensing mapping data in step S1). The negative sign indicates that smaller values ​​represent better carbon emission performance, which meets the goal of minimization optimization. When A2 detects that a certain area has implemented a new energy vehicle promotion policy, it will automatically adjust the value of Pk; Step S44: Taking land use efficiency, traffic carrying capacity and carbon emission intensity as optimization targets, a Pareto optimal urban planning scheme is generated based on constraints through a non-dominated sorting genetic algorithm; further, a dynamic weighted fitness function is designed. , whose expression is:

[0044] Where, For the planning scheme to be optimized, are the sub-objective functions for land use efficiency, traffic carrying capacity, and carbon emission intensity, respectively. Parameters α, β, and γ are dynamically adjusted according to the stage of urban development. New urban areas focus on traffic carrying capacity, with an empirical coefficient of β of 0.6. Old urban reconstruction focuses on land use efficiency, with an empirical coefficient of α of 0.7. Using a multi-objective genetic algorithm, the number of initial planning schemes is 100, each of which contains land zoning, road layout, and building density parameters. The crossover rate is set to 0.8 and the mutation rate is set to 0.1. A new population is generated through crossover and mutation, and 100 generations are iterated to generate the Pareto frontier. Integrating digital twin technology, a 3D city model is generated using the Unity3D engine to display the optimized planning solution. Furthermore, planners can use VR devices (such as the Oculus Rift) to adjust parameters (such as road width and building height) in real time, thereby updating the fitness function value.

[0045] Second, as Figure 4 As shown, this application provides an urban planning decision system based on multimodal remote sensing and knowledge graph, including a multimodal data fusion module, a dynamic knowledge graph engine, a spatiotemporal graph neural network and a multi-objective evolutionary decision module; Figure 5As shown, the multimodal data fusion module is used to integrate multi-source heterogeneous data including those related to urban planning, and adopts deep learning technology to realize the feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generate urban feature vectors; the dynamic knowledge graph engine is communicated with the multimodal data fusion module to associate the urban feature vectors with the urban planning policy database, and track dynamic information in real time through the time series change detection algorithm to construct an urban knowledge graph with spatiotemporal dimensions. Based on the urban knowledge graph, it automatically matches entity state changes with policy rules, and outputs a structured warning report of illegal construction warning event sets and policy compliance labels, forming a data structure for subsequent multi-modal data fusion. Dynamic policy constraints for target optimization; a spatiotemporal graph neural network is communicated with the dynamic knowledge graph engine, a dynamic graph model of the urban road network is constructed based on real-time traffic data, historical traffic flow data is processed based on the dynamic graph model, and the spatiotemporal distribution prediction results of future traffic conditions are output; a multi-objective evolutionary decision-making module is communicated with the multimodal data fusion module, the dynamic knowledge graph engine and the spatiotemporal graph neural network, and is used to take land use efficiency, traffic carrying capacity and carbon emission intensity as optimization targets, combined with the spatiotemporal distribution prediction results of future traffic conditions, and generate a Pareto optimal urban planning scheme through a non-dominated sorting genetic algorithm under the constraints of dynamic policy constraints.

[0046] In one embodiment, the dynamic knowledge graph engine further includes: The feature and policy mapping relationship acquisition unit is used to perform spatiotemporal encoding processing on the urban feature feature vectors. It uses a cross-modal matching algorithm based on the attention mechanism to calculate the semantic similarity between the encoded urban feature feature vectors and the planning control indicators in the policy database, and establish a feature-policy mapping relationship. a change trajectory map acquisition module, which is in communication with the feature and policy mapping relationship acquisition unit and is used to detect abnormal changes in key building parameters based on the feature-policy mapping relationship using an improved ST-DBSCAN spatiotemporal clustering algorithm. When an abnormal change is detected, a change event marker is automatically triggered and a change trajectory map is generated; A knowledge graph acquisition unit is in communication with the change trajectory graph acquisition unit, and is used to construct a city knowledge graph with entities as nodes and spatial relationships as edges, attach state attributes to each node, and set node state weights, thereby constructing a city entity state change matrix marked by timestamps, updating building expansions and road topology changes in real time, and outputting a city knowledge graph; The constraint condition acquisition unit is in communication with the urban knowledge graph acquisition unit and is used to convert and process the urban planning policy database into an executable logical rule set and match it with the knowledge graph node status. When a rule violation is detected, it automatically generates and outputs a structured warning report with an illegal construction warning event set and a policy compliance label, forming dynamic policy constraints for subsequent multi-objective optimization.

[0047] Among them, the functional implementation of each module in the above-mentioned multimodal remote sensing and knowledge graph urban planning decision-making system corresponds to the various steps in the above-mentioned multimodal remote sensing and knowledge graph urban planning decision-making method embodiment, and its functions and implementation process will not be repeated here one by one.

[0048] On the third aspect, an embodiment of the present application provides a multimodal remote sensing and knowledge graph urban planning decision-making device. The multimodal remote sensing and knowledge graph urban planning decision-making device can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0049] Communication interfaces include input / output (I / O), physical, and logical interfaces, which interconnect components within the multimodal remote sensing and knowledge graph urban planning decision-making device. They also interconnect the multimodal remote sensing and knowledge graph urban planning decision-making device with other devices (such as other computing devices or user devices). Physical interfaces can include Ethernet, fiber optic, and ATM interfaces; user devices can include displays and keyboards.

[0050] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0051] The processor can be a general-purpose processor that can call the multimodal remote sensing and knowledge graph urban planning decision-making program stored in the memory and execute the multimodal remote sensing and knowledge graph urban planning decision-making method provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the multimodal remote sensing and knowledge graph urban planning decision-making program is called can be referred to in the various embodiments of the multimodal remote sensing and knowledge graph urban planning decision-making method of this application and will not be repeated here.

[0052] In a fourth aspect, an embodiment of the present application also provides a readable storage medium.

[0053] The readable storage medium of the present application stores an urban planning decision-making program with multimodal remote sensing and knowledge graph, wherein when the urban planning decision-making program with multimodal remote sensing and knowledge graph is executed by a processor, the steps of the urban planning decision-making method with multimodal remote sensing and knowledge graph as described above are implemented.

[0054] Among them, the method implemented when the urban planning decision-making program of multimodal remote sensing and knowledge graph is executed can refer to the various embodiments of the urban planning decision-making method of multimodal remote sensing and knowledge graph of this application, and will not be repeated here.

[0055] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0056] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of this application.

[0057] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multimodal remote sensing and knowledge graph urban planning decision-making method, characterized by: The following steps are involved: Integrate multi-source heterogeneous data related to urban planning, use deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generate urban feature vectors; By associating urban feature vectors with the urban planning policy database, the system uses a temporal change detection algorithm to track dynamic information in real time, constructing an urban knowledge graph with spatiotemporal dimensions. Based on the urban knowledge graph, the system automatically matches entity state changes with policy rules, outputting a structured warning report containing a set of illegal construction warning events and policy compliance labels, and forming dynamic policy constraints for subsequent multi-objective optimization. Build a dynamic graph model of the urban road network based on real-time traffic data, process historical traffic flow data based on the dynamic graph model, and output the spatiotemporal distribution prediction results of future traffic conditions; Taking land use efficiency, traffic carrying capacity and carbon emission intensity as optimization objectives, combined with the spatiotemporal distribution prediction results of future traffic conditions, a Pareto optimal urban planning scheme is generated through a non-dominated sorting genetic algorithm under the constraints of dynamic policy conditions.

2. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 1, characterized in that: The integration includes multi-source heterogeneous data related to urban planning, uses deep learning technology to achieve feature alignment and fusion of optical images and SAR data in satellite remote sensing images, and generates urban feature vectors, specifically including the following steps: Acquire multi-source heterogeneous data including satellite remote sensing images, traffic GPS data, and IoT sensors; Performing radiation correction and geometric correction processing on the acquired optical image to obtain a corrected optical image; Perform denoising and terrain correction on the SAR data to obtain corrected SAR data; Based on the corrected optical image and SAR data, a cross-modal Transformer deep learning model is constructed. The feature fusion weights of the optical image and SAR data are dynamically assigned through a multi-head attention mechanism to obtain cross-modal aligned optical image and SAR data. Perform time synchronization and spatial interpolation on GPS and IoT data to obtain processed traffic GPS data and IoT data; The processed optical images, SAR data, traffic GPS data and IoT sensor data are uniformly represented and the urban feature vector is output.

3. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 2, characterized in that: The radiation correction and geometric correction processing are performed on the acquired optical image to obtain the corrected optical image. The following steps are involved: Input the original optical image data, convert the DN value into apparent reflectance through the radiometric calibration formula, eliminate the sensor response difference, and obtain the optical image data after eliminating the sensor response difference; For the optical image data after eliminating the sensor response differences, a polynomial correction model is used based on the DEM data to eliminate terrain distortion, and the image geometric correction is achieved through ground control points to obtain the geometrically corrected optical image data; The improved dark pixel method is used to perform atmospheric correction on the geometrically corrected optical image data to output an optical image with unified radiation and geometric characteristics.

4. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 2, characterized in that: Based on the corrected optical image and SAR data, a cross-modal Transformer deep learning model is constructed, and the feature fusion weights of the optical image and SAR data are dynamically allocated through a multi-head attention mechanism. Specifically, the following steps are included: Input pre-processed optical images and SAR data, extract multi-scale features through the CNN backbone network, and generate optical feature tensors and SAR feature tensors respectively; The query matrix, key matrix, and value matrix are calculated in the cross-modal attention layer, and the feature fusion weights of the optical feature tensor and the SAR feature tensor are assigned by scaling the dot product attention scores.

5. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 1, characterized in that: The method associates the urban feature vectors with the urban planning policy database, tracks dynamic information in real time through a temporal change detection algorithm, constructs an urban knowledge graph with spatiotemporal dimensions, automatically matches entity state changes with policy rules based on the urban knowledge graph, outputs a structured warning report with a set of illegal construction warning events and policy compliance labels, and forms dynamic policy constraints for subsequent multi-objective optimization. Specifically, the method includes the following steps: Perform spatiotemporal encoding on the urban feature vectors, and use a cross-modal matching algorithm based on the attention mechanism to calculate the semantic similarity between the encoded urban feature vectors and the planning control indicators in the policy database, and establish a feature-policy mapping relationship. Based on the feature-policy mapping relationship, the improved ST-DBSCAN spatiotemporal clustering algorithm is used to detect abnormal changes in key building parameters. When abnormal changes are detected, the change event marker is automatically triggered and a change trajectory map is generated. Using entities as nodes and spatial relationships as edges, each node is assigned a state attribute and a node state weight is set to construct a city knowledge graph. A city entity state change matrix marked with timestamps is used to update building expansions and road topology changes in real time, and output the city knowledge graph. The urban planning policy database is transformed into an executable set of logical rules and matched with the knowledge graph node status. When a rule violation is detected, a structured warning report with an output of illegal construction warning event sets and policy compliance labels is automatically generated to form dynamic policy constraints for subsequent multi-objective optimization.

6. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 5, characterized in that: The improved ST-DBSCAN spatiotemporal clustering algorithm is used to detect abnormal changes in key building parameters, specifically including the following steps: The building height temporal change matrix is ​​obtained, and density accessibility analysis is used to identify spatially continuous and temporally coherent change clusters in the building height temporal change matrix. When the average change rate within a cluster is greater than the change threshold, it is marked as an abnormal event.

7. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 1, characterized in that: The method of constructing a dynamic graph model of an urban road network based on real-time traffic data, processing historical traffic flow data based on the dynamic graph model, and outputting a spatiotemporal distribution prediction result of future traffic conditions specifically includes the following steps: Pre-process the real-time collected traffic data to generate a standardized traffic status matrix; Based on the topological structure of the urban road network, a dynamic graph model is constructed with road sections as nodes, connections between road sections as edges, real-time traffic volume, road width and slope as node attributes, and traffic capacity as edge weight. A graph neural network is used to iteratively update the node states in the constructed dynamic graph model to obtain an iteratively updated dynamic graph model; Based on the iteratively updated dynamic graph model and historical traffic flow time series data, GCN is used to predict node traffic flow changes, identify road sections with traffic exceeding the threshold as traffic hotspots, and output the spatiotemporal distribution prediction results of future traffic conditions.

8. The urban planning decision-making method based on multimodal remote sensing and knowledge graph according to claim 1, characterized in that: The method uses land use efficiency, traffic carrying capacity, and carbon emission intensity as optimization targets, combines the spatiotemporal distribution prediction results of future traffic conditions, and generates a Pareto optimal urban planning scheme through a non-dominated sorting genetic algorithm. Specifically, the method includes the following steps: The land use efficiency is calculated based on the building density and underground space utilization rate characteristics in the urban land feature vector; Combined with the traffic hotspot prediction results, the traffic carrying capacity is calculated with road network saturation as the core indicator; Carbon emission intensity is calculated by combining the energy consumption characteristics in the urban feature vector and the traffic flow data in the spatiotemporal distribution prediction results. Taking land use efficiency, transportation carrying capacity and carbon emission intensity as optimization objectives, a Pareto optimal urban planning scheme is generated through a non-dominated sorting genetic algorithm based on constraints.

9. A multimodal remote sensing and knowledge graph urban planning decision system, characterized by: include: The multimodal data fusion module is used to integrate multi-source heterogeneous data, including those related to urban planning. It uses deep learning technology to align and fuse the features of optical images and SAR data in satellite remote sensing images to generate urban feature vectors. A dynamic knowledge graph engine, in communication with the multimodal data fusion module, is used to associate urban feature vectors with the urban planning policy database, track dynamic information in real time through a temporal change detection algorithm, construct an urban knowledge graph with spatiotemporal dimensions, automatically match entity state changes with policy rules based on the urban knowledge graph, and output a structured warning report with a set of illegal construction warning events and policy compliance labels, thereby forming dynamic policy constraints for subsequent multi-objective optimization; A spatiotemporal graph neural network, communicating with the dynamic knowledge graph engine, constructs a dynamic graph model of the urban road network based on real-time traffic data, processes historical traffic flow data based on the dynamic graph model, and outputs spatiotemporal distribution prediction results of future traffic conditions; The multi-objective evolutionary decision-making module is communicatively connected to the multimodal data fusion module, the dynamic knowledge graph engine and the spatiotemporal graph neural network, and is used to generate a Pareto optimal urban planning scheme through a non-dominated sorting genetic algorithm under the constraints of dynamic policy constraints, taking land use efficiency, traffic carrying capacity and carbon emission intensity as optimization goals, and combining the spatiotemporal distribution prediction results of future traffic conditions.

10. The urban planning decision system based on multimodal remote sensing and knowledge graph according to claim 9, characterized in that: The dynamic knowledge graph engine includes: The feature and policy mapping relationship acquisition unit is used to perform spatiotemporal encoding processing on the urban feature feature vectors. It uses a cross-modal matching algorithm based on the attention mechanism to calculate the semantic similarity between the encoded urban feature feature vectors and the planning control indicators in the policy database, and establish a feature-policy mapping relationship. a change trajectory map acquisition module, which is in communication with the feature and policy mapping relationship acquisition unit and is used to detect abnormal changes in key building parameters based on the feature-policy mapping relationship using an improved ST-DBSCAN spatiotemporal clustering algorithm. When an abnormal change is detected, a change event marker is automatically triggered and a change trajectory map is generated; A knowledge graph acquisition unit is in communication with the change trajectory graph acquisition unit, and is used to construct a city knowledge graph with entities as nodes and spatial relationships as edges, attach state attributes to each node, and set node state weights, thereby constructing a city entity state change matrix marked by timestamps, updating building expansions and road topology changes in real time, and outputting a city knowledge graph; The constraint condition acquisition unit is in communication with the urban knowledge graph acquisition unit and is used to convert and process the urban planning policy database into an executable logical rule set and match it with the knowledge graph node status. When a rule violation is detected, it automatically generates and outputs a structured warning report with an illegal construction warning event set and a policy compliance label, forming dynamic policy constraints for subsequent multi-objective optimization.

Images