Three-dimensional live-action model generation method for city updating

Abstract

The invention relates to a three-dimensional live-action model generation method for city updating. The method comprises the following steps: collecting multi-source city image information, and obtaining semantic data according to the multi-source city image information; performing city modeling visualization according to the multi-source city image information to generate a city model; and matching the semantic data with the city model to obtain a three-dimensional real scene model containing the semantic data. According to the method, city information can be obtained more comprehensively by performing total factor accurate collection through a multi-source three-dimensional monitoring technology, the accuracy of a real scene model is improved, semantic data is matched with a city model, a three-dimensional real scene model containing the semantic data is obtained, multiple materials and textures of the model are combined, and the accuracy of the real scene model is improved. And the model is lightened, so that the purpose of reducing the resource loading time is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of three-dimensional geometric modeling engines, in particular to the field of three-dimensional modeling, and more particularly to a three-dimensional real scene model generation method for urban renewal. BACKGROUND

[0002] China's urbanization is shifting from a period of rapid growth to a period of stable development, and urban development is shifting from a large-scale incremental expansion phase to a phase of mainly improving the quality and efficiency of the stock. Urban development faces many challenges such as dynamic conversion, structural optimization, quality improvement, green transformation, cultural continuity, and governance efficiency. "Urban renewal" is generally considered an important lever to address these challenges.

[0003] Urban renewal requires a comprehensive and accurate understanding of the current situation of the city, including the location, height, structure, appearance of buildings, the direction, width, and slope of roads, and the distribution of public facilities and other information. Three-dimensional real scene models restore the current situation of the city through high-precision digitization, providing a visual and analyzable platform for urban renewal, which helps to improve the efficiency and quality of urban renewal and reduce mistakes and risks in the planning and construction process. At the same time, the complex scenarios of urban renewal have special requirements for three-dimensional real scene models: first, multi-source and multi-modal data need to be collected and fused to improve the information density of the model to meet the requirements of expressing both macro regional patterns and micro architectural details; second, semantic understanding of complex scenarios is needed to automatically identify and classify buildings, infrastructure, temporary structures, and other elements from chaotic urban scenes, while binding attributes such as age, property rights, and purpose; third, low-cost dynamic updates are needed to achieve efficient response to local changes and rapid iteration of the model, and to support real-time invocation and collaboration by terminal users at the grassroots level.

[0004] Current three-dimensional real scene model generation mainly relies on techniques such as oblique photogrammetry, LiDAR, and manual modeling (such as BIM), but these techniques have significant limitations in the context of urban renewal. Oblique photogrammetry can quickly generate large-scale real scene models, but lacks semantic information such as building functions and property boundaries, and has poor reconstruction results for vegetation-shaded areas, making it difficult to support precise analysis in the context of urban renewal. LiDAR has high precision but is expensive and generates large amounts of data, making it inefficient for semantic segmentation in complex scenarios such as historical districts. BIM relies on professional teams for manual construction, which is time-consuming and costly, and cannot meet the frequent local data update needs in the context of urban renewal. SUMMARY

[0005] Therefore, it is necessary to provide a three-dimensional real scene model generation method for urban renewal that balances precision and lightweight.

[0006] The present application provides a three-dimensional real scene model generation method for urban renewal, comprising: S1: Collecting multi-source city image information, and obtaining semantic data according to the multi-source city image information; S2: Visualizing city modeling according to the multi-source city image information, and generating a city model; S3: Matching the semantic data with the city model to obtain a three-dimensional real scene model containing the semantic data.

[0007] Further, in step S1, the multi-source city image information is specifically: On the basis of satellite remote sensing global coverage, for the key areas of city renewal, centimeter-level image information is obtained by using unmanned aerial vehicle oblique photography: building facade image information is obtained by oblique photography, and wall deformation image information is captured in real time by combining crack monitoring sensors; In building reconstruction monitoring, ground laser radar and infrared thermal imaging are fused to identify structure hollowing and steel corrosion area image information; in urban village reconstruction, time and space sequence image information is obtained by periodic oblique photography; dynamic physical sign perception is obtained by an Internet of Things sensor network; In underground pipe network updating, optical fiber grating sensors are implanted to obtain pipe gallery settlement and pressure fluctuation image information; intelligent electric meters are installed in energy reconstruction projects to obtain energy consumption thermal image information.

[0008] Further, in step S1, the multi-source city image information is divided into frequent item sets and non-frequent item sets, wherein the frequent item set refers to multi-source city image information with an update frequency higher than a predetermined threshold in the city update process, which is an important or potential association rule pattern in the city update and management process, and the non-frequent item set refers to multi-source city image information with a frequency lower than a predetermined threshold in the city update process, which is not considered as a potential association rule pattern in the city update and management process, and is defined as follows:

[0009]

[0010] Wherein, Maxlnt i is the maximum value range of the ith item, lnt i is the value interval size of the ith item of the association rule: wherein, UB i and LB i are the upper and lower boundaries of the association rule interval.

[0011] Further, obtaining semantic data according to multi-source city image information is specifically: Estimating semantic data from a single or multiple multi-source city image information, and the semantic data includes various city landscape attributes; obtaining semantic data according to multiple multi-source city image information includes: The semantic data is obtained by increasing the shooting angles or using different direction lightings; the features of the input image are extracted by the filter of the convolutional neural network (CNN), the neurons between two adjacent network layers are locally connected through convolution operation and weight sharing strategy, the input image from multiple sources is further fused into a fixed size feature map through the average pooling layer, the similarity of multiple input image blocks is measured through the sampling number of the loss function to calculate the loss value, and the dimension of the feature map is reduced, so that the processing of any number of input images is realized, and the semantic data is obtained.

[0012] Further, in the estimation of semantic data of multiple images, the multiple images need to be fused, and after the pre-processing of the fusion strategy, it is divided into three stages, each stage has different focuses and loss function combinations, mainly including multi-scale physical consistency loss, anisotropic feature perception loss, adaptive adversarial training framework, closed-loop rendering consistency loss and dynamic loss weighting mechanism; the optimization direction includes: introducing multi-objective joint optimization to capture details at different levels, ensure semantic correlation of cross-scale data, and realize stepless scaling from city-wide to building details.

[0013] Multi-objective joint optimization:

[0014] LMSE: pixel-level material parameter error LSSIM: structural similarity constraint LVGG: VGG-19 feature matching Dynamic weight Using physical constraints to constrain the city landscape in the preset CIM model, reducing the perceptual loss in the landscape iteration process by analyzing the interdependence between the landscape image parameters in the city update and the correlation between the roughness; Through the adaptive adversarial loss function and the dynamic weighted total loss function, the physical constraints, visual quality and rendering accuracy are compared with the real image in the discriminator or multi-scale discriminator of the adversarial training, forming a closed-loop optimization; According to the differentiable renderer, the weights of different city landscape attributes are allocated to ensure that each parameter is optimized in the spatial analysis in the loss function; Further, in step S2, the city modeling visualization according to the multi-source city image information is specifically: S2.1: pre-processing the multi-source city image information to obtain pre-processed multi-source city image information; S2.2: constructing a data set according to the pre-processed multi-source city image information; S2.3: constructing a toon model and a picture quality enhancement model; S2.4: training the text-to-image model based on the LoRA method through the dataset to obtain a trained text-to-image model; training the image quality enhancement model based on the SwinIR method through the dataset to obtain a trained image quality enhancement model; S2.5: based on the trained text-to-image model and the trained image quality enhancement model, a virtual model is established using the digital twin professional 3D modeling, exported in GLTF format, and combined with indexDB technology to establish a city model.

[0015] Further, step S3 is specifically: The semantic data is associated with the city model through a unique identifier, and multiple materials and textures of the city model are merged to reduce the size of the texture material map, thereby achieving the purpose of reducing resource loading time. The specific method includes: Semantic graph construction: a graph neural network is used to establish the correlation of city elements, and the attributes of BIM components are dynamically bound with the UV coordinates of the oblique photography model to form a triple index library of <UUID, spatial coordinates, semantic vector>. The semantic vector dimension is designed to be 512 dimensions, including multiple building function attributes, multiple pipeline material characteristics, and multiple geomechanical parameters; Texture intelligent compression: based on the adaptive texture merging algorithm of the generative adversarial network, the original oblique photography map and the BIM material library are input, and a 4K composite texture map is generated through the StyleGAN2 framework, while the normal map and the specular reflectance parameter are extracted; Deep coupling of oblique photography model and city model is achieved through spatio-temporal attribute dynamic binding technology, including: For the scenario of old community renovation, a structural health semantic encoder is developed to cross-modally correlate the steel corrosion rate, concrete carbonation depth in the BIM model with the oblique photography crack texture to form a dynamic decision vector of <building UUID, safety level, renovation priority>. In the protection of historical blocks, a multi-spectral material analysis algorithm is used to fuse visible light, near-infrared, and thermal imaging oblique photography data to automatically identify traditional building materials and establish a historical building material degradation time series graph. Dynamic texture optimization: a texture synchronization engine for the updating process is developed to support pixel-level comparative analysis of building facades before and after renovation; a generative adversarial network is used to fuse the BIM material of construction design with the oblique photography texture of the current situation to generate a renovation effect preview, thereby improving the compression rate while retaining the texture features of historical buildings.

[0016] Further, step S3 further includes: Through spatio-temporal R-tree indexing, the key areas of urban renewal are detected in real time, and the high-precision modeling range is automatically determined. The grid density of key areas is maintained at 100-300 triangular faces per square meter, and that of non-key areas is reduced to 10-30 faces per square meter. For the urban renewal scenario, a grid simplification priority model is established: The safety risk coefficient is dynamically calculated using structural sensor data, while the historical value coefficient is assigned based on the protection level. To address the needs of existing building renovation, the design retains geometric coding with details preserved, applies non-destructive compression to key components, and applies destructive simplification to non-load-bearing parts.

[0017] Furthermore, step S3 also includes: LOD technology is used to optimize multi-character rendering and simulate lighting and shadows during the update process; First, based on scene interaction, distance, and viewing angle, select an appropriate mesh detail level to enable scene roaming, scaling, rotation, movement, hiding, and other functions in Three.js, reducing the rendering burden; A dynamic detail adjustment model is built based on reinforcement learning to dynamically adjust details according to the user's identity level. Designer's Perspective: Showcasing the Millimeter-Level Integration of BIM MEP Piping with Oblique Photogrammetry Facade From the perspective of construction workers: highlighting the early warning area for construction deviations Public Perspective: Comparing the Effects of Community Renovation By pre-calculating scene lighting distribution through Monte Carlo integration and combining screen space reflection and cascaded shadow mapping, the time spent on shadow calculation is reduced while ensuring visual quality. For complex materials, ray differential tracing technology is used to reduce the amount of reflection and refraction calculations, thereby achieving joint optimization of lighting and shadow. By combining the accurate simulation of light and shadow changes in different seasons and time periods during the renovation of historical districts, and by fusing the lighting information of historical frames in multi-source urban image information with the current frame data, the real-time computing overhead can be reduced.

[0018] Furthermore, step S3 also includes: The creation and update process involves resource pooling management of loading, caching, and reuse models, and periodically cleaning up no longer needed model resources using an LRU caching strategy, including: Resource pooling during the update process involves constructing a perceptive caching model for urban renewal and transformation stages, dividing resources into: planning phase resources, construction phase resources, and operation and maintenance phase resources. The update process uses resource pooling. First, a multi-level cache pool architecture is built: a dynamic priority caching strategy is designed. This strategy allocates data streams to different queues based on different weights during the planning, construction, and operation and maintenance phases. According to the principle that the larger the weight, the higher the priority, different queues will have different priorities.

[0019] In this method, multi-source three-dimensional monitoring technology is used to collect all elements accurately, which can obtain more comprehensive urban information and improve the accuracy of the real scene model. By matching semantic data with the urban model, a three-dimensional real scene model containing semantic data is obtained. At the same time, multiple materials and textures of the model are merged to make the model lightweight and reduce resource loading time. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for generating a 3D reality model for urban renewal in one embodiment; Figure 2 This is a 3D framework diagram that corresponds to an actual urban landscape in one embodiment; Figure 3 Here is a network layer architecture diagram from one embodiment; Figure 4 This is a schematic diagram of the dynamic threshold sparsity mechanism in one embodiment; Figure 5 This is a visual program diagram of one embodiment; Figure 6 A flowchart for generating a city street view rendering based on text description in one embodiment; Figure 7 This is a schematic diagram of the encoder and decoder in the original svbrdf-net model in one embodiment; Figure 8 This is a schematic diagram of the encoder and decoder after the attention module is fused in one embodiment; Figure 9 This is a flowchart of a multimodal image fusion process in one embodiment; Figure 10 This is a schematic diagram of a CIM model in one embodiment; Figure 11 This is a flowchart of an ablation experiment in one embodiment. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0022] To address the aforementioned technological gaps, this method proposes an integrated solution for 3D reality modeling in urban renewal, encompassing "data acquisition → semantic modeling → lightweighting." This includes: using multi-source stereoscopic monitoring technology to accurately collect all elements, comprehensively obtain urban information, and improve the accuracy of the reality model; matching semantic data with the urban model to obtain a 3D reality model containing semantic data; and merging multiple materials and textures of the model to perform lightweight optimization and reduce information loading latency.

[0023] Example 1: This embodiment provides, as follows: Figure 1 The method for generating 3D reality models for urban renewal, as shown, includes: S1: Accurately collect all elements through multi-source three-dimensional monitoring technology to obtain multi-source urban image information, and obtain semantic data based on the multi-source urban image information; S2: Visualize and model cities based on multi-source city image information to generate city models; S3: Match semantic data with the city model to obtain a 3D real-world model containing semantic data.

[0024] In this embodiment, multi-source three-dimensional monitoring technology is used to collect all elements accurately, which can obtain more comprehensive urban information and improve the accuracy of the real scene model. By matching semantic data with the urban model, a three-dimensional real scene model containing semantic data is obtained. At the same time, multiple materials and textures of the model are merged to make the model lightweight and reduce resource loading time.

[0025] Example 2: This embodiment further discloses information based on Embodiment 1: Furthermore, in step S1, the multi-source city image information specifically includes: Based on the full coverage of satellite remote sensing, centimeter-level image information is obtained by using UAV oblique photography for key areas of urban renewal: oblique photography is used to obtain building facade image information, and crack monitoring sensors are combined to capture wall deformation image information in real time. In building renovation monitoring, ground-based lidar and infrared thermal imaging are integrated to identify image information of structural voids and steel corrosion areas; in urban village renovation, spatiotemporal sequence image information is obtained through periodic oblique photography; dynamic vital signs are perceived through IoT sensor networks: geomagnetic vehicle detectors are deployed in traffic renovation areas to obtain image information of traffic organization efficiency. Fiber optic sensors are implanted during underground pipeline network upgrades to obtain image information on pipeline settlement and pressure fluctuations; smart meters are installed in energy renovation projects to obtain image information on energy consumption and heat.

[0026] Further, step S1 includes: multi-source urban image information is divided into frequent itemsets and infrequent itemsets. Frequent itemsets refer to multi-source urban image information with an update frequency higher than a predetermined threshold during the urban renewal process. These are patterns that represent important or potential association rules in the urban renewal and management process. Infrequent itemsets refer to multi-source urban image information with a frequency lower than a predetermined threshold during the urban renewal process. These are patterns that are not considered to have potential association rules in the urban renewal and management process. Their definitions are as follows:

[0027]

[0028] Where MaxInti is the maximum value range of the i-th item, and Inti is the size of the value range of the i-th item of the association rule: In the formula, UB and LB are the upper and lower boundaries of the association rule interval, respectively.

[0029] Furthermore, the semantic data obtained from multi-source urban image information specifically includes: Semantic data is estimated from single or multiple multi-source urban image information. The semantic data includes urban landscape attributes. Semantic data obtained from multiple multi-source urban image information includes: Semantic data is obtained by taking pictures from different angles or using lighting from different directions; features of the input images are extracted using filters of a convolutional neural network (CNN); neurons between two adjacent network layers are locally connected through convolution operations and weight sharing strategies; an average pooling layer further fuses input images from multiple sources into a feature map of a fixed size; and the similarity of multiple input image patches is measured by the number of samples in the loss function to calculate the loss value, thereby reducing the dimensionality of the feature map. This enables the processing of any number of input images to obtain semantic data.

[0030] Furthermore, in estimating semantic data from multiple images, it is necessary to fuse the multiple images. After preprocessing the fusion strategy, it is divided into three stages, each with different focuses and loss function combinations. These mainly include multi-scale physical consistency loss, anisotropic feature perception loss, adaptive adversarial training framework, closed-loop rendering consistency loss, and dynamic loss weighting mechanism. Optimization directions include: introducing multi-objective joint optimization to capture details at different levels, ensuring semantic association of cross-scale data, and achieving stepless scaling from the entire city area to architectural details.

[0031] Multi-objective joint optimization:

[0032] LMSE: Pixel-level material parameter error LSSIM: Structural Similarity Constraint LVGG: VGG-19 Feature Matching For dynamic weights

[0033] Physical constraints are used to constrain the urban landscape in the pre-defined CIM model, and perceptual loss during the landscape iteration process is reduced by analyzing the interdependencies between landscape image parameters and the correlation between roughness in urban renewal. Through the adaptive adversarial loss function and the dynamic weighted total loss function, the physical constraints, visual quality, and rendering accuracy are compared between the discriminator or multi-scale discriminator rendering results and the real images in adversarial training, forming a closed-loop optimization; Allocate weights to different urban landscape attributes according to the differentiable renderer, ensuring that each parameter is optimized in the spatial analysis in the loss function; Furthermore, in step S2, the urban modeling visualization based on multi-source urban image information is specifically as follows: S2.1: Preprocess the multi-source urban image information to obtain the preprocessed multi-source urban image information; S2.2: Construct a dataset according to the preprocessed multi-source urban image information; S2.3: Construct a text-to-image model and an image quality enhancement model; S2.4: Train the text-to-image model based on the LoRA method through the dataset to obtain the trained text-to-image model; train the image quality enhancement model based on the SwinIR method through the dataset to obtain the trained image quality enhancement model; S2.5: Establish a virtual model using digital twin professional 3D modeling based on the trained text-to-image model and the trained image quality enhancement model, export it in the GLTF format, and combine it with the indexDB technology to establish a city model.

[0034] Furthermore, step S3 is specifically as follows: Associate the semantic data with the city model through a unique identifier, merge multiple materials and textures of the city model, reduce the size of the texture material map, and achieve the purpose of reducing the resource loading time. The specific methods include: Semantic graph construction: Use a graph neural network to establish the association relationship of urban elements, dynamically bind the attributes of BIM components with the UV coordinates of the oblique photography model, and form a triple index library of <UUID, spatial coordinates, semantic vector>; the dimension of the semantic vector is designed to be 512 dimensions, including multiple types of building function attributes, multiple pipeline material characteristics, and multiple geomechanical parameters; Intelligent texture compression: Based on the adaptive texture merging algorithm of the generative adversarial network, input the original oblique photography map and the BIM material library, generate a 4K composite texture map through the StyleGAN2 framework, and extract the normal map and the specular reflectivity parameters at the same time; Achieve the deep coupling of the oblique photography model and the city model through the spatio-temporal attribute dynamic binding technology, including: For the renovation of old residential communities, a structural health semantic encoder was developed to cross-modal correlate the steel corrosion rate, concrete carbonation depth and oblique photography crack texture in the BIM model, forming a dynamic decision vector of <building UUID, safety level, renovation priority>. In the protection of historical districts, a multispectral material analysis algorithm is adopted to integrate visible light, near infrared and thermal imaging oblique photography data to automatically identify traditional building materials and establish a time series map of the degradation of ancient building materials. Dynamic texture optimization: Develop a texture synchronization engine for the update process, supporting pixel-level comparative analysis of building facades before and after renovation; use generative adversarial networks to integrate construction design BIM materials with existing textures from oblique photography to generate a preview of the renovation effect, improving the compression rate while preserving the textural features of historical buildings.

[0035] Furthermore, step S3 also includes: Using a spatiotemporal R-tree index, key areas for urban renewal are detected in real time, and the high-precision modeling range is automatically defined. The grid density in key areas is maintained at 100-300 triangles / ㎡, while it is reduced to 10-30 triangles / ㎡ in non-key areas. A grid simplification priority model is established specifically for urban renewal scenarios. The safety risk coefficient is dynamically calculated using structural sensor data, while the historical value coefficient is assigned based on the protection level. To address the needs of existing building renovation, the design retains geometric coding with details preserved, applies non-destructive compression to key components, and applies destructive simplification to non-load-bearing parts.

[0036] Furthermore, step S3 also includes: LOD technology is used to optimize multi-character rendering and simulate lighting and shadows during the update process; First, based on scene interaction, distance, and viewing angle, select an appropriate mesh detail level to enable scene roaming, scaling, rotation, movement, hiding, and other functions in Three.js, reducing the rendering burden; A dynamic detail adjustment model is built based on reinforcement learning to dynamically adjust details according to the user's identity level. Designer's Perspective: Showcasing the Millimeter-Level Integration of BIM MEP Piping with Oblique Photogrammetry Facade From the perspective of construction workers: highlighting the early warning area for construction deviations Public Perspective: Comparing the Effects of Community Renovation By pre-calculating scene lighting distribution through Monte Carlo integration and combining screen space reflection and cascaded shadow mapping, the time spent on shadow calculation is reduced while ensuring visual quality. For complex materials, ray differential tracing technology is used to reduce the amount of reflection and refraction calculations, thereby achieving joint optimization of lighting and shadow. Combined with accurately simulating the light and shadow changes in different seasons and time periods during the renovation of historical blocks, the lighting information of historical frames in multi-source urban image information is fused with the current frame data to reduce the real-time calculation overhead.

[0037] Furthermore, step S3 also includes: Create a pooling management model for loading, caching, and reusing the resources in the update process. Regularly clean up the model resources that are no longer needed through the LRU caching policy, including: Pooling the resources in the update process, constructing a perception-based caching model for the urban renewal and transformation stage, and dividing the resources into: resources in the planning period, resources in the construction period, and resources in the operation and maintenance period Pooling the resources in the update process, first construct a multi-level caching pool architecture: design a dynamic priority caching strategy (DPCS). This strategy distributes the data stream to different queues according to the different weights in the planning period, construction period, and operation and maintenance period. Based on the principle that the greater the weight, the higher the priority, different queues have different priorities, represented by QP, and QPi represents the priority of the i-th queue. And QPi = QWMAXi + QWMINi

[0038] where, QWMAXi is the maximum weight of queue i; QWMINi is the minimum weight in queue i. There are a total of N queues, and the weights are normalized to obtain the output normalized weights that reflect the share of the output resources occupied by the queues.

[0039] In this embodiment, through the multi-source three-dimensional monitoring technology, all-element precise acquisition can obtain more comprehensive urban information, improve the accuracy of the real-scene model, and while matching the semantic data with the urban model to obtain a three-dimensional real-scene model containing semantic data, merge multiple materials and textures of the model to lightweight the model, achieving the purpose of reducing the resource loading time.

[0040] Embodiment Three: This embodiment further discloses on the basis of Embodiment Two: This embodiment realizes the deep coupling of the oblique photography model and CIM data through the original attribute-geometry-texture triple coding mechanism. The specific method includes: Semantic graph construction: Use a graph neural network (GNN) to establish the association relationship of urban elements, and dynamically bind the attribute parameters such as the material parameters of BIM components (IFC4.3 standard) and pipeline pressure values to the UV coordinates of the oblique photography model to form a triple index library of <UUID, spatial coordinates, semantic vector>. The dimension of the semantic vector is designed to be 512 dimensions, including 28 types of building function attributes, 16 types of pipeline material characteristics, and 12 geomechanical parameters.

[0041] Intelligent Texture Compression: An adaptive texture merging algorithm based on Generative Adversarial Network (GAN) was developed. The algorithm takes the original oblique photogrammetric map (8K resolution) and BIM material library as input and generates a 4K composite texture map (PSNR≥42dB) through the StyleGAN2 framework. At the same time, the normal map (accuracy 0.1mm) and specular reflectance parameters (error <3%) are extracted, achieving a texture volume compression of 85% and a visual fidelity improvement of 60%.

[0042] Furthermore, the core of this invention lies in constructing a dedicated data fusion engine for urban renewal. Through dynamic spatiotemporal attribute binding technology, it achieves deep coupling between oblique photogrammetry models and CIM business data, solving the challenge of multi-source heterogeneous data fusion in existing building renovation and historical district preservation. Key application technologies include: 1. Enhanced Semantic Updates: For the renovation of old residential communities, a structural health semantic encoder was developed to cross-modal correlate steel corrosion rate (accuracy 0.1%) and concrete carbonation depth (LiDAR detection) in the BIM model with oblique photography crack texture (0.02mm resolution), forming a dynamic decision vector of <building UUID, safety level, renovation priority>. In the preservation of historical districts, a multispectral material analysis algorithm is adopted, integrating visible light, near-infrared, and thermal imaging oblique photography data to automatically identify traditional building materials (water absorption rate of blue bricks, insect-infested areas of wooden components), and establish a time-series map of ancient building material degradation (annual change rate prediction error <3%).

[0043] 2. Dynamic Texture Optimization: Develop a texture synchronization engine for the update process, supporting pixel-level comparative analysis of building facades before and after renovation. By using Generative Adversarial Network (CycleGAN), the construction design BIM materials are fused with the existing textures from oblique photogrammetry to generate a pre-renovation effect image (PSNR≥45dB), improving the compression rate by 80% while preserving the textural features of the historical building.

[0044] In this embodiment, multi-source three-dimensional monitoring technology is used to collect all elements accurately, which can obtain more comprehensive urban information and improve the accuracy of the real scene model. By matching semantic data with the urban model, a three-dimensional real scene model containing semantic data is obtained. At the same time, multiple materials and textures of the model are merged to make the model lightweight and reduce resource loading time.

[0045] Example 4: This embodiment further discloses information based on Embodiment 1: A three-dimensional framework consistent with the actual urban landscape is established to uniformly and systematically describe the spatial structure of the city. This framework comprises four layers: infrastructure, network, cloud platform, and application. Figure 2 As shown: 1. Infrastructure Layer As the physical sensing foundation for urban renewal, it achieves accurate collection of all elements through multi-source three-dimensional monitoring technology. The main contents include: (1) Based on the full coverage of satellite remote sensing (0.5-meter resolution), for key areas of urban renewal, the UAV oblique photography (3-cm accuracy) is used to generate centimeter-level real-scene models: In the renovation of old residential areas, a fine model of the building facade (200 faces / ㎡ of triangular mesh density) is constructed by 5-lens oblique photography, and the wall deformation is captured in real time by combining crack monitoring sensors (0.02mm accuracy); (2) In the monitoring of building renovation, ground-based lidar (point cloud density 500pts / ㎡) and infrared thermal imaging are integrated to accurately identify structural voids (temperature difference > 3℃ alarm) and areas of steel corrosion (recognition rate 92%); in the renovation of urban villages, a spatiotemporal sequence model is constructed through periodic oblique photography (once a week) to automatically detect changes in illegal construction (change detection accuracy > 95%). At the same time, an Internet of Things sensor network is deployed to realize dynamic vital sign perception: geomagnetic vehicle detectors are deployed in traffic renovation areas (data update frequency 1Hz) to monitor traffic organization efficiency in real time; (3) Fiber optic grating sensors (spacing ≤ 30m) are implanted in the underground pipeline network to dynamically sense the settlement of the pipeline corridor (accuracy ± 0.1mm) and pressure fluctuations; smart meters are installed in the energy transformation project (data is transmitted back at the 15-minute level) to construct an energy consumption heat map.

[0046] This system compresses TB-level data to hundreds of GB-level data through lightweight processing technology (OSGB to 3DTiles) and establishes spatial semantic association rules (RDF triples), enabling automatic matching of building tiles in the oblique photogrammetry model with BIM component IDs in the CIM platform. It meets the requirements of the "Smart City Spatiotemporal Infrastructure Standard" for monitoring point spacing in key areas ≤ 50 meters (20 meters for old residential areas) and dynamic data updates in minutes. It supports multiple application scenarios such as 3D simulation of update schemes (loading delay < 200ms), construction safety early warning (response time ≤ 5s), and evaluation of renovation effects. Compared with traditional methods, it improves data collection efficiency by 300% and reduces on-site inspection costs by 65%.

[0047] 2. Network Layer As the digital transmission hub for urban renewal, it relies on 5G private network slicing technology to build differentiated transmission channels: deploy uRLLC (ultra-reliable low-latency communication) slices in the renovation of old communities to ensure that the transmission latency of structural monitoring data (such as tilt sensors and vibration meters) is ≤8ms and the reliability is 99.999%.

[0048] In urban renewal scenarios employing mMTC (massive machine-type communications) slicing, this invention utilizes an IoT slicing method based on association rules to better extract the required content from repetitive urban renewal tasks. While ensuring core processing capabilities, it minimizes the transmission and storage of invalid information, thereby improving system performance, effectively analyzing massive amounts of data, and reducing costs. The implementation method is as follows: Figure 3 As shown; First, sensor nodes in the Internet of Things (IoT) are modeled to fuse multi-source, heterogeneous, and spatiotemporally sensitive data. This involves associating and processing basic device information, business information, and sensor information, and storing the model results in a sensor node database. Data acquisition involves collecting data from sensors, devices, or other IoT nodes and transmitting it over a network to centralized storage or a cloud platform for further processing and storage. To address the challenges of massive data volume, heterogeneity, spatiotemporal sensitivity, and dynamic streaming characteristics in IoT sensor sampling data management, this invention proposes a sensor node cluster database for managing massive IoT sensor sampling data. The main working process of the sensor access processor is as follows: (1) Numericalization of non-numerical sampling data: The sensor access processor needs to perform relevant multimedia data analysis to obtain numerical exported data that can reflect the physical state of the monitored object. According to the predefined data mapping and transformation relationship, the relevant data is converted into data that conforms to the unified object model specification. This includes scenarios such as historical block protection monitoring, traffic flow dynamic optimization, and urban village renovation spatial analysis, which urgently require multi-source data fusion to support decision-making and high-altitude remote sensing image extraction of spatial data information such as the geographical geometry of historical and cultural protection areas and urban villages. Traffic cameras: extract vehicle density

[0049] Geometric features of historical and cultural conservation areas / urban villages are extracted using a U-Net model from remote sensing images. The area ratio of irregular polygons is calculated, and the density index of urban villages is also calculated.

[0050] (2) Sparsification of densely collected data: Addressing the high-frequency monitoring needs in urban renewal scenarios (such as pedestrian density in historical districts and peak traffic flow on main roads), this invention proposes the following... Figure 4The dynamic threshold sparsity mechanism described above involves extracting key data, dividing the raw data into frequent itemsets and infrequent itemsets, and extracting key frequent itemsets that reflect changes in the physical state of the monitored target from the stream. Only the key frequent itemsets data are uploaded to the sensor spatiotemporal database model for processing. The method involves defining a state change threshold for each type of sensor. If the difference between the new sampled data and the previously uploaded key frequent itemsets data does not exceed the threshold, no processing is required. Only when the difference exceeds the specified threshold does the new sampled data become key frequent itemsets data that needs to be uploaded, thus filtering out a large amount of redundant data.

[0051] By defining a sensor-specific state change threshold δ, key data items reflecting changes in the monitored target's state are intelligently extracted from the raw data stream. When the newly sampled data xt satisfies |xt−xt−Δt|≤δ with the previously uploaded data xt−Δt, it is considered redundant data and discarded; while when |xt−xt−Δt|>δ, xt is uploaded as key state data to the spatiotemporal database. Practice has shown that in urban village renovation monitoring, this mechanism filters out 90% of static building redundancy images, significantly reducing data transmission load.

[0052] (3) For the mobile target monitoring needs in urban renewal (such as drones patrolling historical blocks and vehicles transporting goods at demolition sites), the sensor access processor performs spatiotemporal object modeling. The sensor access processor needs to extract the identifier of the mobile monitoring object and form a sampling record together with the relevant sampling time and sampling location. Then, the sampling record is fused and loaded. In order to quickly query and analyze the sampling information of each monitoring object, the sensor sampling data should be organized by monitoring object, so that all the data of the same monitoring object are stored together and change dynamically over time. Therefore, the sensor sampling data exhibits sequential and dynamically changing streaming characteristics.

[0053] Secondly, regarding the cleaned and preprocessed data: to eliminate errors, noise, and inconsistencies, the model cleans and preprocesses the data collected during the urban renewal process to obtain valuable urban planning information. In-depth analysis of the data can uncover hidden patterns, trends, and correlations, providing more evidence for decision-making.

[0054] Third, rule association is conducted. Association rules refer to a new method used to address the correlation between information in an urban renewal dataset. Association rules encompass multiple perspectives, including building renovation and preservation, cultural and commercial integration, community participation and resident wishes, and cultural heritage protection and utilization, as well as multiple levels such as micro-renovation, comprehensive improvement, protection and restoration, and facility renovation. They study the relationships, correlations, and complex connections between events. These relationships are categorized into frequent itemsets and infrequent itemsets. Frequent itemsets are those with a frequency exceeding a predetermined threshold during the urban renewal process; they represent patterns of important or potential association rules in urban renewal and management. Infrequent itemsets, on the other hand, are those with a frequency below a predetermined threshold; they are patterns not considered to have potential association rules in urban renewal and management. Their definitions are as follows:

[0055] Frequent itemsets are crucial in association rule analysis. By identifying all frequent itemsets, strong association rules can be generated. These rules can be used in areas such as building surveys, land use surveys, population surveys, traffic surveys, public service facility surveys, environmental facility surveys, municipal facility surveys, environmental sanitation surveys, community relationship surveys, and spatial site surveys during urban renewal processes, providing a basis for scientific decision-making.

[0056] Where MaxInti is the maximum value range of the i-th item, and Inti is the size of the value range of the i-th item of the association rule: In the formula, UB and LB are the upper and lower boundaries of the association rule interval, respectively. The smaller the average interval, the higher the accuracy.

[0057] Fourth, a machine learning-based approach is used to build the model, and the items to be processed are grouped. Based on the grouping criteria, they are divided into urban street scene training, historical block micro-update samples, pedestrian density detection, and infrastructure sensor access, which are then used to achieve cross-business time-series collaboration through Time-Sensitive Network (TSN).

[0058] Fifth, deploy edge gateways. Edge gateways are the core devices for building the urban renewal CIM platform. They are used to receive information from urban management IoT devices and send data to the CIM data processing platform using the MQTT protocol. They mainly include functions such as protocol conversion, time calibration, function calculation, data encryption and decryption, and FPGA protection. They can obtain oblique photography models (LOD3 level) and overlay real-time urban pipeline pressure data beyond conventional permissions, realize cross-departmental data exchange, and achieve a 100% interception rate of illegal data tampering, ensuring 100% reliable traceability of urban renewal monitoring data.

[0059] Sixth, a dual-chain fusion architecture is adopted to enhance data credibility. The Hyperledger Fabric technology of blockchain is used to manage low-frequency, high-density data such as design drawings and approval records. The SM4 national cryptographic algorithm is combined to implement transmission encryption. For high-definition oblique photography models, fragmentation encryption and streaming transmission technology are used to build an intelligent protocol gateway to achieve multi-source heterogeneous data fusion. In the building renovation scenario, infrastructure operation status and energy consumption data are aggregated in real time. Finally, by deploying embedded edge computing nodes (MECs), local updates and local processing of oblique photogrammetry models are achieved. In terms of collaboration between edge computing nodes and the cloud platform layer, the edge computing nodes and the cloud platform layer can communicate and interact through a communication network. The edge computing nodes can process and provide feedback to the cloud platform, and the cloud platform can optimize and adjust based on the feedback results from the edge computing nodes to meet the real-time requirements of dynamic simulation of urban renewal.

[0060] 3. Cloud Platform Layer As the intelligent hub of urban renewal, a CIM digital twin engine based on a microservice architecture is constructed to achieve deep understanding and dynamic simulation of all urban elements. The core components utilize a spatiotemporal database cluster (PostgreSQL + TimescaleDB), supporting trillions of spatiotemporal events and possessing millisecond-level spatiotemporal slice query capabilities (response time <50ms). It can integrate oblique photogrammetry models, BIM components, IoT time-series data (such as 100,000+ sensor readings per second), and social big data (population heatmaps, public opinion hotspots). Through a lightweight BIM engine (based on WebAssembly technology), the IFC4.3 building model is compressed to 15% of its original volume (preserving MEP pipeline connections and equipment parameters), achieving lossless conversion with the CityGML 3.0 municipal facility model (geometric error <3mm). A city-level semantic knowledge graph (over 50 million entity relationships) is constructed, covering over 200 categories of elements, including architectural decorative lines (LOD4), underground utility tunnel supports, and geological fault zones.

[0061] The platform offers a matrix of intelligent analysis services: The spatial analysis API supports compliance review of renovation plans (such as setback detection accuracy of ±0.1m), solar radiation simulation (based on ray tracing algorithm, shadow calculation error <5 minutes), and automatic calculation of demolition volume (3D voxel segmentation accuracy of 99%). The multiphysics simulation engine couples structural mechanics (ANSYS kernel), fluid dynamics (OpenFOAM optimized version) and acoustic propagation models, and can deduce reinforcement schemes for historical buildings (deformation prediction error <2mm) and update regional wind and thermal environment (wind speed simulation deviation <0.3m / s). The machine learning pipeline has 20+ pre-trained models built-in, supporting energy consumption prediction (LSTM model MAPE < 8%), construction risk identification (YOLOv7 object detection mAP@0.5 reaches 92%), and material usage optimization (reinforcement learning saves 18%).

[0062] Digital threading technology enables seamless data flow across different phases: BIM models are automatically linked to construction progress photos (CV matching accuracy of 95%) and equipment ledgers during the operation and maintenance period, establishing a full lifecycle traceability chain. A dynamic resource scheduling strategy (Kubernetes + Istio) is adopted, elastically scaling to 500+ container instances during peak periods of renovation plan reviews, ensuring 3D collaborative annotation for 200 concurrent users (latency <100ms). The data governance system complies with ISO / IEC 30182:2017, implementing version control (Git-LFS management model version), metadata traceability (PROV-O standard), and sensitive data anonymization (differential privacy noise Δ=0.03).

[0063] The platform has supported an urban renewal project in Shenzhen, improving the efficiency of three-dimensional compliance review of 1,350 buildings within an 8-square-kilometer area by 400%, achieving a construction conflict early warning accuracy rate of 97%, and reducing design change costs by 65% ​​compared to traditional methods.

[0064] 4. Application Layer Integrating digital twin and blockchain technologies, this initiative empowers the entire lifecycle of urban renewal governance, including smart communities, urban services, secure sharing, and smart healthcare applications. Blockchain-based evidence storage enables reliable traceability throughout the entire process, from planning approvals to construction logs. Specifically, smart communities utilize CIM+BIM fusion to digitize the renovation of old residential areas, AI monitors illegal construction (95% accuracy), and blockchain-based evidence storage of resident voting data reduces disputes by 70%. Urban services dynamically optimize traffic signals (reducing delays by 35%), and smart contracts manage the sharing of charging stations and the trading of NFTs in utility tunnels. Secure sharing employs zero-knowledge proofs to achieve secure circulation of medical images, and blockchain-based evidence storage of emergency commands improves command efficiency by 50%. Smart healthcare optimizes hospital site selection (covering blind spots with an error of <100m), and hashing and recording treatment data on the blockchain ensures audit traceability.

[0065] Blockchain-based end-to-end data storage covers planning to operation and maintenance: changes to drawing versions are traced down to the CAD element level, construction logs and building material information (RFID + quantum tags) are uploaded to the blockchain in real time, and equipment data is encrypted and stored via TEE. Practice shows that the renovation cycle is shortened by 55%, building material waste is reduced by 42%, audit costs are reduced by 90%, and citizen satisfaction reaches 91%, achieving a coordinated upgrade of physical renovation and digital governance.

[0066] Example 5: This embodiment further discloses information based on Embodiment 1: The urban renewal display system platform based on Web API adopts a B / S architecture, enabling cross-platform use. The main business logic is implemented on the server side, forming a three-tier architecture. It utilizes various JavaScript scripting languages ​​and ActiveX technologies in browsers to implement complex functions. The system mainly consists of three modules: digital twin 3D models, data storage, and visualization applications. Figure 5 As shown.

[0067] By integrating 3D urban streetscape data from multiple elements such as street space, urban landscape, historical buildings, geological features, and urban components, the 3D model enables intuitive visualization and real-time interaction, thereby achieving multi-scale urban spatial visualization.

[0068] Digital Twin 3D Model Design This invention utilizes professional 3D modeling and design software (such as Unity and Unreal Engine) to create a virtual model, exports it to GLTF format and performs lightweight processing, and then imports it into a landscape virtual segmentation system. The system's designed clustering and grouping algorithm can group landscape attachments into their respective sections, ensuring they are selected as a whole, allowing the system to better present all street landscape details. Main features include: Model lightweight processing By simplifying the mesh and compressing the texture (such as GLTF format conversion) of complex models such as street space improvement, old residential area renovation and industrial heritage revitalization, this invention achieves lightweight model processing. It innovatively combines GLTF format conversion with indexDB technology to establish a three-dimensional visual model of topography, streets, buildings and green spaces. On the one hand, it solves the problem of lossless migration of BIM data when interfacing with CIM platform; on the other hand, it adopts a hybrid modeling method based on graphics and images and LOD (Level of Detail) and texture mapping technology to realize three-dimensional dynamic visual simulation in the process of urban renewal, achieving a visual effect with high realism.

[0069] IndexedDB is a new concept in HTML5. It is a Web API for supporting lightweight non-relational databases in mainstream browsers. It has advantages such as large capacity, asynchronous operation, transaction processing, and multi-version control. It can directly store various types of data (strings, binary, Blob objects) into its object repository in a "key-value pair" manner, and realize complex queries, transaction processing, cursors and indexes. It maintains only one database to achieve lightweight use. This invention uses IndexedDB technology for local caching of models to avoid page lag and ensure smooth loading on ordinary terminal devices.

[0070] Urban renewal currently faces specific challenges, such as dynamic data updates, multi-party collaboration, protection of historical buildings, and insufficient public participation. This invention combines existing technical solutions to further optimize the integration of oblique photography and CIM to address these challenges.

[0071] Urban renewal typically involves the renovation of old districts, infrastructure upgrades, and historical preservation, requiring comprehensive consideration of overall planning, construction supervision, and public participation. This invention utilizes oblique photography and SVBRDF technology to achieve detailed restoration and multi-dimensional simulation, enabling simultaneous demonstration of the renewal plan's effects.

[0072] The oblique photogrammetry model exists in the form of point clouds and meshes, while the CIM platform uses data formats such as BIM and GIS. This invention adopts the CGCS2000 National Geodetic Coordinate System data format and coordinate system, and unifies the oblique photogrammetry model and CIM platform data coordinates through a seven-parameter transformation model (translation, rotation, scaling), ensuring that the plane error is ≤0.1m and the elevation error is ≤0.3m.

[0073] Coordinate transformation formula:

[0074] Data Format Conversion: Since oblique photogrammetry typically generates large amounts of data, direct integration into the CIM platform may lead to performance issues. Mesh simplification algorithms may be necessary. This invention utilizes modeling export formats from Unity and Unreal Engine software, including FBX, OBJ, GLB, and GLTF2.0. Because GLTF2.0 reduces redundant data and is suitable for OpenGL cluster loading, it can compile scenes, cameras, animations, meshes, materials, and textures into binary content, better enabling Web API layout. This allows for the integration of animation and lighting data into the model, providing richer visual effects for urban renewal scenarios. Therefore, the system uses the GLTF2.0 format for lightweight model processing.

[0075] Example 6: This embodiment further discloses information based on Embodiment 5: This embodiment introduces the deep learning-based SwinIR algorithm for image super-resolution processing. By training on an urban landscape dataset, the model can deeply understand the intrinsic characteristics of urban landscape changes, thereby accurately reproducing the natural changes of the urban landscape over time during image generation. This innovation is crucial for improving the model's generalization ability and enhancing the realism of the generated images. It is particularly suitable for showcasing urban landscape changes during urban renewal scenarios involving evolution, transformation, or other special environmental conditions, making the model training more efficient and resource-saving.

[0076] Finally, in order to fully evaluate the effectiveness of the proposed method, this embodiment, for example... Figure 6 The method is iteratively retrained using LoRA and SwinIR. The performance of the model before and after optimization is compared to verify the effectiveness and advantages of the proposed method in generating urban landscape images, thereby improving the overall visual realism of the urban landscape during the urban renewal process.

[0077] The svbrdf-net technology uses convolutional structures as the basis for feature extraction. In the field of deep learning, convolutional neural networks (CNNs) are commonly used feedforward neural networks in image and graphics processing (composed of convolutional layers, pooling layers, instance normalization layers, and nonlinear activation layers), which have excellent performance in image feature extraction. However, in the process of shooting urban landscapes, the sunlight causes overexposure spots, resulting in obvious artifacts in the CIM model when displaying urban scenes. In order to overcome the problem that the svbrdf-net technology has information loss in processing urban landscapes, resulting in poor visual experience when the CIM model displays urban images, this invention optimizes the display performance of the original svbrdf-net technology in the CIM model by introducing three image technologies for urban renewal: attention mechanism module, multi-image fusion, and landscape dataset. This improves the realism and effectiveness of urban landscapes.

[0078] The fusion attention mechanism simulates the human visual system's attention mechanism in image recognition tasks within the SVBRDF model architecture. In image recognition, it weights different spatial coordinate information in the input, focusing attention on areas relevant to the urban renewal task. This allows the neural network to pay more attention to urban landscape spatial information features, thereby improving the accuracy and efficiency of the CIM model. Furthermore, based on the three convolutional layers in the SVBRDF model encoder, after pooling layers and instance normalization, the fusion attention mechanism enhances its performance and robustness. After the input image is non-linearly activated, a high-dimensional feature vector is obtained for each channel of the feature map. This importance is then used to assign a weight value to each feature, increasing the number of channels in the urban landscape feature map that are useful for the current task. Appropriate selection is necessary when choosing these channels. The encoder and decoder in the original SVBRDF-Net model are as follows: Figure 7 As shown, the encoder and decoder after fusing the attention module are as follows: Figure 8 As shown.

[0079] Secondly, multimodal image fusion. In many cases, the information carried by a single image is insufficient to accurately infer the attributes of a specific urban landscape, especially photos taken by drones from high altitude, which often fail to provide sufficient surface details (such as ground texture and bumpiness). It is necessary to increase the shooting angle or use different lighting directions to reveal subtle geometric structures and reflection characteristics, which increases storage costs and requires improvement in the accuracy and comprehensiveness of the information. To address these issues, this invention extracts features from the input image using filters in a Convolutional Neural Network (CNN). Neurons between two adjacent network layers are locally connected through convolution operations and a weight-sharing strategy. Further, an average pooling layer fuses input images from multiple sources into a fixed-size feature map, significantly reducing the number of free parameters. Then, the similarity of multiple input image patches is measured using the sampling number of the loss function to calculate the loss value, reducing the dimensionality of the feature map. This allows for processing of any number of input images while reducing computational complexity. Specifically, the loss function is evaluated by rendering the real and predicted images under fixed lighting and observation conditions, and then comparing the resulting images pixel-by-pixel to complete the image fusion result evaluation. The multimodal image fusion process is as follows: Figure 9 As shown. The CIM architecture is as follows. Figure 10 As shown; Example 7: This embodiment uses, as follows Figure 11 The ablation experiment shown further verifies the performance of the real-world model: The purpose of conducting the ablation experiment is to combine the specific needs of urban renewal with the effectiveness of integrating attention mechanisms into aspects such as building renovation, environmental optimization, and cultural heritage protection that urban renewal typically involves. It also uses the LoRA module to analyze material restoration of building surfaces, digital modeling of old buildings, and virtual simulation of urban landscapes. This helps urban planners and engineers better understand and optimize their urban renewal models to generate high-precision material parameters for reconstruction or planning in a virtual environment.

[0080] The ablation experiment was divided into four experimental groups: Data acquisition and processing: Urban materials may include brick, concrete, glass, metal, etc. It is necessary to verify the performance of SVBRDF-Net on different materials, which may require adjusting the model structure or training data. This group does not use LoRA and super-resolution modules.

[0081] Using the original SVBRDF-Net architecture, without adding any improved modules. Network structure: Encoder (4 layers ResNet) - Decoder (5 layers transposed convolution) Input / Output: 512x512 RGB image → 9-channel material parameter graph To improve model adaptability, given the diverse types of landscape materials in urban environments, which may be subject to aging and soiling, the group used the LoRA module to adjust data acquisition strategies, such as using drones or mobile scanning devices to acquire multi-angle images of building surfaces.

[0082] Inject the LoRA adapter on top of the Baseline: Injection location: Weight matrix of the first 3 convolutional layers of the encoder. Rank selection: Low-rank decomposition of rank=4 Parameter update: Train only the LoRA incremental matrix, freeze the original weights. In practical applications, such as using optimized material parameters in virtual city simulations or predicting the visual effects of different material replacements in urban redevelopment planning, this group added a super-resolution module to the Baseline front-end.

[0083] The evaluation metrics need to be expanded beyond the original physical and visual metrics to include indicators relevant to practical applications, such as construction feasibility and cost estimation. Therefore, a new experimental group needs to be designed for ablation experiments, utilizing both LoRA and super-resolution modules to address unique urban challenges, such as complex lighting conditions, material aging, and large-scale data processing.

[0084] Improvements to the combinatorial model's adaptability (Exp2) and its practical application scenarios (Exp3): Frontend: Super-resolution module (same as Exp2) Main network: SVBRDF-Net injected with LoRA Experimental control and protection 1. Data Consistency: All experimental groups used the same random seed to partition the dataset. 2. Hardware consistency: All are executed under the same A100 GPU environment. 3. Evaluation Agreement: During testing, the input lighting conditions were kept constant (azimuth 45°, altitude 30°). Normalize the material parameter error before calculation: 4. Statistical significance: Each experiment was repeated 3 times, and the mean was taken. The standard deviation was reported. Key Differences Comparison Table

[0085] This hierarchical experimental design allows for a clear quantification of the independent contribution of each improvement module and their interactions, providing interpretable evidence for model optimization.

[0086] Experimental results

[0087] Key points for verifying the conclusions: LoRA reduces memory consumption by 18% while maintaining 95% of the performance. The super-resolution module improves the material matrix efficiency (MAE) of low-resolution input (256→512) by 37%. The combined use produces a synergistic effect, with an 11% improvement in the retention of high-frequency details.

[0088] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for generating 3D reality models for urban renewal, characterized in that, include: S1: Collect multi-source urban image information and obtain semantic data based on the multi-source urban image information; S2: Visualize and model cities based on multi-source city image information to generate city models; S3: Match semantic data with the city model to obtain a 3D real-world model containing semantic data.

2. The method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, In step S1, the multi-source city image information specifically includes: Based on the full coverage of satellite remote sensing, centimeter-level image information is obtained by using UAV oblique photography for key areas of urban renewal: oblique photography is used to obtain building facade image information, and crack monitoring sensors are combined to capture wall deformation image information in real time. In building renovation monitoring, ground-based lidar and infrared thermal imaging are integrated to identify image information of structural voids and steel corrosion areas; in urban village renovation, spatiotemporal sequence image information is obtained through periodic oblique photography; and dynamic vital signs are perceived through Internet of Things sensor networks. Fiber optic sensors are implanted during underground pipeline network upgrades to obtain image information on pipeline settlement and pressure fluctuations; smart meters are installed in energy renovation projects to obtain image information on energy consumption and heat.

3. The method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, Step S1 includes: Multi-source urban image information is divided into frequent itemsets and infrequent itemsets. Frequent itemsets refer to multi-source urban image information with an update frequency higher than a predetermined threshold during the urban renewal process. These are patterns that represent important or potential association rules in the urban renewal and management process. Infrequent itemsets refer to multi-source urban image information with a frequency lower than a predetermined threshold during the urban renewal process. These are patterns that are not considered to have potential association rules in the urban renewal and management process. Their definitions are as follows: Where MaxInti is the maximum value range of the i-th item, and Inti is the size of the value interval of the i-th item of the association rule: where UB i and LB i These are the upper and lower boundaries of the association rule range, respectively.

4. The method for generating a 3D reality model for urban renewal according to claim 3, characterized in that, The semantic data obtained from multi-source city image information is as follows: Semantic data is estimated from single or multiple multi-source urban image information. The semantic data includes urban landscape attributes. Semantic data obtained from multiple multi-source urban image information includes: Semantic data is obtained by taking pictures from different angles or using lighting from different directions; features of the input images are extracted using filters of a convolutional neural network (CNN); neurons between two adjacent network layers are locally connected through convolution operations and weight sharing strategies; an average pooling layer further fuses input images from multiple sources into a feature map of a fixed size; and the similarity of multiple input image patches is measured by the number of samples in the loss function to calculate the loss value, thereby reducing the dimensionality of the feature map. This enables the processing of any number of input images to obtain semantic data.

5. A method for generating a 3D reality model for urban renewal according to claim 4, characterized in that, In the semantic estimation of multiple images, multiple images need to be fused. After preprocessing of the fusion strategy, it includes multi-scale physical consistency loss, anisotropic feature perception loss, adaptive adversarial training framework, closed-loop rendering consistency loss and dynamic loss weighting mechanism. The optimization direction includes: introducing multi-objective joint optimization to capture details at different levels, ensure semantic association of cross-scale data, and achieve stepless scaling from the whole city to building details. Multi-objective joint optimization: LMSE: Pixel-level material parameter error LSSIM: Structural Similarity Constraint LVGG: VGG-19 Feature Matching For dynamic weights Use physical constraints to constrain the urban landscape in the preset CIM model, and reduce the perceptual loss in the landscape iteration process by reducing the interdependence between landscape image parameters and the correlation between roughness in urban renewal; Through the adaptive adversarial loss function and the dynamic weighted total loss function, the physical constraints, visual quality, and rendering accuracy are compared with the real image in the discriminator or multi-scale discriminator rendering results of adversarial training to form a closed-loop optimization; Allocate weights to different urban landscape attributes according to the differentiable renderer to ensure that each parameter is optimized in the spatial analysis in the loss function.

6. The method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, In step S2, the specific process of urban modeling visualization based on multi-source urban image information is as follows: S2.1: Preprocess the multi-source urban image information to obtain the preprocessed multi-source urban image information; S2.2: Construct a dataset based on the preprocessed multi-source urban image information; S2.3: Construct a text-to-image model and an image quality enhancement model; S2.4: Train the text-to-image model based on the LoRA method through the dataset to obtain the trained text-to-image model; train the image quality enhancement model based on the SwinIR method through the dataset to obtain the trained image quality enhancement model; S2.5: Based on the trained text-to-image model and the trained image quality enhancement model, use digital twin professional 3D modeling to establish a virtual model, export it in GLTF format, and combine it with indexDB technology to establish a city model.

7. The method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, Step S3 is specifically as follows: Associate the semantic data with the city model through a unique identifier, merge multiple materials and textures of the city model, reduce the size of the texture material map, and achieve the purpose of reducing the resource loading time. The specific methods include: Semantic graph construction: Use a graph neural network to establish the association relationship of urban elements, dynamically bind the attributes of BIM components to the UV coordinates of the oblique photography model, and form a triple index library of <UUID, spatial coordinates, semantic vector>; the dimension of the semantic vector is designed to be 512 dimensions, including multiple types of building function attributes, multiple pipeline material characteristics, and multiple geomechanical parameters; Intelligent texture compression: Based on the adaptive texture merging algorithm of the generative adversarial network, input the original oblique photography map and the BIM material library, generate a 4K composite texture map through the StyleGAN2 framework, and extract the normal map and specular reflectivity parameters at the same time; Achieve the deep coupling of the oblique photography model and the city model through the spatio-temporal attribute dynamic binding technology, including: For the renovation scenario of old residential areas, develop a structural health semantic encoder to cross-modally associate the steel bar corrosion rate and concrete carbonation depth in the BIM model with the oblique photography crack texture, and form a dynamic decision vector of <building UUID, safety level, renovation priority>; in the protection of historical blocks, adopt a multi-spectral material analysis algorithm to fuse visible light, near-infrared, and thermal imaging oblique photography data, automatically identify traditional building materials, and establish a temporal degradation map of ancient building materials; Dynamic texture optimization: Develop a texture synchronization engine for the update process, supporting pixel-level comparative analysis of building facades before and after renovation; use generative adversarial networks to integrate construction design BIM materials with existing textures from oblique photography to generate a preview of the renovation effect, improving the compression rate while preserving the textural features of historical buildings.

8. The method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, Step S3 also includes: Using a spatiotemporal R-tree index, key areas for urban renewal are detected in real time, and the high-precision modeling range is automatically defined. The grid density in key areas is maintained at 100-300 triangles / ㎡, while it is reduced to 10-30 triangles / ㎡ in non-key areas. A grid simplification priority model is established specifically for urban renewal scenarios. The safety risk coefficient is dynamically calculated using structural sensor data, while the historical value coefficient is assigned based on the protection level. To address the needs of existing building renovation, the design retains geometric coding with details preserved, applies non-destructive compression to key components, and applies destructive simplification to non-load-bearing parts.

9. The method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, Step S3 also includes: LOD technology is used to optimize multi-character rendering and simulate lighting and shadows during the update process; First, based on scene interaction, distance, and viewing angle, select an appropriate mesh detail level to enable scene roaming, scaling, rotation, movement, and hiding in Three.js, thereby reducing the rendering burden; A dynamic detail adjustment model is built based on reinforcement learning to dynamically adjust details according to the user's identity level. Designer's Perspective: Showcasing the Millimeter-Level Integration of BIM MEP Piping with Oblique Photogrammetry Facade From the perspective of construction workers: highlighting the early warning area for construction deviations Public Perspective: Comparing the Effects of Community Renovation By pre-calculating scene lighting distribution through Monte Carlo integration and combining screen space reflection and cascaded shadow mapping, the time consumption of shadow calculation is reduced while ensuring visual quality. For complex materials, ray differential tracing technology is used to reduce the amount of reflection and refraction calculations, thereby achieving joint optimization of lighting and shadow. By combining the accurate simulation of light and shadow changes in different seasons and time periods during the renovation of historical districts, and by fusing the lighting information of historical frames in multi-source urban image information with the current frame data, the real-time computing overhead can be reduced.

10. A method for generating a 3D reality model for urban renewal according to claim 1, characterized in that, Step S3 also includes: The creation and update process involves resource pooling management of loading, caching, and reuse models, and periodically cleaning up no longer needed model resources using an LRU caching strategy, including: Resource pooling during the update process involves constructing a phase-aware caching model for urban renewal and transformation, dividing resources into: planning phase resources, construction phase resources, and operation and maintenance phase resources. The update process uses resource pooling. First, a multi-level cache pool architecture is built: a dynamic priority caching strategy is designed. This strategy allocates data streams to different queues based on different weights during the planning, construction, and operation and maintenance phases. According to the principle that the larger the weight, the higher the priority, different queues will have different priorities.

Images