3D modeling method based on digital twin cities

Abstract

The invention provides a 3D modeling method based on a digital twin city, and belongs to the technical field of 3D city modeling. Through a multi-source semantic data fusion acquisition means, space-time semantic tags are added to various types of data, the problem of multi-source data isomerism is solved, and semantic unification and efficient fusion of different types of data are realized; by means of hierarchical dynamic modeling means, a building component network is constructed based on triple association of geometry, functions and semantics, so that a model structure better conforms to the logic of the building industry, and the reasonability and efficiency of modeling are improved; a virtual-real two-way mapping means is applied, incremental updating of the model is triggered through a cross-modal semantic comparison algorithm, limitation of traditional static modeling is broken through, and dynamic synchronization of the virtual model and the physical world is achieved; an intelligent optimization means is adopted, a multi-constraint-condition generative adversarial network is combined with a visual attention mechanism, it is ensured that generated textures conform to building geometric features and material specifications, and meanwhile intelligent distribution of rendering resources is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional city modeling, and in particular to a 3D modeling method based on digital twin cities. Background Art

[0002] Traditional urban 3D modeling methods mainly rely on a single data source or manual modeling, and have many technical bottlenecks: the textures generated by traditional methods often only focus on visual effects, ignore the geometric characteristics of buildings and the semantic rules of materials, and often result in problems such as texture distortion and material confusion, which do not comply with construction industry standards.

[0003] The development of digital twin technology has put forward higher requirements for urban 3D modeling: it is necessary to realize the semantic fusion of multi-source data, dynamic updating of models, precise geometric and semantic constraints, and intelligent optimization of the modeling process; however, existing technologies have obvious shortcomings in these aspects, and a more advanced 3D modeling method is urgently needed. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of texture distortion and material disorder in existing three-dimensional city modeling. The present invention proposes a 3D modeling method based on digital twin cities.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is a 3D modeling method based on a digital twin city, comprising the following steps:

[0006] S1. Multi-source semantic data fusion acquisition: Simultaneously acquire LiDAR point clouds with normals, multi-view oblique photography sequences, semantically tagged BIM models, and real-time IoT data streams. Edge computing is used to attach spatiotemporal semantic tags to these data. Specifically, edge computing nodes pre-process IoT data and use a Bayesian classification algorithm to assign semantic tags to data points, achieving semantic unification of multi-source data. The semantic tag format is: spatial coordinates, timestamp, and semantic type; the semantic type is determined by the Bayesian classification algorithm.

[0007] S2. Hierarchical Dynamic Modeling: This involves constructing a macroscopic layer containing terrain semantics, a mesoscopic architectural layer with structural parameters, and a microscopic device layer with integrated sensor interfaces, using a coarse-to-fine progressive modeling strategy. Semantic segmentation is used to extract semantic units from the mesoscopic architectural layer within the BIM model: load-bearing structures, maintenance structures, and decorative components. A building component network is then constructed based on the triple associations of geometry, function, and semantics.

[0008] S3. Bidirectional mapping between the physical and virtual worlds: Establish real-time data channels between the physical and virtual worlds through distributed message queues. Utilize cross-modal semantic comparison mechanisms to trigger incremental model updates, establishing a bidirectional mapping mechanism between the physical world and the virtual model.

[0009] Forward mapping: IoT data drives model status updates in real time through message queues;

[0010] Reverse mapping: Model simulation results generate control instructions that are fed back to the physical device;

[0011] A cross-modal semantic comparison algorithm is used to detect model deviations, triggering local updates when the similarity falls below a dynamic threshold.

[0012] In the cross-modal semantic comparison mechanism, the consistency is measured by calculating the mean intersection-over-union ratio of the image semantic segmentation results and the BIM semantic labels. The formula is: The formula measures the semantic similarity between image I and model B, where IoU is the intersection over union ratio of semantic categories, and m is the total number of regions or samples involved in the calculation. To predict semantics, To achieve true semantics, the algorithm incorporates the semantic standards of the construction industry into the model update trigger mechanism.

[0013] S4. Optimization: Use a multi-constraint generative adversarial network to complete the missing texture, combine it with the visual attention mechanism to achieve rendering optimization, and manage the model version. The loss function of the multi-constraint generative adversarial network integrates three types of constraints: L total =L adv +λ1L geo +λ2L sem ;

[0014] Among them, L adv is a conventional adversarial loss function, which is used to ensure that the generated texture is visually consistent with the real building texture; L geo is a geometric constraint loss function that forces the texture to conform to the building's geometric features by calculating the bidirectional Chamfer distance between the generated texture and the 3D model surface point cloud. The formula is:

[0015] where p i is the sampling point for generating texture, q j is the surface point of the 3D model, N and M represent the number of elements in two different point sets respectively. This constraint introduces the normal vector field information of the building surface into the texture generation process, solving the problem that traditional 2D image restoration cannot handle surface mapping;

[0016] L sem is a semantic constraint loss function. It constructs material layout constraints based on conditional random fields to ensure that the generated texture conforms to the BIM semantic labels, such as the distribution rules of glass curtain walls and concrete walls. The formula is:

[0017] y is the material label sequence for generating textures, x is the input BIM semantic feature, θ cThe material association potential energy function preset for the construction industry, C is a predefined set, where each element c represents a specific semantic group, such as the co-occurrence probability of "glass curtain wall" and "aluminum alloy window frame", η is the spatial association strength coefficient between the control material labels; λ1 and λ2 are the control L geo and L sem The relative importance coefficient in the total loss, dynamically adjusting the constraint weights through reinforcement learning;

[0018] Furthermore, in the intelligent optimization step, the user's focus area is identified through the visual attention mechanism, and refined rendering is applied to the focus area, while the LOD level is reduced in the non-focus area, so as to improve the rendering efficiency while maintaining the visual quality of the key areas.

[0019] Furthermore, model version management specifically includes: using time-space stamps and semantic hashing to generate a unique version number, and calculating the structural change rate between versions based on the semantic adjacency matrix.

[0020] Compared with the existing technology, the beneficial effects of the present invention include: through multi-source semantic data fusion and collection means, spatiotemporal semantic labels are added to various types of data, which solves the problem of multi-source data heterogeneity and realizes semantic unification and efficient fusion of different types of data; with the help of hierarchical dynamic modeling means, a building component network is constructed based on the triple association of geometry, function and semantics, so that the model structure is more in line with the logic of the construction industry, and the rationality and efficiency of modeling are improved; using virtual-reality bidirectional mapping means, the incremental update of the model is triggered by the cross-modal semantic comparison algorithm, breaking through the limitations of traditional static modeling and realizing dynamic synchronization of virtual models and the physical world; adopting intelligent optimization means, using multi-constraint generative adversarial networks combined with visual attention mechanisms to ensure that the generated texture conforms to the building's geometric characteristics and material specifications, while realizing the intelligent allocation of rendering resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The disclosure of the present invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the accompanying drawings, the same reference numerals are used to refer to the same components. Among them:

[0022] Figure 1 The figure schematically shows a flow chart of a 3D modeling method based on a digital twin city proposed according to one embodiment of the present invention. DETAILED DESCRIPTION

[0023] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the present invention and should not be regarded as the entire invention or as a limitation or restriction of the technical solution of the present invention.

[0024] According to the embodiment of the present invention, Figure 1 A 3D modeling method based on a digital twin city includes the following steps:

[0025] S1. Multi-source semantic data fusion acquisition: Simultaneously acquire LiDAR point clouds with normals, multi-view oblique photography sequences, semantically tagged BIM models, and real-time IoT data streams. Edge computing is used to attach spatiotemporal semantic tags to the data. Specifically, a preprocessing module is deployed on the edge server, and IoT data is preprocessed through edge computing nodes. A Bayesian classification algorithm is used to assign semantic tags to data points, achieving semantic unification of multi-source data. The semantic tag format is: spatial coordinates, timestamp, and semantic type. The semantic type is determined by the Bayesian classification algorithm.

[0026] S2. Hierarchical Dynamic Modeling: This involves constructing a macroscopic layer containing terrain semantics, a mesoscopic architectural layer with structural parameters, and a microscopic device layer with integrated sensor interfaces, using a coarse-to-fine progressive modeling strategy. Semantic segmentation is used to extract semantic units from the mesoscopic architectural layer within the BIM model: load-bearing structures, maintenance structures, and decorative components. A building component network is then constructed based on the triple associations of geometry, function, and semantics.

[0027] S3. Bidirectional mapping between the physical and virtual worlds: Establish real-time data channels between the physical and virtual worlds through distributed message queues. Utilize cross-modal semantic comparison mechanisms to trigger incremental model updates, establishing a bidirectional mapping mechanism between the physical world and the virtual model.

[0028] Forward mapping: IoT data drives model status updates in real time through message queues;

[0029] Reverse mapping: Model simulation results generate control instructions that are fed back to the physical device;

[0030] A cross-modal semantic comparison algorithm is used to detect model deviations, triggering local updates when the similarity falls below a dynamic threshold.

[0031] In the cross-modal semantic comparison mechanism, the consistency is measured by calculating the mean intersection-over-union ratio of the image semantic segmentation results and the BIM semantic labels. The formula is: The formula measures the semantic similarity between image I and model B, where IoU is the intersection over union ratio of semantic categories, and m is the total number of regions or samples involved in the calculation. To predict semantics, To ensure true semantics, the algorithm incorporates the semantic standards of the construction industry into the model update trigger mechanism. Compared with traditional geometric comparison algorithms, it can improve the accuracy of semantic consistency detection. Taking the "glass curtain wall" category as an example, if the IoU value is lower than 0.7, the model update is triggered. In densely built areas, that is, with high semantic entropy, the threshold is automatically lowered to avoid false triggering of updates.

[0032] S4. Optimization: A multi-constraint generative adversarial network is used to complete the missing texture. The multi-constraint generative adversarial network uses a three-stage progressive generator:

[0033] Primary generation: Generate 4×4 basic texture structure based on 3D model normal vector;

[0034] Intermediate generation: Integrate BIM material tags to generate 128×128 material distribution;

[0035] Advanced generation: Generate 1024×1024 refined textures by combining multi-view image features;

[0036] Combining the visual attention mechanism to achieve rendering optimization and perform model version management, the loss function of the multi-constraint generative adversarial network integrates three types of constraints: L total =L adv +λ1L geo +λ2L sem ;

[0037] Among them, L adv is a conventional adversarial loss function, which is used to ensure that the generated texture is visually consistent with the real building texture; L geo is a geometric constraint loss function that forces the texture to conform to the building's geometric features by calculating the bidirectional Chamfer distance between the generated texture and the 3D model surface point cloud. The formula is:

[0038] where p i is the sampling point for generating texture, q j is the surface point of the 3D model, N and M represent the number of elements in two different point sets respectively. This constraint introduces the normal vector field information of the building surface into the texture generation process, solving the problem that traditional 2D image restoration cannot handle surface mapping;

[0039] L sem is a semantic constraint loss function. It constructs material layout constraints based on conditional random fields to ensure that the generated texture conforms to the BIM semantic labels, such as the distribution rules of glass curtain walls and concrete walls. The formula is:

[0040] y is the material label sequence for generating textures, x is the input BIM semantic feature, θ cThe material association potential energy function preset for the construction industry, C is a predefined set, where each element c represents a specific semantic group, such as the co-occurrence probability of "glass curtain wall" and "aluminum alloy window frame", η is the spatial association strength coefficient between the control material labels; λ1 and λ2 are the control L geo and L sem The relative importance coefficient in the total loss is used to dynamically adjust the constraint weights through reinforcement learning. For example, curved buildings automatically increase λ1 to strengthen geometric constraints.

[0041] In the intelligent optimization step, the visual attention mechanism is used to identify the user's focus areas, and refined rendering is used for the focus areas, while the LOD level is reduced in the non-focus areas, so as to improve rendering efficiency while maintaining the visual quality of key areas.

[0042] Specifically, it simultaneously acquires LiDAR point clouds, oblique photography, BIM models, and IoT data streams. Edge computing is used to add spatiotemporal semantic tags to each type of data, solving the problem of traditional data heterogeneity. For example, the coordinates of the laser point cloud and the material semantics of the BIM are uniformly labeled, giving data from different sources a common "language."

[0043] Next, the hierarchical dynamic modeling module constructs models based on three layers: macro-terrain, meso-building, and micro-equipment. First, a terrain framework is generated based on the TIN model. Semantic units such as load-bearing structures are extracted from BIM and integrated with photogrammetry data to form a building model. Finally, key facilities are refined and sensor interfaces are reserved, using a "coarse-to-fine" strategy to improve efficiency.

[0044] A distributed message queue is used to establish a physical and virtual data channel. Forward mapping enables IoT data to drive model status updates in real time. Reverse mapping feeds model simulation results back to physical devices. A cross-modal semantic comparison algorithm is used to calculate the similarity between image semantic segmentation results and BIM tags. When the similarity falls below a dynamic threshold, only the local model area is updated, avoiding time-consuming global reconstruction.

[0045] A multi-constraint generative adversarial network is adopted, and geometric constraints, semantic constraints and visual constraints are introduced at the same time to ensure that the texture fits the building surface, force the material to comply with industry standards, and ensure the authenticity of the texture. The visual attention mechanism is combined to identify the user's focus areas for refined rendering, and non-focus areas are lightweight.

[0046] The technical scope of the present invention is not limited to the contents of the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A 3D modeling method based on digital twin city, characterized in that: The following steps are involved: S1. Multi-source semantic data fusion acquisition: Simultaneously acquire LiDAR point clouds with normals, multi-view oblique photography sequences, semantically tagged BIM models, and real-time IoT data streams, and add spatiotemporal semantic tags to the data through edge computing. S2. Hierarchical dynamic modeling: Constructing a macroscopic layer with terrain semantics, a mesoscopic building layer with structural parameters, and a microscopic device layer with integrated sensor interfaces, using a coarse-to-fine progressive modeling strategy; S3. Bidirectional mapping between virtual and real: Establishing real-time data channels between physical and virtual objects through distributed message queues, and using cross-modal semantic comparison mechanisms to trigger incremental model updates. S4. Optimization: Use a multi-constraint generative adversarial network to complete missing textures, combine it with a visual attention mechanism to achieve rendering optimization, and perform model version management.

2. A 3D modeling method based on a digital twin city according to claim 1, characterized in that: In the multi-source semantic data fusion collection step, IoT data is preprocessed through edge computing nodes, and a Bayesian classification algorithm is used to assign semantic labels to data points to achieve semantic unification of multi-source data.

3. The 3D modeling method based on digital twin city according to claim 1, characterized in that: In the hierarchical dynamic modeling step, the meso-architectural layer extracts three types of semantic units in the BIM model, namely load-bearing structure, maintenance structure and decorative components, through semantic segmentation, and constructs a building component network based on the triple association of geometry, function and semantics.

4. The 3D modeling method based on digital twin city according to claim 1, characterized in that: In the virtual-real bidirectional mapping step, a bidirectional mapping mechanism between the physical world and the virtual model is established: Forward mapping: IoT data drives model status updates in real time through message queues; Reverse mapping: Model simulation results generate control instructions that are fed back to the physical device; A cross-modal semantic comparison algorithm is used to detect model deviations, and local updates are triggered when the similarity falls below a dynamic threshold.

5. A 3D modeling method based on a digital twin city according to claim 4, characterized in that: In the cross-modal semantic comparison mechanism, the consistency is measured by calculating the mean intersection-over-union ratio of the image semantic segmentation result and the BIM semantic label. The formula is: The formula measures the semantic similarity between image I and model B, where IoU is the intersection over union ratio of semantic categories, and m is the total number of regions or samples involved in the calculation. To predict semantics, To achieve true semantics, the algorithm incorporates the semantic standards of the construction industry into the model update trigger mechanism.

6. The 3D modeling method based on digital twin city according to claim 1, characterized in that: The loss function of the multi-constraint generative adversarial network integrates three types of constraints: L total =L adv +λ1L geo +λ2L sem ; L adv is the conventional adversarial loss function, L geo is a geometric constraint loss function that forces the texture to conform to the building's geometric features by calculating the bidirectional Chamfer distance between the generated texture and the 3D model surface point cloud. The formula is: p i is the sampling point for generating texture, q j is the surface point of the 3D model, N and M represent the number of elements in two different point sets respectively. This constraint introduces the normal vector field information of the building surface into the texture generation process, solving the problem that traditional 2D image restoration cannot handle surface mapping; L sem is a semantic constraint loss function. It constructs material layout constraints based on conditional random fields to ensure that the generated texture conforms to the BIM semantic labels, such as the distribution rules of glass curtain walls and concrete walls. The formula is: y is the material label sequence for generating textures, x is the input BIM semantic feature, θ c The material association potential energy function preset for the construction industry, η is the spatial association strength coefficient between the control material labels, and C is a predefined set, where each element c represents a specific semantic group.

7. The 3D modeling method based on digital twin city according to claim 6, characterized in that: In the loss function of the multi-constraint generative adversarial network, λ1 and λ2 are the control L geo and L sem The relative importance coefficient in the total loss, dynamically adjusting the constraint weights through reinforcement learning.

8. The 3D modeling method based on digital twin city according to claim 1, characterized in that: In the intelligent optimization step, the user's focus area is identified through the visual attention mechanism, and refined rendering is applied to the focus area, while the LOD level is reduced in the non-focus area.

9. The 3D modeling method based on digital twin city according to claim 1, characterized in that: The model version management specifically includes: using time and space stamps and semantic hashing to generate a unique version number.

10. A 3D modeling method based on a digital twin city according to claim 9, characterized in that: The model version management specifically further includes: calculating the structural change rate between versions based on the semantic adjacency matrix.

Images