Cim model generation method and system based on spatial intelligence and visual knowledge fingerprint

Abstract

This invention provides a CIM model generation method and system based on spatial intelligence and visual knowledge fingerprints, belonging to the field of smart city digital twins; it solves the problem of low efficiency in building texture generation; specifically as follows: edge detection is performed on the top view of the original CIM model, and the plot boundaries are synthesized with the Canny edge map to generate the building layout; planning indicators are extracted from the GIS data of the target plot, and a white model is generated using a modeling engine based on the building layout; the white model is automatically UV unwrapped, and a semantic segmentation map is output based on constraint rules and construction generation logic, and then encoded based on the visual features parsed from the target plot CIM model to generate texture maps; the generated texture maps are remapped back to the surface of the CIM3 level white model to obtain a generative reconstruction model; this invention realizes the continuous regeneration of CIM data features based on spatial intelligence, and solves the contradiction between privacy and utility by resolving spatial logic reasoning and generation.

Description

Technical Field

[0001] This invention relates to a method and system for generating CIM models based on spatial intelligence and visual knowledge fingerprints, belonging to the field of smart city digital twins and spatial intelligence. Background Technology

[0002] Existing anonymization techniques have the following drawbacks when dealing with privacy and security issues in high-fidelity CIM models:

[0003] Loss of utility due to lack of spatial semantic understanding: Traditional visual obfuscation or pixelation only destroys images at the image level and lacks intelligent recognition and understanding of spatial semantics such as windows and load-bearing walls. This approach not only covers up privacy but also completely destroys the spatial analysis value of the model (such as the failure of lighting analysis and field of view analysis).

[0004] The lack of spatial logical reasoning leads to rigid generation: Although simple procedural replacement severs privacy connections, its generation logic is usually based on simple random replacement rules and lacks the ability to reason about the surrounding environment, architectural style and planning indicators (such as plot ratio). This results in the generated city model being like digital building blocks, lacking the organic nature and diversity of the real world.

[0005] Spatial topology defects of end-to-end AIGC models: Although existing text-to-3D models (such as Shap-E) have image generation capabilities, they lack embedded spatial intelligence. The models they generate often have topological errors such as uneven surfaces, non-orthogonality, and even Janus problems, which cannot meet the requirements of CIM engineering for the rigor of geometric structure. Summary of the Invention

[0006] In view of the shortcomings of existing technologies, the purpose of this invention is to provide a CIM model generation method and system based on spatial intelligence and visual knowledge fingerprints, aiming to solve the privacy and security problems of high-fidelity CIM models.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a CIM model generation method based on spatial intelligence and visual knowledge fingerprints includes:

[0008] Read the GIS data of the target plot and extract the plot boundary as the first hard constraint; perform Canny edge detection on the top view of the original CIM model, mask the plot boundary, and obtain the Canny edge map; synthesize the plot boundary and the Canny edge map as the geometric control signal of ControlNet, and then use the LoRA-Layout model to generate the building layout.

[0009] Planning indicators are extracted from the GIS data of the target site. Based on the building layout generated by the LoRA-Layout model, a CIM3 level white model is generated using a modeling engine. The CIM3 level white model is automatically UV unwrapped. Based on constraint rules and construction generation logic, a semantic segmentation map is output. Then, based on the visual features parsed from the CIM model of the target site, the texture map is generated by encoding.

[0010] The generated texture map is remapped back onto the CIM3 level white model surface to obtain a generative reconstructed model.

[0011] Furthermore, the specific steps of Canny edge detection are as follows:

[0012] Obtain the top view corresponding to the original CIM image, and perform Canny edge detection on the top view using an image processing library: apply Gaussian filtering to the top view, then calculate the gradient magnitude and direction of each pixel in the image, and refine the edge pixels with larger gradient magnitudes through non-maximum suppression. The specific process is as follows:

[0013] The Sobel operator is used to calculate the gradient of all pixels in the image along the horizontal and vertical axes, and the gradient direction angle of each pixel is obtained.

[0014] Based on the gradient direction of all pixels in the top view, the overall gradient direction of the top view is discretized into four main directions: 0°, 45°, 90°, and 135°.

[0015] Iterate through and calculate the gradient direction of each pixel in the top view, and compare the gradient magnitude with the pixels in its (3×3) neighborhood:

[0016] Let θ be the gradient direction of the pixel at position (i, j) in the top view. （i,j） The gradient magnitude is G （（i）,（j）） ;

[0017] According to θ （i,j） The size of θ determines the main orientation to which the pixel belongs, and then... （i,j） The main direction of assignment is determined by whether the gradient magnitude of the pixel is the maximum gradient magnitude in its neighborhood. If so, it is retained; otherwise, the gradient magnitude of the pixel is set to zero, and edge pixels with larger gradient magnitudes are refined.

[0018] In the initial refinement of edge pixels using Canny edge detection, high and low thresholds are set, and dual-threshold detection and edge tracking are used to determine the final edge pixels.

[0019] Furthermore, the steps for determining the final edge pixels are as follows:

[0020] Pixels with gradient magnitudes greater than the high threshold are considered strong edge pixels, pixels with gradient magnitudes between the high and low thresholds are considered weak edge pixels, and pixels with gradient magnitudes less than the high threshold are considered non-edge pixels.

[0021] Iterate through each weak edge pixel and detect whether there is a strong edge pixel in the neighborhood of each weak edge pixel (8×8);

[0022] If it exists, then the weak edge pixel is marked as a true edge pixel;

[0023] If it does not exist, its gradient magnitude is set to zero;

[0024] Based on the edge pixels obtained from Canny edge detection, and combined with the land boundary constraints, a control signal without the specific outline of the building is obtained.

[0025] Furthermore, the steps for generating the building layout are as follows:

[0026] Input the hard geographic constraints and control signals into the configured LoRA-Layout model;

[0027] The LoRA-Layout model extracts, transforms, and combines features from the input data through its internal neural network layers;

[0028] During the layout generation process, the LoRA-Layout model incorporates spatial intelligent logic such as orthogonalization and compliance to prevent the generated buildings from exceeding the plot boundaries. While preserving the original plot road network characteristics, the model completely reconstructs the base shape, location, and number of buildings.

[0029] When generating specific building layouts based on rigid geographical constraints, the model does not disrupt the original road network structure. Simultaneously, it adjusts the building distribution according to control signal requirements. The generated layout results undergo anonymization processing.

[0030] For certain sensitive building shapes, blurring is applied;

[0031] For certain special building site locations, a random perturbation algorithm is used to adjust the building location within the area defined by hard geographical constraints;

[0032] The desensitized building layout results are output as the model.

[0033] Furthermore, the specific steps for generating a CIM3 level white model are as follows:

[0034] Extract planning indicators from the GIS data of the target plot: target floor area ratio (FAR), average floor height, and maximum building height.

[0035] The building layout generated by the LoRA-Layout model and the planning indicators extracted from the CIM data are input into the procedural modeling engine;

[0036] According to the formula: In addition to the average floor height and the maximum building height limit, the height of each building is dynamically calculated;

[0037] in, This indicates the area of ​​the land occupied by the building. This indicates the building's footprint area; FAR represents the target floor area ratio.

[0038] The modeling engine automatically performs push-pull and logical segmentation operations to generate a CIM3 level white model.

[0039] The CIM3 level white model is automatically UV unfolded. Based on the constraint rules and construction generation logic, the geometry engine simultaneously outputs a high-precision semantic segmentation map, which identifies the generation area of ​​the component.

[0040] The visual features parsed from the target plot's CIM model are encoded, including architectural style; the encoded visual features are then input into a large language model to generate a structured Prompt.

[0041] Based on the architectural style extracted from the Prompt, the visual features of the building are obtained, and the LoRA-Texture texture model is selected. The selected texture model is imported into the texture generation working environment or software, and the semantic segmentation map of the building is read using the ControlNet-Seg tool. The texture generation system is then used to generate texture maps.

[0042] Furthermore, the steps for generating the texture map are as follows:

[0043] Based on the semantic segmentation map and the visual feature requirements of the building, specific constraint generation rules are set;

[0044] The constraint generation rules, along with the loaded texture model and semantic segmentation map, are used as input to the texture generation system.

[0045] Use ControlNet-Seg to read semantic segmentation maps and perform strong spatial semantic constraints;

[0046] Based on the characteristics of the LoRA-Texture texture model, the strong spatial semantic constraints provided by ControlNet-Seg, and the constraint generation rules, the texture generation system generates texture maps that meet architectural semantics and style requirements through multimodal guidance.

[0047] The generated texture map is subjected to a quality check to see if it meets the constraint generation rules and visual feature requirements. If it does, no processing is performed. If it does not meet the requirements, the constraint generation rules or the parameters related to texture map generation in the texture generation system are adjusted, and then the texture map is regenerated.

[0048] Furthermore, the steps for generating texture maps are as follows:

[0049] The texture generation system extracts the characteristics of the LoRA-Texture texture model, and then reads the position and semantic labels of different components of the building facade based on the semantic information provided by ControlNet-Seg;

[0050] The texture generation system transforms the constraint generation rules into executable parameters and conditions. Based on the characteristics of the LoRA-Texture texture model, the system guides the texture generation in a specific style direction. Combined with the semantic information of ControlNet-Seg, the system checks whether the generated texture matches the semantic region of the building and checks whether the texture map generated by the system conforms to the constraint generation rules at each stage of texture generation.

[0051] During the texture generation process, the texture generation system analyzes the color, shape, and distribution characteristics of the texture map in real time to determine whether it meets the semantic matching and style requirements.

[0052] If the conditions are met, no action is taken; otherwise, the system will adjust the influence weights of the LoRA-Texture model, modify the guidance strength of ControlNet-Seg, or change the specific parameter values ​​in the constraint rules, and then regenerate the texture map until a texture map that meets the requirements is generated.

[0053] The CIM model generation system based on spatial intelligence and visual knowledge fingerprinting includes:

[0054] Data processing module: used to read the GIS data of the target plot, extract the plot boundary as the first hard constraint; perform Canny edge detection on the top view of the original CIM model, mask the plot boundary to obtain the Canny edge map; synthesize the plot boundary and the Canny edge map as the geometric control signal of ControlNet, and then use the LoRA-Layout model to generate the building layout;

[0055] Texture generation module: It is used to extract planning indicators from the GIS data of the target plot, generate a CIM3 level white model based on the building layout generated by the LoRA-Layout model, and use the modeling engine to generate a CIM3 level white model; it performs automatic UV unwrapping on the CIM3 level white model, outputs a semantic segmentation map based on constraint rules and construction generation logic, and then encodes the visual features parsed from the target plot CIM model to generate texture maps.

[0056] Model output module: used to remap the generated texture maps back to the CIM3 level white model surface, to obtain a generative reconstruction model that is physically equivalent to the original scene, maintains continuity in visual style, but achieves complete desensitization in specific geometric details.

[0057] Compared with the prior art, the beneficial effects of the present invention are:

[0058] Endowing CIM systems with "generative spatial intelligence": This invention enables CIM systems to evolve from passive "data viewers" to proactive "spatial creators"; the system can not only store city models, but also understand the planning logic behind them (FAR, coverage, architectural style), and automatically generate an infinite variety of derivative models based on these logics.

[0059] Achieving a win-win situation between privacy protection and spatial analysis utility: The generated model is a completely new geometric entity, completely severing the physical connection with the sensitive original data (privacy and security); however, it maintains a high degree of consistency with the original model in macroscopic spatial indicators (floor area ratio, height) and microscopic visual features (material, window-to-wall ratio). Experiments show that using this desensitized model for urban wind environment simulation and solar radiation analysis yields results with minimal error compared to the original model, demonstrating the advantages of spatial intelligence in preserving data utility.

[0060] Engineering-grade geometric compliance: By introducing a "programmatic modeling engine", this invention solves the problem of "geometric inaccessibility" of objects generated by traditional AIGC models; the generated models are low-polygon and orthogonal, and can be directly imported into GIS, game engines or physics simulation software. Attached Figure Description

[0061] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 This is a schematic diagram of the method of the present invention;

[0063] Figure 2 This is a schematic diagram of the system of the present invention;

[0064] Figure 3 This is a schematic diagram of the pixel comparison range of the present invention;

[0065] Figure 4 This is a schematic diagram of the model of the present invention. Detailed Implementation

[0066] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0067] Example 1

[0068] Please see Figure 1 The CIM model generation method based on spatial intelligence and visual knowledge fingerprinting includes:

[0069] Step S1: Read the GIS data of the target plot and extract the plot boundary as the first hard constraint; perform Canny edge detection on the top view of the original CIM model, mask the plot boundary, and obtain the Canny edge map (such as the pattern characteristics formed by the changes in the shape, number, spacing, direction, and color of lines, such as road grid, entrance and exit orientation, and underground pipeline distribution); synthesize the plot boundary and the Canny edge map as the geometric control signal of ControlNet, and then use the LoRA-Layout model to generate the building layout;

[0070] Canny edge detection processing CIM model top view:

[0071] Convert the original CIM (City Information Model) top view to a format suitable for image processing (such as PNG, JPEG, etc.) and present a two-dimensional planar view from a perspective perpendicular to the ground (i.e., looking down). Obtain the top view corresponding to the original CIM. Use an image processing library (such as OpenCV) to perform Canny edge detection on the top view. The specific steps are as follows:

[0072] Gaussian filtering is applied to the top view to reduce noise in the image. Then, the gradient magnitude and direction of each pixel in the image are calculated. Non-maximum suppression is used to refine edge pixels with larger gradient magnitudes (i.e., pixels whose gradient magnitude is the largest (including those with the same largest gradient) in their neighborhood). The specific process is as follows:

[0073] The Sobel operator is used to calculate the gradients (i.e., the gradient G in the horizontal direction) of all pixels in the image along the horizontal and vertical axes. （横） The gradient G along the vertical axis （纵） ), to obtain the gradient direction angle of each pixel (i.e., gradient direction angle = arctan(G)). （纵） / G （横） ));

[0074] Based on the gradient direction of all pixels in the top view, the overall gradient direction of the top view is discretized into 4 main directions: 0°, 45°, 90°, and 135° (the “4 main directions” here are only for example demonstration; users or relevant technical personnel can increase or decrease the number of “main directions for gradient direction discretization” according to actual needs).

[0075] Please see Figure 3 Iterate through and calculate the gradient direction of each pixel in the top view, and compare the gradient magnitude with the pixels in its (3×3) neighborhood:

[0076] Let θ be the gradient direction of the pixel at position (i, j) in the top view. （i,j） The gradient magnitude is G （（i）,（j）） (The gradient magnitude of a pixel = the gradient magnitude of the pixel along the horizontal axis + the gradient magnitude of the pixel along the vertical axis).

[0077] According to θ （i,j） The size of the pixel determines its primary orientation.

[0078]

[0079] Pixel Assignment Direction Reference Table

[0080] According to θ （i,j） The primary direction of assignment is determined by checking if the gradient magnitude of a pixel is the largest (including those with the same largest gradient) within its neighborhood. If so, it is retained; otherwise, the gradient magnitude of the pixel is set to zero, and edge pixels with larger gradient magnitudes are refined.

[0081] If θ （i,j） If the value belongs to 0°, then compare the gradient magnitude G of that pixel. （（i）,（j）） The gradient magnitude G of the pixels to its left and right （（i）,（j-1）） With G （（i）,（j+1）） The size of the three, G （（i）,（j）） Is it the largest (including those tied for the largest)?

[0082] If G （（i）,（j）） If it is the largest, then keep G. （（i）,（j）） The value;

[0083] If G （（i）,（j）） If it is not the maximum, then G （（i）,（j）） The value is set to zero;

[0084] Similarly, if θ （i,j） Belonging to 45°, compare the gradient magnitude of this pixel with its upper right and lower left adjacent pixels, i.e., compare G. （（i-1）,（j+1）） G （（i）,（j）） G （（i+1）,（j-1）） The size of the three;

[0085] If θ （i,j） Belonging to 90°, compare the gradient magnitude of this pixel with its upper and lower adjacent pixels, that is, compare G. （（i-1）,（j）） G （（i）,（j）） G （（i+1）,（j）） The size of the three;

[0086] If θ （i,j） Belonging to 135°, compare the gradient magnitude of this pixel with its upper-left and lower-right adjacent pixels, i.e., compare G. （（i-1）,（j-1）） G （（i）,（j）） G （（i+1）,（j+1）） The size of the three;

[0087] In the initial refinement of edge pixels using Canny edge detection, high and low thresholds are set, and dual-threshold detection and edge tracking are used to determine the final edge pixels. The specific steps are as follows:

[0088] Pixels with gradient magnitudes greater than the high threshold are considered strong edge pixels, pixels with gradient magnitudes between the high and low thresholds are considered weak edge pixels, and pixels with gradient magnitudes less than the high threshold are considered non-edge pixels.

[0089] Iterate through each weak edge pixel and detect whether there is a strong edge pixel in the neighborhood of each weak edge pixel (8×8);

[0090] If it exists, then the weak edge pixel is marked as a true edge pixel;

[0091] If it does not exist, its gradient magnitude is set to zero;

[0092] Based on the edge pixels obtained by Canny edge detection (i.e., the actual edge pixels), and combined with the plot boundary constraints, a control signal without the specific outline of the building is obtained;

[0093] Load the pre-trained LoRA-Layout model (which uses a large amount of building layout data as training objects and has the ability to generate building layouts), configure the parameters of the model (such as setting the resolution of the generated layout, output type, etc.), and adjust the hyperparameters of the model (such as learning rate, randomness parameters in the generation process, etc.).

[0094] Input the hard geographic constraints and control signals into the configured LoRA-Layout model; during the input process, ensure that the correct order and dimension correspondence of the input data are maintained (i.e., the hard geographic constraints and control signals are arranged in the channel order required by the model).

[0095] The LoRA-Layout model extracts, transforms, and combines features from the input data through internal neural network layers (such as convolutional layers and fully connected layers);

[0096] During the layout generation process, the LoRA-Layout model incorporates spatial intelligent logic such as orthogonalization and compliance to prevent the generated buildings from exceeding the plot boundaries. While preserving the original plot road network characteristics, the model completely reconstructs the base shape, location, and number of buildings.

[0097] When generating specific building layouts based on rigid geographical constraints, the model does not disrupt the original road network structure. Simultaneously, it adjusts the building distribution according to control signal requirements. The generated layout results undergo anonymization processing.

[0098] For certain sensitive building shapes, blurring is performed (if a building's shape has unique identifiers, it is appropriately deformed or simplified) so that it cannot be directly associated with specific individuals or sensitive data through the building's shape;

[0099] For certain special building site locations (such as hospital and school sites), random perturbation algorithms (such as Poisson disk sampling algorithm and Monte Carlo random perturbation algorithm) are used to adjust the building locations within the area defined by hard geographical constraints so that they do not encroach on certain special site locations.

[0100] The desensitized building layout results are used as the model output.

[0101] This invention proposes a generative AI training method for heterogeneous spatial data in the architectural field;

[0102] This method constructs a multi-dimensional Chinese visual fingerprint system, combines it with a Diffusion base model that natively supports Chinese, and utilizes a high-rank low-rank adaptive network (LoRA) to capture high-frequency geometric features. The goal is to "precisely control the generation of building site plans using Chinese professional terminology." The specific implementation steps are as follows:

[0103] Phase 1: Standardization and geometric cleaning of heterogeneous spatial data;

[0104] Step 1.1: Extracting the graph-ground relationship from multi-source data:

[0105] Step-by-step instructions: This step aims to establish a dedicated topological dataset covering various urban fabrics;

[0106] Specific implementation: The system builds an initial image library containing 8,000 to 10,000 samples. The data sources mainly include vector tiles of publicly available map data, CAD master plan export files of historical projects from design institutes, and high-precision satellite images of buildings extracted through semantic segmentation.

[0107] After data collection is completed, the system performs strict figure-ground separation processing: through adaptive threshold segmentation and morphological closing operation, all heterogeneous data from all sources are uniformly transformed into a single-channel binary map, where white pixels represent building base and black pixels represent land background, thereby eliminating the interference of lighting, shadows, material maps and vegetation colors, and forcing the model to focus on learning the figure-ground relationship and the topological structure of building boundaries.

[0108] Step 1.2: Geometric orthogonalization based on the principal direction histogram;

[0109] Step description: In order to ensure that the model learns topological logic that conforms to architectural drawing standards, this step requires strict geometric standardization of the data;

[0110] Specific implementation: The system introduces the Principal Oriented Gradient Histogram (HOG) algorithm to calculate the angular distribution of all line segments in the image. For the identified non-orthogonal images, the algorithm automatically locks their principal axis angles and applies affine transformations to rotate them to a horizontal or vertical alignment. On this basis, the system further performs geometric quality screening, removing non-normal samples with less than 80% of orthogonal line segments (i.e., 90°±5° and 0°±5°), ensuring that the training set contains only high-quality top-view projection data.

[0111] Finally, the system only performs non-destructive data enhancement by rotating and flipping the data by multiples of 90°, and strictly prohibits the use of non-uniform scaling in order to strictly maintain the true physical meaning of the building scale;

[0112] Phase Two: Construction of a Hybrid Chinese Semantic Annotation System

[0113] Step 2.1: Definition of Chinese Visual Knowledge Fingerprint Tag:

[0114] Step Description: In order to achieve precise control of layout generation using Chinese, this invention constructs a "Chinese visual knowledge fingerprint tagging system" to replace the general natural language description;

[0115] Specifically, the "Chinese Visual Knowledge Fingerprint Tagging System" establishes a four-dimensional structured tag library: In the perspective dimension, it is uniformly labeled as "frontal aerial view" or "binarized line drawing"; in the density dimension, it is quantitatively labeled as "high density," "medium density," or "sparse" based on the figure-ground ratio; in the topology dimension, it defines professional layout terms such as "row-and-column," "enclosed," "point-group," and "axial symmetry"; in the building type dimension, it covers specific classifications such as "slab-type high-rise," "point-type tower," "L-shaped building," and "irregular texture." This system provides a standardized semantic foundation for the model to understand complex architectural spatial features.

[0116] Step 2.2: Automated annotation and trigger word implantation based on "geometric operators + VQA";

[0117] Step description: This step performs a precise mapping from image to Chinese text, using a hybrid strategy of "geometric calculation as the main method and visual model as the auxiliary method";

[0118] Specific implementation: The system uses OpenCV geometric operators to automatically calculate the black-and-white pixel ratio and outline bounding rectangle features of the binary image, accurately generating objective geometric labels such as "high density" and "determinant" to avoid the illusion error of large language models;

[0119] The system then uses a visual question answering (VQA) model to identify building type characteristics (such as "slab-type high-rise"); finally, the system integrates the two types of tags mentioned above.

[0120] One type is based on image geometric features, that is, objective geometric labels generated after calculating the black-and-white pixel ratio and the bounding rectangle feature of the binary image using OpenCV geometric operators;

[0121] Another type is the building type feature label identified using a visual question answering (VQA) model;

[0122] The specific term "spatial layout diagram" is set as the trigger word and fixed at the beginning of the prompt word sequence. This trigger word is designed to activate specific architectural graphic distributions in the model's latent space, rather than simply acting as a translation intermediary, thereby establishing a strong association between specific Chinese instructions and architectural geometric features within the model (e.g., "spatial layout diagram, frontal view, binarized line drawing, high density, enclosed layout, slab-type high-rise").

[0123] The aforementioned "spatial layout diagram" refers to a chart or image that can show the distribution and arrangement of the interior or exterior spaces of a building;

[0124] "Spatial layout diagram" is a predefined concept or instruction used to guide the model to process architectural information. It can present information such as the distribution of various functional areas of a building, the arrangement of rooms, and the positional relationship of building components, helping architects to intuitively understand the spatial structure and organization of the building. (Users or relevant technical personnel can pre-set "spatial layout diagram" as a trigger word according to their knowledge and needs in the field of architecture, so as to guide the model to focus on the spatial feature information of the building.)

[0125] For example, in residential buildings, a spatial layout plan can show the location and interconnections of rooms such as the living room, bedroom, kitchen, and bathroom; in commercial buildings, it can show the distribution of spaces such as shops, corridors, elevators, and staircases.

[0126] Phase 3: High-rank LoRA network architecture configuration supporting Chinese semantics;

[0127] Step 3.1: Loading and Feature Space Adaptation of the Bilingual Aligned Base Model:

[0128] Step description: This step builds the underlying architecture of the model; In order to avoid semantic loss and feature misalignment caused by cross-language translation, this invention abandons the traditional "English base + external translation" path and directly selects a multilingual Diffusion model (such as Taiyi-Stable-Diffusion or AltDiffusion) pre-trained with large-scale Chinese image-text pairs as the base for freezing weights.

[0129] Specific implementation: A multilingual Diffusion model (such as Taiyi-Stable-Diffusion or AltDiffusion) pre-trained with large-scale Chinese image-text pairs is selected as the base for freezing weights; the text encoder of this type of base model has implemented latent space alignment between Chinese text vectors and image feature vectors during the pre-training stage, that is, the text encoder of this type of base model has implemented the function of converting the integrated labels (i.e., objective geometric labels and building type feature labels) in step 2.2 above into "Chinese professional terms";

[0130] During training, the input Chinese technical terms (such as “enclosure” and “retreat”) do not need to be converted by external modules. They can be directly parsed by the encoder into high-dimensional semantic conditions that are compatible with the U-Net network, ensuring that the model can natively and losslessly understand complex architectural contexts.

[0131] Step 3.2: Configuration of topology-sensitive high-rank parameters:

[0132] Step-by-step instructions: Given that the base model already possesses a complete language understanding capability, the training focus at this stage will shift entirely to the reconstruction of architectural geometric accuracy;

[0133] Specific implementation: (For architectural engineering drawings with extremely high requirements for edge straightness and spacing accuracy) The rank of LoRA is set to a relatively high 128, and the scaling factor Alpha is also set to 128; the system injects the trainable low-rank matrix into the Transformer module of the U-Net network through matrix factorization, specifically fine-tuning the Key and Value projection matrices of the Cross-Attention layer;

[0134] It should be noted that this high-rank setting is not for language adaptation, but to provide sufficient parameter capacity to store high-frequency spatial geometric information, enabling it to adapt to the extremely complex edge alignment relationships in building layouts, significantly enhancing the model's ability to reconstruct geometric details, and preventing generated straight lines from becoming curved or buildings from sticking together.

[0135] Phase Four: Adversarial Training and Fine-tuning under Geometric Constraints

[0136] Step 4.1: Regularizing the dataset and adversarial training strategy:

[0137] Step description: This step configures the training hyperparameters and environment, uses the mean square error of noisy prediction (MSE Loss) as the loss function, and uses the AdamW optimizer in conjunction with a cosine annealing restart strategy to update the parameters;

[0138] Specific implementation: In order to prevent the model from losing its basic geometric generation ability due to overfitting (i.e., to prevent the model from only drawing specific houses and forgetting the basic line logic), this invention constructs a regularization dataset containing 200 general geometric shapes (such as rectangular grids, circles, and concentric boxes).

[0139] During the training iteration, the system mixes these regularized images and their corresponding basic geometric descriptions with a certain probability (e.g., 10%). Through adversarial training, it maintains the geometric generalization of the underlying model and ensures that the generated building outlines have mathematical regularity.

[0140] Step 4.2: Phased Iterative Training and Geometric Fidelity Verification:

[0141] Implementation details: The training process is divided into two stages: coarse-coarse training and fine-tuning.

[0142] First, a coarse training phase is performed, with a batch size of 4 and a learning rate of 1e. -4The training steps should be no less than 10 epochs to enable the model to learn the basic black-and-white figure-ground relationship and Chinese semantic response;

[0143] This is followed by a fine-tuning phase, reducing the learning rate to 1e. -5 The batch size was adjusted to 1, focusing on fixing minor topological errors such as uneven building corners and inconsistent spacing. In the verification stage, this invention abandons the traditional art and aesthetic scoring and innovatively introduces "Line Straightness & Orthogonality Score" as a verification indicator. The system performs Hough Transform on the images generated in the verification set, calculates the degree of aggregation of line segments in directions that are multiples of 90°, and finally selects the weight file with the highest geometric fidelity score and the lowest loss convergence as the LoRA-Layout model to be released.

[0144] It should be noted that during the training process, the model saves its parameters after each certain number of training steps, and these parameters constitute the weight file.

[0145] When the model corresponding to a certain weight file reaches a convergent state during training, this weight file is called a "loss-converged weight file". It represents the parameter state of the model when it has been trained to a relatively stable stage, and has good performance and generalization ability.

[0146] Furthermore, to quantify the engineering usability of the generated results, the "straightness and orthogonality score" defined in this embodiment is not a general aesthetic score (such as FID), but a specialized indicator based on geometric statistics. Its specific calculation logic is as follows:

[0147] Edge detection and Hough line transform are performed on the generated image. Among all the identified line segments, those with a horizontal angle are counted. ) and vertical ( Within the tolerance range (e.g.) The total length of line segments (pixels) is divided by the total length of all edge pixels in the image; this indicator can objectively reflect the degree of orthogonality of the model when generating building walls and road edges, avoiding the problem that traditional indicators cannot identify line distortion.

[0148] Step S2: Extract planning indicators from the GIS data of the target plot, generate a CIM3 level white model based on the building layout generated by the LoRA-Layout model, and use the modeling engine to generate a CIM3 level white model; perform automated UV unwrapping on the CIM3 level white model, output a semantic segmentation map based on (user-defined) constraint rules and construction generation logic, and then encode the visual features parsed from the target plot CIM model to generate texture maps.

[0149] The specific steps of step S2 are as follows:

[0150] Planning indicators are extracted from the GIS data of the target site, including: target floor area ratio (FAR), average floor height, and maximum building height.

[0151] Target floor area ratio: The ratio of the total building area to the land area;

[0152] The building layout generated from the LoRA-Layout model and the planning indicators extracted from the CIM data are input into the procedural modeling engine (a tool that can automatically generate models based on preset rules and input parameters).

[0153] According to the formula: In addition to the average floor height and the maximum building height limit, the height of each building is dynamically calculated;

[0154] in, This indicates the area of ​​the land occupied by the building. This indicates the building's footprint area; FAR represents the target floor area ratio.

[0155] The modeling engine automatically performs push-pull (extrusion) and logical segmentation operations to generate a CIM3 level white model with standard floors, parapet walls and other structures and a rigorous geometric topology.

[0156] The push-pull operation involves vertically stretching the base shape of a building according to a calculated height to form the basic volume of the building.

[0157] Logical segmentation involves further dividing and adjusting the basic blocks based on the building's functional requirements and structural characteristics (such as dividing the building into different floors and rooms).

[0158] A standard floor refers to a floor in a building that has the same or similar floor plan and function;

[0159] A parapet wall is a low wall surrounding the roof of a building, primarily serving a safety and decorative purpose.

[0160] The CIM3 level white model is automatically UV unfolded. The geometry engine outputs a high-precision semantic segmentation map based on (user-defined) constraint rules and construction generation logic (such as the overall style of the building, the level of texture detail, etc.). This semantic segmentation map identifies the generation areas of components (such as windows, walls, roofs, etc. in the building facade).

[0161] The visual features parsed from the target plot's CIM model are encoded. These visual features include architectural style (such as classical, modern, postmodern, etc.), main color scheme, and material preferences (such as glass, stone, wood, etc.). The encoded visual features are then input into a Large Language Model (LLM), which generates a structured Prompt.

[0162] Based on the Prompt, extract the architectural style to obtain the visual feature requirements of the building, select a LoRA-Texture model (e.g., to generate a cyberpunk style architectural texture, load the "Cyberpunk Style" model; to generate a modern residential style, load the "Modern Residential Style" model); import the selected LoRA-Texture model into the texture generation working environment or software, and use the ControlNet-Seg tool to read the semantic segmentation map of the building;

[0163] Based on the semantic segmentation map and the visual feature requirements of the building, specific constraint generation rules are set, such as stipulating that "glass textures are generated only in red areas and stone textures are generated in blue areas";

[0164] The previously set constraint generation rules, along with the loaded texture model (i.e., the LoRA-Texture texture model selected according to the required architectural style) and the semantic segmentation map, are used as input to the texture generation system (such as Substance 3DDesigner or Quixel Suite).

[0165] Use ControlNet-Seg to read semantic segmentation maps and apply strong spatial semantic constraints (e.g., force glass textures to be generated only in the window area).

[0166] Based on the characteristics of the LoRA-Texture model, the strong spatial semantic constraints provided by ControlNet-Seg, and the constraint generation rules, the texture generation system generates texture maps that conform to architectural semantics and style requirements through multimodal guidance.

[0167] The texture generation system extracts the characteristics of the LoRA-Texture texture model, and then reads the position and semantic labels of different components of the building facade (such as windows, walls, roofs, etc.) based on the semantic information provided by ControlNet-Seg, and the component types represented by these labels;

[0168] Please see Figure 4The texture generation system transforms the constraint generation rules into executable parameters and conditions. Based on the characteristics of the LoRA-Texture texture model, the system guides the texture generation in a specific style direction. Taking the generation of cyberpunk-style architectural textures as an example, the model characteristics will prompt the system to incorporate elements such as neon lighting effects, metallic textures, and futuristic patterns into the textures, so that the generated textures have obvious cyberpunk style characteristics.

[0169] In conjunction with the semantic information of ControlNet-Seg, the system checks whether the generated texture matches the semantic region of the building, and checks whether the texture map generated by the system conforms to the constraint generation rules at each stage of texture generation;

[0170] For example, when semantic information indicates that a certain area is a window, the system will guide the generation of a texture suitable for the window, such as a transparent or semi-transparent glass texture.

[0171] When generating stone textures, the size, shape, and distribution of the particles are controlled to meet the constraint that "the size of the stone texture particles is within a specific range".

[0172] During the texture generation process, the texture generation system analyzes the color, shape, and distribution characteristics of the texture map in real time to determine whether it meets the semantic matching and style requirements.

[0173] If the conditions are met, no action is taken; otherwise, the system will adjust the influence weights of the LoRA-Texture model, modify the guidance strength of ControlNet-Seg, or change the specific parameter values ​​in the constraint rules, and then regenerate the texture map until a texture map that meets the requirements is generated.

[0174] The generated texture map is subjected to a quality check to see if it meets the constraint generation rules and visual feature requirements. If it does, no processing is performed. If it does not meet the requirements, the constraint generation rules or the parameters related to texture map generation in the texture generation system are adjusted, and then the texture map is regenerated.

[0175] Step S3: Remap the texture map generated by the (texture generation system) back to the CIM3 level white model surface to obtain a generative reconstruction model that is equivalent to the original scene in terms of physical indicators, maintains continuity in visual style, but achieves complete desensitization in specific geometric details.

[0176] Example 2

[0177] Please see Figure 2 The CIM model generation system based on spatial intelligence and visual knowledge fingerprints includes:

[0178] The data processing module is used to read the GIS data of the target plot, extract the plot boundary as the first hard constraint; perform Canny edge detection on the top view of the original CIM model, mask the plot boundary, and obtain the Canny edge map (such as the pattern characteristics formed by the changes in the shape, number, spacing, direction, and color of lines, such as road grids, entrance and exit locations, and underground pipeline distribution); synthesize the plot boundary and the Canny edge map as the geometric control signal of ControlNet, and then use the LoRA-Layout model to generate the building layout;

[0179] Texture generation module: used to extract planning indicators from the GIS data of the target plot, generate a CIM3 level white model based on the building layout generated by the LoRA-Layout model, and use the modeling engine to generate a CIM3 level white model; automatically unfold the CIM3 level white model with UVs, output a semantic segmentation map based on (user-defined) constraint rules and construction generation logic, and then encode the visual features parsed from the target plot CIM model to generate texture maps.

[0180] Model output module: used to remap the texture maps generated by the (texture generation system) back to the CIM3 level white model surface, to obtain a generative reconstruction model that is physically equivalent to the original scene, maintains continuity in visual style, but achieves complete desensitization in specific geometric details.

[0181] The above formulas are all dimensionless calculations. The formulas are derived from software simulations using a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation. For example, there are weighting coefficients and proportional coefficients. The values ​​set are to quantify each parameter to obtain a specific value, which is convenient for subsequent comparison. The values ​​of the weighting coefficients and proportional coefficients are only required to not affect the proportional relationship between the parameters and the quantified values.

[0182] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A CIM model generation method based on spatial intelligence and visual knowledge fingerprint, characterized in that, The method is as follows: Read the GIS data of the target plot and extract the plot boundary as the first hard constraint; perform Canny edge detection on the top view of the original CIM model, mask the plot boundary, and obtain the Canny edge map; synthesize the plot boundary and the Canny edge map as the geometric control signal of ControlNet, and then use the LoRA-Layout model to generate the building layout. The steps for generating a building layout are as follows: The first set of hard constraints and control signals are input into the configured LoRA-Layout model; The LoRA-Layout model extracts, transforms, and combines features from the input data through its internal neural network layers; During the layout generation process, the LoRA-Layout model combines spatial intelligent logic to preserve the original plot road network characteristics and reconstruct the base shape, location and number of buildings; In the generated layout result, perform data anonymization: For sensitive building shapes, blurring is applied; For special building site locations, a random perturbation algorithm is used to adjust the building location; The desensitized building layout results are used as the model output. Planning indicators are extracted from the GIS data of the target site, and a CIM3 level white model is generated using a modeling engine based on the building layout. Automated UV unwrapping of CIM Level 3 white model, output semantic segmentation map based on constraint rules and construction generation logic, and then encoding the visual features parsed from the target plot CIM model to generate texture map; The specific steps for generating a CIM3 level white model are as follows: Extract planning indicators from the GIS data of the target plot: target floor area ratio (FAR), average floor height, and maximum building height. The building layout generated by the LoRA-Layout model and the planning indicators extracted from the CIM data are input into the procedural modeling engine; According to the formula: In addition to the average floor height and the maximum building height limit, the height of each building is dynamically calculated; in, This indicates the area of ​​the land occupied by the building. Indicates the area of ​​the building's base; The modeling engine automatically performs push-pull and logical segmentation operations to generate and process CIM3 level white models; The steps for processing CIM3 level white models are as follows: The CIM3 level white model is automatically UV unfolded. Based on the constraint rules and construction generation logic, the geometry engine simultaneously outputs a high-precision semantic segmentation map, which identifies the generation area of ​​the component. The visual features parsed from the target plot's CIM model are encoded, including architectural style; the encoded visual features are then input into a large language model to generate a structured Prompt. Based on the Prompt, the architectural style is extracted to obtain the visual feature requirements of the building. The LoRA-Texture texture model is selected. The selected texture model is imported into the texture generation working environment or software, the semantic segmentation map of the building is read, and the texture generation system is used to generate texture maps. The generated texture map is remapped back onto the CIM3 level white model surface to obtain a generative reconstructed model.

2. The CIM model generation method based on spatial intelligence and visual knowledge fingerprint as described in claim 1, characterized in that, The specific steps of Canny edge detection are as follows: Obtain the top view corresponding to the original CIM image, and perform Canny edge detection on the top view using an image processing library: apply Gaussian filtering to the top view, then calculate the gradient magnitude and direction of each pixel in the image, and refine the edge pixels with larger gradient magnitudes through non-maximum suppression. The specific process is as follows: The Sobel operator is used to calculate the gradient of all pixels in the image along the horizontal and vertical axes, and the gradient direction angle of each pixel is obtained. Based on the gradient direction of all pixels in the top view, the overall gradient direction of the top view is discretized into four main directions: 0°, 45°, 90°, and 135°. Iterate through and calculate the gradient direction of each pixel in the top view, and compare the gradient magnitude with the pixels in its 3×3 neighborhood: Let θ be the gradient direction of the pixel at position (i, j) in the top view. （i,j） The gradient magnitude is G （（i）,（j）） ; According to θ （i,j） The size of the gradient is used to determine the main direction to which the pixel belongs. Then, it is determined whether the gradient magnitude of the pixel is the maximum gradient magnitude in its neighborhood. If it is, it is retained; if not, the gradient magnitude of the pixel is set to zero, and the edge pixels with larger gradient magnitudes are refined. In the initial refinement of edge pixels using Canny edge detection, high and low thresholds are set, and dual-threshold detection and edge tracking are used to determine the final edge pixels.

3. The CIM model generation method based on spatial intelligence and visual knowledge fingerprint as described in claim 2, characterized in that, The steps to determine the final edge pixels are as follows: Pixels with gradient magnitudes greater than the high threshold are considered strong edge pixels, pixels with gradient magnitudes between the high and low thresholds are considered weak edge pixels, and pixels with gradient magnitudes less than the high threshold are considered non-edge pixels. Iterate through each weak edge pixel and check if there are strong edge pixels within the 8×8 neighborhood of each weak edge pixel; If it exists, then the weak edge pixel is marked as a true edge pixel; If it does not exist, its gradient magnitude is set to zero; Based on the edge pixels obtained from Canny edge detection, and combined with the land boundary constraints, a control signal without the specific outline of the building is obtained.

4. The CIM model generation method based on spatial intelligence and visual knowledge fingerprint as described in claim 1, characterized in that, The steps for generating texture maps are as follows: Based on the semantic segmentation map and the visual feature requirements of the building, specific constraint generation rules are set; The constraint generation rules, along with the loaded texture model and semantic segmentation map, are used as inputs to the texture generation system. Read the semantic segmentation map and apply strong spatial semantic constraints; Based on the characteristics of the LoRA-Texture model, strong spatial semantic constraints, and constraint generation rules, the texture generation system generates texture maps that meet architectural semantics and style requirements through multimodal guidance. The generated texture map is subjected to a quality check to see if it meets the constraint generation rules and visual feature requirements. If it does, no processing is performed. If it does not meet the requirements, the constraint generation rules or the parameters related to texture map generation in the texture generation system are adjusted, and then the texture map is regenerated.

5. The CIM model generation method based on spatial intelligence and visual knowledge fingerprint as described in claim 4, characterized in that, The steps to generate a texture map are as follows: The texture generation system extracts the characteristics of the LoRA-Texture model and reads the position and semantic labels of different components on the building facade; The texture generation system transforms the constraint generation rules into executable parameters and conditions, and generates texture maps of a specific style based on the characteristics of the LoRA-Texture texture model. Then, combined with the semantic information of ControlNet-Seg, the system checks whether the generated texture matches the semantic region of the building, and checks whether the texture map generated by the system conforms to the constraint generation rules at each stage of texture generation. During the texture generation process, the texture generation system analyzes the color, shape, and distribution characteristics of the texture map in real time to determine whether it meets the semantic matching and style requirements. If the conditions are met, no action is taken; otherwise, the system will adjust the influence weights of the LoRA-Texture model, modify the guiding strength of ControlNet-Seg, or change the specific parameter values ​​in the constraint rules to regenerate the texture map.

6. The CIM model generation method based on spatial intelligence and visual knowledge fingerprint as described in claim 1, characterized in that, The training steps for the LoRA-Layout model are as follows: An initial image library is constructed, and the images in the initial image library are uniformly converted into single-channel binary images, and then geometric orthogonalized. A Chinese visual knowledge fingerprint labeling system is constructed. OpenCV geometric operators are used to calculate the black-and-white pixel ratio and outline bounding rectangle feature labels of binary images. Then, the VQA model is used to identify building type feature labels and integrate them. By selecting the base for freezing weights, the text encoder converts the integrated labels into Chinese technical terms; during training configuration, the input Chinese technical terms are parsed as: high-dimensional semantic conditions compatible with the U-Net network; The rank of LoRA is set to 128, and the scaling factor is set to 128. The system injects the trainable low-rank matrix into the Transformer module of the U-Net network through matrix factorization to adjust the Key and Value projection matrices of the cross-attention layer. Configure the hyperparameters and environment for model training, use the mean squared error of noise prediction as the loss function, and use the AdamW optimizer and cosine annealing restart strategy to update the parameters in the model; First, a coarse training phase is performed, with a batch size of 4 and a learning rate of 1e. -4 The training steps should be no less than 10 epochs to enable the model to learn the basic black-and-white figure-ground relationship and Chinese semantic response; Then, a fine-tuning phase is performed, reducing the learning rate to 1e. -5 Batch Size was adjusted to 1 to fix minor topology errors; In model validation, straightness and orthogonality scores are introduced, and the weight file with the highest geometric fidelity score and loss convergence is selected as the published LoRA-Layout model.

7. A CIM model generation system based on spatial intelligence and visual knowledge fingerprints, applicable to any one of the CIM model generation methods based on spatial intelligence and visual knowledge fingerprints according to claims 1-6, characterized in that, include: Data processing module: used to read the GIS data of the target plot and extract the plot boundary as the first hard constraint; Canny edge detection is performed on the top view of the original CIM model, and the plot boundaries are masked to obtain the Canny edge map. The plot boundaries and the Canny edge map are synthesized as the geometric control signal of ControlNet, and then the LoRA-Layout model is used to generate the building layout. Texture generation module: It is used to extract planning indicators from the GIS data of the target plot, generate a CIM3 level white model based on the building layout generated by the LoRA-Layout model, and use the modeling engine to generate a CIM3 level white model; it performs automatic UV unwrapping on the CIM3 level white model, outputs a semantic segmentation map based on constraint rules and construction generation logic, and then encodes the visual features parsed from the target plot CIM model to generate texture maps. Model output module: used to remap the generated texture maps back to the CIM3 level white model surface, to obtain a generative reconstruction model that is physically equivalent to the original scene, maintains continuity in visual style, but achieves complete desensitization in specific geometric details.

Images