A method and apparatus for generating a pcg configuration file based on a building image

Abstract

The present disclosure relates to a method and device for generating a PCG configuration file based on a building image. The method receives a building image, pre-processes it to obtain a standardized building facade image and estimates the height-to-length ratio, then uses a multi-modal large language model to perform first semantic recognition to generate an initial configuration object containing initial facade parameters of a first information granularity, then performs second semantic recognition based on the initial facade parameters to obtain a facade configuration array of a second information granularity, and finally aggregates the height-to-length ratio and the facade configuration array to generate a PCG configuration file. This method realizes the end-to-end automatic generation from a single building image to a structured PCG configuration file. By organically integrating image preprocessing, first semantic recognition, second semantic recognition, and configuration aggregation into a complete processing pipeline, the structured PCG configuration information is automatically extracted from the building image, without the need for manual configuration file writing or manual intervention in the processing process, significantly improving the automation level and efficiency of building content generation.

Description

Technical Field

[0001] This disclosure relates to the field of procedural content generation (PCG) technology, and specifically to a method and apparatus for generating PCG configuration files based on architectural images. Background Technology

[0002] PCG (Programmable Graph) refers to a class of technologies that automatically or semi-automatically generate digital content using preset algorithm rules and parameters. In the fields of architectural design and 3D content generation, PCG technology has been widely used in the automated creation of architectural models. Compared to traditional manual modeling, PCG can generate complex 3D architectural models through algorithmic rules and data-driven methods, offering advantages such as high efficiency, strong reusability, and ease of batch generation.

[0003] Currently, PCG-based building generation solutions typically rely on manually writing detailed configuration files, which require defining a large amount of structured information such as floor divisions, component parameters, and arrangement rules. Writing such configuration files demands in-depth architectural knowledge from professionals, and the process is time-consuming, error-prone, and unsuitable for large-scale or rapidly iterative content production needs. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method and apparatus for generating PCG profiles based on architectural images, so as to lower the threshold for using PCG technology and meet the needs of large-scale or rapidly iterating content production.

[0005] In a first aspect, embodiments of this disclosure provide a method for generating a PCG configuration file based on a building image, characterized by comprising: receiving an input building image; preprocessing the building image to obtain a standardized building facade image, and estimating the length-to-height ratio of the main building in the building image during the preprocessing process; performing a first semantic recognition on the standardized building facade image using a multimodal large language model to obtain semantic features; generating an initial configuration object based on the semantic features, wherein the initial configuration object includes initial facade parameters at a first information granularity; performing a second semantic recognition on the standardized building facade image based on the initial facade parameters to obtain a facade configuration array at a second information granularity, wherein the second information granularity is finer than the first information granularity; and aggregating the length-to-height ratio and the facade configuration array to generate a PCG configuration file corresponding to the building image.

[0006] In one optional implementation, the building image is preprocessed to obtain a standardized building facade image. During preprocessing, the aspect ratio of the main building in the building image is estimated. This includes: identifying the main building in the building image using an image segmentation algorithm and separating the main building from the background area to obtain a building mask image; calculating the ratio of the pixel span in the vertical direction to the pixel span in the horizontal direction of the main building based on the building mask image to obtain the aspect ratio; detecting the vanishing point and edge lines of the building facade based on the building mask image; estimating the perspective transformation matrix based on the vanishing point and edge lines; and performing an inverse perspective transformation on the main building area in the building image based on the perspective transformation matrix to eliminate perspective distortion and restore the orthographic projection features of the building facade, thereby obtaining a standardized building facade image.

[0007] In one optional implementation, the semantic features include at least one of the following: (a) contour features, including the overall contour shape and boundary orientation of the building facade in a standardized building facade image; (b) texture patterns, including the material texture features of each area of ​​the building facade; and (c) component distribution, including the positional distribution patterns and arrangement patterns of building components in the building facade.

[0008] In one optional implementation, generating an initial configuration object based on semantic features includes: obtaining a predefined configuration template, which defines the field structure of floors, components, and facade column structures in the PCG configuration file; mapping semantic features to the corresponding configuration fields according to the predefined configuration template to generate an initial configuration object that conforms to the format requirements of the PCG configuration file; wherein the initial facade parameters in the initial configuration object include initial values ​​for floor division, component positioning, and facade column structure, and the initial values ​​belong to the first information granularity.

[0009] In one optional implementation, a second semantic recognition is performed on a standardized building facade image based on initial facade parameters to obtain a facade configuration array with a second information granularity. This includes: using each initial value in the initial facade parameters as a priori constraint, performing a second semantic recognition on the standardized building facade image within a candidate region defined by each initial value to obtain a facade configuration array with a second information granularity; wherein, the second semantic recognition includes floor semantic recognition, component semantic recognition, and facade column structure recognition, and the facade configuration array includes a floor object array, a component object array, and a column object array.

[0010] In one optional implementation, floor semantic recognition includes identifying floor boundaries and calculating the total number of floors by analyzing repetitive texture patterns in the horizontal direction in a candidate area defined by the initial value of floor division; estimating the floor height value of each floor based on the spacing ratio of the floor boundaries and the total number of floors, combined with the prior floor height range of the building type corresponding to the building image; identifying the roof type by analyzing the contour shape and texture features of the building's top area; and generating an array of floor objects, where each floor object contains a floor number, a floor height value, and a roof type, with the floor number increasing sequentially from the bottom to the top. In one optional implementation, component semantic recognition includes identifying the bounding box of each building component and the coordinate position of each building component in the facade coordinate system within a candidate area defined by the initial value of the component; determining the component type of each building component based on the shape characteristics and contextual relationships of each building component; calculating the width and height of each building component based on the bounding box of each building component and the overall facade scale; and generating an array of component objects, wherein each component object contains a globally unique component identifier, component type, width, height, and coordinate position. In one optional implementation, facade column structure identification includes dividing the building facade into several columns within candidate areas defined by the initial values ​​of the facade column structure, and determining the boundary position of each column; determining the column state of each column as normal, convex, or concave based on the concave-convex relationship of each column on the building facade; determining the list of component identifiers referenced by each column based on the boundary position of each column and the component distribution of each floor within the column area; and generating an array of column objects, wherein each column object contains a column index, column state, list of component identifiers, and offset, the offset being used to characterize the depth offset value of the column relative to the facade reference plane in the convex or concave state.

[0011] In one optional implementation, the aspect ratio and facade configuration array are aggregated to generate a PCG configuration file, including: calculating the total number of floors and the overall building height based on the floor object array in the facade configuration array; calculating the overall building width based on the component object array and column object array in the facade configuration array; filling the total number of floors, overall building height, overall building width, and / or aspect ratio into geometric parameter objects; and aggregating the geometric parameter objects with the facade configuration array into a structured PCG configuration file in JSON format.

[0012] In one optional implementation, the aspect ratio and facade configuration arrays are aggregated to generate a PCG profile, including: when the received building images include building images from multiple different perspectives of the same building, the aspect ratio and facade configuration arrays from multiple different perspectives are aggregated to generate a PCG profile containing multiple facade information.

[0013] In an optional implementation, after generating the PCG configuration file corresponding to the building image, the method further includes: receiving an input updated building image, wherein the updated building image is an image obtained after the main building in the building image has been modified; determining the changed area of ​​the main building based on the updated building image and the building image; determining the configuration field corresponding to the changed area in the generated PCG configuration file, wherein the configuration field includes at least one of the affected floor object, component object or column object; and performing a local update on the configuration field corresponding to the changed area to generate an updated PCG configuration file.

[0014] Secondly, embodiments of this disclosure provide an apparatus for generating PCG profiles based on architectural images, comprising: The receiving module is used to receive input building images; The preprocessing module is used to preprocess the building images to obtain standardized building facade images, and to estimate the length-to-height ratio of the main building in the building image during the preprocessing process. The first semantic recognition module is used to perform first semantic recognition on standardized building facade images using a multimodal large language model to obtain semantic features; The initial configuration object generation module is used to generate an initial configuration object based on semantic features, wherein the initial configuration object includes initial facade parameters at the first information granularity; The second semantic recognition module is used to perform second semantic recognition on standardized building facade images based on initial facade parameters to obtain a facade configuration array with a second information granularity, which is finer than the first information granularity. The PCG profile generation module is used to aggregate the aspect ratio and facade configuration arrays to generate the PCG profile corresponding to the building image.

[0015] In one optional implementation, the preprocessing module is configured to: identify the main building in the building image using an image segmentation algorithm, and separate the main building from the background area to obtain a mask image of the main building; calculate the ratio of the pixel span in the vertical direction to the pixel span in the horizontal direction of the main building based on the mask image of the main building to obtain the length-to-height ratio; detect the vanishing point and edge lines of the building facade based on the mask image of the main building; estimate the perspective transformation matrix based on the vanishing point and edge lines; and perform an inverse perspective transformation on the main building area in the building image based on the perspective transformation matrix to eliminate perspective distortion, restore the orthographic projection features of the building facade, and obtain a standardized building facade image.

[0016] In one optional implementation, the semantic features include at least one of the following: (a) contour features, including the overall contour shape and boundary orientation of the building facade in a standardized building facade image; (b) texture patterns, including the material texture features of each area of ​​the building facade; and (c) component distribution, including the positional distribution patterns and arrangement patterns of building components in the building facade.

[0017] In one optional implementation, the initial configuration object generation module is used to obtain a predefined configuration template, which defines the field structure of the PCG configuration file regarding floors, components, and facade column structure; based on the predefined configuration template, the semantic features are mapped to the corresponding configuration fields to generate an initial configuration object that conforms to the PCG configuration file format requirements; wherein, the initial facade parameters in the initial configuration object include initial values ​​regarding floor division, component positioning, and facade column structure, and the initial values ​​belong to the first information granularity.

[0018] In one optional implementation, the second semantic recognition module is used to: use each initial value in the initial facade parameters as a priori constraint, perform second semantic recognition on the standardized building facade image within the candidate area defined by each initial value, and obtain a facade configuration array with a second information granularity; wherein, the second semantic recognition includes floor semantic recognition, component semantic recognition and facade column structure recognition, and the facade configuration array includes a floor object array, a component object array and a column object array.

[0019] In one optional implementation, the second semantic recognition module is used to: identify floor boundaries and calculate the total number of floors within a candidate area defined by the initial value of floor division by analyzing the repetitive texture patterns in the horizontal direction of the standardized building facade image; estimate the floor height value of each floor based on the spacing ratio of the floor boundaries and the total number of floors, combined with the prior floor height range of the building type corresponding to the building image; identify the roof type by analyzing the contour shape and texture features of the building top area; and generate an array of floor objects, wherein each floor object contains a floor number, a floor height value, and a roof type, with the floor number increasing sequentially from the bottom to the top. In one optional implementation, the second semantic recognition module is configured to: identify the bounding box of each building component and the coordinate position of each building component in the facade coordinate system within the candidate area defined by the initial value of the component; determine the component type of each building component based on the shape characteristics and contextual relationships of each building component; calculate the width and height of each building component based on the bounding box of each building component and the overall facade scale; and generate an array of component objects, wherein each component object contains a globally unique component identifier, component type, width, height, and coordinate position. In one optional implementation, the second semantic recognition module is configured to: divide the building facade into several columns within the candidate areas of each column defined by the initial value of the facade column structure, and determine the boundary position of each column; determine the column state of each column as normal, convex, or concave based on the concave-convex relationship of each column on the building facade; determine the list of component identifiers referenced by each column based on the boundary position of each column and the component distribution of each floor within the column area; and generate an array of column objects, wherein each column object contains a column index, a column state, a list of component identifiers, and an offset, the offset being used to characterize the depth offset value of the column relative to the facade reference plane in the convex or concave state.

[0020] In one optional implementation, the PCG configuration file generation module is used to: calculate the total number of floors and the overall building height based on the floor object array in the facade configuration array; calculate the overall building width based on the component object array and column object array in the facade configuration array; fill the total number of floors, overall building height, overall building width, and length-to-height ratio into geometric parameter objects; and aggregate the geometric parameter objects and the facade configuration array into a structured PCG configuration file in JSON format.

[0021] In one optional implementation, the PCG profile generation module is used to: when the received building image includes building images from multiple different perspectives of the same building, aggregate the aspect ratios and facade configuration arrays from multiple different perspectives to generate a PCG profile containing multiple facade information.

[0022] In one optional implementation, the update module is configured to receive an input updated building image, wherein the updated building image is an image obtained after the main building in the building image has been modified; based on the updated building image and the building image, determine the changed area of ​​the main building; determine the configuration field corresponding to the changed area in the generated PCG configuration file, wherein the configuration field includes at least one of the affected floor object, component object or column object; and perform local update on the configuration field corresponding to the changed area to generate an updated PCG configuration file.

[0023] Thirdly, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, characterized in that, when executed by a processor, the program implements the steps of the method described in the first aspect above.

[0024] Fourthly, embodiments of this disclosure provide a computing device, including: Memory, used to store computer program products; A processor for executing a computer program product stored in memory, wherein, when the computer program product is executed, it implements the method described in the first aspect above.

[0025] Fifthly, one or more embodiments of this specification also provide a computer program product, which includes one or more programs that, when run on an electronic device, cause the electronic device to implement the method described in the first aspect.

[0026] The method and apparatus for generating PCG configuration files based on architectural images provided in this disclosure receive architectural images, preprocess them to obtain standardized architectural facade images and estimate the length-to-height ratio, then use a multimodal large language model to perform first semantic recognition to generate an initial configuration object containing initial facade parameters with a first information granularity, then perform second semantic recognition based on the initial facade parameters to obtain a facade configuration array with a second information granularity, and finally aggregate the length-to-height ratio and the facade configuration array to generate a PCG configuration file. This method has the following technical effects: (1) End-to-end automatic generation of structured PCG configuration files from a single building image is realized. By organically integrating image preprocessing, first semantic recognition, second semantic recognition and configuration aggregation into a complete processing pipeline, structured PCG configuration information is automatically extracted directly from building images without the need for manual configuration file writing or manual intervention in the processing process, which significantly improves the automation and efficiency of building content generation.

[0027] (2) The quality of the input image is improved through preprocessing. For example, background interference and perspective distortion in the original building image can be eliminated by orderly cascading processing of background removal, aspect ratio estimation and facade perspective correction, and the orthogonal projection features of the building facade can be restored, providing standardized high-quality input for subsequent semantic recognition and avoiding the impact of image distortion on recognition accuracy.

[0028] (3) The two-level semantic recognition architecture balances global semantic understanding and local measurement accuracy. The first semantic recognition uses a multimodal large language model for macro-level semantic understanding, generating coarse-grained initial facade parameters as prior constraints. Guided by the initial facade parameters, the second semantic recognition performs precise measurement and classification within the narrowed candidate region, generating a fine-grained facade configuration array. This two-level architecture effectively addresses the shortcomings of insufficient local accuracy in a single large model scheme and the lack of global constraints in a single vision scheme, thus balancing global understanding capability and local measurement accuracy. Attached Figure Description

[0029] Figure 1 An overall framework diagram of a method for generating PCG profiles based on architectural images, provided by an embodiment of this disclosure, is shown. Figure 2 A flowchart illustrating a method for generating PCG profiles based on architectural images, provided by an embodiment of this disclosure, is shown. Figure 3A schematic diagram of the data structure of a PCG configuration file provided by an embodiment of this disclosure is shown; Figure 4 A schematic diagram of the structure of an apparatus for generating PCG profiles based on architectural images, provided by an embodiment of the present disclosure, is shown. Figure 5 A schematic diagram of the structure of a computing device provided by an embodiment of the present disclosure is shown. Detailed Implementation

[0030] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Through these descriptions, the features and advantages of the present disclosure will become clearer and more apparent.

[0031] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments. Although various aspects of embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0032] Furthermore, the technical features involved in the different embodiments of this disclosure described below can be combined with each other as long as they do not conflict with each other.

[0033] To facilitate understanding, the technical terms and application scenarios involved in the technical solutions provided in the embodiments of this disclosure will be explained below.

[0034] Procedural content generation (PCG) refers to a type of technology that automatically or semi-automatically generates digital content using preset algorithmic rules and parameters. In the fields of architectural design and 3D content generation, PCG technology has been widely used in the automated creation of architectural models. Compared to traditional manual modeling, PCG can generate complex 3D architectural models through algorithmic rules and data-driven methods, offering advantages such as high efficiency, strong reusability, and ease of batch generation.

[0035] Currently, PCG-based building generation solutions typically rely on manually writing detailed configuration files, which require defining a large amount of structured information such as floor divisions, component parameters, and arrangement rules. Writing such configuration files demands in-depth architectural knowledge from professionals, and the process is time-consuming, error-prone, and unsuitable for large-scale or rapidly iterative content production needs.

[0036] To address the aforementioned technical problems, this disclosure provides a method and apparatus for generating PCG configuration files based on building images. This method automatically extracts structured configuration information of the building's main structure from the building image end-to-end, transforming unstructured image data into a configuration file that can directly drive the PCG system.

[0037] In this way, by converting architectural images end-to-end into structured PCG profiles, users do not need to have professional knowledge of PCG rule writing. They only need to provide architectural images to automatically obtain usable profiles, which significantly reduces the threshold for use and meets the needs of large-scale or rapidly iterating content production.

[0038] This disclosure does not limit the application scenarios of the method for generating PCG configuration files based on architectural images. For example, it can be applied to application scenarios such as architectural design automation, 3D architectural model generation, game scene construction, and urban planning visualization.

[0039] The overall architecture of the method for generating PCG configuration files based on architectural images provided in this disclosure embodiment is described below.

[0040] Figure 1 This is an overall framework diagram of a method for generating PCG configuration files based on architectural images, provided in an embodiment of this disclosure.

[0041] See Figure 1 , Figure 1 This paper presents the end-to-end processing pipeline architecture for a method of generating PCG profiles based on architectural images. The diagram illustrates the complete processing flow from architectural image input to PCG profile output, which can be divided into four stages: image preprocessing stage (stage 1), JSON inference stage (stage 2), automatic semantic recognition stage (stage 3), and PCG profile output stage (stage 4). The output PCG profile can be used by downstream consumer modules.

[0042] Figure 1 The input to the mid-to-end processing flow is architectural images, and the output is a PCG configuration file.

[0043] Phase 1 receives a single building facade image as input and performs three preprocessing operations in sequence to obtain a standardized building facade image.

[0044] Optionally, Stage 1 can include three preprocessing modules executed sequentially: a background removal module, a length-to-height ratio estimation module, and a facade perspective correction module. The background removal module extracts the main building area and removes background interference such as the sky, ground, and surrounding scenery. The length-to-height ratio estimation module estimates the ratio of the building's height to its width and determines the building's overall morphological characteristics based on this ratio. The facade perspective correction module eliminates perspective distortion caused by the shooting angle and restores the orthographic projection characteristics of the building facade. These three modules can be cascaded, with the output of the previous module serving as the input to the next, forming a complete preprocessing pipeline.

[0045] Phase 2 involves inputting the preprocessed standardized building facade images into a multimodal large language model (LLM) and performing JSON inference cascade processing.

[0046] Optionally, Phase Two may include two modules: an image understanding module and a structured JSON output module. The image understanding module performs deep semantic analysis on building facade images, identifying semantic features such as building outlines, texture patterns, and component distribution. The structured JSON output module converts the semantic features extracted by the image understanding module into structured JSON-formatted configuration data. These two modules interact via semantic feature transfer, ensuring accurate mapping of semantic features from the image space to the configuration space.

[0047] Phase 3, based on JSON inference cascading, performs fine-grained automatic semantic recognition, including three parallel recognition tasks: floor semantic recognition, component semantic recognition, and facade column structure recognition.

[0048] Optionally, Phase 3 includes three parallel identification modules: a floor semantic identification module, a component semantic identification module, and a facade column structure identification module. The floor semantic identification module automatically identifies parameters such as the total number of floors, floor height, and roof type. The component semantic identification module automatically identifies parameters such as the location coordinates, component type, width, and height of building components such as doors, windows, and walls on the building facade. The facade column structure identification module automatically identifies the column divisions, column status, and the list of component identifiers referenced by each column on the building facade. These three identification modules share the output of Phase 2 as input, execute independently, and their outputs collectively constitute the complete configuration information at the facade level.

[0049] Phase 4 aggregates the processing results from each phase into a unified structured PCG configuration file, which can be serialized and stored using JSON format.

[0050] Optionally, Phase 4 can be used to aggregate the length-to-height ratio estimated in Phase 1 with the arrays of floor objects, component objects, and column objects generated in Phase 3 to calculate the overall geometric parameters of the building and generate a structured PCG configuration file. This structured PCG configuration file can be in JSON format and contains four top-level fields: floors, components, columns, and geometry.

[0051] The structured PCG configuration file generated in Phase 4 can be used as input data for downstream consumer modules. For example, the structured PCG configuration file can be used to drive the PCG system to generate 3D building models.

[0052] The method for generating PCG configuration files based on architectural images, as provided in the embodiments of this disclosure, will be described in detail below.

[0053] Figure 2 This is a flowchart of a method for generating PCG configuration files based on architectural images, provided in an embodiment of this disclosure.

[0054] See Figure 2 The method may include the following steps: Step S101: Receive the input building image.

[0055] Architectural images are digital images that include the complete building facade, and the image format supports common formats such as JPEG and PNG. Architectural images can be submitted by users through a file upload interface or acquired through external image acquisition devices.

[0056] Optionally, the received building images can be one or more, and this disclosure does not limit this.

[0057] When multiple building images are received, batch processing can be performed using the solution provided in this embodiment. For example, multiple building images can be received in batches by specifying a folder path or a list of image URLs; preprocessing, first semantic recognition, second semantic recognition, and configuration aggregation operations are performed on each building image (see the description of steps S102 to S106 for details) to generate their respective corresponding PCG configuration files; and the generated PCG configuration files are output in batches according to preset naming rules. In batch processing mode, the processing flow of each building image is independent and can be executed in parallel to improve processing efficiency, making it suitable for configuration generation scenarios of large-scale building datasets.

[0058] To facilitate understanding, the following example illustrates the processing flow of a building image.

[0059] Step S102: Preprocess the building image to obtain a standardized building facade image, and estimate the length-to-height ratio of the main building in the building image during the preprocessing process.

[0060] Combination Figure 1 In the first stage of the demonstration, the following three preprocessing operations are performed sequentially on the input architectural image: (1) Background removal operation.

[0061] Background removal is used to extract the main building from the input building image, separating the main building area from the background area to eliminate interference factors such as the sky, ground, and surrounding scenery, so that subsequent processing can focus on the building facade itself.

[0062] For example, the outline of the main building in a building image can be identified by an image segmentation algorithm, and the main building can be separated from the background area to obtain a mask image of the main building.

[0063] For example, a residential building photograph might contain a large area of ​​sky, greenery, and adjacent buildings. After background removal, the sky, ground, and surrounding scenery are removed, leaving only the main building area, resulting in a masked image containing only the main building.

[0064] In this way, background removal eliminates interference information in the image that is unrelated to the building facade, allowing subsequent aspect ratio estimation and facade perspective correction to focus on the main building area, avoiding interference from background elements on building outline detection and edge extraction, thereby improving the accuracy of each preprocessing step.

[0065] (2) Estimation of length-to-height ratio.

[0066] The length-to-height ratio refers to the ratio of the building's height to its width. It can be used to determine the overall morphological characteristics of a building, such as whether it is tall or flat, serving as a reference for subsequent configuration.

[0067] For example, the length-to-height ratio can be obtained by calculating the ratio of the pixel span in the vertical direction to the pixel span in the horizontal direction of the building body mask image obtained after background removal.

[0068] For example, in the masked image of a building after background removal, if the building spans 1800 pixels vertically and 1000 pixels horizontally, then the length-to-height ratio is 1800 / 1000 = 1.8. This ratio is greater than 1.5, indicating that the building is a tall residential building. If a commercial building has a vertical span of 600 pixels and a horizontal span of 500 pixels, then the length-to-height ratio is 1.2, indicating a flat building.

[0069] In this way, the length-to-height ratio parameter provides a priori constraints on the overall building form for subsequent configuration generation, enabling floor semantic recognition and facade column structure recognition to be carried out under reasonable form expectations. At the same time, this parameter is also an important component of the geometric parameters in the final PCG configuration file.

[0070] (3) Facade perspective correction operation.

[0071] The facade perspective correction operation is used to eliminate perspective distortion in the main building area of ​​a building image after estimating the aspect ratio, and restore the orthogonal projection characteristics of the building facade.

[0072] For example, based on the building body mask image after estimating the aspect ratio, the vanishing point and edge line of the building facade can be detected; based on the detected vanishing point and edge line, the perspective transformation matrix can be estimated; based on the perspective transformation matrix, the inverse perspective transformation of the building body area in the building image can be performed to eliminate the perspective distortion caused by the shooting angle, and a standardized building facade image can be obtained.

[0073] For example, in a photograph of a building taken from below, the originally parallel vertical lines of the building facade appear to converge upwards, while the originally horizontal floor slab lines appear to be tilted. After facade perspective correction, it was detected that the vanishing point of the vertical lines in the facade is located at the top of the image, and the horizontal edges are tilted to the upper right. Based on this, the perspective transformation matrix was estimated, and an inverse perspective transformation was performed on the main building area in the architectural image. In the corrected image, the vertical lines are restored to be parallel to each other and perpendicular to the ground, and the horizontal lines are restored to be parallel to each other and parallel to the ground.

[0074] In this way, facade perspective correction eliminates the geometric distortion introduced by the shooting angle, making the vertical and horizontal edges of the building facade in the corrected standardized building facade image perpendicular to each other, which conforms to the visual effect when the human eye looks at the building directly. This provides a standardized input with accurate geometric relationships for the first and second semantic recognition of the subsequent multimodal large language model, and avoids the errors caused by perspective distortion to the detection of floor boundaries, component size measurement and column boundary positioning.

[0075] It should be noted that the above three preprocessing steps are executed in a cascaded manner, with the output of the previous operation serving as the input of the next operation, gradually improving the image quality. The final output is a standardized building facade image that has eliminated background interference and perspective distortion and includes the overall building shape parameters, providing high-quality input data for the subsequent second and third stages of processing.

[0076] Combination Figure 1 Phase two, as demonstrated, involves inputting the preprocessed standardized building facade images into a large language model (LLM). Leveraging the LLM's image understanding capabilities, the preprocessed standardized building facade images are transformed into structured JSON configuration data, achieving intelligent mapping from image semantics to configuration parameters. The specific implementation of phase two is described in steps S103 and S104.

[0077] Step S103: Use a multimodal large language model to perform first semantic recognition on the standardized building facade image to obtain semantic features.

[0078] In this embodiment, the multimodal large language model has image understanding capabilities and can perform deep semantic analysis on the input standardized building facade image to identify high-level semantic features.

[0079] For example, semantic features may include at least one of the following: (a) Outline features, including the overall outline shape and boundary orientation of the building facade in standardized building facade images.

[0080] (b) Texture patterns, including material texture features of various areas of the building facade, such as brick wall texture, glass curtain wall texture, etc.

[0081] (c) Component distribution, including the location distribution pattern and arrangement of building components in the building facade.

[0082] In this embodiment of the disclosure, building components are identifiable and parameterizable independent constituent elements on a building facade, and are structurally described in the PCG configuration file in the form of Component objects. For example, building components may include, but are not limited to, doors, windows, and walls.

[0083] Step S104: Generate an initial configuration object based on semantic features, wherein the initial configuration object includes initial facade parameters at the first information granularity.

[0084] Combination Figure 2 The image understanding module passes semantic features to the structured JSON output module, which can then map the semantic features into configuration data in JSON format.

[0085] In one optional implementation, generating the initial configuration object based on semantic features can be achieved as follows: A predefined configuration template is obtained, which defines the field structures for floors, components, and facade column structures in the PCG configuration file; semantic features are mapped to the corresponding configuration fields according to the predefined configuration template to generate an initial configuration object that conforms to the PCG configuration file format requirements; wherein, the initial facade parameters in the initial configuration object include initial values ​​for floor division, component positioning, and facade column structure, and these initial values ​​belong to the first information granularity. That is, the first information granularity refers to the granularity of the facade parameters represented in the form of intervals / ranges / categories.

[0086] For example, after the multimodal large language model identifies semantic features from standardized building facade images, such as the building facade being roughly rectangular in shape, the facade material being mainly brick texture, windows being evenly distributed in a grid pattern, and the main entrance being located in the center of the ground floor, the structured JSON output module maps these semantic features into an initial configuration object based on a predefined configuration template. The initial value for floor division can include the estimated total number of floors (e.g., "approximately 6 to 8 floors") and the approximate height range of each floor; the initial value for component location can include the approximate location and general category of doors and windows; and the initial value for facade column structure can include the estimated number of columns and the preliminary arrangement of each column. This initial configuration object provides a structured prior framework for subsequent fine-grained recognition.

[0087] In this way, by using the first semantic recognition and templated mapping of the multimodal large language model, unstructured architectural images are transformed into structured initial configuration objects, providing spatial positioning guidance and parameter initialization constraints for the accurate measurement of the second semantic recognition, and reducing the search space and computational complexity of fine-grained recognition.

[0088] In addition, the image understanding module and the structured JSON output module exchange data by passing semantic features to ensure that the semantic information extracted from the image space can be accurately and completely mapped to the structured data in the configuration space, avoiding information loss or distortion.

[0089] Step S105: Perform second semantic recognition on the standardized building facade image based on the initial facade parameters to obtain a facade configuration array with a second information granularity. The second information granularity is finer than the first information granularity.

[0090] Combination Figure 1 The third stage of the demonstration involves performing finer-grained automatic semantic recognition on standardized building facade images, i.e., second semantic recognition, based on the first semantic recognition, to obtain a facade configuration array with a second information granularity.

[0091] An optional implementation method for the second semantic recognition may include: using the initial values ​​in the initial facade parameters as prior constraints, performing floor semantic recognition, component semantic recognition, and facade column structure recognition on the standardized building facade image respectively.

[0092] For example, floor semantic recognition can be implemented as follows: within the candidate area defined by the initial value of floor division, the floor boundary lines are identified and the total number of floors is calculated by analyzing the repetitive texture patterns in the horizontal direction of the standardized building facade image; based on the spacing ratio of the floor boundary lines and the total number of floors, combined with the prior floor height range of the building type corresponding to the building image, the floor height value of each floor is estimated; the roof type is identified by analyzing the contour shape and texture features of the building top area; and an array of floor objects is generated, where each floor object contains a floor number, a floor height value, and a roof type, with the floor number increasing sequentially from the bottom to the top.

[0093] Among these features, repeating texture patterns can be characteristics that identify floor boundaries, such as floor slab lines and window sills. Roof types can include flat roofs, gable roofs, and hipped roofs.

[0094] The results of floor semantic recognition can be stored as an array of floor objects, such as... Figure 3 As shown, each floor object can contain attributes such as floor_id (floor number, 0, 1, 2, ... from the bottom to the top), floor_height (floor height in meters), roof_type (roof type enumeration value), and column_count (number of columns on the facade of this floor). The column_count parameter can be filled in after the facade column structure is identified.

[0095] For example, in the initial facade parameters output by the first semantic recognition, the initial value for floor division indicates that the building facade roughly contains 6 to 8 floors, and the roof area has a triangular outline. Under the constraint of this initial value, the floor semantic recognition detects 7 clear floor slab lines in the specified vertical candidate area, thus determining the total number of floors to be 7; the spacing between adjacent floor slab lines is 150 pixels, with a spacing ratio of 1:1:1:1:1:1:1. Combining this with the prior knowledge that the standard floor height for this building type (such as residential) is approximately 3 meters, the floor height of each floor is estimated to be 3 meters; the top area is confirmed to be a gable roof through contour matching. The final generated floor object array contains 7 floor objects, with the first floor numbered 0 and the top floor numbered 6, and the top floor roof type being a gable roof.

[0096] In this way, by using the initial floor division values ​​in the initial facade parameters as a guide, floor semantic recognition does not need to blindly search for floor lines on the entire image, but can perform precise detection within a narrowed candidate area, which improves recognition efficiency and reduces missed detections and false detections.

[0097] For example, component semantic recognition can be implemented as follows: within the candidate area defined by the initial value of the component, identify the bounding box of each building component and the coordinate position of each building component in the facade coordinate system; determine the component type of each building component based on the shape characteristics and contextual relationships of each building component; calculate the width and height of each building component based on the bounding box of each building component and the overall facade scale; generate an array of component objects, where each component object contains a globally unique component identifier, component type, width, height and coordinate position.

[0098] The results of component semantic recognition can be stored in the form of an array of component objects, such as... Figure 3 As shown, each component object can contain attributes such as component_id (globally unique component identifier), component_type (component type enumeration value, such as door / window / wall), width (component width in meters), height (component height in meters), and position (the component's two-dimensional coordinate position in the facade, including two sub-attributes x and y).

[0099] For example, in the initial facade parameters output by the first semantic recognition, the initial component localization value indicates that there is a door in the central area of ​​the ground floor of the building facade, and windows are evenly distributed on each floor. Guided by this initial value, the component semantic recognition accurately locates a door component with a width of 2 meters and a height of 3 meters in the candidate area of ​​the ground floor, and determines the component type to be a door; it detects window bounding boxes one by one in the candidate areas of each floor, and determines the component type to be a window. The final generated component object array contains 1 door object and 14 window objects, each object containing precise size and coordinate information.

[0100] In this way, by using the initial values ​​of component positioning in the initial facade parameters as a guide, component semantic recognition narrows the target detection range from the entire image to each candidate region, significantly reducing the false recognition rate, while improving the accuracy of component bounding boxes and size measurements.

[0101] For example, facade column structure identification can be achieved as follows: within the candidate areas of each column defined by the initial value of the facade column structure, the building facade is divided into several columns, and the boundary position of each column is determined; based on the concave-convex relationship of each column on the building facade, the column state of each column is determined to be normal, convex, or concave; based on the boundary position of each column and the component distribution of each floor in the column area, the list of component identifiers referenced by each column is determined; an array of column objects is generated, wherein each column object contains a column index, column state, list of component identifiers, and offset, the offset being used to characterize the depth offset value of the column relative to the facade reference plane in the convex or concave state.

[0102] In a building facade, the state of a column—whether it is ordinary, protruding, or recessed—describes the spatial relationship between the wall plane containing that column and the reference plane of the main building. It is a core parameter that constitutes the three-dimensionality of the facade.

[0103] In a normal state, the wall is flush with the main reference plane of the building, neither protruding forward nor recessed backward. A protruding state means the wall extends forward (towards the observer) and beyond the main reference plane of the building, such as an entrance porch or bay window on the building's central axis. A recessed state means the wall is recessed backward (away from the observer) and below the main reference plane of the building, such as a recessed balcony area.

[0104] Taking a three-column building as an example: the middle column protrudes outward to form an entrance porch, with a protrusion depth of 0.5 meters, that is, a forward offset of 0.5 meters, and the column is in a protruding state; the left and right columns are flush with the main wall of the building, with a forward offset of 0, and the column is in a normal state; if a column is recessed inward to form a balcony, its wall is recessed 0.3 meters behind the main wall, that is, a backward offset of 0.3 meters, and the column is in a recessed state.

[0105] The recognition results of the facade column structure can be stored in the form of an array of column objects, such as... Figure 3 As shown, each column object can contain attributes such as column_index (column index, from left to right: 0, 1, 2, ...), column_state (column state enumeration value: normal, convex, or concave), component_refs (an array of component identifiers referenced by the column, arranged in top-to-bottom order), offset_front (front offset, depth offset value in convex state), and offset_back (back offset, depth offset value in concave state).

[0106] For example, in the initial facade parameters output by the first semantic recognition, the initial value of the facade column structure indicates that the building facade is roughly divided into 3 columns, with the middle column possibly being a protruding state. Under the constraint of this initial value, the facade column structure recognition detects 2 vertical dividing lines in the horizontal direction, dividing the facade into 3 columns, with column indices from left to right being 0, 1, and 2. Through concavity and convexity analysis, it is determined that the first column is in a protruding state, while the 0th and 2nd columns are in a normal state. The protrusion depth of the first column is 0.5 meters, so its forward offset is 0.5 meters. Based on the association between components and columns, the component identifier list referenced by the 0th column is determined to be [comp_0,comp_1,…], the component identifier list referenced by the 1st column is [comp_7,comp_8,…], and the component identifier list referenced by the 2nd column is [comp_14,comp_15,…].

[0107] In this way, by using the initial values ​​of the facade column structure in the initial facade parameters as a guide, the facade column structure identification accurately determines the boundaries, states and component affiliations of each column within the defined column candidate area, providing accurate parameter basis for the PCG system to generate the three-dimensional concave and convex structure of the facade.

[0108] The aforementioned arrays of floor objects, component objects, and column objects together constitute the facade configuration array, which is the configuration data at the second information granularity. In other words, the second information granularity refers to the granularity at which facial parameters are represented by defined values ​​and complete fields.

[0109] It should be noted that the above-mentioned floor semantic recognition, component semantic recognition and facade column structure recognition can be performed in parallel or in sequence, and this disclosure does not limit this.

[0110] Step S106: Aggregate the length-to-height ratio and facade configuration arrays to generate the PCG configuration file corresponding to the building image.

[0111] This step aggregates the processing results from the preceding stages into a unified structured PCG configuration file.

[0112] In one alternative implementation, the total number of floors and the overall building height can be calculated based on the floor object array in the facade configuration array; the overall building width can be calculated based on the component object array and column object array in the facade configuration array; the total number of floors, the overall building height, the overall building width, and the length-to-height ratio can be filled into geometric parameter objects; and the geometric parameter objects and the facade configuration array can be aggregated into a structured PCG configuration file in JSON format.

[0113] The overall building width is calculated based on the building facade image scale and the column boundary positions in the column object array; the overall building height is calculated by summing the floor height values ​​of each floor object; the total number of floors is equal to the length of the floor object array; the length-to-height ratio can be estimated during the image preprocessing stage or calculated based on the overall building height and overall building width, and this embodiment does not limit this.

[0114] Alternatively, the total number of floors, the overall building height, and the overall building width can be filled as properties in the geometry parameter object; or the total number of floors and the length-to-height ratio can be filled as properties in the geometry parameter object.

[0115] For example, the array of floor objects, the array of component objects, the array of column objects, and the height ratio generated during the image preprocessing stage can be aggregated into a complete JSON object.

[0116] like Figure 3As shown, the top-level structure of this JSON object (PCG_Config) contains four fields: floors, components, columns, and geometry. Then, the properties in the geometry object are filled in, including building_width (overall building width), building_height (overall building height), floor_count (total number of floors), and aspect_ratio (length-to-height ratio). Next, the aggregated JSON object undergoes basic format validation to ensure all required fields are filled, data types conform to predefined requirements, and array lengths match nominal values. Finally, the validated JSON object is serialized into a JSON text string and output as a file to a specified path with the extension .json and UTF-8 encoding.

[0117] For example, the image preprocessing stage yields an array containing 7 floor objects, a component semantic recognition stage yields an array containing 15 component objects, and a facade column structure recognition stage yields an array containing 3 column objects. The length-to-height ratio estimated during image preprocessing is 2.1. During data aggregation, these three arrays and the length-to-height ratio are aggregated into a single JSON object. During parameter filling, the overall building height is calculated to be 21 meters based on the 7 floor objects, and the overall building width is calculated to be 10 meters based on the boundary positions of the 3 column objects and the width of the component objects. The total number of floors is filled to 7, and the length-to-height ratio is verified to be 21 / 10 = 2.1, consistent with the preprocessing estimate. After passing format validation, the data is serialized into a UTF-8 encoded JSON file and output.

[0118] In this way, by aggregating the geometric parameters extracted in the image preprocessing stage with the refined facade configuration array generated in the semantic automatic recognition stage, a complete structured configuration file that meets the input requirements of the PCG system is formed, realizing end-to-end automated processing from a single building image to a PCG configuration file that can directly drive the generation of building models.

[0119] In other embodiments, this disclosure also provides a multi-view fusion configuration generation mode.

[0120] In one alternative implementation, when the received building images include building images from multiple different perspectives of the same building, the aspect ratios and facade configuration arrays from multiple different perspectives can be aggregated to generate a PCG configuration file containing multiple facade information.

[0121] For example, in step S101, building images of the same building from multiple different perspectives are received. For example, multiple different perspectives include facade perspectives such as front, side, and rear. Steps S102 to S105 are performed independently for each perspective of the building image. Then, in step S106, the recognition results of multiple perspectives (such as length-to-height ratio, facade configuration array, etc.) are fused to generate a comprehensive PCG configuration file containing multiple facade information.

[0122] For example, in step S101, architectural images of the same building from three different perspectives are received: a front elevation image, a side elevation image, and a rear elevation image. Preprocessing, first semantic recognition, and second semantic recognition are performed on the front elevation image to obtain the aspect ratio and facade configuration array, which includes 7 floor objects, 15 component objects, and 3 column objects. The same processing is performed on the side elevation image to obtain the aspect ratio and facade configuration array, which includes 7 floor objects, 6 component objects, and 1 column object. The same processing is performed on the rear elevation image to obtain the aspect ratio and facade configuration array, which includes 7 floor objects, 10 component objects, and 2 column objects. In step S106, the aspect ratios and facade configuration arrays from the three perspectives are aggregated into a single comprehensive PCG configuration file. The top-level structure of this integrated PCG configuration file is organized according to the facade orientation, for example, it includes three sections: front, side, and back. Each section contains its own floor data, component data, column data, and geometric parameters.

[0123] In this way, the multi-view fusion mode can integrate the recognition results of multiple facades of a building to generate a complete configuration covering the entire building, which is particularly suitable for 3D modeling and digital twin scenarios that require accurate reproduction of the overall appearance of a building.

[0124] In other embodiments, this disclosure also provides an incremental configuration update mode.

[0125] In one optional implementation, after generating the PCG configuration file corresponding to the building image, an updated building image is received as input. The updated building image is an image obtained after the main building in the building image has been modified. Based on the updated building image and the building image, the changed area of ​​the main building is determined. The configuration field corresponding to the changed area in the generated PCG configuration file is determined. The configuration field includes at least one of the affected floor object, component object, or column object. The configuration field corresponding to the changed area is locally updated to generate the updated PCG configuration file.

[0126] For example, after processing an image of the front facade of a building using the method of this disclosure, a corresponding structured PCG configuration file is generated, which includes 7 floor objects, 15 component objects, and 3 column objects. Subsequently, the building undergoes partial renovations, replacing the central door on the ground floor with a wider glass door and changing the windows on both sides to floor-to-ceiling windows. The user takes an updated image of the renovated building and inputs it into the system. The system compares the updated building image with the original building image and detects that the changed area is located within the first column of the bottom layer (floor 0). It then determines that the changed area corresponds to the following configuration fields in the generated configuration file: the floor object with floor number 0 in the floor object array; the component objects with component identifiers comp_0 (original door component) and comp_1 and comp_2 (original window components) in the component object array; and the component reference list in the column object with column index 1 in the column object array. The system performs partial updates to the affected configuration fields: keeping the component type of comp_0 as "door" and updating its width from 2 meters to 3 meters; updating the component types of comp_1 and comp_2 from "window" to "floor-to-ceiling window" and their height from 1.5 meters to 2.5 meters; and updating the corresponding component reference list in the column object. The configuration data for the remaining 6 floors, 12 components, and 2 columns remains unchanged.

[0127] In this way, the incremental configuration update mode only needs to update the configuration fields corresponding to the areas of the building facade that have undergone local changes, avoiding the reprocessing of the entire building image and the regeneration of the full configuration file, which significantly improves the efficiency of configuration updates. It is particularly suitable for configuration maintenance and version iteration in scenarios where building facades are frequently modified locally.

[0128] Once the structured PCG configuration file is output, it can be used by downstream consumer modules.

[0129] For example, refer to Figure 1 For downstream consumption, the output of this disclosure can be used by the following types of external modules: (a) 3D assembly module: reads column data and component data in the PCG configuration file and performs automatic assembly of parametric 3D buildings; (b) Visual editing module: reads the PCG configuration file and presents it in the editor interface, supporting users to visually edit the configuration; (c) Regeneration closed-loop module: manages the version lifecycle of the PCG configuration file and supports one-click regeneration and configuration rewriting after editing.

[0130] It should be noted that the specific implementation of the downstream consumption module is not the focus of this disclosure, but is only provided as an example and will not be further elaborated.

[0131] The following two specific examples illustrate the solutions provided in the embodiments of this disclosure.

[0132] Example 1: Generate the PCG configuration file corresponding to the facade of a residential building.

[0133] Using an image of the facade of a six-story residential building as input, the following processing is performed according to the method provided in this embodiment of the disclosure: 1. Input stage: Receive a JPEG image of a residential building facade with a resolution of 1920×1080 pixels.

[0134] 2. Preprocessing stage: Perform background removal to extract the main building area; perform length-to-height ratio estimation to obtain a length-to-height ratio of approximately 1.8; perform facade perspective correction to eliminate perspective distortion caused by shooting angle and output a standardized orthogonal facade image.

[0135] 3. JSON Inference Phase: The image understanding module of the multimodal large language model identifies semantic features of the residence, such as a rectangular facade, a brick-concrete wall texture, and a regular grid arrangement of components. The structured JSON output module generates an initial configuration object based on these semantic features. The initial facade parameters contain initial values ​​for floor division, component positioning, and facade column structure, representing the first level of information granularity.

[0136] 4. Automatic Semantic Recognition Stage: Using the initial values ​​in the initial facade parameters as prior constraints, perform floor semantic recognition, component semantic recognition, and facade column structure recognition respectively.

[0137] Among them, the semantic recognition of the floors determines that there are a total of 6 floors, each floor is 3.0 meters high, the roof type is flat, and generates a floor object array containing 6 floor objects; The component semantic recognition identified a total of 6 door components and 42 window components. The type, width, height and position coordinates of each component were completely recorded and an array of component objects was generated. The facade column structure identification determines that the building facade is divided into 7 columns in the horizontal direction. Each column is in a normal state. Each column contains 6 layers and a total of 6 components (each layer corresponds to 1 component, the bottom part of the column is the door component, and the other layers are the window components), generating a column object array containing 7 column objects.

[0138] The three object arrays mentioned above together constitute the facade configuration array for the second information granularity.

[0139] 5. Configuration Output Stage: Aggregate the results from each stage to generate a complete structured PCG configuration file. In the geometric parameters, the overall building width is 10 meters, the overall building height is 18.0 meters, the total number of floors is 6, and the length-to-height ratio is 1.8. The configuration file is output in JSON text format.

[0140] Example 2: Generate PCG configuration file for commercial building facades.

[0141] Use a photograph of the facade of a three-story commercial building as input: 1. Input stage: Receive a facade photo of a three-story commercial building (such as a small shopping mall). The ground floor of the building has a large glass curtain wall, and the upper floor has a regular arrangement of windows.

[0142] 2. Preprocessing stage: After background removal, length-to-height ratio estimation (length-to-height ratio is about 1.2, which is a flat building), and facade perspective correction, a standardized orthogonal facade image is output.

[0143] 3. JSON Inference Phase: The image understanding module identifies semantic features such as a large-area glass curtain wall texture at the bottom layer, a brick wall texture at the top layer, and a component distribution pattern of sparse bottom layer and dense top layer. The structured JSON output module generates an initial configuration object based on these semantic features.

[0144] 4. Semantic Automatic Recognition Stage: The floor semantic recognition determines that there are a total of 3 floors, with a floor height of 4.5 meters on the ground floor, 3.6 meters on the second floor, and 3.6 meters on the third floor. The roof type is a gable roof. The component semantic recognition identifies large-area glass curtain wall components on the ground floor and standard window components on the upper floors. The facade column structure recognition determines that the building facade is divided into 12 columns, with 4 columns in the middle area being recessed and 4 columns on each side being normal.

[0145] 5. Configuration Output Stage: In the generated PCG configuration file, the column status field of the column data includes two values: normal and recessed. The length-to-height ratio in the geometric parameters is 1.2, reflecting the flattened morphological characteristics of commercial buildings. The configuration file is output in JSON text format.

[0146] In summary, the method for generating PCG profiles based on architectural images provided in this disclosure has the following beneficial effects: (1) Achieve end-to-end automatic generation from a single building image to a structured PCG configuration file.

[0147] This disclosure presents for the first time a complete end-to-end processing pipeline from a single architectural image to a structured PCG configuration file. It organically integrates four stages: image preprocessing, multimodal AI inference, automatic semantic recognition, and structured configuration output. This achieves fully automated conversion from architectural images to PCG configurations, eliminating the need for manual configuration file writing or intervention in the processing. This pipeline solves the problem of separation between image processing and PCG configuration generation in existing technologies, significantly improving the automation and efficiency of architectural content generation.

[0148] (2) Improve the quality of the input image through preprocessing cascade to provide a reliable basis for subsequent reasoning.

[0149] This disclosure proposes a cascaded preprocessing method for architectural images. By sequentially chaining three operations—background removal, aspect ratio estimation, and facade perspective correction—it transforms raw architectural images acquired under arbitrary shooting conditions into standardized architectural facade images. Background removal eliminates interference factors such as the sky, ground, and surrounding scenery, allowing subsequent processing to focus on the building itself. Aspect ratio estimation provides parameters for the overall morphological characteristics of the building, offering a reference for configuration generation. Facade perspective correction eliminates perspective distortion caused by the shooting angle, restoring the orthographic projection characteristics of the building facade. The three preprocessing steps are executed in cascade, progressively improving image quality and ensuring standardized, high-quality input for subsequent AI inference stages.

[0150] (3) The two-level semantic recognition architecture balances global understanding and local accuracy.

[0151] This disclosure innovatively employs a two-level semantic recognition architecture combining a multimodal large language model and fine-grained visual recognition. The first semantic recognition utilizes the multimodal large language model to perform macroscopic semantic understanding of architectural images, generating an initial configuration object containing initial facade parameters at the first information granularity. This provides structured spatial positioning guidance and parameter initialization constraints for fine-grained recognition. The second semantic recognition uses the initial facade parameters as prior constraints to perform precise measurement and classification within a defined candidate region, generating a facade configuration array at the second information granularity. This two-level architecture addresses the shortcomings of pure large model solutions (insufficient local accuracy) and pure visual solutions (lack of global constraints), balancing global semantic understanding capabilities with local measurement accuracy.

[0152] (4) Achieve multi-dimensional fine-grained automatic identification of floors, components and column structures.

[0153] This disclosure enables the automatic identification of multi-dimensional semantic information at the floor, component, and column structure levels from architectural images. Floor semantic recognition can automatically determine the total number of floors, estimate the floor height of each floor, and classify roof types; component semantic recognition can automatically identify the location, type, and size parameters of building components such as doors and windows; facade column structure recognition can automatically parse the column division, column status, and component affiliation relationships of the building facade. The identification results are stored in a standardized object array format, achieving accurate conversion of architectural facade semantic information into structured configuration data, providing complete architectural facade parameters for the PCG system.

[0154] (5) A unified structured configuration data representation ensures good readability and interoperability.

[0155] This disclosure defines a unified JSON-formatted structured PCG configuration file specification, containing four top-level fields: floor data, component data, column data, and geometric parameters. It comprehensively covers information such as the floors, components, arrangement rules, and overall dimensions of the building facade. This standardized data representation ensures the configuration file has good readability, scalability, and cross-system interoperability, facilitating consumption by downstream external systems such as 3D assembly modules and visual editing modules.

[0156] (6) Supports multi-view fusion and incremental updates to adapt to diverse application scenarios.

[0157] This disclosure supports a multi-view fusion configuration generation mode, which can aggregate architectural image processing results from multiple views of the same building into a comprehensive PCG configuration file containing information on multiple facades. This is suitable for 3D modeling and digital twin scenarios that require accurate reconstruction of the entire building's appearance. Furthermore, this disclosure also supports an incremental configuration update mode. When a building facade undergoes partial modification, only the configuration fields corresponding to the changed area need to be updated, without reprocessing the entire image, significantly improving the efficiency of configuration updates.

[0158] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0159] Based on the same concept, this disclosure also provides an apparatus for generating PCG profiles based on architectural images. Since the principle by which this apparatus solves the problem is similar to the aforementioned method for generating PCG profiles based on architectural images, the implementation of this apparatus can refer to the implementation of the aforementioned method for generating PCG profiles based on architectural images, and repeated details will not be described again.

[0160] Figure 4 This is a schematic diagram of the device for generating PCG profiles based on architectural images. (See attached diagram) Figure 4 The PCG profile generation device 200 based on architectural images includes: Receiver module 201 is used to receive input building images; The preprocessing module 202 is used to preprocess the building image to obtain a standardized building facade image, and to estimate the length-to-height ratio of the main building in the building image during the preprocessing process. The first semantic recognition module 203 is used to perform first semantic recognition on standardized building facade images using a multimodal large language model to obtain semantic features; The initial configuration object generation module 204 is used to generate an initial configuration object based on semantic features, wherein the initial configuration object includes initial facade parameters at the first information granularity. The second semantic recognition module 205 is used to perform second semantic recognition on the standardized building facade image based on the initial facade parameters to obtain a facade configuration array with a second information granularity, which is finer than the first information granularity. PCG profile generation module 206 is used to aggregate the aspect ratio and facade configuration array to generate the PCG profile corresponding to the building image.

[0161] In an optional implementation, the preprocessing module 202 is configured to: identify the main building in the building image using an image segmentation algorithm, and separate the main building from the background area to obtain a mask image of the main building; calculate the ratio of the pixel span in the vertical direction to the pixel span in the horizontal direction of the main building based on the mask image of the main building to obtain the length-to-height ratio; detect the vanishing point and edge lines of the building facade based on the mask image of the main building; estimate the perspective transformation matrix based on the vanishing point and edge lines; and perform an inverse perspective transformation on the main building area in the building image based on the perspective transformation matrix to eliminate perspective distortion, restore the orthographic projection features of the building facade, and obtain a standardized building facade image.

[0162] In one alternative implementation, the semantic features include at least one of the following: (a) contour features, including the overall contour shape and boundary orientation of the building facade in a standardized building facade image; (b) Texture pattern, including the material texture characteristics of each area of ​​the building facade; (c) Component distribution, including the positional distribution pattern and arrangement of building components in the building facade.

[0163] In one optional implementation, the initial configuration object generation module 204 is used to obtain a predefined configuration template, which defines the field structure of floors, components, and facade column structure in the PCG configuration file; the semantic features are mapped to the corresponding configuration fields according to the predefined configuration template to generate an initial configuration object that conforms to the format requirements of the PCG configuration file; wherein, the initial facade parameters in the initial configuration object include initial values ​​for floor division, component positioning, and facade column structure, and the initial values ​​belong to the first information granularity.

[0164] In one optional implementation, the second semantic recognition module 205 is used to: use each initial value in the initial facade parameters as a priori constraint, perform second semantic recognition on the standardized building facade image within the candidate area defined by each initial value, and obtain a facade configuration array with a second information granularity; wherein, the second semantic recognition includes floor semantic recognition, component semantic recognition and facade column structure recognition, and the facade configuration array includes a floor object array, a component object array and a column object array.

[0165] In one optional implementation, the second semantic recognition module is used to: identify floor boundaries and calculate the total number of floors within a candidate area defined by the initial value of floor division by analyzing the repetitive texture patterns in the horizontal direction of the standardized building facade image; estimate the floor height value of each floor based on the spacing ratio of the floor boundaries and the total number of floors, combined with the prior floor height range of the building type corresponding to the building image; identify the roof type by analyzing the contour shape and texture features of the building top area; and generate an array of floor objects, wherein each floor object contains a floor number, a floor height value, and a roof type, with the floor number increasing sequentially from the bottom to the top. In one optional implementation, the second semantic recognition module is configured to: identify the bounding box of each building component and the coordinate position of each building component in the facade coordinate system within the candidate area defined by the initial value of the component; determine the component type of each building component based on the shape characteristics and contextual relationships of each building component; calculate the width and height of each building component based on the bounding box of each building component and the overall facade scale; and generate an array of component objects, wherein each component object contains a globally unique component identifier, component type, width, height, and coordinate position.

[0166] In one optional implementation, the second semantic recognition module is configured to: divide the building facade into several columns within the candidate areas of each column defined by the initial value of the facade column structure, and determine the boundary position of each column; determine the column state of each column as normal, convex, or concave based on the concave-convex relationship of each column on the building facade; determine the list of component identifiers referenced by each column based on the boundary position of each column and the component distribution of each floor within the column area; and generate an array of column objects, wherein each column object contains a column index, a column state, a list of component identifiers, and an offset, the offset being used to characterize the depth offset value of the column relative to the facade reference plane in the convex or concave state.

[0167] In one optional implementation, the PCG configuration file generation module 206 is used to: calculate the total number of floors and the overall building height based on the floor object array in the facade configuration array; calculate the overall building width based on the component object array and column object array in the facade configuration array; fill the total number of floors, overall building height, overall building width, and length-to-height ratio into geometric parameter objects; and aggregate the geometric parameter objects and the facade configuration array into a structured PCG configuration file in JSON format.

[0168] In one optional implementation, the PCG profile generation module 206 is configured to: when the received building image includes building images from multiple different perspectives of the same building, aggregate the aspect ratios and facade configuration arrays from multiple different perspectives to generate a PCG profile containing multiple facade information.

[0169] In one optional implementation, the update module is configured to receive an input updated building image, wherein the updated building image is an image obtained after the main building in the building image has been modified; based on the updated building image and the building image, determine the changed area of ​​the main building; determine the configuration field corresponding to the changed area in the generated PCG configuration file, wherein the configuration field includes at least one of the affected floor object, component object or column object; and perform local update on the configuration field corresponding to the changed area to generate an updated PCG configuration file.

[0170] It should be noted that the specific data processing methods of each unit in the above-mentioned apparatus 200 for generating PCG configuration files based on architectural images can be referred to the method embodiments described above, and will not be repeated here.

[0171] This disclosure also provides a computing device. Figure 5 This is a structural block diagram of a computing device provided in an embodiment of this disclosure. Figure 5 As shown, the computing device 300 may include a processor 301 and a memory 302; the memory 302 may be coupled to the processor 301. It is worth noting that... Figure 5 This is exemplary; other types of structures can also be used to supplement or replace this structure to achieve telecommunications functions or other functions. Optionally, the computing device 300 can be a server or a local computing device, thereby enabling the execution of a method for generating PCG profiles based on building images, either in the cloud or locally.

[0172] In one possible implementation, the functionality of the apparatus 200 for generating PCG profiles based on architectural images can be integrated into the processor 301. The processor 301 can be configured to execute the embodiments described above.

[0173] In another possible implementation, the apparatus 200 for generating PCG profiles based on architectural images can be configured separately from the processor 301. For example, the apparatus 200 for generating PCG profiles based on architectural images can be configured as a chip connected to the processor 301, and the aforementioned embodiments can be implemented through the control of the processor 301.

[0174] Furthermore, in some alternative implementations, the computing device 300 may also include: a communication module, an input unit, an audio processor, a display, a power supply, etc. It is worth noting that the computing device 300 is not necessarily required to include these components. Figure 5 All components shown; in addition, the computing device 300 may also include Figure 5 For components not shown, please refer to existing technology.

[0175] In some alternative implementations, processor 301, sometimes also referred to as controller or operation control, may include a microprocessor or other processor device and / or logic device, which receives input and controls the operation of various components of computing device 300.

[0176] The memory 302 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store information related to the apparatus 200 for generating PCG profiles based on architectural images, and may also store programs for executing that information. The processor 301 may execute the program stored in the memory 302 to perform information storage or processing, etc.

[0177] An input unit can provide input to the processor 301. This input unit may be, for example, a keypad or touch input device. A power supply can be used to provide power to the computing device 300. A display can be used to display images and text, etc. This display may be, for example, an LCD display, but is not limited to this.

[0178] Memory 302 can be a solid-state memory, such as read-only memory (ROM), random access memory (RAM), SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROM, etc. Memory 302 can also be some other type of device. Memory 302 includes buffer memory (sometimes referred to as a buffer). Memory 302 may include an application / function storage unit for storing application programs and function programs or processes for executing operations of computing device 300 via processor 301.

[0179] The memory 302 may also include a data storage unit for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the computing device. The driver storage unit of the memory 302 may include various drivers for the computer device for communication functions and / or for performing other functions of the computer device (such as messaging applications, address book applications, etc.).

[0180] The communication module is a transmitter / receiver that sends and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor 301 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0181] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various processes of the above-described method embodiment for generating PCG configuration files based on architectural images, and achieves the same technical effects. To avoid repetition, it will not be described again here.

[0182] The aforementioned readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0183] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and system embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0184] While this specification provides method operation steps as shown in the embodiments or flowcharts through one or more embodiments, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual device or client product execution, the method may be executed sequentially as shown in the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0185] In the description of this disclosure, it should be noted that the terms "upper", "lower", "inner", "outer", "front", "back", "left", "right", etc., indicate the orientation or positional relationship based on the orientation or positional relationship in the working state of this disclosure, and are only for the convenience of describing this disclosure and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure.

[0186] In the description of this disclosure, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. Those skilled in the art can understand the specific meaning of these terms in this disclosure based on the specific circumstances.

[0187] The present disclosure has been described above with reference to preferred embodiments; however, these embodiments are merely exemplary and illustrative. Various substitutions and modifications can be made to the present disclosure based on these embodiments, all of which fall within the protection scope of the present disclosure.

Claims

1. A method for generating PCG profiles based on architectural images, characterized in that, include: Receive input building images; The building image is preprocessed to obtain a standardized building facade image, and the length-to-height ratio of the main building in the building image is estimated during the preprocessing process. The standardized building facade image is subjected to first semantic recognition using a multimodal large language model to obtain semantic features; An initial configuration object is generated based on the semantic features, wherein the initial configuration object includes initial facade parameters at a first information granularity; Based on the initial facade parameters, the standardized building facade image is subjected to second semantic recognition to obtain a facade configuration array with a second information granularity, wherein the second information granularity is finer than the first information granularity; The length-to-height ratio and the facade configuration array are aggregated to generate the PCG configuration file corresponding to the building image.

2. The method according to claim 1, characterized in that, The building image is preprocessed to obtain a standardized building facade image, and the length-to-height ratio of the main building in the building image is estimated during the preprocessing process, including: The building body in the building image is identified by an image segmentation algorithm, and the building body is separated from the background area to obtain a building body mask image; Based on the building body mask image, the ratio of the pixel span in the vertical direction to the pixel span in the horizontal direction of the building body is calculated to obtain the length-to-height ratio. Based on the building body mask image, detect the vanishing point and edge line of the building body facade; Estimate the perspective transformation matrix based on the vanishing point and the edge line; Based on the perspective transformation matrix, an inverse perspective transformation is performed on the main building area in the building image to eliminate perspective distortion, restore the orthogonal projection characteristics of the building facade, and obtain a standardized building facade image.

3. The method according to claim 1, characterized in that, The semantic features include at least one of the following: (a) Contour features, including the overall contour shape and boundary orientation of the building facade in the standardized building facade image; (b) Texture pattern, including the material texture features of each area of ​​the building facade; (c) Component distribution, including the location distribution pattern and arrangement of building components in the building facade.

4. The method according to claim 1, characterized in that, Generate an initial configuration object based on the semantic features, including: Obtain a predefined configuration template, which defines the field structure of the floor, component, and facade column structure in the PCG configuration file; The semantic features are mapped to the corresponding configuration fields according to the predefined configuration template to generate an initial configuration object that conforms to the PCG configuration file format requirements; The initial facade parameters in the initial configuration object include initial values ​​for floor division, component positioning, and facade column structure, and the initial values ​​belong to the first information granularity.

5. The method according to claim 4, characterized in that, Based on the initial facade parameters, a second semantic recognition is performed on the standardized building facade image to obtain a facade configuration array with a second information granularity, including: Using each initial value in the initial facade parameters as a priori constraint, the standardized building facade image is subjected to second semantic recognition within the candidate region defined by each initial value to obtain a facade configuration array with a second information granularity. The second semantic recognition includes floor semantic recognition, component semantic recognition, and facade column structure recognition. The facade configuration array includes a floor object array, a component object array, and a column object array.

6. The method according to claim 5, characterized in that, Using the initial values ​​of the initial facade parameters as prior constraints, the standardized building facade image is subjected to second semantic recognition within the candidate regions defined by each initial value, including: Within the candidate area defined by the initial value of the floor division, the floor boundary line is identified and the total number of floors is calculated by analyzing the repeating texture pattern in the horizontal direction of the standardized building facade image. Based on the spacing ratio of the floor boundary lines and the total number of floors, combined with the prior floor height range of the building type corresponding to the building image, the floor height value of each floor is estimated. Roof type can be identified by analyzing the outline shape and texture features of the building's top area; Generate an array of floor objects, where each floor object contains a floor number, a floor height value, and a roof type, with the floor numbers increasing sequentially from the bottom to the top.

7. The method according to claim 5, characterized in that, Using the initial values ​​of the initial facade parameters as prior constraints, the standardized building facade image is subjected to second semantic recognition within the candidate regions defined by each initial value, including: Within the candidate region defined by the initial values ​​of the components, identify the bounding box of each building component and the coordinate position of each building component in the elevation coordinate system; Based on the shape characteristics and contextual relationships of each building component, determine the component type of each building component; Calculate the width and height of each building component based on its boundary frame and overall facade scale. Generate an array of component objects, where each component object contains a globally unique component identifier, component type, width, height, and coordinate position.

8. The method according to claim 5, characterized in that, Using the initial values ​​of the initial facade parameters as prior constraints, the standardized building facade image is subjected to second semantic recognition within the candidate regions defined by each initial value, including: Within the candidate areas of each column defined by the initial values ​​of the facade column structure, the building facade is divided into several columns, and the boundary position of each column is determined; Based on the concave-convex relationship of each column on the building facade, determine the column status of each column as ordinary, convex, or concave; Based on the boundary positions of each column and the distribution of components within the column area on each floor, determine the list of component identifiers referenced in each column; Generate an array of column objects, where each column object contains a column index, column status, a list of component identifiers, and an offset, the offset being used to characterize the depth offset of the column relative to the elevation reference plane in a convex or concave state.

9. The method according to claim 5, characterized in that, The length-to-height ratio and the facade configuration array are aggregated to generate a PCG configuration file, including: Based on the array of floor objects in the facade configuration array, calculate the total number of floors and the overall building height; Calculate the overall width of the building based on the component object array and column object array in the facade configuration array; Fill the total number of floors, the overall height of the building, the overall width of the building, and / or the length-to-height ratio into geometric parameter objects; The geometric parameter object and the facade configuration array are aggregated into a structured PCG configuration file in JSON format.

10. The method according to claim 1 or 9, characterized in that, The length-to-height ratio and the facade configuration array are aggregated to generate a PCG configuration file, including: When the received building images include building images from multiple different perspectives of the same building, the length-to-height ratios and facade configuration arrays from the multiple different perspectives are aggregated to generate a PCG configuration file containing multiple facade information.

11. The method according to claim 1, characterized in that, The method further includes: Receive an input updated building image, wherein the updated building image is an image obtained after modifying the main building in the building image; Based on the updated building image and the building image, determine the changed areas of the main building; Determine the configuration field corresponding to the changed area in the generated PCG configuration file, wherein the configuration field includes at least one of the affected floor object, component object or column object; The configuration fields corresponding to the changed areas are locally updated to generate the updated PCG configuration file.

12. An apparatus for generating PCG profiles based on architectural images, characterized in that, include: The receiving module is used to receive input building images; The preprocessing module is used to preprocess the building image to obtain a standardized building facade image, and to estimate the length-to-height ratio of the main building in the building image during the preprocessing process. The first semantic recognition module is used to perform first semantic recognition on the standardized building facade image using a multimodal large language model to obtain semantic features; An initial configuration object generation module is used to generate an initial configuration object based on the semantic features, wherein the initial configuration object includes initial facade parameters at a first information granularity. The second semantic recognition module is used to perform second semantic recognition on the standardized building facade image based on the initial facade parameters to obtain a facade configuration array with a second information granularity, wherein the second information granularity is finer than the first information granularity. The PCG configuration file generation module is used to aggregate the length-to-height ratio and the facade configuration array to generate the PCG configuration file corresponding to the building image.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 11.

14. A computing device, characterized in that, include: Memory, used to store computer program products; A processor is configured to execute a computer program product stored in the memory, wherein when the computer program product is executed, it implements the method described in any one of claims 1 to 11.

Images