Facade photovoltaic layout generation and power generation potential assessment method based on generative seeding and differentiable optimization

Abstract

The present application belongs to the technical field of BIPV design automation, and relates to a facade photovoltaic layout generation and power generation potential evaluation method based on generative seeding and differentiable optimization. Firstly, a symbolic scene definition is generated by geometric correction and semantic analysis preprocessing on the facade image. Then, a programmatic generative layout seeding is performed based on a large language model. Then, a layout refinement reconstruction based on differentiable optimization is performed. Gradient iterative optimization is performed on the initial layout by using a composite loss function containing physical constraint loss, geometric regularization loss and semantic adhesion loss. Finally, annual total power generation is obtained based on performance simulation and power generation evaluation of a physical engine. The present application effectively overcomes the problems of physical unfeasibility, geometric precision loss and inability to directly guide engineering deployment of existing AI layout schemes, and realizes end-to-end efficient generation of a photovoltaic array arrangement scheme from a single 2D image to a high-geometric-precision layout and deployment-level photovoltaic installation.

Description

Technical Field

[0001] This invention belongs to the field of BIPV design automation technology, and relates to a method for generating and evaluating the power generation potential of facade photovoltaic layouts based on generative seeding and differentiable optimization. Background Technology

[0002] With the increasing demand for a global energy transition towards sustainable energy, solar photovoltaic (PV) has become the cornerstone of renewable energy strategies. Building-integrated photovoltaics (BIPV) technology, by integrating photovoltaic modules into the building envelope (such as roofs, facades, and windows), transforms buildings from passive energy consumers into active energy producers, playing a crucial role in supporting sustainable urban development and green transformation. In high-density urban environments, building facades possess vast and underutilized solar-facing areas. Related research indicates that in densely populated urban areas with high-rise buildings, the installed capacity potential of facade PV can often be several times that of rooftops; neglecting facade assessments will lead to a significant underestimation of the city's total renewable energy capacity.

[0003] Traditional studies assessing the potential of building facade photovoltaics typically rely on 3D building models, digital surface models (DSMs), or LiDAR data from Geographic Information Systems (GIS). However, these methods face significant challenges in practical applications: firstly, acquiring high-precision 3D data is costly and lacks standardized criteria; secondly, traditional assessment methods often simplify the building envelope, neglecting the influence of semantic barriers such as windows, doors, and balconies, leading to a significant overestimation of the actual installable area, with errors typically ranging from 25% to 40%.

[0004] In recent years, advancements in advanced foundational models (such as the Segment Anything Model, SAM) and open-set object detection technologies have made it possible to perform zero-shot semantic parsing and installable region identification using single 2D street view images. Nevertheless, existing deep learning-based semantic awareness methods still heavily rely on large-scale, manually labeled datasets. However, in the BIPV (Building Integrated Vehicle) field, the obstacle features of heterogeneous building facades are extremely complex and lack standardized annotations, resulting in a severe shortage of large-scale, precise datasets suitable for training. Despite these breakthroughs in semantic awareness, a core bottleneck remains in the transition from semantic evaluation to automated layout synthesis.

[0005] Existing automated evaluation processes typically stop at identifying a raw, continuous mask of installable areas, neglecting the fact that photovoltaic systems in engineering practice consist of discrete, standard-sized hardware modules. Optimizing the arrangement of these discrete modules within non-convex polygon constraints is a highly challenging combinatorial optimization problem. Furthermore, while Large Language Models (LLMs) exhibit high-level spatial reasoning capabilities, their directly generated layout proposals often lack the geometric precision required for physical deployment, easily leading to issues such as component collisions, overlaps, or alignment failures with building edges, rendering the generated solutions physically unfeasible. Therefore, there is currently a lack of technical means to deeply integrate high-level semantic reasoning with low-level geometric precision optimization, thereby automatically generating engineering-grade, high-geometric-precision photovoltaic array layout schemes.

[0006] The underlying reasons why existing heuristic algorithms (such as MaxRects) or random search strategies (such as MOGA) produce cluttered and fragmented layouts when dealing with non-convex constrained facades are as follows:

[0007] (1) Limitations of independent decision-making in local greedy search and disconnect from global logic: Algorithms represented by MaxRects adopt a local greedy search model, only searching for the largest empty rectangle in the current step to fill. When there are complex obstacles such as windows and air conditioner units, the algorithm lacks forward-looking analysis of the overall modularity and symmetry of the building. Early-placed components often cut off the global alignment axis required for global alignment, resulting in messy filling in the remaining fragmented space in the later stages, which is the root cause of layout fragmentation.

[0008] (2) The “Search Sparsity” Trap in Discrete Solution Space: Genetic algorithms such as MOGA perform random jump searches at discrete coordinate points. Under non-convex constraints, solutions that satisfy both collision-free and globally equal-gap alignment are extremely sparse and discontinuous in a very large solution space. Due to the lack of mesh adsorption constraints to guide components toward the mesh, random search is almost impossible to achieve sub-pixel-level alignment accuracy in a finite time when dealing with large-scale components, resulting in the final solution exhibiting high entropy and disordered characteristics, which cannot meet the standardized requirements of bracket installation in engineering.

[0009] For example, the paper (Estimating surface utilization factors for BIPV applications using pix2pix on street captured façade images [J]. Journal of Physics: Conference Series, 2023, 2600: 042005.) utilizes a method for identifying usable BIPV surfaces and assessing solar energy potential based on street view facade images. It estimates the facade surface utilization coefficient and annual solar energy potential through pixel-level semantic segmentation and subsequent analysis. However, this paper primarily focuses on the installable areas and potential assessment of the facade itself, lacking an automatic layout synthesis mechanism for discrete standard-sized photovoltaic modules. It still fails to address the key technological gap between "semantic perception" and "engineering-level layout generation."

[0010] Therefore, it is of great significance to study a method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization in order to solve the problems existing in the current technology. Summary of the Invention

[0011] The purpose of this invention is to solve the problems existing in the prior art and to provide a method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentially optimized methods.

[0012] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0013] A method for evaluating the generation and power generation potential of facade photovoltaic layouts based on generative seeding and differentiable optimization is characterized by the following steps:

[0014] (1) Geometric correction and semantic parsing preprocessing;

[0015] First, a homography matrix H is constructed by detecting orthogonal feature lines on the facade. The original 2D street view image (a 2D image containing buildings obtained through the street view map API or taken on-site) is corrected to an orthogonal plane to obtain a corrected image (i.e., the facade image, which is an orthophoto image obtained after geometric correction to eliminate perspective distortion). The physical scale s is calculated by combining prior knowledge of building components (specifically, prior knowledge of building height or window frame dimensions). Then, the open set detection model and the segmentation base model are integrated (integrating the open set detection model and the segmentation base model means using the open set detection model as a region recognition operator and the segmentation base model as a pixel-level mask generator) to identify facade boundaries and obstacle areas (such as windows, doors, air conditioner outdoor units, etc.) and generate symbolic scene definitions. ;

[0016] (2) Programmatic generative layout seeding based on LLM;

[0017] Define the symbolic scene generated in step (1). The data is serialized to text, guiding the LLM to perform spatial logic partitioning and dynamic allocation of multi-specification photovoltaic modules, and outputting the initial layout state in Python code. ;

[0018] (3) Fine-grained layout reconstruction based on differentiable optimization;

[0019] Initial layout state The centroid coordinates of each photovoltaic module are used as continuous optimization variables. A composite loss function is constructed for these continuous optimization variables, which includes physical constraint loss, geometric regularization loss, and semantic adhesion loss. Geometric regularization loss is the alignment loss, and semantic adhesion loss is the anchor point loss. Geometric regularization loss and semantic adhesion loss are functional classifications of loss functions defined in this invention, while alignment loss and anchor point loss are the specific mathematical implementations of these two types of losses, i.e., alignment loss belongs to geometric regularization loss, and anchor point loss belongs to semantic adhesion loss. The gradient of the centroid coordinates of each photovoltaic module is calculated based on the composite loss function, and the gradient descent algorithm is used to iteratively update the centroid coordinates of each photovoltaic module to correct the module pose (i.e., using the composite loss function). To achieve the overall optimization goal, the gradient of this function with respect to the current component's centroid coordinates is calculated. Then, the coordinate variables are guided to iteratively move in the opposite direction of the gradient (correcting the pose) until the loss function converges, generating an optimized layout state with high geometric accuracy. ;

[0020] (4) Performance simulation and capacity assessment based on physics engine;

[0021] The optimized high geometric precision layout state Geographic parameters and meteorological data are input into a known physical modeling and simulation system (such as pvlib) to calculate the effective solar radiation on the surface of each photovoltaic module. The Sandia array performance model is used to estimate the battery operating temperature, and the final AC power output is calculated through a single diode model and a thermal correction model. The calculated effective solar radiation and AC power output on the surface of each photovoltaic module are aggregated over the entire year to obtain the total annual power generation, which is used to assess the energy utilization potential of the facade.

[0022] The above-generated optimized high geometric accuracy layout state The total annual power generation is used to characterize the physical deployability, geometric regularity, and installation feasibility of the component arrangement, while the final total annual power generation is used to characterize the power generation performance and energy utilization potential of the layout scheme under the target geographical and meteorological conditions.

[0023] As a preferred technical solution:

[0024] As described above, in the method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization, in step (1), let the pixel coordinates of any point in the original two-dimensional street view image be... Then the corresponding pixel coordinates in the corrected image The homography matrix .

[0025] As described above, the method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization, in step (1), the specific process of constructing the homography matrix H by detecting the orthogonal feature lines of the facade is as follows: using the line segment detection algorithm to extract four feature edge segments in the facade of the original two-dimensional street view image, the four feature edge segments directly enclose a rectangular area in the actual physical space or are extended to enclose a rectangular area, the four feature edge segments directly enclose a quadrilateral area on the plane of the original two-dimensional street view image or are extended to enclose a quadrilateral area, and the homography matrix is ​​constructed by using the geometric constraint relationship between the vertices of the quadrilateral area and the homography transformation. .

[0026] As described above, in the method for generating and assessing the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization, step (1) integrates the open set detection model and the segmentation base model to identify facade boundaries and obstacle areas and generate symbolic scene definitions. The specific process is as follows: Natural language prompts and corrected images are input into an open set detection model, which outputs a corrected image with a target bounding box. This corrected image with the target bounding box is then input into a segmentation base model, which outputs a corrected image with a pixel-level mask. A polygon fitting algorithm is used to extract the vertex coordinate sequence of the mask contour of the corrected image with the pixel-level mask, and this sequence is encapsulated into a shape containing the facade boundary. Area with obstacles The geometric topological description yields the symbolic scene definition. .

[0027] The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization, as described above, includes the following specific steps in step (2):

[0028] (i) Semantic space partitioning: The complex non-convex installable area is decomposed into several rectangular sub-regions using LLM, and a grid reference consistent with the building symmetry and window arrangement is established; the grid reference is achieved by prompting LLM to identify the alignment features of obstacles (such as the width of the wall between windows) and reserving equal gap reference lines in the generated code;

[0029] (ii) Hybrid component ratio: In view of the geometric constraints of each rectangular sub-region, the photovoltaic module set of large, medium and small specifications is obtained by dynamically combining the LLM to improve the fill rate; This is one of the core links of the LLM chain thinking reasoning. The LLM acts as a global reasoning agent. By analyzing the geometric size constraints of the rectangular sub-region, it autonomously decides the optimal combination strategy of large, medium and small specifications of photovoltaic modules.

[0030] (iii) Programmatic Output: LLM does not directly output images. Instead, it synthesizes Python code to assign the centroid coordinates (x, y) of each independent module unit in the photovoltaic module set (i.e., any specific photovoltaic module instance selected from the large, medium, and small specification libraries) based on the intersection points of the grid reference in (i), thereby achieving the initial layout. The specific implementation method for assigning the centroid coordinates (x, y) of each independent module unit in the photovoltaic module set based on the intersection points of the grid reference is as follows: First, determine the row and column layout reference of the module array according to the grid reference established in step (i); Second, combine the specification dimensions of each independent module unit in the photovoltaic module set to be laid out and the position of the rectangular sub-region where it is located, as determined in step (ii), and determine the centroid coordinates (x, y) of each independent module unit in the local coordinate system of the facade through geometric calculation.

[0031] As described above, in the method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization, the composite loss function in step (3) The mathematical definition of is as follows:

[0032] ;

[0033] in, For collision damage, For boundary loss, For alignment loss, For anchor point loss, , , and These are the weighting coefficients for each type of loss.

[0034] As described above, the method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization includes physical constraint losses such as collision losses and boundary losses.

[0035] Collision loss is defined as the sum of the overlapping areas between components and between a component and an obstacle, and the calculation formula is as follows:

[0036] ;

[0037] in, and Representing the first The and the first A rectangular area of ​​components whose layout needs to be optimized; The centroid coordinates are The The geometric envelope rectangle of a photovoltaic module; This indicates the calculation of the area of ​​the intersection of two geometrically bounded rectangular regions; Indicates the first The geometric region of the obstacle;

[0038] Boundary loss ensures that the component is strictly within the installable facade. The penalty value for component vertices exceeding the facade boundary is calculated using the signed distance field (SDF). The calculation formula is as follows:

[0039] ;

[0040] in, For the first The first photovoltaic module vertex coordinates ; As vertex To the facade boundary The distance.

[0041] The above-described method for assessing the generation and power generation potential of facade photovoltaic layouts based on generative seeding and differentiable optimization, aligns with the loss... By minimizing the horizontal and vertical gap vectors of adjacent components The variance is realized, guiding the array to form a regular mesh structure. The calculation formula is:

[0042] ;

[0043] in, and These represent the horizontal and vertical gap vectors between adjacent components in the array, respectively. This is a variance operator used to measure the gap vector. The degree of fluctuation is used to characterize the irregularity of the array.

[0044] As described above, the method for assessing the generation and power generation potential of facade photovoltaic layouts based on generative seeding and differentiable optimization, anchor point loss Used to limit the optimized position Without deviating from the initial LLM scheme The calculation formula is:

[0045] ;

[0046] in, This refers to the total number of photovoltaic modules installed within the facade. To optimize the real-time centroid coordinates during the process, For the first determined by LLM The initial centroid coordinates of each photovoltaic module.

[0047] Invention principle:

[0048] This invention fundamentally addresses the inherent limitations of LLM in numerical perception and geometric precision by constructing a comprehensive framework that integrates procedural generative layout seeding and continuous geometric optimization based on LLM. Instead of directly improving the numerical output accuracy of the model itself, it employs a paradigm shift approach, decoupling the layout task into two stages: initial layout state generation and refined layout reconstruction based on differentiable optimization. In the first stage, LLM is used as a global inference agent to perform procedural generative layout seeding. Through LLM's chain-like reasoning, semantic features of the building facade are identified and a mesh reference system is established, producing an initial layout state that is globally reasonable but contains physical conflicts. In the second stage, this invention innovatively introduces refined layout reconstruction based on differentiable optimization. The initial layout is used as the starting point for optimization, and gradient descent iterations are performed on a continuous manifold to achieve calibration.

[0049] This invention is not a linear superposition of existing algorithms, but rather achieves a paradigm shift through programmatic generative layout seeding based on LLM and refined layout reconstruction based on differentiable optimization. Specifically:

[0050] (1) Technological Synergy: This invention uses LLM to perform five-step chain thinking (CoT), guiding the model to perform procedural generative layout seeding, playing the role of a senior engineer in strategic planning. LLM pre-performs semantic space partitioning, identifies vertical rectangular sub-regions formed by windows or horizontal strip-shaped regions between floors, and establishes a grid reference system aligned with architectural semantics. By analyzing the physical topological distribution of obstacles, complex non-convex regions are decomposed into a series of rectangular sub-regions, and grid axes consistent with architectural aesthetics are established, thereby ensuring that the generated layout has a reasonable topological structure and global alignment logic in the seeding stage, effectively avoiding layout fragmentation caused by local greedy search. This strategy of guiding local deployment by global topological planning enables the layout to have a reasonable topological structure from the seeding stage, which cannot be spontaneously generated by discrete algorithms based solely on geometric computing power (such as MOGA). LLM is essentially a model that processes text and language probabilities, and it inherently lacks the concept of two-dimensional / three-dimensional physical absolute coordinates. This invention does not simply call LLM, but rather invents a five-step chain thinking engineering framework specifically for photovoltaic layout. This framework cleverly translates complex architectural physical space constraints into textual logic rules that LLM can understand, thereby forcing LLM to think about symmetry and divide rectangular sub-regions like a senior engineer, and finally accurately output layout code with coordinates, solving the problem of multi-specification component layout under non-convex constraints that traditional algorithms find difficult to handle.

[0051] (2) Geometric Regularization Principle: A non-convex constrained facade refers to the installable area in a three-dimensional building facade. Non-convexity mainly stems from semantic obstacles such as windows and doors in the facade, which create spatial constraints with concave geometric features on what was originally a continuous facade. If a building facade is a standard rectangle without any obstacles, it is geometrically a convex set; however, in actual urban BIPV assessment scenarios, due to the need to consider the impact of obstacles on the installation area, most facades exhibit non-convex constrained features in terms of geometric topology. This invention introduces a specific alignment loss. This invention utilizes the gradient of the variance of the gap vector between adjacent components to generate geometric constraints that guide the components to align with a virtual mesh. This mathematical property of using second-order statistics to guide geometric self-alignment forces all scattered components back onto a uniform, equally spaced virtual mesh line. Compared to MOGA's random coordinate jumps, this invention performs pose smoothing correction on a continuous manifold, solving the long-standing technical problem of layout disorder and improving regularity. This output method based on spatial logical partitioning and mesh reference effectively solves the problem of disordered arrangement caused by the sparsity of search in the discrete solution space of random search algorithms, thus producing an initial layout scheme with high regularity and meeting standardized installation requirements.

[0052] The alignment loss designed in this invention is based on the core mathematical calculation of variance. When the variance of a set of data is continuously reduced and approaches 0, it means that every value in this set of data becomes exactly the same. In this invention, when the variance of the gap is optimized to the minimum, the gaps between all photovoltaic panels will inevitably become completely equal, thus achieving equidistant spacing.

[0053] Beneficial effects:

[0054] (1) The present invention provides a method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization. It proposes a complete end-to-end process and successfully transforms the abstract semantic suggestions generated by AI into specific discrete component layouts that can be used for engineering deployment. This solves the problem that existing research only stays at the area evaluation level and cannot directly guide construction, and fills the technical gap from semantic perception to layout synthesis.

[0055] (2) The present invention provides a method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization. By using LLM for "generative seeding", a multi-specification ratio strategy with global rationality is obtained. Then, a differentiable optimization engine is used for sub-pixel level fine adjustment, which effectively solves the physical infeasibility problem of high collision rate (about 9.7%) in the AI ​​direct generation scheme and reduces it to below 0.7%.

[0056] (3) The present invention provides a method for generating and evaluating the power generation potential of a facade photovoltaic layout based on generative seeding and differentially optimized method. The introduced adaptive equidistant regularization can force the array to automatically align and maintain uniform gaps, thereby improving the geometric regularity index by more than 10%. This not only conforms to architectural aesthetics but also reduces the engineering complexity of bracket installation and wiring, significantly improving the geometric regularity and engineering quality of the photovoltaic array.

[0057] (4) The present invention provides a method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization. It relies only on a single 2D street view image and basic semantic data, without the need for expensive LiDAR measurement. The processing time for a single building is only about 80 seconds, which greatly improves the efficiency of large-scale urban-scale BIPV potential assessment and layout planning, and has extremely high engineering universality and efficiency.

[0058] (5) The method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization of the present invention is not a linear superposition of existing algorithms, but a paradigm shift achieved through global semantic seeding and continuous manifold optimization.

[0059] (6) The present invention provides a method for generating and evaluating the power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization. Through a zero-shot learning architecture, the system has a strong generalization ability when dealing with heterogeneous and non-standard building facades. It can produce high-precision semantic parsing results without large-scale training data, thus solving the problem of scarce labeled data in the BIPV field. Attached Figure Description

[0060] Figure 1 This is a flowchart of an automatic generation method for photovoltaic layout of three-dimensional building facades based on generative seeding and differentiable optimization according to the present invention.

[0061] Figure 2 This is the overall network architecture of an embodiment of the present invention;

[0062] Figure 3 This is the Segmentation Basic Model (SAM) network architecture of this invention embodiment;

[0063] Figure 4 The figures show a comparison of the effects of the MaxRects algorithm, the MOGA genetic algorithm, and the method of this invention on actual buildings. In the figures, blue rectangles represent 1.2 m × 1.2 m components, green rectangles represent 1.0 m × 1.0 m components, and cyan rectangles represent 0.8 m × 0.8 m components. Among them, (a), (b), and (c) are the layout results of building facade A using MaxRects, MOGA genetic algorithm, and the method of this invention, respectively; (d), (e), and (f) are the layout results of building facade B using MaxRects, MOGA genetic algorithm, and the method of this invention, respectively; and (g), (h), and (i) are the layout results of building facade C using MaxRects, MOGA genetic algorithm, and the method of this invention, respectively.

[0064] Figure 5 The images shown are elevation views of the two-dimensional street scene before and after correction, respectively. (a) is before correction, and (b) is after correction. Detailed Implementation

[0065] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0066] A method for assessing the generation and power generation potential of facade photovoltaic layouts based on generative seeding and differentiable optimization, such as Figure 1 As shown, the specific steps are as follows:

[0067] (1) Geometric correction and semantic parsing preprocessing;

[0068] (1.1) Construct the homography matrix H by detecting orthogonal feature lines of the facade;

[0069] A line segment detection algorithm is used to extract four feature edge segments from the facade of the original 2D street view image. These four feature edge segments directly enclose a rectangular region in the actual physical space, or enclose a rectangular region after extension. They also directly enclose a quadrilateral region on the plane of the original 2D street view image, or enclose a quadrilateral region after extension. A homography matrix is ​​constructed using the geometric constraints between the vertices of the quadrilateral regions and the homography transformation. ;

[0070] (1.2) Correct the original two-dimensional street scene image to an orthogonal plane to obtain the corrected image and calculate the physical scale s (meters / pixels) by combining the prior knowledge of building components, and establish the mapping relationship from pixel space to physical measurement space;

[0071] Let the pixel coordinates of any point in the original 2D street view image be... Then the corresponding pixel coordinates in the corrected image The homography matrix ;

[0072] (1.3) At the semantic parsing level, the Grounding-DINO open set detection model and the Segmentation Base Model (SAM) are integrated to identify facade boundaries and obstacle regions and generate symbolic scene definitions. ;

[0073] Natural language prompts and corrected images are input into an open set detection model, which outputs a corrected image with a target bounding box. This corrected image with the target bounding box is then input into a segmentation base model, which outputs a corrected image with a pixel-level mask. A polygon fitting algorithm is used to extract the vertex coordinate sequence of the mask contour of the corrected image with the pixel-level mask, and this sequence is encapsulated into a structure containing the facade boundary. Area with obstacles The geometric topological description yields the symbolic scene definition. ;

[0074] This step employs a zero-shot segmentation strategy, aiming to leverage the visual features learned by the segmentation base model SAM on large-scale general datasets such as SA-1B to achieve high generalization perception of unknown and complex facade semantics without requiring domain-specific data annotation and secondary training for BIPV scenarios.

[0075] like Figure 3The diagram shows the network architecture of the segmentation basic model, SAM. Its overall process mainly includes four parts: image feature extraction, cue information encoding, segmentation prediction, and result output. First, the corrected image undergoes an image encoder to extract multi-scale visual features. During encoding, residual connections enhance feature transfer, and deep convolution improves the representation of local features. Simultaneously, the target bounding box is fed into the cue encoder and mask decoder for encoding. Then, the two feature streams are fed into the segmentation head for fusion and prediction. After the segmentation head, the model generates a segmentation mask through convolution and outputs the corresponding confidence score, ultimately synthesizing to obtain a corrected image with a pixel-level mask.

[0076] Generated symbolic scene definition It provides strict geometric constraints for subsequent layout synthesis, effectively avoiding the problem of overestimation of installation area caused by ignoring semantic obstacles in traditional methods.

[0077] (2) Programmatic generative layout seeding based on LLM (e.g., gpt4o);

[0078] Define the symbolic scene generated in step (1). The data is serialized to text, guiding the LLM to perform spatial logic partitioning and dynamic allocation of multi-specification photovoltaic modules, and outputting the initial layout state in Python code. The specific steps are as follows:

[0079] (2.1) In-depth analysis of multi-dimensional information on the facade;

[0080] LLM first defines the symbolic scene output from step (1). Perform a global scan, which includes physical scale calculation, obstacle topology recognition, and coordinate system alignment;

[0081] ① Physical scale calculation: Analyze the overall pixel size of the facade and convert it into a physical measurement space (e.g., 20 m wide, 15 m high) in combination with the physical scale s.

[0082] ② Obstacle topology recognition: Accurately identifies the relative position and geometric dimensions of all prohibited areas such as windows, doors, and balconies;

[0083] ③ Coordinate system alignment: Establish a coordinate system with the upper left corner of the elevation as the origin. A two-dimensional Cartesian coordinate system with the X-axis pointing to the right and the Y-axis pointing downwards serves as the sole reference system for all subsequent components.

[0084] (2.2) Semantic space partitioning: In order to reduce the solution complexity of non-convex regions, LLM defines the symbolic scene output in step (1). Perform semantic space partitioning:

[0085] ① Rectangular decomposition: Based on the physical distribution of obstacles, LLM is used to decompose the complex non-convex installable area into several rectangular sub-regions;

[0086] ②Architectural semantic alignment: Unlike simple geometric cutting, the model analyzes the symmetry of the building (e.g., vertical columnar areas between windows or horizontal strips between floors) and establishes a grid reference consistent with the building's symmetry and window arrangement to ensure that the generated layout has visual harmony.

[0087] (2.3) Hybrid module ratio: In view of the geometric constraints of each rectangular sub-region, the photovoltaic module set is obtained by dynamically combining large, medium and small photovoltaic modules using LLM to improve the fill rate;

[0088] Large-sized components (e.g., 1.2 m × 1.2 m): Evaluate their coverage advantages in open areas (e.g., at the top skyline or large blank facades);

[0089] Small / medium-sized components (e.g., 1.0 m × 1.0 m or 0.8 m × 0.8 m): Analyze their geometric flexibility in handling narrow edges, window gaps, and non-standard corners;

[0090] For each defined sub-region, LLM develops a differentiated "hybrid layout strategy";

[0091] Priority assignment: Establish a filling logic of large to small; prioritize the arraying of large-size components in large areas;

[0092] Gap compensation mechanism: In narrow gaps that are insufficient to accommodate large-sized components, smaller-sized components are automatically matched to fill the gaps, thereby maximizing area utilization (AUR). This process simulates the manual selection logic of a senior BIPV engineer, effectively overcoming the resource waste caused by traditional single-specification layouts.

[0093] (2.4) Programmatic Output: LLM does not directly output images, but instead synthesizes Python code to assign the centroid coordinates (x, y) of each independent module unit in the photovoltaic module set based on the intersection of the grid reference in (2.2), thereby achieving the initial layout. The specific implementation logic of this process is as follows:

[0094] ① Coordinate Assignment: Based on the logical division of the elevation space in the aforementioned steps, LLM utilizes its global spatial reasoning capabilities to directly calculate the precise centroid physical coordinates of each independent module unit in the photovoltaic module assembly (i.e., any specific module instance selected from the 1.2 m, 1.0 m, and 0.8 m specification libraries) in a two-dimensional Cartesian coordinate system. ;

[0095] ② Code encapsulation: LLM writes the centroid coordinate parameters and specification type ID of each independent component unit into the data structure of the Python script, and forces the model to avoid rigid loop traversal algorithms;

[0096] ③ Layout Instantiation: The generated Python script is called and executed by the backend environment. Based on the preset coordinate sequence and specification parameters in the code, pose assignment and spatial object instantiation are performed on each independent component unit on the virtual plane, thereby outputting the initial layout state. .

[0097] Although the layout generated at this stage has extremely high semantic rationality and preliminary engineering feasibility, due to the inherent error of LLM in perceiving continuous spatial numerical values, the initial scheme usually has a local physical conflict rate of about 9.7% (e.g., very small overlap or deviation), which provides a key initial layout state for the subsequent step (3) differentiable optimization.

[0098] (3) Fine-grained layout reconstruction based on differentiable optimization;

[0099] Initial layout state The centroid coordinates of each photovoltaic module are used as continuous optimization variables to construct a composite loss function that includes physical constraint loss, geometric regularization loss, and semantic adhesion loss. The gradient of the centroid coordinates of each photovoltaic module is calculated based on the composite loss function. Then, the gradient descent algorithm (the gradient value is the gradient vector obtained by taking the partial derivative of the composite loss function with respect to the centroid coordinates of each photovoltaic module, which changes dynamically with the current layout state and the iteration process) is used to iteratively update the centroid coordinates of each photovoltaic module to correct the module pose and generate an optimized layout state with high geometric accuracy. ;

[0100] Composite loss function The mathematical definition of is as follows:

[0101] ;

[0102] in, For collision damage, For boundary loss, For alignment loss, For anchor point loss, , , and These are the weighting coefficients for each type of loss;

[0103] The constraint loss includes collision loss and boundary loss;

[0104] Collision loss is defined as the sum of the overlapping areas between components and between a component and an obstacle, and the calculation formula is as follows:

[0105] ;

[0106] in, and They represent the first The and the first Real-time centroid coordinates of each photovoltaic module during the optimization process and Representing the first The and the first A rectangular area of ​​components whose layout needs to be optimized. The centroid coordinates are The The geometric envelope rectangle of a photovoltaic module; This indicates the calculation of the area of ​​the intersection of two geometrically bounded rectangular regions; Indicates the first The geometric region of the obstacle;

[0107] Boundary loss is calculated using the signed distance field (SDF) to determine the penalty for component vertices exceeding the facade boundary. The calculation formula is as follows:

[0108] ;

[0109] in, For the first The first photovoltaic module vertex coordinates ; As vertex To the facade boundary The distance;

[0110] Geometric regularization loss is the alignment loss, and semantic adhesion loss is the anchor point loss.

[0111] Alignment loss By minimizing the horizontal and vertical gap vectors of adjacent components The variance is realized, and the calculation formula is:

[0112] ;

[0113] in, and These represent the horizontal and vertical gap vectors between adjacent components in the array, respectively. This is a variance operator used to measure the gap vector. The degree of fluctuation is used to characterize the irregularity of the array;

[0114] Anchor point loss Used to limit the optimized position Without deviating from the initial LLM scheme The calculation formula is:

[0115] ;

[0116] in, This refers to the total number of photovoltaic modules installed within the facade. To optimize the real-time centroid coordinates during the process, For the first determined by LLM The initial centroid coordinates of each photovoltaic module.

[0117] (4) Performance simulation and capacity assessment based on physics engine;

[0118] The optimized high geometric precision layout state Geographic parameters and meteorological data are input into a physical modeling and simulation system (such as pvlib) to calculate the effective solar radiation on the surface of each photovoltaic module, and the final AC power output is calculated through a single diode model and a thermal correction model. The calculated effective solar radiation and AC power output on the surface of each photovoltaic module are aggregated over the entire year to obtain the total annual power generation, which is used to assess the energy utilization potential of the facade.

[0119] The following specific embodiments illustrate the method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization of the present invention, using the geographical coordinates of Tianjin area and the latest typical meteorological year (TMY) in 2025 as an example, as follows:

[0120] (1) such as Figure 2 and Figure 5 As shown, the original 2D street view image, after geometric correction and semantic parsing preprocessing, yields the parsed orthogonal facade (corrected image). The corrected facade has a width of 18 m, a height of 14 m, and a total area of ​​252 m². 2 The physical scale s is 0.02 meters per pixel. Obstacles on the facade include 6 windows and 1 door. The facade boundaries and the aforementioned obstacle regions are identified using an open set detection model and a segmentation baseline model. Vertex coordinate sequences of the obstacle outlines are extracted to generate a symbolic scene definition containing the facade boundary set and the obstacle regions.

[0121] (2) In the procedural generative layout seeding stage, the symbolic scene definition is serialized and filled into our language instructions, and then used as text input to LLM. The input content includes the physical dimensions of the facade, obstacle location and size information, installable area information, and photovoltaic module specification library; among which, the PV panel candidate set includes three specifications: 1.2 m × 1.2 m, 1.0 m × 1.0 m, and 0.8 m × 0.8 m. The LLM output content includes rectangular sub-regions, grid references, the set of photovoltaic modules to be deployed, and the initial centroid coordinates of each independent module unit.

[0122] In this embodiment, the initial layout includes 180 photovoltaic modules, comprising 92 1.2 m × 1.2 m modules, 36 1.0 m × 1.0 m modules, and 52 0.8 m × 0.8 m modules, with a total installation area of ​​201.76 m². 2 The initial area utilization rate (initial area utilization rate (total installation area - area of ​​obstacles such as doors and windows) / total installation area) is 87.20%.

[0123] (3) In the layout refinement and reconstruction stage, the centroid coordinates of the photovoltaic modules are used as continuous optimization variables. The initial layout state, facade boundaries, obstacle geometry, and composite loss function parameters are input to construct a composite loss function including collision loss, boundary loss, alignment loss, and anchor point loss. The Adam optimizer is then used for iterative optimization. The optimization parameters are set as follows: learning rate 0.01, maximum iteration count 500, and convergence threshold of less than 1×10⁻⁶ for the total loss change between two adjacent iterations. -4 The weighting coefficients for collision loss, boundary loss, alignment loss, and anchor point loss are 1.0, 2.0, 0.5, and 0.2, respectively. The output of this stage is the optimized, high-geometric-precision layout state, including the final specification type, final centroid coordinates, and final arrangement position of each photovoltaic module.

[0124] (4) In the power generation potential assessment stage, the optimized high-geometric-precision layout, target area geographic parameters, and corresponding TMY meteorological data are input into the pvlib physical modeling simulation system. The facade orientation is set to due south, the tilt angle is 90°, and the surface albedo is 0.2. The Sandia array performance model is used to estimate the battery operating temperature, and the final AC power output is calculated using a single diode model and a thermal correction model. The output of this stage includes the effective solar radiation on the surface of each photovoltaic module, the AC power output at each moment, and the total annual power generation obtained by aggregating throughout the year. In this embodiment, the total annual power generation is 28,430 kWh.

[0125] Collision rate The test is based on geometric interference analysis, and the evaluation criterion is defined as the ratio of the conflict area (overlapping area) to the total installation area. The calculation formula is as follows:

[0126] ;

[0127] Represents all components and The overlapping area between them;

[0128] Represents all components facade obstacles The overlapping area between (such as windows and doors);

[0129] Total area of ​​all planned installation components.

[0130] After iterative optimization, the layout collision rate was significantly reduced from 9.7% to 0.69%.

[0131] Geometric regularity is a commonly used evaluation metric in architectural computational design and computational geometry, used to measure the degree of deviation of an array of objects from an ideally aligned mesh. Before optimization, the geometric regularity metric was 0.85. After iterative optimization, the geometric regularity metric improved to 0.94, representing an improvement of 10.6%.

[0132] Results and Validation: This invention underwent large-scale validation on a heterogeneous dataset of 80 buildings, including low-rise, mid-rise, and high-rise structures. The results show that the average area utilization rate (AUR) of the layout generated by this invention reaches 86.5%, outperforming traditional MaxRects (77.4%) and MOGA genetic algorithm (44.1%). The end-to-end processing time for a single building averages only 79.52 seconds, significantly improving the efficiency of urban-scale BIPV potential assessment, and the generated layout has practical value for directly guiding engineering installation.

[0133] Figure 4 The image shows a comparison of the effects of the MaxRects algorithm, the MOGA genetic algorithm, and the method of this invention on actual buildings (building facade A, building facade B, and building facade C). For building facade A: Figure 4 The AUR values ​​for (a) to (c) are 84.9%, 51.3%, and 92.1%, respectively. For building facade B: Figure 4 The AUR values ​​for (d) to (f) are 68.4%, 41.2%, and 89.4%, respectively; for building facade C: Figure 4The AUR values ​​for (g) to (i) are 71.8%, 27.7%, and 76.7%, respectively. The MaxRects algorithm, due to its local sequential filling logic, only fills empty spaces in the current step. When facing complex obstacles such as windows, it lacks a global analysis of the overall building module and symmetry, leading to severe fragmentation in the later stages of the layout. The MOGA genetic algorithm performs random jump searches at discrete coordinate points. Due to the lack of constraints guiding components towards the grid, the generated schemes exhibit obvious disorder, making it difficult to meet standardized installation requirements. In contrast, this invention performs global policy planning through LLM, identifying vertical regions or horizontal strips of the building and pre-establishing a grid reference system aligned with the building's semantics. Subsequently, in the process of refined layout reconstruction based on differentiable optimization, the alignment loss in the composite loss function is used to smoothly correct the component poses, guiding them to converge to a unified virtual grid line.

Claims

1. A method for assessing the generation and power generation potential of facade photovoltaic layouts based on generative seeding and differentiable optimization, characterized in that... Includes the following steps: (1) Geometric correction and semantic parsing preprocessing; First, a homography matrix H is constructed by detecting orthogonal feature lines on the facade. The original two-dimensional street scene image is then corrected to an orthogonal plane to obtain a corrected image. The physical scale s is then calculated by combining prior knowledge of building components. Then, the open set detection model and the segmentation base model are integrated to identify facade boundaries and obstacle regions and generate symbolic scene definitions. ; (2) Procedural generative layout seeding based on large language models; Define the symbolic scene generated in step (1). The data is serialized to text, guiding a large language model to perform spatial logic partitioning and dynamic allocation of multi-specification photovoltaic modules, and outputting the initial layout state in Python code. ; (3) Fine-grained layout reconstruction based on differentiable optimization; Initial layout state The centroid coordinates of each photovoltaic module are used as continuous optimization variables to construct a composite loss function that includes physical constraint loss, geometric regularization loss, and semantic adhesion loss. Geometric regularization loss is used as alignment loss, and semantic adhesion loss is used as anchor point loss. The gradient of the centroid coordinates of each photovoltaic module is calculated based on the composite loss function, and the gradient descent algorithm is used to iteratively update the centroid coordinates of each photovoltaic module to correct the module pose and generate an optimized layout state with high geometric accuracy. ; (4) Performance simulation and capacity assessment based on physics engine; The optimized high geometric precision layout state Geographic parameters and meteorological data are input into the physical modeling and simulation system to calculate the effective solar radiation on the surface of each photovoltaic module, and the final AC power output is calculated through a single diode model and a thermal correction model. The calculated effective solar radiation and AC power output on the surface of each photovoltaic module are aggregated over the entire year to obtain the total annual power generation, which is used to assess the energy utilization potential of the facade.

2. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization as described in claim 1, characterized in that, In step (1), let the pixel coordinates of any point in the original two-dimensional street view image be... Then the corresponding pixel coordinates in the corrected image The homography matrix .

3. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 2, characterized in that, In step (1), the specific process of constructing the homography matrix H by detecting the orthogonal feature lines of the facade is as follows: Four feature edge segments are extracted from the facade of the original two-dimensional street view image using a line segment detection algorithm. These four feature edge segments directly enclose a rectangular region in the actual physical space, or are extended to enclose a rectangular region. They also directly enclose a quadrilateral region on the plane of the original two-dimensional street view image, or are extended to enclose a quadrilateral region. The homography matrix is ​​then constructed using the geometric constraint relationship between the vertices of the quadrilateral region and the homography transformation. .

4. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 3, characterized in that, In step (1), the open set detection model and the segmentation base model are integrated to identify facade boundaries and obstacle areas and generate symbolic scene definitions. The specific process is as follows: Natural language prompts and corrected images are input into an open set detection model, which outputs a corrected image with a target bounding box. This corrected image with the target bounding box is then input into a segmentation base model, which outputs a corrected image with a pixel-level mask. A polygon fitting algorithm is used to extract the vertex coordinate sequence of the mask contour of the corrected image with the pixel-level mask, and this sequence is encapsulated into a shape containing the facade boundary. Area with obstacles The geometric topological description yields the symbolic scene definition. .

5. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 4, characterized in that, Step (2) specifically includes the following steps: (i) Semantic space partitioning: The complex non-convex installable area is decomposed into several rectangular sub-regions using a large language model, and a grid reference consistent with the building symmetry and window arrangement is established. (ii) Hybrid component ratio: In view of the geometric constraints of each rectangular sub-region, the photovoltaic modules of large, medium and small specifications are dynamically combined using a large language model to obtain a photovoltaic module set in order to improve the fill rate; (iii) Programmatic output: The large language model does not directly output images, but instead synthesizes Python code to assign the centroid coordinates (x,y) of each independent component unit in the photovoltaic module set based on the intersection of the grid reference in (i), thereby achieving the initial layout.

6. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 5, characterized in that, The composite loss function in step (3) The mathematical definition of is as follows: ； in, For collision damage, For boundary loss, For alignment loss, For anchor point loss, , , and These are the weighting coefficients for each type of loss.

7. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 6, characterized in that, Physical constraint losses include collision losses and boundary losses; Collision loss is defined as the sum of the overlapping areas between components and between a component and an obstacle, and the calculation formula is as follows: ； in, The centroid coordinates are The The geometric envelope rectangle of a photovoltaic module; This indicates the calculation of the area of ​​the intersection of two geometrically bounded rectangular regions; Indicates the first The geometric region of the obstacle; The boundary loss uses the signed distance field to calculate the penalty value for component vertices exceeding the facade boundary. The calculation formula is as follows: ； in, For the first The first photovoltaic module vertex coordinates ; As vertex To the facade boundary The distance.

8. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 7, characterized in that, Alignment loss By minimizing the horizontal and vertical gap vectors of adjacent components The variance is realized, and the calculation formula is: ； in, and These represent the horizontal and vertical gap vectors between adjacent components in the array, respectively.

9. The method for evaluating the generation and power generation potential of facade photovoltaic layout based on generative seeding and differentiable optimization according to claim 8, characterized in that, Anchor point loss Used to limit the optimized position Without deviating from the initial scheme of the large language model The calculation formula is: ； in, This refers to the total number of photovoltaic modules installed within the facade. The first one determined by the large language model The initial centroid coordinates of each photovoltaic module.

Images