Building facade photovoltaic module arrangement method and system based on image and point cloud

Abstract

The invention provides a building facade photovoltaic module arrangement method and system based on images and point clouds. The method comprises the following steps: firstly, identifying targets, such as windows, doors, air conditioner outdoor units, ventilation openings, electric boxes and the like, which cannot be mounted on a building facade by using Mamba-Yolo; then, internal and external parameter calibration between a camera and a laser radar (LiDAR) is utilized to project bounding box pixel coordinates to a three-dimensional point cloud so as to obtain the real position and size of each obstacle in a laser radar coordinate system. And finally, calculating the characteristics of the building facade based on a point cloud processing algorithm, and realizing the arrangement of the photovoltaic modules based on an image processing algorithm. According to the method, the installable area of the photovoltaic module in the target area can be accurately evaluated in a short time, technical support is provided for reasonably formulating a distributed photovoltaic power generation plan, and the construction of a renewable energy source city is accelerated.

Description

Technical Field

[0001] This invention discloses a method and system for arranging photovoltaic modules on building facades based on images and point clouds, belonging to the field of photovoltaic artificial intelligence technology. Background Technology

[0002] With the increasing demand for renewable energy and green buildings, consumer demand for environmentally friendly and energy-efficient building products is also rising. Facade photovoltaics is a perfect combination of building and renewable energy production technology. Covering the front of a building with photovoltaic panels means having a larger collection surface than the roof surface, thus enabling the production of more renewable energy.

[0003] The layout and resource assessment of photovoltaic systems typically require consideration of facade feature parameters such as slope and available area. Due to the influence of various factors including building appearance, window arrangement, structural load-bearing capacity, and electrical wiring, traditional manual measurement and scheme design suffer from high costs, long processing times, and susceptibility to errors. In the area of ​​visual automated recognition of building facade elements (such as windows, doors, air conditioning units, and decorative components), current approaches primarily focus on pixel-level semantic segmentation and object detection. Pixel-level segmentation provides accurate boundary masks, suitable for scenarios requiring precise contours (e.g., accurate calculation of installable area, complex window frame shapes). Recent work has focused on high-quality facade segmentation algorithms and dataset construction, proposing joint segmentation or multi-task learning methods combining 2D / 3D data to improve accuracy. While this segmentation method offers precise boundaries and facilitates direct calculation, its pixel-level annotation costs are high, model inference and training overhead are significant, and its generalization ability is limited when rapid on-site deployment is needed and annotation is insufficient. Furthermore, for engineering design applications, the scale of annotation and output needs to be mapped to meter-level coordinates and combined with point cloud data to advance subsequent design. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a method and system for arranging photovoltaic (PV) modules on building facades based on images and point clouds. This method can efficiently and accurately detect obstacles on building facades and calculate the actual area of ​​the facade suitable for PV panel installation. It requires minimal computation and has strong universality. Specifically, the method first uses Mamba-Yolo to identify uninstallable targets on the building facade, such as windows, doors, air conditioner units, vents, and electrical boxes. Then, using intrinsic and extrinsic parameter calibration between the camera and LiDAR, the bounding box pixel coordinates are projected onto a 3D point cloud to obtain the true position and size of each obstacle in the LiDAR coordinate system. Finally, the features of the building facade are calculated based on point cloud processing algorithms, and the PV modules are arranged based on image processing algorithms. Experiments show that this invention can accurately assess the installable area of ​​PV modules within a target region in a short time, providing technical support for the rational formulation of distributed photovoltaic power generation plans and accelerating the construction of renewable energy cities.

[0005] The present invention adopts the following technical solution: A method for arranging photovoltaic modules on building facades based on images and point clouds includes the following steps: (1) A robot equipped with a lidar and an RGB camera acquires color images of the target building wall and point clouds synchronized with it in time, and uses the color image data as a building facade image dataset; (2) Input the dataset into the object detection network to identify obstacles on the wall. The object detection network outputs the bounding box and category confidence of each identified obstacle. (3) Based on the intrinsic parameters of the RGB image and the extrinsic parameters of the camera relative to the lidar, the bounding box in step (2) is matched with the point cloud in step (1), the bounding box is projected and transformed to the lidar coordinate system to obtain the obstacle point cloud falling within the bounding box, that is, the three-dimensional bounding representation of the identified object. (4) Calculate the obstacle point cloud based on image processing and point cloud processing algorithms to obtain the characteristic parameters of the building wall, such as slope, usable area and arrangement of distributed photovoltaic modules; The method and system for arranging photovoltaic modules on building facades based on images and point clouds are characterized in that the target detection network adopts the Mamba-Yolo image target detection network.

[0006] The method and system for arranging photovoltaic modules on building facades based on images and point clouds are characterized in that the step of projecting and transforming the bounding box to the lidar coordinate system to obtain the obstacle point cloud falling within the bounding box and the three-dimensional bounding representation of the identified object includes: filtering point cloud points projected onto the image plane and falling within the bounding box, and filtering out point clouds within the bounding box and stray point clouds to obtain the effective area point cloud of the building wall.

[0007] The method and system for arranging photovoltaic modules on building facades based on images and point clouds are characterized in that the image processing and point cloud processing algorithms are used to calculate the obstacle point cloud to obtain characteristic parameters of the building wall, such as slope and usable area, including: The Random Sampling Consensus (RANSAC) algorithm is used to fit the plane of the building wall and solve the parameters of the plane equation. Then, the slope parameters of the building wall are calculated based on the angle between the normal vector of the plane and the normal vector of the XOY plane. The Random Sampling Consensus (RANSAC) algorithm is used to fit the point cloud of each selected obstacle to a plane. The corresponding point cloud is then projected onto the plane fitted by RANSAC. The Alpha Shape algorithm is used to extract the boundary points of the plane. The disordered boundary points of the obtained patches are sorted using the four-quadrant tangent sorting method. The boundary points are then projected onto the pixel map and filled. The minimum bounding rectangle of the region is calculated.

[0008] The maximum bounding rectangle obtained from the projection of each obstacle is buffered and expanded to meet construction and safety distance requirements, and all buffered rectangles are merged to generate the total uninstallable area.

[0009] A preliminary installable area is obtained by performing a set difference operation between the wall surface region and the uninstallable area. Connected components are extracted and their area, perimeter, and minimum bounding rectangle are calculated. Areas with a maximum bounding rectangle area less than 1m2 are filtered out.

[0010] Each region is rasterized to obtain a binary matrix (which can be placed = 1). The maximum axial rectangle is calculated on the binary matrix using the maximum rectangle algorithm, and the distributed photovoltaic module arrangement is designed based on it.

[0011] The method and system for arranging photovoltaic modules on building facades based on images and point clouds are characterized in that the image processing and point cloud processing algorithms are used to calculate the installable area to obtain characteristic parameters of the building walls, such as roof slope, usable area, and the arrangement of distributed photovoltaic modules, including: The Random Sampling Consensus (RANSAC) algorithm is used to fit the plane of the building wall and solve the parameters of the plane equation. Then, the slope parameters of the building wall are calculated based on the angle between the normal vector of the plane and the normal vector of the XOY plane. The installable area is represented as a set of connected polygons. The area of ​​the maximum bounding rectangle of each polygon is calculated, and areas with a maximum bounding rectangle less than 1m are filtered out. 2 The region is defined, and the arrangement of distributed photovoltaic modules is designed accordingly.

[0012] The present invention also provides a system for performing the above-described method, comprising: The data acquisition module is used to acquire color images of the target building's walls and point clouds synchronized with them in time by a robot equipped with a LiDAR and an RGB camera; The obstacle detection module is used to run a Mamba-Yolo network to obtain the obstacle outlines of building walls; The projection module is used to obtain the obstacle point cloud of the building wall using the intrinsic and extrinsic parameters of the radar camera; The parameter calculation module is used to calculate the characteristic parameters of building walls and the arrangement of photovoltaic modules; Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This technical solution proposes a method and system for arranging photovoltaic modules on building facades based on images and point clouds, which can replace manual calculation and installation, reduce safety hazards, and greatly reduce workload.

[0013] (2) This technical solution is based on the Mamba-Yolo network for detecting obstacles on building facades. This network model has good robustness and high efficiency.

[0014] This technical solution, through explicit camera-sensor calibration and pixel-depth joint projection, can accurately map visual inspection results to meter-level coordinates, meeting the dimensional accuracy requirements of engineering design. Attached Figure Description

[0015] Figure 1 This is a basic flowchart of a method and system for arranging photovoltaic modules on building facades based on images; Figure 2 This is a schematic diagram of the Mamba-Yolo network model structure. Detailed Implementation

[0016] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention. Example

[0017] A method for arranging photovoltaic modules on building facades based on images and point clouds, such as Figure 1 As shown.

[0018] Step 1: City color image and point cloud data acquisition and preprocessing A robot equipped with a LiDAR and an RGB camera acquires color images of the target building's walls and point clouds synchronized with them over time. The color image data is used as a building facade image dataset, labeled, and then used for subsequent image target detection.

[0019] Step 2: Building facade image detection Based on Mamba-Yolo, obstacles such as building walls in the projected image are extracted. After obtaining the bounding boxes of the obstacles, the bounding boxes are projected and transformed to the LiDAR coordinate system to obtain the point cloud of the obstacles falling within the bounding boxes, as well as the three-dimensional bounding representation of the identified object.

[0020] Step 3: Calculation of non-obstacle point cloud parameters for building walls Within the bounding box, select point cloud points that are projected onto the image plane and fall within the bounding box, and filter out point clouds within the bounding box and stray point clouds to obtain the effective area point cloud of the building wall.

[0021] The Random Sampling Consensus (RANSAC) algorithm is used to fit the plane of the building wall and solve the parameters of the plane equation. Then, the slope parameters of the building wall are calculated based on the angle between the normal vector of the plane and the normal vector of the XOY plane. Step 4: Calculation of point cloud parameters for building wall obstacles The Random Sampling Consensus (RANSAC) algorithm is used to fit the point cloud of each selected obstacle to a plane. The corresponding point cloud is then projected onto the plane fitted by RANSAC. The Alpha Shape algorithm is used to extract the boundary points of the plane. The disordered boundary points of the obtained patches are sorted using the four-quadrant tangent sorting method. The boundary points are then projected onto the pixel map and filled. The minimum bounding rectangle of the region is calculated.

[0022] The maximum bounding rectangle obtained from the projection of each obstacle is buffered and expanded to meet construction and safety distance requirements, and all buffered rectangles are merged to generate the total uninstallable area.

[0023] The preliminary installable area is obtained by performing a set difference operation between the wall surface region and the uninstallable area. Connected components are extracted and their area, perimeter, and minimum bounding rectangle are calculated. The maximum bounding rectangle area is filtered out to be less than 1m². 2 The area.

[0024] Each region is rasterized to obtain a binary matrix (which can be placed = 1). The maximum axial rectangle is calculated on the binary matrix using the maximum rectangle algorithm, and the distributed photovoltaic module arrangement is designed based on it.

[0025] Through the above method, the present invention provides a method and system for arranging photovoltaic modules on building facades based on images and point clouds. It can efficiently and accurately detect obstacles on building facades and calculate the actual area of ​​the building facade that can be used to arrange photovoltaic panels. The calculation is small, which provides technical support for the rational formulation of distributed photovoltaic power generation plans and accelerates the construction of renewable energy cities.

[0026] Those skilled in the art should understand that the above embodiments are merely illustrative of the technical solutions of the present invention and not limitations. Several improvements and modifications can be made without departing from the spirit and scope of the present invention, and these improvements and modifications should also be considered within the protection scope of the present invention.

[0027] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0028] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0029] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0030] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0031] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for arranging photovoltaic modules on building facades based on images and point clouds, characterized in that, Includes the following steps: (1) A robot equipped with a lidar and an RGB camera acquires color images of the target building wall and point clouds synchronized with it in time, and uses the color image data as a building facade image dataset; (2) Input the dataset into the object detection network to identify obstacles on the wall. The object detection network outputs the bounding box and category confidence of each identified obstacle. (3) Based on the intrinsic parameters of the RGB image and the extrinsic parameters of the camera relative to the lidar, the bounding box in step (2) is matched with the point cloud in step (1), the bounding box is projected and transformed to the lidar coordinate system to obtain the obstacle point cloud falling within the bounding box, that is, the three-dimensional bounding representation of the identified object. (4) The obstacle point cloud is calculated based on image processing and point cloud processing algorithms to obtain the characteristic parameters of the building wall, such as slope, usable area and arrangement of distributed photovoltaic modules.

2. The method for arranging photovoltaic modules on a building facade based on images and point clouds as described in claim 1, characterized in that: The target detection network mentioned above uses the Mamba-Yolo image target detection network.

3. The method for arranging photovoltaic modules on building facades based on images and point clouds as described in claim 1, characterized in that: The step of projecting and transforming the bounding box to the lidar coordinate system to obtain the obstacle point cloud falling within the bounding box and the three-dimensional bounding representation of the identified object includes: filtering point cloud points projected onto the image plane and falling within the bounding box, and filtering out point clouds and stray point clouds within the bounding box to obtain the effective area point cloud of the building wall.

4. The method for arranging photovoltaic modules on building facades based on images and point clouds as described in claim 1, characterized in that: The algorithm based on image processing and point cloud processing is used to calculate the obstacle point cloud to obtain the characteristic parameters of the building wall, including slope and usable area. The Random Sampling Consensus (RANSAC) algorithm is used to fit the plane of the building wall and solve the parameters of the plane equation. Then, the slope parameters of the building wall are calculated based on the angle between the normal vector of the plane and the normal vector of the XOY plane. The Random Sampling Consensus (RANSAC) algorithm is used to fit the point cloud of each selected obstacle to a plane. The corresponding point cloud is then projected onto the plane fitted by RANSAC. The Alpha Shape algorithm is used to extract the boundary points of the plane. The disordered boundary points of the obtained patches are sorted using the four-quadrant tangent sorting method. The boundary points are then projected onto the pixel map and filled. The minimum bounding rectangle of the region is calculated. The maximum bounding rectangle obtained from the projection of each obstacle is buffered and expanded to meet construction and safety distance requirements, and all buffered rectangles are merged to generate the total uninstallable area; A preliminary installable area is obtained by performing a set difference operation between the wall surface region and the uninstallable area. Connected components are extracted and their area, perimeter, and minimum bounding rectangle are calculated. Areas with a maximum bounding rectangle area less than 1m2 are filtered out. Each region is rasterized to obtain a binary matrix (which can be placed = 1). The maximum axial rectangle is calculated on the binary matrix using the maximum rectangle algorithm, and the distributed photovoltaic module arrangement is designed based on it.

5. The method and system for arranging photovoltaic modules on building facades based on images and point clouds as described in claim 1, characterized in that... The calculation of the installable area based on image processing and point cloud processing algorithms to obtain characteristic parameters of the building walls, roof slope, usable area, and arrangement of distributed photovoltaic modules includes: The Random Sampling Consensus (RANSAC) algorithm is used to fit the plane of the building wall and solve the parameters of the plane equation. Then, the slope parameters of the building wall are calculated based on the angle between the normal vector of the plane and the normal vector of the XOY plane. The installable area is represented as a set of connected polygons. The area of ​​the maximum bounding rectangle of each polygon is calculated, and areas with a maximum bounding rectangle less than 1m are filtered out. 2 The region is defined, and the arrangement of distributed photovoltaic modules is designed accordingly.

6. A system for performing the method as described in any one of claims 1 to 5, characterized in that, include: The data acquisition module is used to acquire color images of the target building's walls and point clouds synchronized with them in time by a robot equipped with a LiDAR and an RGB camera; The obstacle detection module is used to run a Mamba-Yolo network to obtain the obstacle outlines of building walls; The projection module is used to obtain the obstacle point cloud of the building wall using the intrinsic and extrinsic parameters of the radar camera; The parameter calculation module is used to calculate the characteristic parameters of building walls and the arrangement of photovoltaic modules.

Images