Planar prior guided 3DGS three-dimensional reconstruction method and system for photovoltaic scene

Abstract

The present application relates to a kind of plane priori guide 3DGS three-dimensional reconstruction method and system for photovoltaic scene.The main body area mask of photovoltaic panel in image is automatically extracted by target detection and image segmentation method, the real physical size of target photovoltaic panel is combined with camera parameter information, the plane structure priori of the target photovoltaic panel under world coordinate system is recovered.The plane seed three-dimensional Gaussian is generated on the plane of target photovoltaic panel using the plane structure priori, and it is added in 3DGS initialization process, to improve the problem of insufficient initial Gaussian distribution of photovoltaic panel large plane weak texture area.Secondly, in the training process of 3DGS, plane consistency constraint is introduced to initial Gaussian set, which can improve the convergence efficiency, geometric stability and reconstruction quality of photovoltaic scene three-dimensional reconstruction, and generate the real 3D model of target photovoltaic panel.Provide efficient and stable technical support for photovoltaic station digital twin modeling and unmanned aerial vehicle inspection three-dimensional reconstruction.

Description

Technical Field

[0001] This invention relates to the field of 3D reconstruction technology, and in particular to a planar prior-guided 3DGS 3D reconstruction method and system for photovoltaic scenarios. Background Technology

[0002] With the rapid construction of large-scale photovoltaic power plants and the Shagohuang new energy base, the demand for intelligent operation and maintenance of photovoltaic power plants is constantly increasing. Rapid, stable, and high-quality 3D reconstruction of photovoltaic power plants is a crucial foundation for realizing digital twins, equipment inspection, fault location, condition assessment, and visual management.

[0003] Traditional 3D reconstruction methods mainly include photogrammetry based on structure-of-motion / multi-view stereo vision (SfM / MVS), point cloud or mesh modeling methods, and the neural radiance field method developed in recent years. SfM / MVS methods can recover camera pose and sparse point cloud structure from multi-view images, but the weak surface texture of photovoltaic panels results in a limited number of feature points to be extracted and matched. The reconstructed sparse point cloud suffers from insufficient coverage and extremely uneven distribution in the planar area of ​​the photovoltaic panel, making it difficult to support high-quality surface reconstruction. While implicit neural representation methods such as NeRF (Neural Radiance Fields) can achieve good view synthesis results, their high training and rendering costs make them difficult to meet the needs of rapid modeling and real-time interaction in photovoltaic power plants.

[0004] 3D Gaussian Splatting (3DGS) is an explicit 3D scene representation method proposed in recent years. It represents the scene using a large number of 3D Gaussian elements and utilizes differentiable rasterization to achieve efficient training and real-time rendering. This method has advantages such as fast training speed, high rendering efficiency, and easy editing of explicit representations, making it suitable for large-scale engineering scene reconstruction such as photovoltaic scenes. The initialization process of standard 3DGS heavily relies on the sparse point cloud recovered by SfM to generate the initial 3D Gaussian elements. Since the sparse point cloud in the photovoltaic panel area is relatively sparse and unevenly distributed, the model faces a large number of gaps in the initial Gaussian coverage of this area at the beginning of training. Therefore, standard 3DGS needs to rely on a subsequent adaptive Gaussian densification process, which gradually fills the entire planar area from a small number of initial points through repeated cloning and splitting of Gaussians.

[0005] However, photovoltaic scenes exhibit distinct planar structural characteristics. Photovoltaic panels are typically arranged in a regular pattern, with weak surface texture, and large areas are nearly planar. Standard 3DGS primarily optimizes the appearance by fitting the rendering result to the real image based on the difference between the rendered result and the real image, using disordered Gaussian thickening and morphological distortion. It lacks explicit geometric constraints on the planar structure of photovoltaic panels. Specifically, the Gaussian center drifts along the plane's normal direction, disrupting the plane's flatness; the Gaussian expands along the normal direction, resulting in significant planar thickening and out-of-plane artifacts in the reconstructed photovoltaic panel; simultaneously, some Gaussians over-expand their scale within the plane to compensate for large areas of weak texture, further exacerbating boundary blurring and local structural distortion. These factors collectively lead to poor geometric stability and low training convergence efficiency in the photovoltaic panel region. Summary of the Invention

[0006] In view of this, in order to at least partially improve the above-mentioned problems, the present invention provides a planar prior-guided 3DGS three-dimensional reconstruction method and system for photovoltaic scenarios.

[0007] Firstly, this invention provides a planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios, including: Multiple multi-view photovoltaic scene images of the target photovoltaic panel are acquired and target detection and segmentation are performed one by one to obtain the main region mask of the target photovoltaic panel in each of the multi-view photovoltaic scene images; Based on the mask of each subject region, an approximate two-dimensional geometric observation of the target photovoltaic panel is constructed in each of the multi-view photovoltaic scene images; The true physical dimensions of the target photovoltaic panel and the camera parameter information corresponding to each of the multi-view photovoltaic scene images are obtained. Based on multiple approximate two-dimensional geometric observations, the true physical dimensions, and the camera parameter information, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered. Based on the prior knowledge of the planar structure, regularly distributed planar seed points are generated on the plane of the target photovoltaic panel, and a corresponding planar seed three-dimensional Gaussian is initialized for each of the planar seed points. Based on COLMAP technology, a primitive Gaussian is generated for the original sparse points on the target photovoltaic panel, and multiple seed 3D Gaussians in the plane are merged with the primitive Gaussians as initial Gaussians to obtain an initial Gaussian set. During 3DGS training, a plane consistency constraint is applied to the initial Gaussian set, and joint optimization is performed in conjunction with the rendering reconstruction loss to output the 3D model of the target photovoltaic panel.

[0008] In one embodiment, constructing an approximate two-dimensional geometric observation of the target photovoltaic panel in each of the multi-view photovoltaic scene images based on each of the main region masks includes: Based on each subject region mask, the subject contour of the target photovoltaic panel in each multi-view photovoltaic scene image is extracted and the minimum bounding rotation rectangle of the subject contour is calculated. The four vertices of the minimum bounding rotation rectangle are used as approximate two-dimensional geometric observations of the corresponding multi-view photovoltaic scene image.

[0009] In one embodiment, the process of recovering the prior planar structure of the target photovoltaic panel in the world coordinate system based on multiple approximate two-dimensional geometric observations, the actual physical dimensions, and camera parameter information includes: For each of the aforementioned multi-view photovoltaic scene images: Based on the actual physical dimensions, four coplanar three-dimensional reference points are established in the local coordinate system of the target photovoltaic panel; Obtain at least four two-dimensional vertices corresponding to the approximate two-dimensional geometric observation of the current multi-view photovoltaic scene image, and sort the four two-dimensional vertices so that they are arranged in a clockwise or counterclockwise direction; The candidate correspondences between the arranged two-dimensional vertices and the three-dimensional reference points that satisfy the cyclic order are enumerated. For each candidate correspondence, the PnP pose estimation algorithm is used to solve the candidate rotation matrix and translation vector from the local coordinate system of the target photovoltaic panel to the camera coordinate system, in combination with the camera parameter information. The candidate rotation matrix and the translation vector are used as a set of candidate poses. For each set of candidate poses, the four three-dimensional reference points are reprojected onto the image plane of the current multi-view photovoltaic scene image to obtain four reprojection points; Calculate the average reprojection error between the four two-dimensional vertices and the four reprojection points; From all the candidate poses, the candidate pose with the smallest average reprojection error and that satisfies the preset positive depth condition is selected as the photovoltaic panel candidate planar pose corresponding to the current multi-view photovoltaic scene image. Based on the candidate planar poses of the photovoltaic panel corresponding to each of the multiple multi-view photovoltaic scene images, and combined with the camera extrinsic parameters recovered by COLMAP, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered.

[0010] In one embodiment, the step of recovering the prior planar structure of the target photovoltaic panel in the world coordinate system based on the candidate planar poses of the photovoltaic panel corresponding to each of the multiple multi-view photovoltaic scene images, combined with the camera extrinsic parameters recovered by COLMAP, includes: The quality of the candidate planar pose of the photovoltaic panel corresponding to each of the multi-view photovoltaic scene images is evaluated, and the multi-view photovoltaic scene images that meet the preset quality conditions are determined as effective viewpoints. Based on the camera extrinsic parameters from the world coordinate system to the camera coordinate system recovered by COLMAP, the candidate planar poses of the photovoltaic panels corresponding to each effective viewpoint are transformed to the COLMAP world coordinate system to obtain the poses of the target photovoltaic panels in the world coordinate system under each effective viewpoint. Based on the pose of the target photovoltaic panel in the world coordinate system under each effective viewpoint, determine the plane center, plane normal vector and attitude parameters corresponding to each effective viewpoint; For each effective viewpoint, based on the corresponding photovoltaic panel candidate planar pose, the position of the camera center in the local physical coordinate system of the photovoltaic panel under that effective viewpoint is calculated in reverse. Based on the camera extrinsic parameters recovered by COLMAP, the second camera center position of the camera center in the COLMAP world coordinate system is obtained under each effective viewpoint; For any two effective viewpoints, a scale candidate value is constructed based on the ratio of the distance between the center positions of the first camera and the distance between the center positions of the second camera; For multiple effective viewpoints, multiple scale candidate values ​​are obtained by combining them in pairs. Abnormal scale candidate values ​​are removed from the multiple scale candidate values ​​using the interquartile range method. The median of the remaining scale candidate values ​​is taken as the scale factor. Based on the scale factor, the actual physical dimensions of the target photovoltaic panel are transformed to the scale of the COLMAP world coordinate system; A multi-frame consistency screening is performed on the plane center, plane normal vector and attitude parameters corresponding to multiple effective viewpoints to eliminate effective viewpoints whose differences from other effective viewpoints exceed a preset threshold. The plane center, plane normal vector and attitude parameters corresponding to the effective viewpoints that have passed the multi-frame consistency screening are fused across multiple frames to obtain a unified planar structure prior in the world coordinate system.

[0011] In one embodiment, the step of generating regularly distributed planar seed points on the plane of the target photovoltaic panel based on the planar structure prior, and initializing a corresponding planar seed 3D Gaussian for each of the planar seed points, includes: The planar center, two mutually orthogonal principal directions, plane width, plane height, plane normal vector, and preset in-plane sampling interval are obtained from the prior information of the planar structure. The number of samples along the two mutually orthogonal principal directions is determined, and planar seed points are generated uniformly distributed in the plane of the target photovoltaic panel. For each of the planar seed points, a planar seed 3D Gaussian is initialized, wherein the Gaussian center of the planar seed 3D Gaussian is set to the position of the planar seed point, and the rotation attitude of the 3D Gaussian is such that two axes of the 3D Gaussian are aligned with the two mutually orthogonal principal directions, and the third axis is aligned with the plane normal vector. Furthermore, the initial scale of the 3D Gaussian along the plane normal vector direction is smaller than the initial scale along the two principal directions, so that the initialized planar seed 3D Gaussian has a flat ellipsoidal shape that fits the surface of the target photovoltaic panel.

[0012] In one embodiment, the preset in-plane sampling interval is determined based on the average nearest neighbor distance of the original Gaussian points generated on the target photovoltaic panel using COLMAP technology within the planar region of the target photovoltaic panel, and the in-plane sampling interval is not greater than a preset multiple of the average nearest neighbor distance.

[0013] In one embodiment, the original Gaussian sparse points generated on the target photovoltaic panel based on COLMAP technology are merged with multiple planar seed 3D Gaussians and the original Gaussians as initial Gaussians to obtain an initial Gaussian set, including: For each of the planar seed 3D Gaussians, the nearest neighbor original Gaussian with the closest spatial position is determined in the original Gaussians, and the spatial distance between the planar seed 3D Gaussian and the nearest neighbor original Gaussian is calculated; When the spatial distance is less than the preset deduplication threshold, the position of the nearest original Gaussian is merged and updated with the position of the planar seed 3D Gaussian, or the planar seed 3D Gaussian is not added to the initial Gaussian set. When the spatial distance is greater than or equal to the preset deduplication threshold, the planar seed 3D Gaussian is added as a new initial Gaussian to the initial Gaussian set; At the same time, planar attribute labels are set for all the planar seed 3D Gaussians added to the initial Gaussian set to distinguish them from ordinary Gaussians composed of the original Gaussians.

[0014] In one embodiment, the preset deduplication threshold is determined based on the average nearest neighbor distance of the original Gaussian within the planar region of the target photovoltaic panel, or based on the in-plane sampling interval used when generating the planar seed point; For the nearest-neighbor original Gaussian, if the spatial distance between it and the plane seed 3D Gaussian is less than the preset deduplication threshold, and the projection position of the nearest-neighbor original Gaussian on the plane of the target photovoltaic panel is within the effective range of the plane setting, then the plane attribute label is also set for the nearest-neighbor original Gaussian.

[0015] In one embodiment, applying a plane consistency constraint to the initial Gaussian set during 3DGS training includes: Detect whether the initial Gaussians in the initial Gaussian set carry a planar attribute label, and use the initial Gaussians carrying the planar attribute label as planar related Gaussians; A plane consistency loss is applied to the plane-correlated Gaussian, the plane consistency loss comprising at least one of the following: Point-to-plane distance loss is used to limit the normal distance from the Gaussian center of the plane-related Gaussian to the plane of the target photovoltaic panel; An attitude consistency loss is used to align the minimum scale axis direction of the plane-dependent Gaussian with the plane normal vector direction of the target photovoltaic panel. Normal thickness suppression loss is used to limit the scale of the plane-correlated Gaussian along the plane normal vector direction to not exceed a preset upper limit of normal scale; In-plane scale loss is used to limit the scale of the plane-correlated Gaussian within the plane of the target photovoltaic panel to no more than a preset upper limit for in-plane scale.

[0016] This invention also provides a planar prior-guided 3DGS three-dimensional reconstruction system for photovoltaic scenarios, the system being used to implement the method described in any one of the above, comprising: The image segmentation module is used to acquire multiple multi-view photovoltaic scene images of the target photovoltaic panel and perform target detection and segmentation one by one to obtain the main region mask of the target photovoltaic panel in each of the multi-view photovoltaic scene images; A construction module is used to construct an approximate two-dimensional geometric observation of the target photovoltaic panel in each of the multi-view photovoltaic scene images based on each of the main body region masks; The coordinate recovery module is used to obtain the true physical size of the target photovoltaic panel and the camera parameter information corresponding to each of the multi-view photovoltaic scene images. Based on multiple approximate two-dimensional geometric observations, the true physical size, and the camera parameter information, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered. The seed generation module is used to generate regularly distributed planar seed points on the plane of the target photovoltaic panel based on the prior of the planar structure, and initialize the corresponding planar seed three-dimensional Gaussian for each of the planar seed points; The Gaussian merging module is used to generate the original Gaussian of the original sparse points on the target photovoltaic panel based on COLMAP technology, and merge the multiple planar seed 3D Gaussians with the original Gaussians as the initial Gaussians to obtain the initial Gaussian set. The 3DGS training module is used to apply a plane consistency constraint to the initial Gaussian set during the 3DGS training process, and perform joint optimization in conjunction with the rendering reconstruction loss to output the 3D model of the target photovoltaic panel.

[0017] Beneficial Effects: This application automatically extracts the main region mask of the target photovoltaic panel in multiple multi-view photovoltaic scene images using target detection and image segmentation methods to determine the approximate outline and approximate two-dimensional geometric observation of the target photovoltaic panel. Then, combining the actual physical dimensions of the target photovoltaic panel and camera parameter information, it reconstructs the planar structure prior of the target photovoltaic panel in the world coordinate system. Using this planar structure prior, a planar seed 3D Gaussian is generated on the plane of the target photovoltaic panel and added to the 3DGS initialization process to improve the problem of insufficient initial Gaussian distribution in large, weakly textured areas of the photovoltaic panel. Secondly, during 3DGS training, a planar consistency constraint is introduced to the initial Gaussian set to suppress out-of-plane drift, normal diffusion, and planar thickening phenomena along the plane normal direction, reducing redundant Gaussian generation. This improves the convergence efficiency, geometric stability, and reconstruction quality of the photovoltaic scene 3D reconstruction, generating a realistic 3D model of the target photovoltaic panel. This provides efficient and stable technical support for digital twin modeling of photovoltaic power plants, 3D reconstruction of UAV inspections, and intelligent operation and maintenance visualization. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart illustrating a planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the structure of a planar prior-guided 3DGS three-dimensional reconstruction system for photovoltaic scenarios, according to another embodiment of the present invention. Detailed Implementation

[0020] The terms "first," "second," and "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects and not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to these processes, methods, products, or apparatuses.

[0021] Please see Figure 1 , Figure 2 One embodiment of the present invention provides a planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios, which can be applied to scenarios requiring digital twin modeling of photovoltaic power plants, 3D reconstruction of UAV inspections, photovoltaic panel inspection and maintenance, and visual management of photovoltaic power plants. The method includes: S1. Acquire multiple multi-view photovoltaic scene images of the target photovoltaic panel and perform target detection and segmentation one by one to obtain the main region mask of the target photovoltaic panel in each of the multi-view photovoltaic scene images; S2. Based on the mask of each main region, construct an approximate two-dimensional geometric observation of the target photovoltaic panel in each of the multi-view photovoltaic scene images; S3. Obtain the true physical dimensions of the target photovoltaic panel and the camera parameter information corresponding to each of the multi-view photovoltaic scene images. Based on multiple approximate two-dimensional geometric observations, the true physical dimensions, and the camera parameter information, recover the prior planar structure of the target photovoltaic panel in the world coordinate system. S4. Based on the prior knowledge of the planar structure, generate regularly distributed planar seed points on the plane of the target photovoltaic panel, and initialize the corresponding planar seed three-dimensional Gaussian for each planar seed point; S5. Based on COLMAP technology, generate the original Gaussian of the original sparse points on the target photovoltaic panel, and merge the multiple planar seed three-dimensional Gaussians with the original Gaussians as the initial Gaussians to obtain the initial Gaussian set. S6. During the 3DGS training process, a plane consistency constraint is applied to the initial Gaussian set, and joint optimization is performed in conjunction with the rendering reconstruction loss to output the 3D model of the target photovoltaic panel.

[0022] As described above, firstly, the main region mask of the target photovoltaic panel in multiple multi-view photovoltaic scene images is automatically extracted using target detection and image segmentation methods to determine the approximate outline and approximate two-dimensional geometric observation of the target photovoltaic panel. Then, combined with the actual physical size of the target photovoltaic panel and camera parameter information, the planar structure prior of the target photovoltaic panel in the world coordinate system is recovered. Using this planar structure prior, a planar seed 3D Gaussian is generated on the plane of the target photovoltaic panel and added to the 3DGS initialization process to improve the problem of insufficient initial Gaussian distribution in the large, weakly textured areas of the photovoltaic panel. Secondly, during the 3DGS training process, a planar consistency constraint is introduced into the initial Gaussian set to suppress out-of-plane drift, normal diffusion, and plane thickening phenomena along the plane normal direction, reducing redundant Gaussian generation. This improves the convergence efficiency, geometric stability, and reconstruction quality of the 3D reconstruction of the photovoltaic scene, generating a realistic 3D model of the target photovoltaic panel. This provides efficient and stable technical support for digital twin modeling of photovoltaic power plants, 3D reconstruction of UAV inspections, and intelligent operation and maintenance visualization.

[0023] It should be noted that approximate two-dimensional geometric observation is a two-dimensional geometric description extracted from a single two-dimensional image, used to characterize the projection range and main extension direction of the target photovoltaic panel's main plane in the current multi-view photovoltaic scene image.

[0024] In one embodiment, step S1, which involves acquiring multiple multi-view photovoltaic scene images of the target photovoltaic panel and performing target detection and segmentation on each image to obtain the main region mask of the target photovoltaic panel in each multi-view photovoltaic scene image, includes: For each input multi-view photovoltaic scene image, a text-guided object detection model (Grounding DINO) is first used to detect target photovoltaic panels based on preset photovoltaic panel category prompts, obtaining candidate bounding boxes and corresponding detection confidence scores. Low-confidence candidate results are then eliminated based on a preset confidence threshold to obtain the effective photovoltaic panel candidate bounding boxes in the current multi-view photovoltaic scene image.

[0025] Subsequently, the valid candidate bounding boxes of photovoltaic panels are used as box cues input into the SAM segmentation model to obtain the corresponding candidate masks of photovoltaic panels. To reduce the impact of similar background textures, shadows, supports, adjacent photovoltaic panels, and segmentation spillover on subsequent geometric estimation, the candidate masks are cropped within the bounding boxes using the candidate bounding boxes. Then, connected component analysis and contour filtering are performed on the cropped binary masks to remove small isolated regions and retain the largest main connected regions.

[0026] Furthermore, the candidate masks are evaluated for quality based on the mask area, the completeness of the main outline determined by the main connected region, the rectangle fill rate, and the consistency of the aspect ratio. Observations with mask areas that are too small, severely fragmented outlines, excessively low rectangle fill rates, or obviously abnormal shapes are judged as low-quality results and discarded, thus obtaining the optimized target photovoltaic panel main area mask.

[0027] In one embodiment, step S2, which constructs an approximate two-dimensional geometric observation of the target photovoltaic panel in each of the multi-view photovoltaic scene images based on each of the subject region masks, includes: Based on each subject region mask, the subject contour of the target photovoltaic panel in each multi-view photovoltaic scene image is extracted and the minimum bounding rotation rectangle of the subject contour is calculated. The four vertices of the minimum bounding rotation rectangle are used as approximate two-dimensional geometric observations of the corresponding multi-view photovoltaic scene image.

[0028] As described above, the minimum circumscribed rectangle is used to characterize the two-dimensional projection range, center position, and main extension direction of the target photovoltaic panel's main plane in the current image, and its four vertices serve as approximate geometric observations used for subsequent pose estimation. When a drone or camera photographs the target photovoltaic panel, it is difficult to guarantee that each target photovoltaic panel is photographed from a vertical perspective. Therefore, when the drone or moving camera photographs the target photovoltaic panel from a non-vertical perspective, the true rectangular boundary of the target photovoltaic panel may appear as an irregular perspective quadrilateral in the image due to perspective projection. Therefore, this embodiment does not limit the four vertices of the minimum circumscribed rectangle to the precise projection of the four real physical corner points of the photovoltaic panel in the image, but rather uses them as a stable approximate description of the projection range, center position, and main extension direction of the target photovoltaic panel's main plane in the current image. This processing method avoids the problem of amplified corner point extraction errors caused by unstable segmentation edges, providing a more stable input for subsequent pose estimation.

[0029] Furthermore, photovoltaic panel segmentation masks may suffer from local defects, boundary noise, and partial shading. Directly extracting discrete corner points from the contour can easily lead to problems such as unstable corner point numbers, inconsistent vertex arrangements, and positional jumps. The minimum bounding rectangle of rotation can provide four-point observations with a fixed number, consistent order, and relatively insensitive to local contour disturbances, facilitating the subsequent construction of candidate correspondences between two-dimensional vertices and the three-dimensional reference points.

[0030] In one embodiment, step S3, which reconstructs the prior planar structure of the target photovoltaic panel in the world coordinate system based on multiple approximate two-dimensional geometric observations, the actual physical dimensions, and camera parameter information, includes: S31. For each of the aforementioned multi-view photovoltaic scene images: Based on the actual physical dimensions, four coplanar three-dimensional reference points are established in the local coordinate system of the target photovoltaic panel; Obtain at least four two-dimensional vertices corresponding to the approximate two-dimensional geometric observation of the current multi-view photovoltaic scene image, and sort the four two-dimensional vertices so that they are arranged in a clockwise or counterclockwise direction; The candidate correspondences between the arranged two-dimensional vertices and the three-dimensional reference points that satisfy the cyclic order are enumerated. For each candidate correspondence, the PnP pose estimation algorithm is used to solve the candidate rotation matrix and translation vector from the local coordinate system of the target photovoltaic panel to the camera coordinate system, in combination with the camera parameter information. The candidate rotation matrix and the translation vector are used as a set of candidate poses. For each set of candidate poses, the four three-dimensional reference points are reprojected onto the image plane of the current multi-view photovoltaic scene image to obtain four reprojection points; Calculate the average reprojection error between the four two-dimensional vertices and the four reprojection points; From all the candidate poses, the candidate pose with the smallest average reprojection error and that satisfies the preset positive depth condition is selected as the candidate planar pose of the photovoltaic panel corresponding to the current multi-view photovoltaic scene image.

[0031] S32. Based on the candidate planar poses of the photovoltaic panel corresponding to each of the multiple multi-view photovoltaic scene images, and combined with the camera extrinsic parameters recovered by COLMAP, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered.

[0032] As mentioned above, to ensure the consistency of the arrangement of the four two-dimensional vertices from different perspectives, they are uniformly sorted according to their positional relationship relative to the center of the rotating rectangle, arranging them in a clockwise or counterclockwise direction. Subsequently, based on the actual physical width W and height H of the target photovoltaic panel, four coplanar three-dimensional reference points are established in the local coordinate system of the target photovoltaic panel: ; ; ; ; Where the superscript 'b' represents the local coordinate system of the target photovoltaic panel, This represents the position of the first 3D reference point in the local coordinate system, ordered in either counter-clockwise or clockwise order. This represents the position of the second 3D reference point in the local coordinate system, ordered in either a counter-clockwise or clockwise sequence. This represents the position of the third 3D reference point in the local coordinate system, ordered in either a counter-clockwise or clockwise sequence. This represents the position of the fourth 3D reference point in the local coordinate system, ordered in either counter-clockwise or clockwise order. The origin of this local coordinate system is located at the center of the photovoltaic panel plane, the local x-axis and local y-axis are along the width and height directions of the target photovoltaic panel, respectively, and the local z-axis is perpendicular to the surface of the photovoltaic panel.

[0033] Considering that different shooting angles may cause changes in the starting position and arrangement direction of 2D vertices, this invention enumerates candidate correspondences between 2D vertices and 3D reference points that satisfy a cyclic order. As an example, assume that the four 2D vertices are arranged clockwise as p1, p2, p3, p4, and the 3D reference points are arranged clockwise as follows: , , , Since we don't know which corresponds to which, we enumerate four candidate correspondences: Candidate 1: ; Candidate 2:

[0034] Candidate 3: ; Candidate 4: ; For each candidate correspondence, combining the camera intrinsic parameter matrix from the camera parameter information, the set of 2D vertices and the set of 3D reference points obtained above are used as inputs to PnP. The PnP pose estimation algorithm is used to solve for the candidate rotation matrix and translation vector from the photovoltaic panel local coordinate system to the camera coordinate system. The coordinate transformation relationship is as follows: ; Among them, X b Represents a three-dimensional reference point in the local coordinate system (e.g.) - ), X c R represents the 3D coordinates of the same 3D reference point in the camera coordinate system. bc,t T represents a 3×3 matrix corresponding to the multi-view photovoltaic scene image in the current t-th frame, describing the rotation attitude of the photovoltaic panel's local coordinate system relative to the camera coordinate system, i.e., the candidate rotation matrix. bc,t This represents a 3×1 column vector corresponding to the multi-view photovoltaic scene image in the current t-th frame, describing the spatial translation distance of the origin of the local coordinate system of the photovoltaic panel relative to the optical center of the camera (the origin of the camera coordinate system), i.e., the translation vector.

[0035] For each set of candidate poses, four 3D reference points are reprojected onto the plane of the current multi-view photovoltaic scene image, and the average reprojection error between the 2D vertices and the reprojected points is calculated. The reprojection position of the k-th 3D reference point is represented as: ,in, Let π be the k-th reprojection point of the multi-view photovoltaic scene image in frame t obtained from the calculated current candidate pose, where π is the projection function and K is the reprojection point. t Let be the camera intrinsic parameter matrix corresponding to the multi-view photovoltaic scene image in frame t. The coordinates of the k-th 3D reference point in the camera coordinate system are calculated from the PnP results of the multi-view photovoltaic scene image of frame t.

[0036] The corresponding average reprojection error is expressed as:

[0037] Among them, e t Let u be the average reprojection error of the multi-view photovoltaic scene image in frame t. t,k This represents the k-th two-dimensional vertex of the multi-view photovoltaic scene image in frame t. It is an L2 norm.

[0038] For all candidate correspondences, their average reprojection error is calculated, and a positive depth check is performed to ensure that the transformed 3D reference point is in front of the camera. From all candidate poses, the result with the smallest average reprojection error and satisfying the positive depth condition is selected as the candidate planar pose of the photovoltaic panel corresponding to the current multi-view photovoltaic scene image.

[0039] S32 includes: S321. Perform quality evaluation on the candidate planar pose of the photovoltaic panel corresponding to each of the multi-view photovoltaic scene images, and determine the multi-view photovoltaic scene images that meet the preset quality conditions as effective viewpoints. Specifically, since the minimum circumscribed rotation rectangle provides an approximate two-dimensional observation, this invention does not directly rely on the single-frame PnP result to determine the final planar prior. Instead, it further combines the mask area, rectangle fill rate, aspect ratio consistency, and average reprojection error to conduct a comprehensive quality evaluation of the current frame observation. The comparison value in the quality evaluation method can be set as needed and is not limited here. When the mask area is too small (e.g., less than 30% of the mask area of ​​other images), the rectangle fill rate is too low (e.g., less than 30% of the mask area of ​​other images), the aspect ratio deviation is too large (e.g., the ratio is greater than the preset ratio), or the average reprojection error exceeds the preset threshold, the frame is judged as a low-quality observation and is removed or its weight in subsequent multi-frame fusion is reduced, and the multi-view photovoltaic scene image that meets the requirements is taken as the effective viewpoint.

[0040] S322. Based on the camera extrinsic parameters from the world coordinate system to the camera coordinate system recovered by COLMAP, the candidate planar pose of the photovoltaic panel corresponding to each effective viewpoint is transformed to the COLMAP world coordinate system to obtain the pose of the target photovoltaic panel in the world coordinate system under each effective viewpoint. As mentioned above, a 3D point in the world coordinate system and a 3D point in the camera coordinate system satisfy the first rigid body transformation relationship: ; Among them, X c X represents the three-dimensional coordinates of the same photovoltaic panel candidate planar pose in the camera coordinate system. w This refers to the pose of the same photovoltaic panel candidate planar pose in the world coordinate system reconstructed by COLMAP. wc,t Let T be the rotation matrix from the world coordinate system to the camera coordinate system in the effective viewpoint of frame t. wc,t Let represent the translation vector from the world coordinate system to the camera coordinate system under the effective viewpoint of frame t.

[0041] S323. Based on the pose of the target photovoltaic panel in the world coordinate system under each effective viewpoint, determine the plane center, plane normal vector and attitude parameters corresponding to each effective viewpoint; As described above, based on the aforementioned PnP pose estimation algorithm, point X in the local physical coordinate system (with real physical scale) of the photovoltaic panel has been obtained. b The second rigid body transformation relationship to the camera coordinate system: ; Therefore, by simultaneously establishing the two transformation relationships mentioned above, a direct mapping between the local physical coordinate system of the photovoltaic panel and the COLMAP world coordinate system can be established, thereby extracting the position of the center of the photovoltaic panel plane in the world coordinate system corresponding to the effective viewpoint of frame t (i.e., the origin of the local coordinate system is (0,0,0)). T The local coordinate system is mapped to the world coordinate system, and the plane normal vector is mapped to the world coordinate system, thus obtaining complete local attitude information of the photovoltaic panel.

[0042] S324. For each effective viewpoint, based on the corresponding photovoltaic panel candidate planar pose, calculate the position of the camera center in the local physical coordinate system of the photovoltaic panel under that effective viewpoint. Specifically, the purpose of this step is to determine the absolute position of the camera's optical center in the local coordinate system of the photovoltaic panel with real physical scale, denoted as the center position C of the first camera. b,t Since the camera center's coordinates in its own camera coordinate system are always the origin, that is... Substituting this into the second rigid body transformation relation above, we get: ; Performing rearrangement and matrix inversion operations on the above equation, since the rotation matrix R... bc,t Since is an orthogonal matrix, its inverse is equal to its transpose. Therefore, the formula for calculating the center position of the first camera can be obtained as follows: ; Through this step, the present invention transforms the camera center into a physical space with actual length and width dimensions (such as millimeters and meters) centered on the photovoltaic panel.

[0043] S325. Based on the camera extrinsic parameters recovered from COLMAP, obtain the second camera center position C of the camera center in the COLMAP world coordinate system under each effective viewpoint. w,t ; As mentioned above, when performing sparse reconstruction, the COLMAP algorithm outputs the camera extrinsic parameters corresponding to each frame of the image, namely the rotation matrix R from the COLMAP world coordinate system to the camera coordinate system. wc,t Translation vector T wc,t Similarly, following the inverse calculation logic of the first camera center position in step S324, the inverse calculation formula for the first camera center position can be obtained as follows: ; It should be noted that the center position C of the second camera obtained at this time... w,t Existing in the relative space constructed by COLMAP, its values ​​lack true physical scale significance.

[0044] S326. For any two effective viewpoints, construct a scale candidate value based on the ratio of the distance between the center positions of the first camera and the distance between the center positions of the second camera. Specifically, considering that the reconstruction results of pure vision 3D reconstruction algorithms such as COLMAP are usually determined only in the sense of similar transformation, that is, the world coordinate system they construct lacks a real absolute physical scale, this embodiment uses the geometric distance relationship between the center position of the first camera with a real physical scale and the center position of the second camera without a real scale to estimate and recover the global scale factor.

[0045] For any two selected valid viewpoints (denoted as frame i and frame j), their positions as the center of the first camera in the local physical coordinate system of the photovoltaic panel are denoted as C. b,i and C b,j The center position of the second camera in the COLMAP world coordinate system is denoted as C. w,i and C w,j Then, by calculating the ratio of the Euclidean distances between the centers of the same pair of cameras in different coordinate systems, a candidate scale value S is constructed. ij The calculation formula is as follows: ; in, This represents the straight-line distance (L2 norm) between two points in three-dimensional space. Since the denominator represents the absolute distance in the real physical world, and the numerator represents the relative distance in COLMAP space, this ratio reflects the scale relationship between the real physical space and the COLMAP reconstructed space.

[0046] S327. For multiple effective viewpoints, multiple scale candidate values ​​are obtained by combining them in pairs. Abnormal scale candidate values ​​are removed from the multiple scale candidate values ​​using the interquartile range method. The median of the retained scale candidate values ​​is taken as the scale factor. As mentioned above, to avoid inaccurate scale calculations due to PnP pose calculation errors or feature point extraction deviations from individual viewpoints, this embodiment employs a robust mechanism of multi-viewpoint cross-validation and statistical filtering. First, multiple valid viewpoints are traversed, and each viewpoint is paired with another. The scale is then calculated using the aforementioned formula to obtain a set of multiple S... ij A set of candidate scaling values ​​is formed. Then, the interquartile range (IQR) method is introduced to remove outliers from this set. The first quartile (Q1) and the third quartile (Q3) of the candidate scaling value set are calculated, and the interquartile range IQR = Q3 - Q1 is obtained. Scale candidate values ​​less than Q1 - 1.5 × IQR or greater than Q3 + 1.5 × IQR are identified as outliers and removed. Finally, the median is selected from the remaining candidate scaling values ​​after outlier removal as the global scaling factor adopted by the system. By using the median instead of the arithmetic mean, the robustness of the system against extreme outliers is further enhanced.

[0047] S328. Based on the scale factor, transform the actual physical dimensions of the target photovoltaic panel to the scale of the COLMAP world coordinate system; As described above, after determining the global scale factor, the known physical properties of the photovoltaic panel are scaled using this scale factor. Specifically, this involves multiplying the actual physical width W and physical height H of the photovoltaic panel, as well as various positional parameters in the local coordinate system, by the scale factor to transform and align them to the scale system of the COLMAP world coordinate system. This provides an accurate benchmark for subsequently embedding the photovoltaic panel into the 3D reconstructed scene at the correct size.

[0048] S329. Perform multi-frame consistency screening on the plane center, plane normal vector and attitude parameters corresponding to multiple effective viewpoints, and remove effective viewpoints whose differences from other effective viewpoints exceed a preset threshold. As mentioned above, considering that approximating the main outline of the target photovoltaic panel using the minimum bounding rectangle during single-frame image processing may introduce approximation errors, coupled with perturbations at image segmentation boundaries and pose jitter inherent in the PnP algorithm, all of these factors will affect the accuracy of planar prior information. Therefore, this embodiment introduces multi-frame consistency screening: For the plane center coordinates, plane normal vectors, and local pose parameters recovered from multiple effective viewpoints, the system calculates parameter deviation metrics between different effective frames. For example, it calculates the Euclidean distance deviation between the plane centers of each frame, the angular deviation between the plane normal vectors, the degree of pose change (such as the Lie algebra difference in the rotation matrix), and the PnP average reprojection error for each frame. When one or more of the above metrics for a frame differ significantly from the average level of other effective frames, or exceed a pre-set tolerance threshold, the pose result of that frame is judged as an abnormal observation / unstable state and removed from the effective viewpoint set.

[0049] S3210. Perform multi-frame fusion on the plane center, plane normal vector and attitude parameters corresponding to the effective viewpoint that has passed the multi-frame consistency screening to obtain the unified planar structure prior in the world coordinate system.

[0050] As described above, for the stable and valid results retained after consistency screening, this embodiment uses multi-frame fusion to output the final accurate parameters. The fusion algorithm can employ median averaging, weighted averaging (e.g., using the reciprocal of the reprojection error as weight), or other robust estimation methods. Furthermore, before fusing the planar normal vectors of each frame, orientation unification and normalization processing are required (e.g., specifying that the normal vectors all point to the camera's optical center) to avoid mutual cancellation or incorrect results due to opposite signs of the normal vectors.

[0051] Through the multi-frame fusion described above, a unified and accurate prior knowledge of the photovoltaic panel's planar structure in the world coordinate system is finally obtained. This planar structure prior knowledge includes complete explicit information describing the three-dimensional spatial state of the target photovoltaic panel, including at least: the center coordinates of the target photovoltaic panel's plane, the unit normal vector, the principal direction vectors in both planes, the plane's width and height after scale unification, the global scale factor, and the precise rotation and translation parameters from the target photovoltaic panel's local physical coordinate system to the world coordinate system. Preferably, the planar structure prior knowledge is persistently stored in the form of a structured file (such as JSON or XML format).

[0052] In summary, this embodiment transforms pixel-level observations of photovoltaic panels from multiple effective viewpoints into a high-precision explicit planar structure prior in the world coordinate system. This process effectively mitigates errors caused by perspective distortion, minimum bounding rectangle approximation, and inaccurate segmentation masks in single-frame processing. Specifically, the minimum bounding rectangle is only used to construct a stable approximate two-dimensional geometric observation for a single frame and is not considered a precise projection of the actual physical corner points of the target photovoltaic panel. The combination of steps such as mask main region selection, vertex candidate arrangement enumeration, IQR scale filtering, and multi-frame consistency fusion suppresses the risk of 3D reconstruction failure caused by prior pose jitter, thereby obtaining a stable planar structure prior that can be used for 3DGS initialization and training optimization.

[0053] It should be noted that the t-th frame image mentioned in the embodiments of the present invention may be a continuous image frame from a video stream, or it may be one of multiple independent images captured discretely. The present invention does not limit this.

[0054] In one embodiment, step S4, which generates regularly distributed planar seed points on the plane of the target photovoltaic panel based on the planar structure prior, and initializes the corresponding planar seed 3D Gaussian for each of the planar seed points, includes: S41. Obtain the plane center, two mutually orthogonal main directions in the plane, plane width, plane height, plane normal vector, and preset in-plane sampling interval from the prior information of the planar structure. Determine the number of samples along the two mutually orthogonal main directions and generate planar seed points uniformly distributed in the plane of the target photovoltaic panel. As mentioned above, standard 3DGS typically initializes 3D Gaussian units with sparse point clouds recovered using SfM or COLMAP. However, the surface texture of photovoltaic panels is weak, and the number of feature points extracted and matched is limited, resulting in insufficient coverage and uneven distribution of sparse point clouds in the planar region of the photovoltaic panel. Consequently, the model needs to rely on Gaussian densification processes to gradually fill in the planar region in the early stages of training. To improve the above problems, this embodiment uses the planar structure prior of the photovoltaic panel obtained in the aforementioned steps to generate regularly distributed planar seed points on the target photovoltaic panel plane.

[0055] Let c be the a priori plane center of the target photovoltaic panel in the world coordinate system, and let e be the two mutually orthogonal principal directions in the plane. u and e v Let the unit normal vector of the plane be n, and let W and H be the actual physical width and height of the plane, respectively. Based on the principles of spatial geometry, the above three direction vectors strictly satisfy the orthogonality relation: ; ; First, the system uses a preset in-plane sampling interval Δ uand Δ v Calculate the number N of sampling points along the width and height directions respectively. u and N v : ; ; Where, Δ u Along the width direction of the target photovoltaic panel, i.e., the principal direction e in the plane u Above, the distance between two adjacent sampling points, Δ v Along the height direction of the target photovoltaic panel, i.e., the principal direction in the plane e v Above, the distance between two adjacent sampling points. N u N represents the total number of seed points that need to be generated along the width of the target photovoltaic panel. v This represents the total number of seed points that need to be generated along the height of the target photovoltaic panel. This represents the floor operation, performed for the m width-direction sampling positions and the nth height-direction sampling position (m∈[0,N). u -1],n∈[0,N v -1]), its two-dimensional relative coordinates (u) in the plane of the local coordinate system of the target photovoltaic panel. m ,u n ) can be represented as: ; ; Furthermore, by vector translation and superposition, the coordinates are transformed to the world coordinate system to obtain the absolute three-dimensional coordinates p of the (m, n)th plane seed point. m,n : ; Furthermore, to ensure that the generated planar seed points can fill the areas insufficiently covered by the original COLMAP sparse points, and to prevent memory overflow due to an excessively large initial number of Gaussian points, this embodiment adjusts the sampling interval Δ. u and Δ v The value of is adaptively constrained. Specifically, the upper limit of this sampling interval is dynamically linked to the "average nearest neighbor distance" of the original sparse points generated based on the COLMAP algorithm within the target photovoltaic panel area. The system forcibly sets Δ u and Δ v It should not be greater than a preset multiple of the average nearest neighbor distance (e.g., between 1.0 and 1.5 times).

[0056] S42. For each of the plane seed points, initialize a plane seed 3D Gaussian, wherein the Gaussian center of the plane seed 3D Gaussian is set to the position of the plane seed point, and the rotation attitude of the 3D Gaussian is such that the two axes of the 3D Gaussian are aligned with the two mutually orthogonal principal directions respectively, and the third axis is aligned with the plane normal vector. The initial scale of the 3D Gaussian along the plane normal vector direction is smaller than the initial scale along the two principal directions, so that the initialized plane seed 3D Gaussian has a flat ellipsoid shape that fits the surface of the target photovoltaic panel.

[0057] Specifically, after generating a uniformly distributed set of planar seed points, the system assigns a seed point p to each planar seed point. m,n Explicitly initialize the core parameters of a 3D Gaussian, including center position, rotation orientation, and scale parameters. The center position is determined by centering the Gaussian. Set to the corresponding seed point coordinates p m,n Since the seed is sampled entirely within the physical plane of the photovoltaic panel, its geometric offset along the normal vector n is 0; the three orthogonal axes of the Gaussian local coordinate system of the rotational attitude are forcibly aligned to e. u e v And n. The scale parameter includes scale components in three orthogonal directions (let's call them n). , , The initial scale along the normal direction of the plane in this embodiment For a small value, the initial scale along the directions in both planes and Then assign a value based on the sampling interval (e.g., equal to or proportional to the sampling interval) to satisfy:

[0058] ; As an optional specific implementation, in order to avoid Gaussian volume expansion in the normal direction due to improper initial scale setting, the initial scale in the normal direction is... The relative proportions can be set according to the in-plane scale, for example, by... Set to no more than or The value can be 1% to 5%, or, assuming the coordinate system scale is standardized, [the value can be...]. Set directly to the preset minimum floating-point constant (e.g., 10). -3 Up to 10 -5 (magnitude).

[0059] This results in the initialized planar seed 3D Gaussian being a flattened ellipsoid that fits the surface of the photovoltaic panel. The initial color of the planar seed 3D Gaussian can be obtained by sampling its projection position in one or more input viewpoints, or it can be assigned a value based on the colors of neighboring COLMAP sparse points; the initial opacity and spherical harmonic features are set using parameters that are the same as or adapted to the standard 3DGS initialization process.

[0060] In one embodiment, step S5, which involves generating a primitive Gaussian set of sparse points on the target photovoltaic panel based on COLMAP technology and merging multiple seed 3D Gaussians from the planar model with the primitive Gaussian set to obtain an initial Gaussian set, includes: S51. For each of the planar seed 3D Gaussians, determine the nearest neighbor original Gaussian in the original Gaussians, and calculate the spatial distance between the planar seed 3D Gaussian and the nearest neighbor original Gaussian. Specifically, let P be the set of original Gaussian points (i.e., original sparse points) generated from the initialization of the COLMAP sparse point cloud. s The set of photovoltaic panel planar seed Gaussians (i.e., planar seed points) generated through the aforementioned steps is P. p Before performing Gaussian merging, it is necessary to perform a spatial coverage check. For the planar seed set P... p Each plane seed point p i In the original sparse point set P s The algorithm uses nearest neighbor search algorithms (such as KD-Tree) to find the nearest neighbor original point q with the closest spatial location. i And calculate the Euclidean distance d between them. i , which is the nearest neighbor spatial distance. Its distance metric formula is expressed as: ; S52. When the spatial distance is less than the preset deduplication threshold, the position of the nearest original Gaussian is merged and updated with the position of the planar seed 3D Gaussian, or the planar seed 3D Gaussian is not added to the initial Gaussian set. Specifically, a deduplication distance threshold is preset. When the calculated nearest neighbor spatial distance < At that time, its physical meaning indicates that at the seed point in the plane Within the local spatial region, there is already a valid original sparse point coverage extracted by COLMAP. To avoid Gaussian redundancy, the planar seed point can be directly discarded and not added again. Alternatively, the position and color information of the original sparse point can be extracted and fused with the planar attribute label of the planar seed point to optimize the Gaussian features of the local region.

[0061] S53. When the spatial distance is greater than or equal to the preset deduplication threshold, the planar seed 3D Gaussian is added as a new initial Gaussian to the initial Gaussian set. Specifically, when the calculated nearest neighbor spatial distance ≥ This indicates that the local planar region belongs to a "feature-deficient area" due to the weak texture of the photovoltaic panel surface, and the original COLMAP sparse points failed to effectively cover this area. At this point, the system seeds the planar points... The corresponding initialized planar seed 3D Gaussian is used as an effective supplementary element and is formally added to the original Gaussian set as a new initial Gaussian.

[0062] This yields the initial Gaussian set after fusion. : ; in, Let Gaussian set be initialized with COLMAP sparse points. This is the effective planar seed Gaussian set retained after removing duplicates of the original sparse points.

[0063] S54. At the same time as adding, set a planar attribute label for all the planar seed 3D Gaussians added to the initial Gaussian set, so as to distinguish them from the ordinary Gaussians composed of the original Gaussians.

[0064] As described above, in order to enable the network to effectively distinguish which Gaussians belong to the "planar dependent Gaussians" attached to the target photovoltaic panel and which belong to the background or non-planar "ordinary scene Gaussians" during the subsequent differentiable rendering training and Gaussian densification process, this embodiment assigns planar attribute labels (e.g., setting label values) to the planar seed 3D Gaussians generated from planar priors and added to the initial Gaussian set. = 1); while for ordinary Gaussians initialized only by regular COLMAP sparse points, ordinary attribute labels are assigned (e.g., setting label values). = 0). Furthermore, for the original Gaussian points located within the effective area of ​​the photovoltaic panel plane in step S52 and whose information is fused with the planar seed points, the system performs a tolerance judgment based on their vertical geometric distance to the target mathematical plane and their local planar range. If they meet the coplanarity requirement, their planar attribute label can also be set as a planar attribute label. That is, for the nearest neighbor original Gaussian point, if the spatial distance between it and the planar seed 3D Gaussian point is less than the preset deduplication threshold, and the projection position of the nearest neighbor original Gaussian point on the plane of the target photovoltaic panel is within the effective area of ​​the planar setting, then the planar attribute label is also set for the nearest neighbor original Gaussian point. The preset deduplication threshold is determined based on the average nearest neighbor distance of the original Gaussian point within the planar region of the target photovoltaic panel, or based on the in-plane sampling interval used when generating the planar seed points; In summary, compared to the problem of insufficient initial sparse points in existing methods, the traditional approach relies on the model to adjust positions, clone, and split through multiple rounds of gradient feedback during the long training process, gradually increasing the number of Gaussians to achieve effective coverage of large-area planes. This embodiment, before training begins, generates uniform, fitted, and directional 3D Gaussian seed planes based on the prior knowledge of the planar structure in real physical space, actively filling the coverage blind spots caused by weak textures. Through distance threshold deduplication in this embodiment, the photovoltaic panel area achieves complete initial spatial coverage while avoiding redundancy caused by dense sampling. Therefore, it can significantly reduce the invalid gradient search of Gaussians for planar positions in the early stages of training, reduce the compensation requirement for insufficient planar representation in the subsequent densification process, fundamentally reduce Gaussian redundancy and out-of-plane artifacts caused by disordered densification, and improve the training convergence efficiency and stability of the 3DGS model's 3D geometric representation.

[0065] In one embodiment, step S6, which involves applying a plane consistency constraint to the initial Gaussians in the initial Gaussian set during 3DGS training and performing joint optimization in conjunction with the rendering reconstruction loss to output a three-dimensional (3D) model of the target photovoltaic panel, includes: Detect whether the initial Gaussians in the initial Gaussian set carry a planar attribute label, and use the initial Gaussians carrying the planar attribute label as planar related Gaussians; A plane consistency loss is applied to the plane-correlated Gaussian, the plane consistency loss comprising at least one of the following: Point-to-plane distance loss is used to limit the normal distance from the Gaussian center of the plane-related Gaussian to the plane of the target photovoltaic panel; An attitude consistency loss is used to align the minimum scale axis direction of the plane-dependent Gaussian with the plane normal vector direction of the target photovoltaic panel. Normal thickness suppression loss is used to limit the scale of the plane-correlated Gaussian along the plane normal vector direction to not exceed a preset upper limit of normal scale; In-plane scale loss is used to limit the scale of the plane-correlated Gaussian within the plane of the target photovoltaic panel to no more than a preset upper limit for in-plane scale.

[0066] As mentioned above, traditional standard 3DGS algorithms are prone to Gaussian diffusion, drift, and planar thickening phenomena along the normal direction when processing weakly textured regions. To address this issue, this embodiment introduces a planar consistency constraint that applies only to the target planar region. Specifically, let Q be the set of all 3D Gaussians during the current training process. For the i-th Gaussian in the set, its core parameters include: center position... Rotation parameters (such as rotation matrices or quaternions), three-dimensional scale components and planar attribute tags Extract Gaussians with a plane attribute label value of 1 and form a plane-related Gaussian set. ={ =1}.

[0067] Furthermore, when the planar correlated Gaussian performs a cloning or splitting operation, the newly generated Gaussian inherits the planar attribute label; when the Gaussian carrying the planar attribute label is pruned, the planar attribute label is deleted synchronously. In the adaptive density control process of 3DGS, the number and distribution of Gaussians are dynamically changing. This embodiment ensures that when Gaussians generated from planar priors are cloned or split, their offspring Gaussians are still governed by planar constraints by inheriting and synchronously deleting the labels. This avoids the problem of traditional static regularization methods easily failing in dynamic point clouds, allowing planar constraints to remain effective throughout the entire training lifecycle as the number of Gaussians dynamically changes.

[0068] Furthermore, based on the normal distance from the Gaussian center to the target photovoltaic panel plane and the projection position of the Gaussian center in the plane, the Gaussian carrying the plane attribute label is dynamically verified, and the plane attribute label of Gaussians exceeding the preset range is removed. Specifically, let c be the a priori plane center of the target photovoltaic panel's planar structure in the world coordinate system, and let e be the two mutually orthogonal unit principal directions in the plane. u and e v Let the unit normal vector of the plane be n, and the actual physical width and height of the plane be W and H, respectively. For the i-th Gaussian, calculate the normal distance from its center to the target plane.

[0069] ; Simultaneously, calculate the local projected coordinates 'a' of its Gaussian center in the two principal directions of the target plane. i and b i : ; ; Set the judgment condition: when the Gaussian simultaneously satisfies the normal distance... ≤ (Allowable normal distance threshold), and projected coordinates | |≤ and (in and When the Gaussian center exceeds the specified tolerance (for planar boundaries), its planar attribute label is preserved. Conversely, if the center of Gaussian continuously exceeds the specified range, it is removed from the planar-related Gaussian set Q. p Removed from the middle.

[0070] During training, some Gaussians carrying planar attribute labels may deviate from the target photovoltaic panel's planar region due to gradient anomalies. If their planar attribute labels are not removed in time, the planar consistency constraint will continue to apply to these deviating Gaussians, attempting to force them to the target plane, which may introduce rendering artifacts such as local area ghosting or tearing. This embodiment continuously monitors the normal distance of the Gaussian center relative to the target photovoltaic panel's plane and its in-plane projection position, automatically removing the planar attribute labels from Gaussians that exceed a preset spatial range. This avoids the planar constraint acting on Gaussians that have deviated from the target region, ensuring that the spatial range of the planar constraint is always limited to the effective area of ​​the target photovoltaic panel.

[0071] Furthermore, after constructing an accurate plane-dependent Gaussian set Q... p Then, a plane consistency loss is applied to the plane-correlated Gaussian, the plane consistency loss including at least one of the following: point-to-plane distance loss, attitude consistency loss, normal thickness suppression loss, and in-plane scale loss.

[0072] ① Point-to-plane distance loss, used to limit the normal distance L from the Gaussian center of the plane-correlated Gaussian to the plane of the target photovoltaic panel. dist : ; in, Let ρ be the total number of plane-dependent Gaussians, and ρ(·) be the robust error function (which can be an absolute value function, a square function, or a Smooth L1 function, etc.). This constraint can reduce the normal distance from the center of the Gaussians to the target plane and suppress the drift of the center of the Gaussians in the forward and backward directions of the plane.

[0073] ② Attitude consistency loss, used to ensure that the minimum scale axis direction of the plane-dependent Gaussian is aligned with the plane normal vector direction of the target photovoltaic panel. Let R be the rotation matrix of the i-th Gaussian. i The smallest scale direction unit vector determined by the scale parameter is v. i,min Then the attitude consistency loss L normal for: ; This formula evaluates orientation alignment by calculating the absolute value of the vector dot product. When the smallest scale axis v... i,min When parallel to or opposite to the target plane normal n, As the value approaches 1, the loss approaches zero. This constraint allows the thin axes of the Gaussian to align along the plane normal, while the other two larger-scale axes are mainly distributed within the plane, thereby enhancing the consistency between the Gaussian shape and the planar structure of the photovoltaic panel.

[0074] ③ Normal thickness suppression loss L thin This is used to limit the scale of the plane-dependent Gaussian along the plane normal vector direction to not exceed a preset upper limit of the normal scale: ; in, The equivalent scale of the i-th Gaussian along the normal direction of the target plane (can be calculated by projecting the covariance matrix onto the normal n, or approximated by the local scale component closest to the normal). This represents the upper limit of the permissible normal scale. By limiting the normal scale, excessive expansion of the Gaussian plane in the front-to-back direction on the photovoltaic panel surface can be suppressed, reducing planar thickness and out-of-plane artifacts.

[0075] ④ In-plane scale loss L scale This is used to limit the size of the plane-dependent Gaussian within the plane of the target photovoltaic panel to no more than a preset upper limit for in-plane size: ; in, and These are the equivalent scales of Gauss in the two principal directions in the plane; and This represents the corresponding upper limit of the scale. This constraint prevents a small number of Gaussians from excessively magnifying the in-plane scale and covering a large area, reducing the risk of boundary blurring and local structural distortion.

[0076] By weighting and combining the above four constraints, we obtain the total plane consistency loss L. plane : ( (These are the weights of each planar constraint.) Finally, the planar consistency loss is jointly optimized with the rendering reconstruction loss using dynamic weights, wherein the dynamic weights are zero or less than a preset value in the early stage of training, gradually increase to a preset maximum value in the middle stage of training, and gradually decrease in the later stage of training.

[0077] Let L be the rendering and reconstruction loss of standard 3DGS. render If the current iteration number is a, then the total training loss L total for: ; Where λ(a) is the planar constraint weight dynamically scheduled with the number of training iterations.

[0078] To avoid imposing excessively strong planar constraints in the early stages of training due to the unstable Gaussian parameters, this embodiment employs a phased weight scheduling approach. In the initial warm-up phase, the planar constraint weights are set to zero or a small value, allowing the model to first establish basic color and visibility relationships based on multi-view images. In the constraint enhancement phase, λ(a) is gradually increased, causing the planar correlated Gaussians to converge towards the target plane and form a stable, flat structure. In the mid-term stabilization phase, λ(a) is maintained at a preset maximum value. In the later stages of training, λ(a) is gradually decreased, ensuring that the final reconstruction result is still primarily refined by the consistency of multi-view rendering.

[0079] To reduce additional computational overhead, the planar consistency loss can be calculated at a preset iteration interval, while the standard rendering reconstruction loss is calculated normally in each iteration. The planar consistency loss only applies to the set Q. p The Gaussian is used for calculation, so a planarization constraint is not uniformly applied to all scenes.

[0080] The technical advantages of the above training method are as follows: point-to-plane distance constraints limit the out-of-plane drift of the Gaussian center; attitude consistency constraints ensure that the thin axis of the Gaussian is aligned with the plane normal; normal thickness suppression constraints limit the expansion of the Gaussian along the normal; and in-plane scale constraints prevent excessive expansion of the Gaussian in the direction of the panel surface. Simultaneously, through the initialization, inheritance, pruning synchronization, and dynamic verification of planar labels, these constraints can continuously and selectively apply to the target photovoltaic panel area during the 3DGS densification and pruning process. This reduces the disordered densification, out-of-plane drift, and planar thickening phenomena caused by the Gaussian to compensate for weak texture photometric errors, thereby improving the geometric stability of the photovoltaic panel area, training convergence efficiency, and final reconstruction quality.

[0081] In 3DGS representation, a scene is represented by a set of three-dimensional Gaussian ellipsoids (specifically referred to as planar seed 3D Gaussians, primordial Gaussians, or initial Gaussians, depending on their generation source or stage). Each Gaussian ellipsoid is defined by a set of optimizable parameters, including: a center position parameter for determining the position of the Gaussian center in the world coordinate system; a rotation parameter for determining the orientation of the Gaussian local coordinate system; a scale parameter for determining the scale of the 3D Gaussian ellipsoid along the three axes of the local coordinate system; an opacity parameter for controlling the transparency of the Gaussian during rendering; and spherical harmonic coefficients for encoding viewpoint-related color information. These optimizable parameters together constitute the parameterized representation of the 3D Gaussian ellipsoid (3D Gaussian element), which is iteratively updated during training using a gradient descent algorithm.

[0082] During standard 3DGS training, the system iteratively updates the optimizable parameters constituting the initial Gaussian to make the rendered image approximate the input multi-view photovoltaic scene image. Based on the aforementioned initialization, this invention introduces a plane consistency constraint during the training optimization phase, which, together with the standard rendering reconstruction loss, guides the joint optimization of the aforementioned optimizable parameters. The optimizable parameters include the center position, rotation parameters, scale parameters, opacity, and spherical harmonic coefficients of each initial Gaussian.

[0083] The process of jointly optimizing and outputting the 3D model of the target photovoltaic panel based on the aforementioned planar consistency loss and rendering reconstruction loss specifically includes: First, the total training loss is calculated, and based on this total training loss, the optimizable parameters of each initial Gaussian in the initial Gaussian set are jointly optimized using the backpropagation algorithm. In each training iteration, the system renders the current initial Gaussian set to generate a predicted image corresponding to the viewpoint, and compares the differences between the real pixel values ​​of the multi-view photovoltaic scene image and the pixel values ​​of the predicted image to calculate the standard 3DGS rendering reconstruction loss. This rendering reconstruction loss typically includes L1 loss and D-SSIM loss.

[0084] Subsequently, the planar consistency loss is calculated for the initial Gaussian bearing the planar attribute label according to a preset iteration interval. To reduce computational overhead, this planar consistency loss can be calculated non-iteratively. The rendering reconstruction loss and the planar consistency loss are weighted and summed to obtain the total training loss for this iteration.

[0085] Based on the total training loss, the gradient of the total training loss with respect to the optimizable parameters of each initial Gaussian is calculated using the gradient descent optimizer, and the optimizable parameters are updated through backpropagation, thereby achieving joint optimization of multi-view rendering consistency and planar structure constraints.

[0086] In the joint optimization process, a phased dynamic scheduling strategy is adopted for the weight λ(a) of the plane consistency loss to avoid imposing excessively strong plane constraints in the early stages of training when the optimizable parameters have not yet fully converged, which could lead to the optimization getting trapped in local optima. The phased dynamic scheduling strategy includes: Initial warm-up phase: Set the weight λ(a) of the plane consistency loss to zero or a preset minimum value. At this time, the total training loss is mainly composed of the rendering reconstruction loss, so that the model can prioritize the establishment of the basic color distribution and visibility geometry of the scene. Constraint enhancement phase: As the number of training iterations increases, λ(a) is gradually increased according to a linear or exponential law, guiding the initial Gaussian carrying the planar attribute label to move closer to the plane of the target photovoltaic panel and gradually forming a stable flat structure; Mid-term stabilization stage: λ(a) is maintained at the preset maximum weight value. During this stage, 3DGS's adaptive densification and pruning operations are used to suppress the generation of off-surface noise by utilizing planar constraints and limit the normal thickness expansion during the densification process. In the later refinement stage of training: as the training tends to converge, λ(a) is gradually reduced to relax the rigid constraints of the plane, so that the final reconstruction result can be refined by fitting local textures and small deformations based on the consistency of multi-view photometric intensity.

[0087] When the preset maximum number of iterations is reached or the total training loss meets the convergence condition, the optimization stops, and the optimized initial Gaussian set is output as the three-dimensional model of the target photovoltaic panel.

[0088] Through the aforementioned multi-stage joint optimization, the feature gaps in the weakly textured areas of the target photovoltaic panel are filled, improving the flatness and geometric accuracy of the panel surface. The optimized initial Gaussian set contains multiple initial Gaussians with converged parameters. Each initial Gaussian records its spatial location, flattened ellipsoid shape, color, and texture features, collectively constituting a high-fidelity explicit radiation field of the target photovoltaic panel in three-dimensional space. The system saves and outputs this optimized Gaussian set as a point cloud file (e.g., .ply format) or other three-dimensional structured data formats. The output three-dimensional model exhibits low noise and no obvious thickening artifacts in terms of visual rendering quality, and its underlying geometry maintains a high degree of consistency with the actual physical plane of the target photovoltaic panel. It can be directly applied to subsequent tasks such as photovoltaic panel surface defect inspection, three-dimensional dimension measurement, or scene roaming on a digital twin platform.

[0089] It is understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0090] Based on the same inventive concept, embodiments of the present invention also provide a system for implementing the planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios described above. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more system embodiments provided below can be found in the limitations of the method described above, and will not be repeated here.

[0091] In one exemplary embodiment, such as Figure 2 As shown, a planar prior-guided 3DGS three-dimensional reconstruction system for photovoltaic scenarios is provided, including: image segmentation module 1, construction module 2, coordinate recovery module 3, seed generation module 4, Gaussian merging module 5, and 3DGS training module 6.

[0092] The above system corresponds to the aforementioned method embodiments and can implement the corresponding method steps. Its implementation principle and technical effect are similar, and will not be repeated here.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein, and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios, characterized in that, include: Multiple multi-view photovoltaic scene images of the target photovoltaic panel are acquired and target detection and segmentation are performed one by one to obtain the main region mask of the target photovoltaic panel in each of the multi-view photovoltaic scene images; Based on the mask of each subject region, an approximate two-dimensional geometric observation of the target photovoltaic panel is constructed in each of the multi-view photovoltaic scene images; The true physical dimensions of the target photovoltaic panel and the camera parameter information corresponding to each of the multi-view photovoltaic scene images are obtained. Based on multiple approximate two-dimensional geometric observations, the true physical dimensions, and the camera parameter information, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered. Based on the prior knowledge of the planar structure, regularly distributed planar seed points are generated on the plane of the target photovoltaic panel, and a corresponding planar seed three-dimensional Gaussian is initialized for each of the planar seed points. Based on COLMAP technology, the original Gaussian of the original sparse points is generated on the target photovoltaic panel, and multiple planar seed 3D Gaussians and the original Gaussian are merged as initial Gaussians to obtain an initial Gaussian set; During 3DGS training, a plane consistency constraint is applied to the initial Gaussian set, and joint optimization is performed in conjunction with the rendering reconstruction loss to output the 3D model of the target photovoltaic panel.

2. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 1, characterized in that, The step of constructing an approximate two-dimensional geometric observation of the target photovoltaic panel in each of the multi-view photovoltaic scene images based on the mask of each of the main regions includes: Based on each subject region mask, the subject contour of the target photovoltaic panel in each multi-view photovoltaic scene image is extracted and the minimum bounding rotation rectangle of the subject contour is calculated. The four vertices of the minimum bounding rotation rectangle are used as approximate two-dimensional geometric observations of the corresponding multi-view photovoltaic scene image.

3. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 1, characterized in that, The process of reconstructing the planar structure prior of the target photovoltaic panel in the world coordinate system based on multiple approximate two-dimensional geometric observations, the actual physical dimensions, and camera parameter information includes: For each of the aforementioned multi-view photovoltaic scene images: Based on the actual physical dimensions, four coplanar three-dimensional reference points are established in the local coordinate system of the target photovoltaic panel; Obtain at least four two-dimensional vertices corresponding to the approximate two-dimensional geometric observation of the current multi-view photovoltaic scene image, and sort the four two-dimensional vertices so that they are arranged in a clockwise or counterclockwise direction; The candidate correspondences between the arranged two-dimensional vertices and the three-dimensional reference points that satisfy the cyclic order are enumerated. For each candidate correspondence, the PnP pose estimation algorithm is used to solve the candidate rotation matrix and translation vector from the local coordinate system of the target photovoltaic panel to the camera coordinate system, in combination with the camera parameter information. The candidate rotation matrix and the translation vector are used as a set of candidate poses. For each set of candidate poses, the four three-dimensional reference points are reprojected onto the image plane of the current multi-view photovoltaic scene image to obtain four reprojection points; Calculate the average reprojection error between the four two-dimensional vertices and the four reprojection points; From all the candidate poses, the candidate pose with the smallest average reprojection error and that satisfies the preset positive depth condition is selected as the photovoltaic panel candidate planar pose corresponding to the current multi-view photovoltaic scene image. Based on the candidate planar poses of the photovoltaic panel corresponding to each of the multiple multi-view photovoltaic scene images, and combined with the camera extrinsic parameters recovered by COLMAP, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered.

4. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 3, characterized in that, The process of recovering the prior planar structure of the target photovoltaic panel in the world coordinate system based on the candidate planar poses of the photovoltaic scene images corresponding to each of the multiple views, combined with the camera extrinsic parameters recovered by COLMAP, includes: The quality of the candidate planar pose of the photovoltaic panel corresponding to each of the multi-view photovoltaic scene images is evaluated, and the multi-view photovoltaic scene images that meet the preset quality conditions are determined as effective viewpoints. Based on the camera extrinsic parameters from the world coordinate system to the camera coordinate system recovered by COLMAP, the candidate planar poses of the photovoltaic panels corresponding to each effective viewpoint are transformed to the COLMAP world coordinate system to obtain the poses of the target photovoltaic panels in the world coordinate system under each effective viewpoint. Based on the pose of the target photovoltaic panel in the world coordinate system under each effective viewpoint, determine the plane center, plane normal vector and attitude parameters corresponding to each effective viewpoint; For each effective viewpoint, based on the corresponding photovoltaic panel candidate planar pose, the position of the camera center in the local physical coordinate system of the photovoltaic panel under that effective viewpoint is calculated in reverse. Based on the camera extrinsic parameters recovered by COLMAP, the second camera center position of the camera center in the COLMAP world coordinate system is obtained under each effective viewpoint; For any two effective viewpoints, a scale candidate value is constructed based on the ratio of the distance between the center positions of the first camera and the distance between the center positions of the second camera; For multiple effective viewpoints, multiple scale candidate values ​​are obtained by combining them in pairs. Abnormal scale candidate values ​​are removed from the multiple scale candidate values ​​using the interquartile range method. The median of the remaining scale candidate values ​​is taken as the scale factor. Based on the scale factor, the actual physical dimensions of the target photovoltaic panel are transformed to the scale of the COLMAP world coordinate system; A multi-frame consistency screening is performed on the plane center, plane normal vector and attitude parameters corresponding to multiple effective viewpoints to eliminate effective viewpoints whose differences from other effective viewpoints exceed a preset threshold. The plane center, plane normal vector and attitude parameters corresponding to the effective viewpoints that have passed the multi-frame consistency screening are fused across multiple frames to obtain a unified planar structure prior in the world coordinate system.

5. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 1, characterized in that, The step of generating regularly distributed planar seed points on the plane of the target photovoltaic panel based on the prior knowledge of the planar structure, and initializing a corresponding planar seed 3D Gaussian for each of the planar seed points, includes: The planar center, two mutually orthogonal principal directions, plane width, plane height, plane normal vector, and preset in-plane sampling interval are obtained from the prior information of the planar structure. The number of samples along the two mutually orthogonal principal directions is determined, and planar seed points are generated uniformly distributed in the plane of the target photovoltaic panel. For each of the planar seed points, a planar seed 3D Gaussian is initialized, wherein the Gaussian center of the planar seed 3D Gaussian is set to the position of the planar seed point, and the rotation attitude of the 3D Gaussian is such that two axes of the 3D Gaussian are aligned with the two mutually orthogonal principal directions, and the third axis is aligned with the plane normal vector. Furthermore, the initial scale of the 3D Gaussian along the plane normal vector direction is smaller than the initial scale along the two principal directions, so that the initialized planar seed 3D Gaussian has a flat ellipsoidal shape that fits the surface of the target photovoltaic panel.

6. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 5, characterized in that, The preset in-plane sampling interval is determined based on the average nearest neighbor distance of the original Gaussian points generated on the target photovoltaic panel using COLMAP technology within the planar region of the target photovoltaic panel, and the in-plane sampling interval is not greater than a preset multiple of the average nearest neighbor distance.

7. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 1, characterized in that, The method involves generating a primitive Gaussian set based on COLMAP technology using sparse points on the target photovoltaic panel, and merging multiple seed 3D Gaussians from the planar model with the primitive Gaussian set to obtain an initial Gaussian set, including: For each of the planar seed 3D Gaussians, the nearest neighbor original Gaussian with the closest spatial position is determined in the original Gaussians, and the spatial distance between the planar seed 3D Gaussian and the nearest neighbor original Gaussian is calculated; When the spatial distance is less than the preset deduplication threshold, the position of the nearest original Gaussian is merged and updated with the position of the planar seed 3D Gaussian, or the planar seed 3D Gaussian is not added to the initial Gaussian set. When the spatial distance is greater than or equal to the preset deduplication threshold, the planar seed 3D Gaussian is added as a new initial Gaussian to the initial Gaussian set; At the same time, planar attribute labels are set for all the planar seed 3D Gaussians added to the initial Gaussian set to distinguish them from ordinary Gaussians composed of the original Gaussians.

8. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 7, characterized in that, The preset deduplication threshold is determined based on the average nearest neighbor distance of the original Gaussian within the planar region of the target photovoltaic panel, or based on the in-plane sampling interval used when generating the planar seed point; For the nearest-neighbor original Gaussian, if the spatial distance between it and the plane seed 3D Gaussian is less than the preset deduplication threshold, and the projection position of the nearest-neighbor original Gaussian on the plane of the target photovoltaic panel is within the effective range of the plane setting, then the plane attribute label is also set for the nearest-neighbor original Gaussian.

9. The planar prior-guided 3DGS three-dimensional reconstruction method for photovoltaic scenarios according to claim 1, characterized in that, The process of applying a plane consistency constraint to the initial Gaussian set during 3DGS training includes: Detect whether the initial Gaussians in the initial Gaussian set carry a planar attribute label, and use the initial Gaussians carrying the planar attribute label as planar related Gaussians; A plane consistency loss is applied to the plane-correlated Gaussian, the plane consistency loss comprising at least one of the following: Point-to-plane distance loss is used to limit the normal distance from the Gaussian center of the plane-related Gaussian to the plane of the target photovoltaic panel; An attitude consistency loss is used to align the minimum scale axis direction of the plane-dependent Gaussian with the plane normal vector direction of the target photovoltaic panel. Normal thickness suppression loss is used to limit the scale of the plane-correlated Gaussian along the plane normal vector direction to not exceed a preset upper limit of normal scale; In-plane scale loss is used to limit the scale of the plane-correlated Gaussian within the plane of the target photovoltaic panel to no more than a preset upper limit for in-plane scale.

10. A planar prior-guided 3DGS three-dimensional reconstruction system for photovoltaic scenarios, characterized in that, The system is used to implement the method as described in any one of claims 1-9, comprising: The image segmentation module is used to acquire multiple multi-view photovoltaic scene images of the target photovoltaic panel and perform target detection and segmentation one by one to obtain the main region mask of the target photovoltaic panel in each of the multi-view photovoltaic scene images; A construction module is used to construct an approximate two-dimensional geometric observation of the target photovoltaic panel in each of the multi-view photovoltaic scene images based on each of the main body region masks; The coordinate recovery module is used to obtain the true physical size of the target photovoltaic panel and the camera parameter information corresponding to each of the multi-view photovoltaic scene images. Based on multiple approximate two-dimensional geometric observations, the true physical size, and the camera parameter information, the prior planar structure of the target photovoltaic panel in the world coordinate system is recovered. The seed generation module is used to generate regularly distributed planar seed points on the plane of the target photovoltaic panel based on the prior of the planar structure, and initialize the corresponding planar seed three-dimensional Gaussian for each of the planar seed points; The Gaussian merging module is used to generate the original Gaussian of the original sparse points on the target photovoltaic panel based on COLMAP technology, and merge the multiple planar seed 3D Gaussians with the original Gaussians as the initial Gaussians to obtain the initial Gaussian set. The 3DGS training module is used to apply a plane consistency constraint to the initial Gaussian set during the 3DGS training process, and perform joint optimization in conjunction with the rendering reconstruction loss to output the 3D model of the target photovoltaic panel.

Images