A multi-source data fusion photovoltaic power station digital modeling method

Abstract

The application discloses a kind of multi-source data fusion's photovoltaic power station digital modeling method, comprising the following steps: according to the design document of photovoltaic power station, the topological structure and number of photovoltaic module in photovoltaic power station are extracted;Unmanned aerial vehicle is carried laser radar and survey camera, collects the three-dimensional point cloud and visible light image of photovoltaic power station;Combining photovoltaic module features and semantic segmentation model, identify photovoltaic module in visible light image;Combining three-dimensional coordinates and image matrix, the three-dimensional point cloud corresponding to the surface of each photovoltaic module is extracted;According to the three-dimensional point cloud corresponding to photovoltaic module surface, construct the three-dimensional model of photovoltaic module;The multi-source data of photovoltaic module is fused, and the digital model of photovoltaic power station is constructed.The application can be combined with unmanned aerial vehicle path planning technology, realize the intelligent inspection of photovoltaic power station;Combined with fault diagnosis technology and panorama splicing technology, realize photovoltaic module fault positioning and visualization, provide important basis for the wisdom operation of photovoltaic power station.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of multi-source data fusion's photovoltaic power station digital modeling technique, belong to the three-dimensional reconstruction and intelligent inspection technical field of photovoltaic system. BACKGROUND

[0002] Traditional photovoltaic operation and maintenance has three major common industry problems of low efficiency, great difficulty and high cost. Since artificial inspection consumes a lot of time and labor, spot inspection has strong randomness and poor pertinence. At present, the operation and maintenance level of photovoltaic power station is generally insufficient. By mounting a pan-tilt head on a drone, visible light images and infrared thermal images of photovoltaic modules are collected to realize fault diagnosis and positioning analysis of photovoltaic modules, which helps to improve the operation and maintenance level of photovoltaic power station. Intelligent inspection and fault diagnosis of photovoltaic power station need to abstract photovoltaic power station in three-dimensional space into a digital model. The three-dimensional model of photovoltaic power station can provide an important basis for drone path planning and image collection; the planar model of photovoltaic power station can realize fault positioning and visualization of photovoltaic modules by combining fault diagnosis method and panoramic stitching method. Therefore, digital modeling of photovoltaic power station is crucial.

[0003] The prior art has studied photovoltaic power station three-dimensional modeling method based on point cloud, and photovoltaic power station planar modeling method combining image and GPS information. However, photovoltaic modules in photovoltaic array are usually closely arranged, making it difficult to separate the three-dimensional point cloud of each photovoltaic module surface. The photovoltaic power station three-dimensional modeling method based on point cloud can usually only model photovoltaic array, but not photovoltaic modules. The photovoltaic power station planar modeling method based on image and GPS information loses three-dimensional information, making it difficult to obtain the accurate position of photovoltaic modules in three-dimensional coordinate system. The prior art does not fully utilize the advantages of multi-source data, and has not accurately realized the three-dimensional reconstruction of photovoltaic power station. How to combine design documents, three-dimensional point cloud and visible light images to fully extract effective features from multi-source data is an important problem to be considered in digital modeling of photovoltaic power station. SUMMARY

[0004] Technical problem: The present application aims to solve the deficiencies in the prior art, and combines the design documents, three-dimensional point cloud and visible light images of photovoltaic power station to match the topological structure, three-dimensional model, image matrix and number information of photovoltaic modules, and provides a multi-source data fusion photovoltaic power station digital modeling method.

[0005] Technical solution: A multi-source data fusion photovoltaic power station digital modeling method of the present application comprises the following steps in sequence:

[0006] S1: According to the design documents of photovoltaic power station, the topological structure and number of photovoltaic modules in photovoltaic power station are extracted;

[0007] S2: The unmanned aerial vehicle carries a laser radar and a survey camera to collect three-dimensional point clouds and visible light images of the photovoltaic power station;

[0008] S3: The photovoltaic module features and the semantic segmentation model are combined to identify the photovoltaic module in the visible light image;

[0009] S4: The three-dimensional point clouds corresponding to the surface of each photovoltaic module are extracted in combination with the three-dimensional coordinates and the image matrix;

[0010] S5: The three-dimensional model of the photovoltaic module is constructed according to the three-dimensional point clouds corresponding to the surface of the photovoltaic module;

[0011] S6: The multi-source data of the photovoltaic module is fused to construct a digital model of the photovoltaic power station.

[0012] Further, in the step S1, the topological structure of the photovoltaic module is represented by an undirected graph, each photovoltaic module is taken as a node of the graph, and the number of the photovoltaic module is taken as the number of the node; if two photovoltaic modules are adjacent, there is an edge between the corresponding two nodes; if two photovoltaic modules are not adjacent, there is no edge between the corresponding two nodes; the undirected graph is divided into several connected subgraphs, and the positional relationship between the subgraphs is recorded.

[0013] Further, in the step S2, the laser radar and the survey camera are located in the same gimbal, and the relative position is kept unchanged; the yaw angle of the gimbal is 0°, the roll angle is 0°, and the pitch angle is -90°; the three-dimensional point clouds are collected by a non-repeating scanning mode, and at least 10 points are collected in any 10x10 cm 2 region on the surface of the photovoltaic module; the visible light images are collected by an equal time interval shooting mode, and any photovoltaic module is at least completely present in one visible light image.

[0014] Further, in the step S3, W represents the number of visible light images, G h represents the image matrix obtained after the radial distortion of the hth visible light image is corrected, where h=1, 2, …, W; M h represents a perspective transformation matrix, I h represents the image matrix obtained after the perspective deformation of G h is corrected, I h =M h ·G h ; the surface of the photovoltaic module is rectangular, the silicon cell presents a deep blue color, the aluminum alloy frame presents a silver-white color, the silicon cell surface has mutually perpendicular main grid lines and fine grid lines, and there are connecting lines between the silicon cells; in combination with the geometric features, color features, and texture features of the photovoltaic module and the semantic segmentation model, the complete photovoltaic module is identified in I h , and the image matrix of the photovoltaic module is extracted.

[0015] Further, in the step S4, based on the three-dimensional point cloud of the photovoltaic power station, the three-dimensional point cloud corresponding to the surface of all photovoltaic modules is extracted as Π; (p1, p2, p3) T represents the three-dimensional coordinates of a point θ in Π, and θ is matched with the e-th visible light image, where l∈{1, 2, …, W}; θ corresponds to G l in the coordinates ψ=(g1, g2) T , G l represents the image matrix obtained after correcting the radial distortion of the l-th visible light image; τ=(t1, t2, t3) T represents the position of the camera in the three-dimensional coordinate system when the l-th visible light image is taken; g1 and g2 are solved according to the following coordinate transformation formula:

[0016]

[0017] where f1, f2, e1, e2 are known camera intrinsic parameters; λ is an unknown variable.

[0018] Further, in the step S4, the coordinates Φ=M l ·ψ of θ in I l are calculated, and I l represents the image matrix obtained after correcting the perspective distortion of G l ; d j represents the distance from Φ to the center of the j-th photovoltaic module in I l , Ω represents the value range of j, and θ is solved as θ is matched with the corresponding photovoltaic module image matrix; N represents the number of photovoltaic modules in the photovoltaic power station, and Π k represents the three-dimensional point cloud corresponding to the surface of the k-th photovoltaic module, where k=1, 2, …, N; combining the three-dimensional coordinates of each point in Π with the corresponding image matrix, Π is divided into Π1, Π2, …, Π N .

[0019] Further, in the step S5, A k ·x+B k ·y+C k ·z+D k =0 represents the plane equation corresponding to the surface of the k-th photovoltaic module, and each point in Π k is substituted into the plane equation to obtain an over-determined equation group; the over-determined equation group is solved to obtain the unit normal vector n k =(A k ,B k ,C k ) T of the surface of the k-th photovoltaic module; the length of the photovoltaic module is L, the width is W, and the height is H; the center of the k-th photovoltaic module is O k , and the unit vector uk =(α k ,β k ,γ k ) T Parallel to the long side of the kth photovoltaic module, unit vector v k =n k ×u k Normal vector perpendicular to the surface and long side of the kth photovoltaic module.

[0020] Furthermore, in step S5, m represents Π k The number of three-dimensional points in R i =(x i ,y i ,z i ) T Represents π k For the i-th point in , calculate R i The boundary coefficient δ i and σ i as follows:

[0021]

[0022] Where i = 1, 2, ..., m; when δ i >0 or σ i >0, R i Beyond the boundary of the kth PV module.

[0023] Furthermore, in step S5, the boundary optimization target F is as follows:

[0024]

[0025] In satisfying α k 2 +β k 2 +γ k 2 =1, solve O k , α k , β k , γ k , let the boundary optimization target F obtain the minimum value; with O k As the center, with u k , v k , n k As the direction vectors of length, width and height respectively, a three-dimensional model of the k-th photovoltaic module is constructed.

[0026] Further, in the step S6, the image matrix of the photovoltaic module is matched with the three-dimensional model according to the GPS information and the camera parameter of the visible light image; the three-dimensional model is matched with the design document according to the topological structure of the photovoltaic module, and the three-dimensional model of the photovoltaic module is automatically numbered; the image matrix, the three-dimensional model, the topological structure and the numbering information of the photovoltaic module are fused as the digital model of the photovoltaic power station.

[0027] Beneficial effects: the application provides a multi-source data fusion photovoltaic power station digital modeling method, which combines the design document, the three-dimensional point cloud and the visible light image of the photovoltaic power station, matches the topological structure, the three-dimensional model, the image matrix and the numbering information of the photovoltaic module, and generates the digital model of the photovoltaic power station. The method can realize intelligent inspection of the photovoltaic power station by combining the unmanned aerial vehicle path planning technology, realizes photovoltaic module fault positioning and visualization by combining the fault diagnosis technology and the panoramic splicing technology, and provides an important basis for intelligent operation and maintenance of the photovoltaic power station. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 It is a whole flowchart of the embodiment;

[0029] Figure 2 It is a three-dimensional point cloud diagram of the photovoltaic power station;

[0030] Figure 3 It is a visible light image diagram of the photovoltaic power station;

[0031] Figure 4 It is a topological structure diagram of the photovoltaic module;

[0032] Figure 5 It is a matching diagram of the image matrix and the three-dimensional point cloud of the photovoltaic module, wherein (a) is the image matrix of the photovoltaic module; (b) is the corresponding three-dimensional point cloud on the surface of the photovoltaic module. DETAILED DESCRIPTION

[0033] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings, but the protection scope of the application is not limited to the embodiments. As shown in the drawings, the multi-source data fusion photovoltaic power station digital modeling method of the embodiment comprises the following steps in sequence: Figure 1

[0034] S1: according to the design document of the photovoltaic power station, the topological structure and the numbering of the photovoltaic module in the photovoltaic power station are extracted.

[0035] ​In this embodiment, an undirected graph is used to represent the topological structure of photovoltaic modules. Each photovoltaic module is used as a node of the graph, and the number of the photovoltaic module is used as the node number. If two photovoltaic modules are adjacent, there is an edge between the two corresponding nodes; if two photovoltaic modules are not adjacent, there is no edge between the two corresponding nodes. The undirected graph is divided into several connected subgraphs, and the positional relationship between the subgraphs is recorded.

[0036] S2: The drone is equipped with a lidar and a mapping camera to collect 3D point clouds and visible light images of the photovoltaic power station.

[0037] In this embodiment, the laser radar and the mapping camera are located on the same gimbal, and their relative positions remain unchanged; the gimbal's yaw angle is 0°, the roll angle is 0°, and the pitch angle is -90°; the 3D point cloud is collected by non-repeating scanning mode, and any 10×10 cm on the surface of the photovoltaic module is scanned. 2 At least 10 points are collected in the area; visible light images are collected by taking pictures at equal time intervals, and any photovoltaic module appears completely in at least one visible light image. In this embodiment, Figure 2 This is a three-dimensional point cloud diagram of a photovoltaic power station; Figure 3 Schematic diagram of visible light image of a photovoltaic power station.

[0038] S3: Combine photovoltaic module features with semantic segmentation models to identify photovoltaic modules in visible light images.

[0039] In this embodiment, W represents the number of visible light images, G h represents the image matrix obtained after the radial distortion of the h-th visible light image is corrected, where h = 1, 2, ..., W; M h represents the perspective transformation matrix, I h Represents G h The image matrix obtained after correcting the perspective deformation, I h =M h ·G h The surface of the photovoltaic module is rectangular, the silicon cell is dark blue, the aluminum alloy frame is silvery white, there are main grid lines and fine grid lines perpendicular to each other on the surface of the silicon cell, and there are connecting lines between the silicon cells; Combining the geometric features, color features, texture features of the photovoltaic module, and the semantic segmentation model, in I h The complete photovoltaic module is identified and the image matrix of the photovoltaic module is extracted.

[0040] S4: Combine the three-dimensional coordinates with the image matrix to extract the three-dimensional point cloud corresponding to the surface of each photovoltaic module.

[0041] In this embodiment, based on the three-dimensional point cloud of the photovoltaic power station, the three-dimensional point cloud Π corresponding to the surface of all photovoltaic modules is extracted; (p1, p2, p3) Tdenotes the three-dimensional coordinates of a point θ in Π, which is matched with the lth visible light image, where l ∈ {1, 2, …, W}; θ corresponds to G l denotes the coordinates ψ = (g1, g2) in G T , G l denotes the image matrix obtained after correcting the radial distortion of the lth visible light image; τ = (t1, t2, t3) T denotes the position of the camera in the three-dimensional coordinate system when the lth visible light image is taken; g1 and g2 are solved according to the following coordinate conversion formula:

[0042]

[0043] where f1, f2, e1, e2 are known camera intrinsic parameters; λ is an unknown variable.

[0044] The coordinates Φ = M l · ψ in I l are calculated, where I l denotes the image matrix obtained after correcting the perspective distortion of G l ; d j denotes the distance from Φ to the center of the jth photovoltaic component in I l , Ω represents the value range of j, and is solved θ is matched with the corresponding photovoltaic component image matrix; N represents the number of photovoltaic components in the photovoltaic power station, and Π k denotes the three-dimensional point cloud corresponding to the surface of the kth photovoltaic component, where k = 1, 2, …, N; combined with the three-dimensional coordinates of each point in Π and the corresponding image matrix, Π is divided into Π1, Π2, …, Π N .

[0045] S5: According to the three-dimensional point cloud corresponding to the surface of the photovoltaic component, a three-dimensional model of the photovoltaic component is constructed.

[0046] In this embodiment, A k · x + B k · y + C k · z + D k = 0 represents the plane equation corresponding to the surface of the kth photovoltaic component, and each point in Π k is substituted into the plane equation to obtain an overdetermined equation group; the overdetermined equation group is solved to obtain the unit normal vector n k = (A k , B k , C k ) T of the surface of the kth photovoltaic component; the length of the photovoltaic component is L, the width is W, and the height is H; the center of the kth photovoltaic component is O k , and the unit vector u k = (α k , βk ,γ k ) T Parallel to the long side of the kth photovoltaic module, unit vector v k =n k ×u k Normal vector perpendicular to the surface and long side of the kth photovoltaic module.

[0047] m represents π k The number of three-dimensional points in R i =(x i ,y i ,z i ) T Represents π k For the i-th point in , calculate R i The boundary coefficient δ i and σ i as follows:

[0048]

[0049] Where i = 1, 2, ..., m; when δ i >0 or σ i >0, R i Beyond the boundary of the kth PV module.

[0050] The boundary optimization objective F is as follows:

[0051]

[0052] In satisfying α k 2 +β k 2 +γ k 2 =1, solve O k , α k , β k , γ k , let the boundary optimization target F obtain the minimum value; with O k As the center, with u k , v k , n k As the direction vectors of length, width and height respectively, a three-dimensional model of the k-th photovoltaic module is constructed.

[0053] S6: Integrate multi-source data of photovoltaic modules to build a digital model of the photovoltaic power station.

[0054] In the embodiment, the image matrix of the photovoltaic module is matched with the three-dimensional model according to the GPS information and the camera parameters of the visible light image; the three-dimensional model is matched with the design document according to the topological structure of the photovoltaic module, and the three-dimensional model of the photovoltaic module is automatically numbered; the image matrix, the three-dimensional model, the topological structure and the numbering information of the photovoltaic module are fused to serve as the digital model of the photovoltaic power station. Figure 4 FIG. 4 is a schematic diagram of the topological structure of the photovoltaic module, wherein each node represents a photovoltaic module, and an edge between two nodes represents that the two photovoltaic modules are adjacent. Figure 5 FIG. 5 is a schematic diagram of the matching of the image matrix and the three-dimensional point cloud of the photovoltaic module, wherein (a) is the image matrix of the photovoltaic module; and (b) is the three-dimensional point cloud corresponding to the surface of the photovoltaic module.

[0055] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application but not to limit it, although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A method for digital modeling of a multi-source data fusion photovoltaic power station, characterized in that: The steps include the following in sequence: S1: According to the design document of the photovoltaic power station, the topological structure and number of the photovoltaic module in the photovoltaic power station are extracted; S2: The unmanned aerial vehicle is equipped with a laser radar and a survey camera, and three-dimensional point clouds and visible light images of the photovoltaic power station are collected; S3: The photovoltaic module in the visible light image is identified in combination with the photovoltaic module features and the semantic segmentation model; S4: In combination with the three-dimensional coordinates and the image matrix, the three-dimensional point cloud corresponding to the surface of each photovoltaic module is extracted; S5: According to the three-dimensional point cloud corresponding to the surface of the photovoltaic module, a three-dimensional model of the photovoltaic module is constructed; S6: The multi-source data of the photovoltaic module is fused to construct a digital model of the photovoltaic power station; In step S5, A k ·x+B k ·y+C k ·z+D k = 0 represents the plane equation corresponding to the surface of the kth photovoltaic module, and each point in Π k is substituted into the plane equation respectively to obtain an over-determined equation group, where Π k represents the three-dimensional point cloud corresponding to the surface of the kth photovoltaic module; the over-determined equation group is solved to obtain the unit normal vector n k =(A k ,B k ,C k ) T of the surface of the kth photovoltaic module; the length of the photovoltaic module is L, the width is W, and the height is H; the center of the kth photovoltaic module is O k , the unit vector u k =(α k ,β k ,γ k ) T is parallel to the long side of the kth photovoltaic module, and the unit vector v k =n k ×u k is perpendicular to the normal vector of the surface of the kth photovoltaic module and the long side. m denotes the number of three-dimensional points in Π k R i = (x i ,y i ,z i ) T denotes the i-th point in Π k The boundary coefficients δ i and σ i for R i are computed as follows: wherein i = 1, 2,..., m; when δ i > 0 or σ i > 0, R i exceeds the boundary of the kth photovoltaic component; The boundary optimization objective F is as follows: Under the constraint condition of α k 2 + β k 2 + γ k 2 = 1, solve O k , α k , β k , γ k , so that the boundary optimization target F obtains the minimum value; take O k as the center, and take u k , v k , n k as the direction vectors of length, width and height respectively to construct the three-dimensional model of the kth photovoltaic component.

2. The photovoltaic power station digital modeling method of claim 1, wherein: In the step S1, the topological structure of the photovoltaic module is represented by an undirected graph, each photovoltaic module is taken as a node of the graph, and the number of the photovoltaic module is taken as the number of the node; if two photovoltaic modules are adjacent, there is an edge between the corresponding two nodes; if two photovoltaic modules are not adjacent, there is no edge between the corresponding two nodes; the undirected graph is divided into several connected subgraphs, and the positional relationship between the subgraphs is recorded.

3. The photovoltaic power station digital modeling method of claim 2, wherein: In the step S2, the laser radar and the mapping camera are located in the same holder, and the relative position is kept unchanged; the yaw angle of the holder is 0°, the roll angle is 0°, and the pitch angle is -90°; the three-dimensional point cloud is collected through the non-repeated scanning mode, and at least 10*10 cm 2 areas of the photovoltaic assembly are collected; the visible light image is collected through the equal time interval photographing mode, and the photovoltaic assembly appears completely in at least one visible light image.

4. The photovoltaic power station digital modeling method of claim 3, wherein: S3, W represents the number of visible light images, represents the first image matrix obtained after correcting the radial distortion of the visible light images, represents the perspective transformation matrix, represents image matrix obtained after correcting the perspective distortion, The surface of the photovoltaic module is rectangular, the silicon cell presents a deep blue color, the aluminum alloy frame presents a silver-white color, the silicon cell surface has mutually perpendicular main grid lines and fine grid lines, and there are connecting lines between the silicon cells; combined with the geometric features, color features and texture features of the photovoltaic module, and the semantic segmentation model, the complete photovoltaic module is identified in , and the image matrix of the photovoltaic module is extracted.

5. The photovoltaic power station digital modeling method of claim 4, wherein: In the step S4, based on the three-dimensional point cloud of the photovoltaic power station, the three-dimensional point cloud corresponding to the surface of all photovoltaic modules is extracted Π; (p1,p2,p3) T Represents the three-dimensional coordinates of a point θ in π, and compares θ with the first The visible light images are matched, where θ corresponds to The coordinates in ψ=(g1,g2) T , Indicates the The image matrix obtained after the radial distortion of the visible light image is corrected; τ = (t1, t2, t3) T Indicates the shooting When taking a visible light image, the camera's position in the three-dimensional coordinate system is calculated; according to the following coordinate transformation formula, g1 and g2 are solved: Where f1, f2, e1, e2 are known camera intrinsic parameters; λ is an unknown variable.

6. The photovoltaic power station digital modeling method of claim 5, wherein: In step S4, the calculation of θ corresponds to Coordinates in express The image matrix obtained after correcting the perspective deformation; d j Indicates Φ to The distance to the center of the jth photovoltaic module in , Ω represents the value range of j, solve Match θ with the corresponding PV module image matrix; N represents the number of PV modules in the PV power station, Π k Represents the three-dimensional point cloud corresponding to the surface of the k-th photovoltaic module, where k = 1, 2, ..., N; combining the three-dimensional coordinates of each point in Π with the corresponding image matrix, divide Π into Π1, Π2, ..., Π N .

7. The photovoltaic power station digital modeling method of claim 6, wherein: In the step S6, according to the GPS information and camera parameters of the visible light image, the image matrix and the three-dimensional model of the photovoltaic module are matched; according to the topological structure of the photovoltaic module, the three-dimensional model and the design document are matched, and the three-dimensional model of the photovoltaic module is automatically numbered; the image matrix, the three-dimensional model, the topological structure and the number information of the photovoltaic module are fused to serve as the digital model of the photovoltaic power station.

Images