Multi-source data fusion photovoltaic power station digital modeling method

Abstract

The invention discloses a multi-source data fusion photovoltaic power station digital modeling method, which comprises the following steps of extracting topological structures and serial numbers of photovoltaic modules in a photovoltaic power station according to a design document of the photovoltaic power station; the unmanned aerial vehicle carries a laser radar and a surveying and mapping camera and collects three-dimensional point cloud and visible light images of the photovoltaic power station; identifying the photovoltaic module in the visible light image by combining the photovoltaic module characteristics and the semantic segmentation model; in combination with the three-dimensional coordinates and the image matrix, extracting three-dimensional point clouds corresponding to the surfaces of the photovoltaic modules; constructing a three-dimensional model of the photovoltaic module according to the three-dimensional point cloud corresponding to the surface of the photovoltaic module; and fusing the multi-source data of the photovoltaic module, and constructing a digital model of the photovoltaic power station. According to the invention, an unmanned aerial vehicle path planning technology can be combined to realize intelligent inspection of the photovoltaic power station; in combination with a fault diagnosis technology and a panoramic splicing technology, fault positioning and visualization of the photovoltaic module are realized, and an important basis is provided for intelligent operation and maintenance of a photovoltaic power station.

Description

Technical Field

[0001] The present invention relates to a multi-source data fusion digital modeling technology for a photovoltaic power station, belonging to the technical field of three-dimensional reconstruction and intelligent inspection of photovoltaic systems. Background Art

[0002] Traditional photovoltaic operation and maintenance has three common industry problems: low efficiency, high difficulty, and high cost. Since manual inspections consume a lot of time and labor, and spot inspections are highly random and poorly targeted, the current operation and maintenance level of photovoltaic power stations is generally insufficient. By using drones equipped with gimbals to collect visible light images and infrared thermal imaging of photovoltaic components, fault diagnosis and positioning analysis of photovoltaic components can be achieved, which will help improve the operation and maintenance level of photovoltaic power stations. Intelligent inspection and fault diagnosis of photovoltaic power stations require the photovoltaic power station in three-dimensional space to be abstracted into a digital model. The three-dimensional model of the photovoltaic power station can provide an important basis for drone path planning and image acquisition; the plane model of the photovoltaic power station can combine fault diagnosis methods with panoramic stitching methods to achieve photovoltaic component fault positioning and visualization. Therefore, digital modeling of photovoltaic power stations is crucial.

[0003] The prior art has studied the three-dimensional modeling method of photovoltaic power stations based on point clouds, and the planar modeling method of photovoltaic power stations combining images and GPS information. However, the photovoltaic modules in the photovoltaic array are usually arranged closely, which makes it difficult to separate the three-dimensional point clouds on the surface of each photovoltaic module. The three-dimensional modeling method of photovoltaic power stations based on point clouds can usually only achieve modeling for photovoltaic arrays, but it is difficult to achieve modeling for photovoltaic modules; the planar modeling method of photovoltaic power stations based on images and GPS information loses the three-dimensional information, and it is difficult to obtain the accurate position of photovoltaic modules in the three-dimensional coordinate system. The prior art has not fully utilized the advantages of multi-source data, and has not yet accurately realized the three-dimensional reconstruction of photovoltaic power stations. How to combine design documents, three-dimensional point clouds and visible light images, and fully extract effective features from multi-source data is an important issue that needs to be considered in the digital modeling of photovoltaic power stations. Summary of the invention

[0004] Technical problem: The present invention aims to solve the deficiencies in the prior art by combining the design documents, three-dimensional point clouds and visible light images of photovoltaic power stations, matching the topological structure, three-dimensional model, image matrix and numbering information of photovoltaic modules, and providing a digital modeling method for photovoltaic power stations with multi-source data fusion.

[0005] Technical solution: A digital modeling method for a photovoltaic power station by multi-source data fusion of the present invention comprises the following steps in sequence:

[0006] S1: Extract the topological structure and numbering of photovoltaic modules in the photovoltaic power station according to the design documents of the photovoltaic power station;

[0007] S2: The drone is equipped with a laser radar and a mapping camera to collect 3D point clouds and visible light images of the photovoltaic power station;

[0008] S3: Combine photovoltaic module features with semantic segmentation models to identify photovoltaic modules in visible light images;

[0009] S4: Combining the three-dimensional coordinates with the image matrix, extracting the three-dimensional point cloud corresponding to the surface of each photovoltaic module;

[0010] S5: construct a three-dimensional model of the photovoltaic module according to the three-dimensional point cloud corresponding to the surface of the photovoltaic module;

[0011] S6: Integrate multi-source data of PV modules to build a digital model of the PV power station.

[0012] Furthermore, in step S1, an undirected graph is used to represent the topological structure of the photovoltaic module, each photovoltaic module is used as a node of the graph, and the number of the photovoltaic module is used as the number of the node; if two photovoltaic modules are adjacent, there is an edge between the corresponding two nodes; if the two photovoltaic modules are not adjacent, there is no edge between the corresponding two nodes; the undirected graph is divided into a number of connected subgraphs, and the positional relationship between the subgraphs is recorded.

[0013] Furthermore, in step S2, the laser radar and the mapping camera are located on the same gimbal, and their relative positions remain unchanged; the yaw angle of the gimbal is 0°, the roll angle is 0°, and the pitch angle is -90°; the three-dimensional point cloud is collected by non-repetitive scanning mode, and any 10×10 cm 2 At least 10 points are collected in the area; visible light images are collected by taking photos at equal time intervals, and any photovoltaic module appears completely in at least one visible light image.

[0014] Furthermore, in step S3, W represents the number of visible light images, G h represents the image matrix obtained after the radial distortion of the hth visible light image is corrected, where h = 1, 2, ..., W; M h represents the perspective transformation matrix, I h Represents G h The image matrix obtained after correcting the perspective deformation, I h =M h ·G h The surface of the photovoltaic module is rectangular, the silicon cell is dark blue, the aluminum alloy frame is silvery white, there are main grid lines and thin grid lines perpendicular to each other on the surface of the silicon cell, and there are connecting lines between the silicon cells; Combining the geometric features, color features, texture features of the photovoltaic module, and the semantic segmentation model, in I h The complete photovoltaic module is identified and the image matrix of the photovoltaic module is extracted.

[0015] Furthermore, in step S4, based on the three-dimensional point cloud of the photovoltaic power station, the three-dimensional point cloud Π corresponding to the surface of all photovoltaic modules is extracted; (p1, p2, p3) T Represents the three-dimensional coordinates of a point θ in Π, and matches θ with the e-th visible light image, where l∈{1, 2, …, W}; θ corresponds to G l The coordinates in ψ=(g1,g2) T , G l represents the image matrix obtained after the radial distortion of the lth visible light image is corrected; τ = (t1, t2, t3) T represents the position of the camera in the three-dimensional coordinate system when the lth visible light image is taken; solve g1 and g2 according to the following coordinate transformation formula:

[0016]

[0017] Among them, f1, f2, e1, e2 are known camera intrinsic parameters; λ is an unknown variable.

[0018] Furthermore, in step S4, the corresponding I of θ is calculated. l The coordinate Φ in the l ·ψ, I l Represents G l The image matrix obtained after correcting the perspective deformation; d j Indicates Φ to I l The distance from the center of the jth photovoltaic module in , Ω represents the value range of j, solve Match θ with the corresponding PV module image matrix; N represents the number of PV modules in the PV power station, Π k Represents the 3D point cloud corresponding to the surface of the kth photovoltaic module, where k = 1, 2, ..., N; combining the 3D coordinates of each point in Π with the corresponding image matrix, Π is divided into Π1, Π2, ..., Π N .

[0019] Furthermore, in step S5, A k ·x+B k y+C k ·z+D k =0 represents the plane equation corresponding to the surface of the kth photovoltaic module, and Π k Substitute each point in into the plane equation to obtain an overdetermined system of equations; solve the overdetermined system of equations to obtain the unit normal vector n of the kth photovoltaic module surface k =(A k ,B k ,C k ) T ; The length of the photovoltaic module is L, the width is W, and the height is H; the center of the kth photovoltaic module is O k , the unit vector uk =(α k ,β k ,γ k ) T Parallel to the long side of the kth PV module, unit vector v k =n k ×u k The normal vector perpendicular to the surface and long side of the kth PV module.

[0020] Further, in step S5, m represents Π k The number of three-dimensional points in R i =(x i ,y i ,z i ) T Represents Π k For the i-th point in i The boundary coefficient δ i With σ i as follows:

[0021]

[0022] Where i = 1, 2, ..., m; when δ i >0 or σ i >0, R i Beyond the boundary of the kth PV module.

[0023] Furthermore, in step S5, the boundary optimization target F is as follows:

[0024]

[0025] In satisfying α k 2 +β k 2 +γ k 2 = 1, solve O k , α k , β k , γ k , let the boundary optimization objective F obtain the minimum value; k Centered on u k , v k , n k They are used as the direction vectors of length, width and height respectively to construct the three-dimensional model of the kth photovoltaic module.

[0026] Furthermore, in step S6, the image matrix of the photovoltaic module is matched with the three-dimensional model according to the GPS information and camera parameters of the visible light image; the three-dimensional model is matched with the design document according to the topological structure of the photovoltaic module, and the three-dimensional model of the photovoltaic module is automatically numbered; the image matrix, three-dimensional model, topological structure, and numbering information of the photovoltaic module are integrated as a digital model of the photovoltaic power station.

[0027] Beneficial effects: The present invention provides a digital modeling method for photovoltaic power stations with multi-source data fusion, which combines the design documents, three-dimensional point clouds and visible light images of photovoltaic power stations, matches the topological structure, three-dimensional model, image matrix and numbering information of photovoltaic modules, and generates a digital model of photovoltaic power stations. This method can be combined with drone path planning technology to realize intelligent inspection of photovoltaic power stations; combined with fault diagnosis technology and panoramic stitching technology, it can realize the location and visualization of photovoltaic module faults, providing an important basis for the intelligent operation and maintenance of photovoltaic power stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall process of the embodiment;

[0029] Figure 2 This is a three-dimensional point cloud schematic diagram of a photovoltaic power station;

[0030] Figure 3 This is a schematic diagram of a visible light image of a photovoltaic power station;

[0031] Figure 4 Schematic diagram of the topological structure of photovoltaic modules;

[0032] Figure 5 Schematic diagram of matching the image matrix and three-dimensional point cloud of a photovoltaic module, where (a) is the image matrix of the photovoltaic module; (b) is the three-dimensional point cloud corresponding to the surface of the photovoltaic module. DETAILED DESCRIPTION

[0033] The specific implementation modes of the present invention will be further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments. Figure 1 As shown, a digital modeling method for a photovoltaic power station by multi-source data fusion in this embodiment includes the following steps in sequence:

[0034] S1: Extract the topological structure and numbering of photovoltaic modules in the photovoltaic power station according to the design documents of the photovoltaic power station.

[0035] In this embodiment, an undirected graph is used to represent the topological structure of the photovoltaic module, each photovoltaic module is used as a node of the graph, and the number of the photovoltaic module is used as the number of the node; if two photovoltaic modules are adjacent, there is an edge between the corresponding two nodes; if the two photovoltaic modules are not adjacent, there is no edge between the corresponding two nodes; the undirected graph is divided into a number of connected subgraphs, and the positional relationship between the subgraphs is recorded.

[0036] S2: The drone is equipped with a lidar and a mapping camera to collect 3D point clouds and visible light images of the photovoltaic power station.

[0037] In this embodiment, the laser radar and the mapping camera are located on the same gimbal, and their relative positions remain unchanged; the yaw angle of the gimbal is 0°, the roll angle is 0°, and the pitch angle is -90°; the three-dimensional point cloud is collected by non-repetitive scanning mode, and any 10×10cm on the surface of the photovoltaic module 2 At least 10 points are collected in the area; visible light images are collected by taking pictures at equal time intervals, and any photovoltaic module appears completely in at least one visible light image. In this embodiment, Figure 2 This is a three-dimensional point cloud schematic diagram of a photovoltaic power station; Figure 3 Schematic diagram of visible light image of a photovoltaic power station.

[0038] S3: Combine PV module features with the semantic segmentation model to identify PV modules in visible light images.

[0039] In this embodiment, W represents the number of visible light images, G h represents the image matrix obtained after the radial distortion of the hth visible light image is corrected, where h = 1, 2, ..., W; M h represents the perspective transformation matrix, I h Represents G h The image matrix obtained after correcting the perspective deformation, I h =M h ·G h The surface of the photovoltaic module is rectangular, the silicon cell is dark blue, the aluminum alloy frame is silvery white, there are main grid lines and thin grid lines perpendicular to each other on the surface of the silicon cell, and there are connecting lines between the silicon cells; Combining the geometric features, color features, texture features of the photovoltaic module, and the semantic segmentation model, in I h The complete photovoltaic module is identified and the image matrix of the photovoltaic module is extracted.

[0040] S4: Combine the three-dimensional coordinates with the image matrix to extract the three-dimensional point cloud corresponding to the surface of each photovoltaic module.

[0041] In this embodiment, based on the three-dimensional point cloud of the photovoltaic power station, the three-dimensional point cloud Π corresponding to the surface of all photovoltaic modules is extracted; (p1, p2, p3) TRepresents the three-dimensional coordinates of a point θ in Π, and matches θ with the lth visible light image, where l∈{1, 2, …, W}; θ corresponds to G l The coordinates in ψ=(g1,g2) T , G l represents the image matrix obtained after the radial distortion of the lth visible light image is corrected; τ = (t1, t2, t3) T represents the position of the camera in the three-dimensional coordinate system when the lth visible light image is taken; solve g1 and g2 according to the following coordinate transformation formula:

[0042]

[0043] Among them, f1, f2, e1, e2 are known camera intrinsic parameters; λ is an unknown variable.

[0044] Calculate θ corresponding to I l The coordinate Φ in the l ·ψ, I l Represents G l The image matrix obtained after correcting the perspective deformation; d j Indicates Φ to I l The distance from the center of the jth photovoltaic module in , Ω represents the value range of j, solve Match θ with the corresponding PV module image matrix; N represents the number of PV modules in the PV power station, Π k Represents the 3D point cloud corresponding to the surface of the kth photovoltaic module, where k = 1, 2, ..., N; combining the 3D coordinates of each point in Π with the corresponding image matrix, Π is divided into Π1, Π2, ..., Π N .

[0045] S5: Construct a three-dimensional model of the photovoltaic module according to the three-dimensional point cloud corresponding to the surface of the photovoltaic module.

[0046] In this embodiment, A k ·x+B k y+C k ·z+D k =0 represents the plane equation corresponding to the surface of the kth photovoltaic module, and Π k Substitute each point in into the plane equation to obtain an overdetermined system of equations; solve the overdetermined system of equations to obtain the unit normal vector n of the kth photovoltaic module surface k =(A k ,B k ,C k ) T ; The length of the photovoltaic module is L, the width is W, and the height is H; the center of the kth photovoltaic module is O k , the unit vector u k =(α k ,βk ,γ k ) T Parallel to the long side of the kth PV module, unit vector v k =n k ×u k The normal vector perpendicular to the surface and long side of the kth PV module.

[0047] m represents Π k The number of three-dimensional points in R i =(x i ,y i ,z i ) T Represents Π k For the i-th point in i The boundary coefficient δ i With σ i as follows:

[0048]

[0049] Where i = 1, 2, ..., m; when δ i >0 or σ i >0, R i Beyond the boundary of the kth PV module.

[0050] The boundary optimization objective F is as follows:

[0051]

[0052] In satisfying α k 2 +β k 2 +γ k 2 = 1, solve O k , α k , β k , γ k , let the boundary optimization objective F obtain the minimum value; k Centered on u k , v k , n k They are used as the direction vectors of length, width and height respectively to construct the three-dimensional model of the kth photovoltaic module.

[0053] S6: Integrate multi-source data of PV modules to build a digital model of the PV power station.

[0054] In this embodiment, the image matrix of the photovoltaic module is matched with the three-dimensional model according to the GPS information and camera parameters of the visible light image; the three-dimensional model is matched with the design document according to the topological structure of the photovoltaic module, and the three-dimensional model of the photovoltaic module is automatically numbered; the image matrix, three-dimensional model, topological structure and numbering information of the photovoltaic module are integrated as a digital model of the photovoltaic power station. Figure 4 Schematic diagram of the topological structure of photovoltaic modules. Each node represents a photovoltaic module. The existence of an edge between two nodes indicates that the corresponding two photovoltaic modules are adjacent. Figure 5 Schematic diagram of matching the image matrix and three-dimensional point cloud of a photovoltaic module, where (a) is the image matrix of the photovoltaic module; (b) is the three-dimensional point cloud corresponding to the surface of the photovoltaic module.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A digital modeling method for a photovoltaic power station by fusion of multi-source data, characterized by: The following steps are included in sequence: S1: Extract the topological structure and numbering of photovoltaic modules in the photovoltaic power station according to the design documents of the photovoltaic power station; S2: The drone is equipped with a laser radar and a mapping camera to collect 3D point clouds and visible light images of the photovoltaic power station; S3: Combine photovoltaic module features with semantic segmentation models to identify photovoltaic modules in visible light images; S4: Combining the three-dimensional coordinates with the image matrix, extracting the three-dimensional point cloud corresponding to the surface of each photovoltaic module; S5: construct a three-dimensional model of the photovoltaic module according to the three-dimensional point cloud corresponding to the surface of the photovoltaic module; S6: Integrate multi-source data of PV modules to build a digital model of the PV power station.

2. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 1, characterized in that: In the step S1, an undirected graph is used to represent the topological structure of the photovoltaic module, each photovoltaic module is used as a node of the graph, and the number of the photovoltaic module is used as the number of the node; if two photovoltaic modules are adjacent, there is an edge between the corresponding two nodes; if the two photovoltaic modules are not adjacent, there is no edge between the corresponding two nodes; the undirected graph is divided into a number of connected subgraphs, and the positional relationship between the subgraphs is recorded.

3. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 2 is characterized by: In step S2, the laser radar and the mapping camera are located on the same gimbal, and their relative positions remain unchanged; the yaw angle of the gimbal is 0°, the roll angle is 0°, and the pitch angle is -90°; the three-dimensional point cloud is collected by non-repetitive scanning mode, and any 10×10 cm on the surface of the photovoltaic module 2 At least 10 points are collected in the area; visible light images are collected by taking photos at equal time intervals, and any photovoltaic module appears completely in at least one visible light image.

4. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 3 is characterized by: In step S3, W represents the number of visible light images, G h represents the image matrix obtained after the radial distortion of the hth visible light image is corrected, where h = 1, 2, ..., W; M h represents the perspective transformation matrix, I h Represents G h The image matrix obtained after correcting the perspective deformation, I h =M h ·G h The surface of the photovoltaic module is rectangular, the silicon cell is dark blue, the aluminum alloy frame is silvery white, there are main grid lines and thin grid lines perpendicular to each other on the surface of the silicon cell, and there are connecting lines between the silicon cells; Combining the geometric features, color features, texture features of the photovoltaic module, and the semantic segmentation model, in I h The complete photovoltaic module is identified and the image matrix of the photovoltaic module is extracted.

5. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 4 is characterized by: In the step S4, based on the three-dimensional point cloud of the photovoltaic power station, the three-dimensional point cloud UI corresponding to the surface of all photovoltaic modules is extracted; (p1,p2,p3) T Represents the three-dimensional coordinates of a point θ in Π, and compares θ with the first The visible light images are matched, where θ corresponds to The coordinates in ψ=(g1,g2) T , Indicates The image matrix obtained after the radial distortion of the visible light image is corrected; τ = (t1, t2, t3) T Indicates shooting When the visible light image is taken, the position of the camera in the three-dimensional coordinate system; solve g1 and g2 according to the following coordinate transformation formula: Among them, f1, f2, e1, e2 are known camera intrinsic parameters; λ is an unknown variable.

6. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 5, characterized in that: In step S4, the calculation of θ corresponds to Coordinates in express The image matrix obtained after correcting the perspective deformation; d j Indicates Φ to The distance from the center of the jth photovoltaic module in , Ω represents the value range of j, solve Match θ with the corresponding PV module image matrix; N represents the number of PV modules in the PV power station, Π k represents the 3D point cloud corresponding to the surface of the k-th photovoltaic module, where k = 1, 2, ..., N; combining the 3D coordinates of each point in Π with the corresponding image matrix, Π is divided into Π1, Π2, ..., Π N .

7. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 6, characterized in that: In step S5, A k ·x+B k y+C k ·z+D k =0 represents the plane equation corresponding to the surface of the kth photovoltaic module, and Π k Substitute each point in the plane equation to obtain an overdetermined system of equations; solve the overdetermined system of equations to obtain the unit normal vector n of the kth photovoltaic module surface k =(A k ,B k ,C k ) T ; The length of the photovoltaic module is L, the width is W, and the height is H; the center of the kth photovoltaic module is O k , the unit vector u k =(α k ,β k ,γ k ) T Parallel to the long side of the kth PV module, unit vector v k =n k ×u k The normal vector perpendicular to the surface and long side of the kth PV module.

8. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 7, characterized in that: In step S5, m represents Π k The number of three-dimensional points in R i =(x i ,y i ,z i ) T Represents Π k For the i-th point in i The boundary coefficient δ i With σ i as follows: Where i = 1, 2, ..., m; when δ i >0 or σ i >0, R i Beyond the boundary of the kth PV module.

9. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 8, characterized in that: In step S5, the boundary optimization target F is as follows: In satisfying α k 2 +β k 2 +γ k 2 = 1, solve O k , α k , β k , γ k , let the boundary optimization objective F obtain the minimum value; k Centered on u k , v k , n k They are used as the direction vectors of length, width and height respectively to construct the three-dimensional model of the kth photovoltaic module.

10. The method for digital modeling of a photovoltaic power station by multi-source data fusion according to claim 9, characterized in that: In step S6, the image matrix of the photovoltaic module is matched with the three-dimensional model according to the GPS information and camera parameters of the visible light image; the three-dimensional model is matched with the design document according to the topological structure of the photovoltaic module, and the three-dimensional model of the photovoltaic module is automatically numbered; the image matrix, three-dimensional model, topological structure and numbering information of the photovoltaic module are integrated as a digital model of the photovoltaic power station.

Images