Photovoltaic power station air-ground integrated intelligent operation and maintenance method, system, equipment and medium

Abstract

The invention discloses an air-ground integrated intelligent operation and maintenance method, system and device for a photovoltaic power station and a medium, and belongs to the technical field of photovoltaic power station inspection, and the method comprises the steps: carrying out the collaborative data collection and edge preprocessing through dispatching heterogeneous equipment; optimizing an inspection path based on a three-dimensional geographic information system and a task allocation algorithm; fusing multi-source data at a center end, performing intelligent analysis by utilizing a defect knowledge graph, and mapping a result to a group cascade coordinate database to realize accurate positioning; finally, an inspection report is automatically generated, and an intelligent operation and maintenance closed loop is completed. According to the invention, through collaborative operation of the multi-rotor unmanned aerial vehicle cluster and the track type inspection robot, based on a multi-source data fusion engine and a defect knowledge graph, time-space alignment and intelligent analysis are carried out on infrared, visible light and other multi-modal data, so that accurate defect identification and processing scheme matching are realized; the problems of coverage blind areas, detection lag, resource waste and data islands are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photovoltaic power station inspection, and particularly relates to a photovoltaic power station air-ground integrated intelligent operation and maintenance method, system, device and medium. BACKGROUND

[0002] With the increase of operation time, the faults of photovoltaic components begin to occur frequently, which greatly affects the power generation efficiency of the power station, and even may cause serious accidents such as fire. The types of faults of photovoltaic components are various, and a small part of them can be distinguished by appearance, and most of them need to be checked by special infrared equipment, which also brings additional difficulties to the operation and maintenance work.

[0003] The conventional manual mode not only needs to consume a large amount of labor cost, but also is easily affected by the terrain, and appears an unreachable area, thereby generating an inspection blind spot. At the same time, in the power station relay protection room, the inspection frequency of manual inspection is difficult to meet the requirements, so that many faults cannot be discovered in time, and the power loss continues to occur. The heavy inspection task and the backward inspection mode constitute the main contradiction of the current photovoltaic power station operation and maintenance. In order to improve the inspection efficiency, the photovoltaic inspection gradually develops from the manual dominant inspection to the hardware dominant intelligent inspection, which is an irreversible trend, and therefore the present application proposes a photovoltaic power station air-ground integrated intelligent operation and maintenance system. SUMMARY

[0004] In view of the above problems, the present application is proposed.

[0005] Therefore, the technical problem solved by the present application is that the traditional manual inspection mode cannot effectively cope with the operation and maintenance challenges brought by the large-scale development of photovoltaic power stations. The specific performance is that the faults of photovoltaic components occur frequently and are various (need to rely on special equipment such as infrared for detection), and the manual inspection has problems such as high cost, easy to produce blind area due to terrain limitation, insufficient inspection frequency of key areas such as relay protection room, etc., which leads to that the defects are not discovered in time, not only causes the power loss, but also hides the risk of serious safety accidents such as fire. The contradiction between the low efficiency of the existing inspection mode and the high requirement of the operation and maintenance task is prominent.

[0006] To solve the above technical problems, the present application provides the following technical scheme: a photovoltaic power station air-ground integrated intelligent operation and maintenance method, comprising,

[0007] The heterogeneous devices are scheduled, the power station operation unit is cooperatively data collected and preprocessed, a joint inspection task is allocated to the heterogeneous devices based on a heterogeneous device cooperative task allocation algorithm to realize multi-objective optimization, in the data collection process, a three-dimensional obstacle avoidance model is constructed by fusing multi-source sensing information to realize dynamic obstacle avoidance, and a YOLOv5 model is deployed for online pre-identification; three-dimensional geographic information system is used to provide the inspection path planning for the heterogeneous devices and control the heterogeneous devices to identify the equipment state; the fixed monitoring data is aggregated, the multi-source data fusion engine is used in the central scheduling server to perform time and space alignment and fusion; the matched analysis of the defect knowledge graph is performed on the fused data, and the defect information obtained is mapped to the string level coordinate database; the intelligent report generator is used to automatically generate the inspection report, and the intelligent operation and maintenance closed loop is completed.

[0008] As a preferred scheme of the photovoltaic power station air-ground integrated intelligent operation and maintenance method, the cooperative data collection and preprocessing of the power station operation unit comprises,

[0009] For each frame of image in the visible light video stream, a feature extraction algorithm is used to extract feature points.

[0010] Using BFMatcher, the Hamming distance is used as the similarity measure, the visible light image feature points are preliminarily matched with the feature points in the infrared image, and the coarse matching points are obtained.

[0011] The RANSAC algorithm is applied to screen the coarse matching points, and a homography matrix is calculated.

[0012] The homography matrix is used to perform affine transformation on the original infrared image.

[0013] As a preferred scheme of the photovoltaic power station air-ground integrated intelligent operation and maintenance method, the three-dimensional obstacle avoidance model constructed by fusing multi-source sensing information comprises,

[0014] The visual odometry is run, the ORB feature points are extracted, and the pose transformation matrix between adjacent frames is solved by minimizing the re-projection error optimization.

[0015] The laser point cloud assisted closed loop detection is performed, and the laser point cloud is rasterized into a three-dimensional voxel map.

[0016] The occupancy probability of each voxel is updated based on the truncated signed distance function model.

[0017] The long short-term memory network is used to predict the motion trajectory of the dynamic obstacle.

[0018] As a preferred scheme of the photovoltaic power station air-ground integrated intelligent operation and maintenance method, the YOLOv5 model is deployed for online pre-identification, comprising,

[0019] Adaptive histogram equalization is performed on the input image, and normalization is performed.

[0020] The preprocessed image is sent into the YOLOv5 network for forward propagation calculation, and single inference is accelerated by using FPGA.

[0021] The initial detection result output by the YOLOv5 model is subjected to non-maximum suppression, and unreliable detection boxes are filtered.

[0022] As a preferred scheme of the photovoltaic power station air-ground integrated intelligent operation and maintenance method, the method comprises the steps of,

[0023] Oblique photogrammetry is performed to generate a digital surface model with a centimeter-level resolution.

[0024] An improved genetic algorithm is used to cooperatively optimize the flight paths of multiple unmanned aerial vehicles.

[0025] In the MySQL relational database, a binding relationship between the photovoltaic component number and the WGS-84 geographic coordinates is established and maintained, and a string-level coordinate mapping database is constructed.

[0026] Based on the path, map and coordinate database, the unmanned aerial vehicle is dispatched and controlled to fly to the target position, and a device state recognition algorithm library is started.

[0027] The improved genetic algorithm is represented as,

[0028]

[0029] Wherein, min is the minimum value of the objective function, T is the traversal summation of n unmanned aerial vehicle inspection tasks, i T is the flight time of the i-th unmanned aerial vehicle, i E is the energy consumption index of the i-th unmanned aerial vehicle, i R is the obstacle risk coefficient of the i-th inspection task.

[0030] As a preferred scheme of the photovoltaic power station air-ground integrated intelligent operation and maintenance method, the method comprises the steps of,

[0031] An intelligent fire and smoke detection module based on LSTM is used to construct a temperature field prediction model.

[0032] ViBe background modeling and KLT optical flow method are used to support human and vehicle classification tracking.

[0033] The temperature field prediction model is represented as,

[0034] P fire =σ(W·[T t-10 ,…,T t ]+b)

[0035] Wherein, P fire is a real probability, represents a target probability value predicted by a temperature field prediction model, sigma is a Sigmoid activation function, a linear transformation result is carried out nonlinear mapping, W is a weight matrix, T is a temperature sequence, and b is a bias term.

[0036] As a preferred scheme of the photovoltaic power station air-ground integrated intelligent operation and maintenance method, the time-space alignment and fusion using the multi-source data fusion engine is represented as,

[0037] The time-space alignment algorithm is adopted to aggregate the unmanned aerial vehicle, robot and fixed node data.

[0038] Based on the defect knowledge graph of the Neo4j graph database, the association relationship between historical defect characteristics and processing schemes is stored.

[0039] An HTML5 inspection report containing a defect distribution thermodynamic diagram and a repair priority evaluation matrix is automatically generated.

[0040] The application provides a photovoltaic power station air-ground integrated intelligent operation and maintenance system.

[0041] To solve the above technical problems, the application provides the following technical scheme: a photovoltaic power station air-ground integrated intelligent operation and maintenance system, comprising: a multi-rotor unmanned aerial vehicle cluster, a track inspection robot, a central dispatching server and a distributed fixed monitoring network module.

[0042] The multi-rotor unmanned aerial vehicle cluster is equipped with a dual-optical camera, an on-board edge computing unit integrated with an FPGA processor and a laser radar obstacle avoidance module.

[0043] The track inspection robot moves along a preset slide wire track, carries a six-degree-of-freedom mechanical arm and an infrared vision module.

[0044] The central dispatching server is configured with a multi-modal data fusion processing module and a three-dimensional geographic information system.

[0045] The distributed fixed monitoring network module comprises a 360-degree rotating holder and an infrared-visible light dual-mode camera array.

[0046] The application provides a computer device, which comprises a memory and a processor, and the memory stores a computer program.

[0047] The application provides a computer readable storage medium, which stores a computer program, and the computer program is used to realize the steps of the photovoltaic power station air-ground integrated intelligent operation and maintenance method when executed by a processor.

[0048] The multi-rotor unmanned aerial vehicle cluster of the application optimizes a path through an improved genetic algorithm, 95% of a photovoltaic area is covered for detection, and the track inspection robot moves along a slide wire track, covers a booster station and a power collection line fixed area, improves inspection efficiency, and guarantees full coverage of inspection.

[0049] The multi-source data fusion engine adopts a space-time alignment algorithm, aggregates infrared temperature fields and visible light image data, constructs a defect knowledge graph from a Neo4j graph database, stores a correlation relationship of historical defect features and processing schemes, and realizes multi-source data cooperation.

[0050] Through air-ground-fixed cooperative detection, intelligent algorithm optimization and multi-modal data fusion, four core problems of coverage blind area, detection lag, resource waste and data island in photovoltaic power station operation and maintenance are systematically solved, and full life cycle protection is provided for safe and efficient operation of the photovoltaic power station. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 A general flowchart of a photovoltaic power station air-ground integrated intelligent operation and maintenance method provided by an embodiment of the application.

[0053] Figure 2 A general framework diagram of a photovoltaic power station air-ground integrated intelligent operation and maintenance system provided by an embodiment of the application. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor should be within the protection scope of the application.

[0055] Embodiment 1, refer to Figure 1 For an embodiment of the application, the embodiment provides a photovoltaic power station air-ground integrated intelligent operation and maintenance method, which comprises:

[0056] S1, scheduling heterogeneous devices, cooperative data collection and preprocessing of power station operation units.

[0057] S2, based on the heterogeneous device cooperative task allocation algorithm, the joint inspection task is allocated to the heterogeneous device to realize multi-objective optimization.

[0058] S3, in the data collection process, a three-dimensional obstacle avoidance model is constructed by fusing multi-source perception information to realize dynamic obstacle avoidance, and a YOLOv5 model is deployed for online pre-recognition.

[0059] S4, using three-dimensional geographic information system, providing inspection path planning for heterogeneous devices and controlling heterogeneous devices to identify device status.

[0060] S5, aggregate fixed monitoring data, use multi-source data fusion engine in central dispatch server for spatio-temporal alignment and fusion.

[0061] S6, defect knowledge graph matching analysis is performed on the fused data, and the obtained defect information is mapped to the string level coordinate database.

[0062] S7, through the intelligent report generator, the inspection report is automatically generated, and the intelligent operation and maintenance closed loop is completed.

[0063] The overall technical scheme of the present application realizes the full-process automation and intelligentization of photovoltaic power station operation and maintenance through air-ground heterogeneous device cooperation, intelligent edge computing and center cloud brain fusion. Its beneficial effects are: first, through the multi-source data cooperative collection of unmanned aerial vehicles, robots and fixed monitoring networks and FPGA accelerated online identification, the millisecond level discovery and accurate positioning of photovoltaic panel hot spot, component damage and other defects are realized, the inspection efficiency is improved and the blind area is eliminated; secondly, based on the intelligent scheduling of three-dimensional geographic information system and multi-objective optimization algorithm, the optimal planning and dynamic obstacle avoidance of the inspection path are realized, the operation energy consumption and risk are reduced; finally, relying on knowledge graph and multi-source data fusion technology, defect diagnosis, coordinate mapping and report generation can be automatically completed, a closed-loop operation and maintenance system from perception, analysis to decision is constructed, and the overall goal of reducing operation and maintenance cost, reducing power generation loss and preventing and controlling safety risk is finally achieved.

[0064] Embodiment 2 is an embodiment of the present application, which provides a kind of photovoltaic power station air-ground integrated intelligent operation and maintenance method based on the last embodiment, comprising:

[0065] S1 in the cooperative data collection and preprocessing of power station operation units includes the following steps A1-A4:

[0066] A1, for each frame of image in visible light video stream, feature extraction algorithm is used to extract feature points.

[0067] A2, using BFMatcher, taking Hamming distance as similarity measure, to preliminarily match the visible light image feature points with the feature points in the infrared image to obtain coarse matching points.

[0068] A3, applying RANSAC algorithm to screen the coarse matching point pairs and calculating a homography matrix.

[0069] The homography matrix is expressed as,

[0070]

[0071] wherein, argmin H To find the matrix H that minimizes the objective function, is expressed as selecting the matrix that minimizes the sum of squared errors from all possible H; if the input data dimension is n x 1, H is an m x n matrix, which maps the input x i to the m-dimensional space; ∑ i is the sum of all samples i, i = 1, 2,..., N, representing N samples in the data set; x i is the input feature vector of the i-th sample, dimension: n x 1, in image registration, it is the feature point coordinates; represents the target output vector of the i-th sample; is the linear transformation result of the matrix H to the input x i ;

[0072] A4, using the homography matrix to perform affine transformation on the original infrared image.

[0073] In the embodiment of the application, the feature extraction algorithm in A1 is ORB feature extraction. The method is aimed at the characteristics of high texture repeatability and easy occurrence of mismatch of the photovoltaic panel array, and through dynamically adjusting the scale factor of the image pyramid and the feature point distribution density, and introducing illumination invariance constraint in the feature description stage, the robustness of extraction and matching of key features such as photovoltaic panel corner points and mounting brackets under complex illumination is improved.

[0074] In an alternative embodiment, SIFT algorithm can be used. First, a Gaussian pyramid is constructed and extreme point detection is performed in different scale spaces, then the feature point position and scale are precisely located by fitting a three-dimensional quadratic function, and a principal direction is assigned to each feature point based on the local image gradient direction; finally, the gradient direction histogram is calculated in the neighborhood of the feature point to generate a 128-dimensional SIFT feature descriptor for matching.

[0075] In another alternative embodiment, the FAST corner detector can be combined with the BRIEF descriptor. Specifically, first, the FAST-9 corner detector is used to quickly locate key points in the image, then the binary coded BRIEF descriptor is generated based on the gray value comparison relationship of the key point neighborhood pixels, and finally the feature matching is completed by calculating the Hamming distance between the descriptors.

[0076] In the embodiment of the present application, the heterogeneous device cooperative task allocation algorithm in S2 is device capability matrix modeling, which quantifies the detection accuracy, moving speed and working range of each device; through a multi-objective optimization function, the detection coverage, working time and energy consumption indicators are simultaneously optimized; the distributed task scheduling mechanism supports online task re-allocation and device fault tolerance.

[0077] The multi-objective optimization function is expressed as,

[0078] max (aC cover - bT total - gE total )

[0079] Wherein, max represents the maximum value of the objective function, C cover represents the detection coverage, T represents the total working time, E represents the total energy consumption, a, b, g represent the weighting coefficients, if a increases, the system tends to improve the coverage, if b increases, the system tends to reduce the temperature, if g increases, the system tends to energy saving mode, and a+b+g=1.

[0080] In an alternative embodiment, the heterogeneous device cooperative task allocation algorithm can be a greedy task allocation algorithm based on fixed priority. Specifically, a fixed task priority is set for each type of heterogeneous device (such as unmanned aerial vehicle, robot) in advance (such as unmanned aerial vehicle priority higher than robot), the system generates a task queue according to the inspection area, and always allocates the task with the highest priority to the idle device with the highest priority until all tasks are allocated.

[0081] In another alternative embodiment, the heterogeneous device cooperative task allocation algorithm can also be a distributed negotiation algorithm based on auction mechanism. Specifically, each inspection task is regarded as an auction item, and each heterogeneous device is regarded as a bidder, and the bidding is made according to the current location, remaining power and task cost of the device; the central scheduler acts as an auctioneer, and the task is awarded to the device with the lowest bid (i.e. the smallest cost).

[0082] In the embodiment of the present application, the multi-source perception information in S3, i.e., fused laser point cloud data and ORB-SLAM vision algorithm, runs the ORB-SLAM vision algorithm, extracts ORB feature points, and solves the pose transformation matrix between adjacent frames by minimizing the re-projection error optimization.

[0083] In an alternative embodiment, the multi-source perception information can be a pure vision scheme. The specific implementation is: using visual inertial odometry, relying only on monocular or binocular camera and IMU data for fusion. The high-frequency attitude data provided by the IMU is used to integrate and correct the motion estimation between images, and the bag-of-words model is used for loop detection and repositioning, and finally the six-degree-of-freedom pose estimation is output.

[0084] In another alternative embodiment, the multi-source perception information can also be a pure laser radar scheme. The specific implementation is: using laser radar odometry and mapping algorithm framework. This scheme extracts corner and plane point features in the laser point cloud, and uses feature matching between two consecutive point clouds to calculate the motion transformation of the radar itself, and then gradually accumulates to construct a point cloud map of the environment, and performs positioning based on the map.

[0085] Further, the fusion of multi-source perception information in S3 to construct a three-dimensional obstacle avoidance model includes the following steps B1-B4:

[0086] B1, running the visual odometry, extracting the ORB feature points, and solving the pose transformation matrix between adjacent frames by minimizing the re-projection error optimization.

[0087] B2, performing laser point cloud assisted loop detection, and rasterizing the laser point cloud into a three-dimensional voxel map.

[0088] The cloud rasterization (resolution 0.1m 3 ).

[0089] B3, updating the occupancy probability of each voxel based on the truncated signed distance function model.

[0090] Obstacle probability update based on TSDF:

[0091]

[0092] Wherein, TSDF(x) is a truncated signed distance function value, indicating the truncated distance of point x to the target surface, usually [-1, 1], used to describe the relative position relationship of points in three-dimensional space to the surface; d(x) is a signed distance function (Signed Distance Function, SDF), indicating the actual distance of point x to the nearest surface, d(x)>0: point x is outside the surface; d(x)<0: point x is inside the surface; d(x)=0: point x is exactly on the surface; trunc is the abbreviation of the truncation threshold, which defines the effective range of distance.

[0093] B4, using a long short-term memory network to predict the motion trajectory of a dynamic obstacle.

[0094] Dynamic obstacle prediction (LSTM network):

[0095]

[0096] In the formula, is the dynamic obstacle prediction value at time t; LSTM is a long short-term memory network (Long Short-Term Memory); v t-3 ,v t-2 ,v t-1 is the input feature of the historical time step, which is the input sequence of the LSTM, and is the observation value of the continuous time point.

[0097] Predicting the trajectory 1 second in the future, with an error of <0.3m.

[0098] In the embodiments of the present application, the visual odometry in B1, i.e., the improved algorithm of ORB-SLAM, extracts ORB feature points (1000 per frame), and motion estimation:

[0099]

[0100] Wherein, T k,k-1 is the transformation matrix from time k-1 to k, and argmin T is to find the matrix T that minimizes the objective function, is to apply the transformation T to the three-dimensional point of the last time to obtain the coordinates in the current coordinate system; π is a projection function, and X is a 3D point.

[0101] In an alternative embodiment, the visual odometry can be a direct sparse odometry (Direct Sparse Odometry, DSO) based visual odometry. The specific implementation is as follows: this scheme does not extract feature points, but directly uses the gray gradient information of image pixels, constructs a photometric error function containing a large number of points, and minimizes the photometric error of these points between adjacent frames to directly solve the camera pose.

[0102] In another alternative implementation, visual odometry can also be based on optical flow. Specifically, the KLT optical flow tracking algorithm is first used to track the motion trajectory of a set of sparse feature points between consecutive image frames to obtain their pixel displacement vector field.

[0103] Furthermore, deploying the YOLOv5 model in S3 for online pre-identification includes the following steps C1-C3:

[0104] C1. Perform adaptive histogram equalization on the input image and then normalize it.

[0105] Further input image preprocessing: Adaptive histogram equalization (CLAHE, clip_limit = 2.0); normalization: pixel / 255.0 - mean [0.485, 0.456, 0.406].

[0106] C2. The preprocessed image is fed into the YOLOv5 network for forward propagation calculation, and the FPGA is used to accelerate single inference.

[0107] Furthermore, the forward propagation time is 11ms (FPGA accelerated); the output tensor resolution is as follows:

[0108] (x,y,w,h,conf,cls)=decode(tensor).

[0109] C3. Perform non-maximum suppression on the initial detection results output by the YOLOv5 model and filter out unreliable detection boxes.

[0110] NMS threshold: IoU = 0.5; Confidence filtering: conf ≥ 0.7 (false negative rate < 5%).

[0111] Furthermore, S4 provides inspection path planning for heterogeneous devices and controls the heterogeneous devices to identify their equipment status, including the following steps D1-D4:

[0112] D1. Perform oblique photogrammetry to generate a digital surface model with centimeter-level resolution.

[0113] D2. An improved genetic algorithm is used to collaboratively optimize the flight paths of multiple UAVs.

[0114] The improved genetic algorithm is represented as follows:

[0115]

[0116] Where min represents the minimum value of the objective function. To iterate and sum over n drone inspection tasks, T iFlight time of the ith UAV, E i Energy consumption index of the ith UAV, R i Obstacle risk coefficient of the ith inspection task.

[0117] Wherein, the time weight 0.4: ensure single task ≤2 hours (avoid illumination condition change affecting infrared detection); energy consumption weight 0.3: balance lithium battery endurance limit (≥45 minutes margin); risk weight 0.3: prefer to avoid live equipment area (safety distance ≥5m).

[0118] D3, in the MySQL relational database, the binding relationship between the photovoltaic component number and the WGS-84 geographic coordinates is established and maintained, and the string level coordinate mapping database is constructed.

[0119] D4, based on the path, map and coordinate database, the UAV is dispatched and controlled to fly to the target position, and the device state recognition algorithm library is started.

[0120] Further, the aggregated fixed monitoring data in S5 includes the following steps E1-E2:

[0121] E1, the intelligent smoke and fire detection module based on LSTM, a temperature field prediction model is constructed.

[0122] The temperature field prediction model is represented as,

[0123] P fire =σ(W·[T t-10 ,…,T t ]+b)

[0124] Wherein, P fire is the true probability, indicating the target probability value predicted by the temperature field prediction model, σ is the Sigmoid activation function, which performs nonlinear mapping on the linear transformation result, W is the weight matrix, T is the temperature sequence, and b is the bias term.

[0125] E2, ViBe background modeling and KLT optical flow method are used to support human and vehicle classification tracking;

[0126] Further, the spatio-temporal alignment and fusion using the multi-source data fusion engine in S6 includes the following steps F1-F3.

[0127] F1, the spatio-temporal alignment algorithm is used to aggregate UAV, robot and fixed node data.

[0128] Further, the spatio-temporal alignment algorithm includes spatial alignment, because the spatial coordinate system and the perspective of the data collected by different devices are different. For example, the UAV uses WGS-84 global coordinates, the orbital robot uses local coordinates with the starting point of the orbit as the origin, and the fixed camera has its own image pixel coordinate system.

[0129] So the following operations are needed:

[0130] Coordinate system 1: Through the coordinate transformation matrix, the positioning data of all devices (such as the GPS coordinates of the unmanned aerial vehicle, the track encoder reading of the robot) are uniformly converted into the same global coordinate system, which is usually based on the engineering coordinate system of the three-dimensional geographic information system (3D GIS) or the WGS-84 coordinate system.

[0131] Image data registration: For visible light, infrared and other image data, feature point matching (such as SIFT, ORB) and perspective transformation algorithms are used to accurately align the images taken by different devices at different angles to the corresponding positions on the three-dimensional geographic information model.

[0132] The space-time alignment algorithm also includes time alignment. Since each heterogeneous device has an independent clock, the data collection timestamps are not synchronized, resulting in a millisecond-level or even second-level deviation of the "same time" data.

[0133] So the following operations are needed:

[0134] Time synchronization: Network time protocol (NTP) or PTP (precise time protocol) is used to provide unified time synchronization services for all devices, and the absolute error of the clock of each device is minimized.

[0135] F2, defect knowledge graph based on Neo4j graph database, stores the association relationship between historical defect features and processing schemes.

[0136] F3, automatically generate an HTML5 inspection report containing a defect distribution heat map and a repair priority evaluation matrix.

[0137] Embodiment 3, refer to Figure 2 As an embodiment of the present application, the embodiment provides an integrated intelligent operation and maintenance system for photovoltaic power station, comprising: a multi-rotor unmanned aerial vehicle cluster, a track inspection robot, a central dispatching server and a distributed fixed monitoring network module:

[0138] The multi-rotor unmanned aerial vehicle cluster is equipped with a dual-optical camera, an on-board edge computing unit integrated with an FPGA processor and a laser radar obstacle avoidance module.

[0139] The on-board edge computing unit comprises:

[0140] The space-time synchronization compression module based on FPGA realizes millisecond-level synchronous acquisition and H.265 encoding of dual-optical images.

[0141] The dynamic obstacle avoidance decision module fuses laser point cloud data and ORB-SLAM visual algorithm to construct a three-dimensional obstacle avoidance model.

[0142] The light-weight YOLOv5 model is deployed to identify defects in the pre-recognition module for online detection of photovoltaic panel hot spots and component damage. The detection threshold is set to 0.7 confidence.

[0143] The track inspection robot moves along the preset sliding contact line track, carrying a six-degree-of-freedom mechanical arm and an infrared vision module.

[0144] The track inspection robot is configured as follows:

[0145] The millimeter wave radar and ultrasonic composite navigation system achieves a track positioning accuracy of ±5mm.

[0146] The adaptive multi-spectral lighting module includes a 450-650nm visible light LED array and an 850nm near-infrared fill light.

[0147] The device state recognition algorithm library integrates a MobileNetV3-based indicator light state classification model and a CRNN-CTC instrument OCR recognition model.

[0148] The central dispatch server is configured with a multi-modal data fusion processing module and a three-dimensional geographic information system.

[0149] The central dispatch server is deployed as follows:

[0150] The multi-source data fusion engine uses a spatio-temporal alignment algorithm to aggregate data from drones, robots, and fixed nodes.

[0151] The defect knowledge graph based on the Neo4j graph database stores the association between historical defect features and treatment plans.

[0152] The intelligent report generator automatically generates an HTML5 inspection report containing a defect distribution heat map and a repair priority assessment matrix.

[0153] The three-dimensional geographic information system includes:

[0154] The centimeter-level resolution digital surface model is generated through oblique photogrammetry using a DJI P1 aerial camera.

[0155] The intelligent flight path planning module uses an improved genetic algorithm to optimize the paths of multiple drones.

[0156] The string-level coordinate mapping database uses a MySQL relational database to store the binding relationship between component numbers and WGS-84 geographic coordinates.

[0157] The distributed fixed monitoring network module includes a 360° rotating gimbal and an infrared-visible light dual-mode camera array.

[0158] The distributed fixed monitoring network module includes:

[0159] Intelligent firework detection module based on LSTM, construct temperature field prediction model.

[0160] Intrusion detection module using ViBe background modeling and KLT optical flow method, supporting human-vehicle classification and tracking.

[0161] The embodiment also provides an electronic device suitable for the case of the photovoltaic power station airfield integrated intelligent operation and maintenance method, and the electronic device comprises a memory and a processor.

[0162] The embodiment also provides a storage medium having a computer program stored thereon, and the program is executed by a processor to implement the photovoltaic power station airfield integrated intelligent operation and maintenance method.

[0163] The storage medium provided by the embodiment and the photovoltaic power station airfield integrated intelligent operation and maintenance method provided by the above embodiment belong to the same inventive concept, and the technical details not described in the embodiment can be referred to the above embodiment, and the embodiment has the same beneficial effects as the above embodiment.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk or an optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of various embodiments of the present application.

[0165] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit the present application, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, and all should be covered in the scope of the claims of the present application.

Claims

1. An integrated intelligent operation and maintenance method for photovoltaic power station air and land, characterized in that: The method comprises the steps of: Scheduling heterogeneous devices, collecting and preprocessing data of power station operation units cooperatively; Based on the heterogeneous device cooperative task allocation algorithm, the joint inspection tasks are allocated to the heterogeneous devices to realize multi-objective optimization; In the data collection process, a three-dimensional obstacle avoidance model is constructed by fusing multi-source perception information to realize dynamic obstacle avoidance, and a YOLOv5 model is deployed for online pre-identification; Using three-dimensional geographic information system, the inspection path planning for the heterogeneous devices is provided, and the device state recognition for the heterogeneous devices is controlled; The fixed monitoring data is aggregated, and the spatio-temporal alignment and fusion are performed by using a multi-source data fusion engine in the central dispatching server; The matched analysis of the defect knowledge graph is performed on the fused data, and the obtained defect information is mapped to the string level coordinate database; Through the intelligent report generator, the inspection report is automatically generated, and the intelligent operation and maintenance closed loop is completed. 2.The photovoltaic power station land-air integrated intelligent operation and maintenance method of claim 1, wherein: The method comprises the steps of: For each frame of image in the visible light video stream, feature points are extracted by using a feature extraction algorithm; Using BFMatcher, the visible light image feature points are preliminarily matched with the feature points in the infrared image by taking Hamming distance as the similarity measure, and coarse matching points are obtained; The RANSAC algorithm is applied to screen the coarse matching point pairs, and a homography matrix is calculated; The original infrared image is subjected to affine transformation by using the homography matrix. 3.The photovoltaic power station land-air integrated intelligent operation and maintenance method of claim 2, characterized in that: The method comprises the steps of: Running a visual odometry, extracting ORB feature points, and optimizing and solving the pose transformation matrix between adjacent frames by minimizing the re-projection error; Performing laser point cloud assisted closed loop detection, and rasterizing the laser point cloud into a three-dimensional voxel map; Updating the occupancy probability of each voxel based on the truncated signed distance function model; The motion trajectory of the dynamic obstacle is predicted by using a long short-term memory network.

4. The photovoltaic power station airfield integrated intelligent operation and maintenance method of claim 3, wherein: The method comprises the steps of: Performing adaptive histogram equalization on the input image, and performing normalization; The preprocessed image is sent into the YOLOv5 network for forward propagation calculation, and the single inference is accelerated by using FPGA; The initial detection result output by the YOLOv5 model is subjected to non-maximum suppression, and unreliable detection boxes are filtered. 5.The photovoltaic power station land-air integrated intelligent operation and maintenance method of claim 4, characterized in that: The method comprises the steps of: Performing oblique photogrammetry to generate a digital surface model with a centimeter-level resolution; Using an improved genetic algorithm to cooperatively optimize the flight path of multiple unmanned aerial vehicles; In the MySQL relational database, the binding relationship between the photovoltaic component number and the WGS-84 geographic coordinates is established and maintained, and a string level coordinate mapping database is constructed; Based on the path, map and coordinate database, the unmanned aerial vehicle is dispatched and controlled to fly to the target position, and a device state recognition algorithm library is started; The improved genetic algorithm is represented as: wherein min is the minimum value of the objective function, T is the traversal sum of n UAV inspection tasks, i E is the flight time of the ith UAV, i E is the energy consumption index of the ith UAV, i R is the obstacle risk coefficient of the ith inspection task. 6.The photovoltaic power station land-air integrated intelligent operation and maintenance method of claim 4, wherein: The method comprises the steps of: An intelligent smoke and fire detection module based on LSTM is used to construct a temperature field prediction model; ViBe background modeling and KLT optical flow method are used to support human and vehicle classification tracking; The temperature field prediction model is represented as: P fire = σ(W·[T t-10 , …, T t ] + b) where P fire is the true probability, represents the target probability value predicted by the temperature field prediction model, σ is a Sigmoid activation function for nonlinear mapping of the linear transformation result, W is a weight matrix, T is a temperature sequence, and b is a bias term.

7. The photovoltaic power plant airfield integrated intelligent operation and maintenance method of claim 4, wherein: The spatio-temporal alignment and fusion using the multi-source data fusion engine comprises, The spatio-temporal alignment algorithm is used to aggregate the data of unmanned aerial vehicles, robots and fixed nodes; A defect knowledge graph based on a Neo4j graph database stores the association between historical defect features and treatment schemes; An HTML5 inspection report containing a defect distribution heat map and a repair priority evaluation matrix is automatically generated.

8. A photovoltaic power station airfield integrated intelligent operation and maintenance system, applying the photovoltaic power station airfield integrated intelligent operation and maintenance method of any one of claims 1-7, characterized in that, The system comprises a multi-rotor unmanned aerial vehicle cluster, an orbital inspection robot, a central dispatch server and a distributed fixed monitoring network module. The multi-rotor unmanned aerial vehicle cluster is equipped with a dual-light camera, an on-board edge computing unit integrated with an FPGA processor and a laser radar obstacle avoidance module. The orbital inspection robot moves along a preset slide wire track, carries a six-degree-of-freedom mechanical arm and an infrared vision module. The central dispatch server is configured with a multi-modal data fusion processing module and a three-dimensional geographic information system. The distributed fixed monitoring network module comprises a 360° rotating cloud platform and an infrared-visible light dual-mode camera array. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the method for intelligent operation and maintenance of a photovoltaic power station air-ground integrated system according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method for intelligent operation and maintenance of a photovoltaic power station air-ground integrated system according to any one of claims 1 to 7.

Images