An AI intelligent operation and maintenance system and method for a photovoltaic power station based on image recognition and unmanned aerial vehicle linkage

Abstract

The application provides a photovoltaic power station AI intelligent operation and maintenance system and method based on image recognition and unmanned aerial vehicle linkage, relates to the technical field of photovoltaic power station intelligent operation and maintenance, and calibrates time stamps through a clock synchronization mechanism by collecting images and meteorological data, and stores data fusion matrices according to a unified time base. By sliding window analysis, a Kalman filter model is constructed to predict normal state parameters, an abnormal flag is triggered when the parameter deviation exceeds a dynamic threshold, and the false alarm rate is reduced through multi-sensor cross verification. If it is a bird invasion event, the trajectory is predicted based on the Kalman filter, the path is corrected combined with the wind speed and direction, and the laser positioning algorithm is used to drive away the birds in advance to protect the photovoltaic modules; if it is a device failure, a constrained Markov decision process optimization model is constructed, the optimal unmanned aerial vehicle inspection path and task sequence are calculated by combining the abnormal distribution and energy consumption constraints. The accuracy of photovoltaic power station operation and maintenance anomaly detection and the inspection efficiency are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent operation and maintenance technology for photovoltaic power plants, and more specifically, to an AI-based intelligent operation and maintenance system and method for photovoltaic power plants based on image recognition and drone linkage. Background Technology

[0002] With the continuous growth of global energy demand and the gradual strengthening of environmental protection awareness, photovoltaic power generation, as a form of renewable energy, has become one of the important development directions in the energy sector. To improve the operating efficiency and power generation capacity of photovoltaic power plants, large-scale, distributed photovoltaic power plants have been widely deployed globally. However, due to the long-term exposure of photovoltaic modules to the outdoors, they are affected by weather conditions, pollutant accumulation, bird interference, and equipment aging, which can significantly reduce module efficiency and even lead to potential safety risks. Therefore, efficient and intelligent operation and maintenance technologies for photovoltaic power plants have gradually become the focus of industry attention. In recent years, intelligent operation and maintenance methods based on artificial intelligence (AI), image recognition, and drone technology have developed rapidly, enabling refined monitoring and efficient inspection of large-scale photovoltaic power plants. In particular, the combination of the flexible deployment capabilities of drones and image recognition technology can significantly improve the detection accuracy of equipment faults and abnormal events. However, existing technical solutions still face many technical bottlenecks, such as weak real-time data acquisition, poor multi-sensor fusion effects, and a high false alarm rate in anomaly detection. These problems still need to be addressed in actual operation and maintenance.

[0003] Currently, intelligent operation and maintenance of photovoltaic (PV) power plants largely relies on single data acquisition methods, such as monitoring the status of PV modules through thermal imaging cameras or equipment operation data. While this traditional approach achieves some degree of visual management, its ability to dynamically perceive and predict abnormal states is limited due to the limited variety of sensors and the lack of multi-dimensional data fusion capabilities. Furthermore, existing methods are relatively passive in handling environmental interference factors (such as shading caused by bird activity), often misjudging surface image anomalies as module failures, thus increasing unnecessary processing costs. In addition, while the widespread use of drones in PV operation and maintenance can cover a wider inspection range, shortcomings in inspection path optimization and task sequence scheduling models result in low energy utilization and low inspection efficiency. Therefore, addressing challenges such as data acquisition and fusion, anomaly labeling and classification, and multi-objective task scheduling based on multi-sensor integration technology and artificial intelligence algorithms is crucial for achieving smarter and more efficient operation and maintenance of PV power plants. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention is proposed. This invention provides an AI-powered intelligent operation and maintenance system and method for photovoltaic power plants based on image recognition and drone-based collaboration. This system can, to some extent, solve the problems of low anomaly detection accuracy due to the inability to achieve real-time fusion of multi-dimensional data, false alarms and missed alarms caused by environmental interference, and low inspection efficiency and high energy consumption due to insufficient optimization of drone inspection paths and task scheduling.

[0005] According to one aspect of the present invention, an AI-based intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage is provided, comprising: Images and meteorological data are collected by laser PTZ camera and sensor NCU, a hardware clock synchronization mechanism is established to realize timestamp calibration, and the data is stored in pairs according to a unified time base to form a multi-dimensional data fusion matrix. A sliding window analysis is performed on the fusion matrix to construct a Kalman filter model to predict normal state parameters. When the actual parameter deviation exceeds the dynamic threshold, an anomaly marker is triggered, and false alarms are eliminated through multi-sensor cross-validation. When an abnormal event is marked as bird intrusion, the bird trajectory is predicted based on Kalman filtering, and the lead compensation parameters are calculated by combining wind speed and wind direction. The predictive laser positioning algorithm is then used to drive the birds away before they reach the photovoltaic modules. When an abnormal event is marked as a device failure, a constrained Markov decision process optimization model is constructed. Taking into account the distribution of abnormal areas and energy consumption constraints, the optimal access sequence is calculated to guide the UAV to perform inspection tasks.

[0006] Furthermore, the method for constructing the multidimensional data fusion matrix includes: Perform grayscale processing on each image and extract image brightness feature values, texture feature values, and motion feature values; The extracted image feature values ​​are combined with the wind speed, wind direction, and irradiance values ​​at the same moment to form a multi-element data vector; Data is organized using preset time windows. All data vectors collected within each time window are arranged in chronological order to form a fusion matrix, with each column of the matrix corresponding to a different feature dimension.

[0007] Furthermore, the method for constructing the Kalman filter model prediction includes: Set a sliding window of a preset length, containing a complete multidimensional data vector at multiple consecutive time points; Based on data from consecutive moments within a sliding window, the state transition coefficients are determined by calculating the numerical changes of each dimension between adjacent moments. Construct a state transition matrix, and perform calculations based on this matrix and the actual state vector at the current time step to obtain the predicted values ​​of each dimension parameter at the next time step.

[0008] Furthermore, the predictive laser positioning algorithm includes: By comparing image frames at three consecutive time points, the positional changes of birds in the image coordinate system are identified, and pixel coordinates are converted into actual spatial coordinates. The instantaneous velocity and acceleration vectors of birds were calculated using the numerical difference method; Predict the future location of birds using kinematic equations.

[0009] Furthermore, the calculation of the control parameters for the laser bird deterrent device includes: Calculate the estimated time for birds to reach the surface of the photovoltaic module; calculate the total response delay time of the laser bird deterrent device; determine the optimal start-up time of the laser; calculate the horizontal turning angle and vertical pitch angle of the laser.

[0010] Furthermore, the constrained Markov decision process UAV inspection sequence optimization model includes: Construct the state space of a discrete-time Markov decision process, where each state consists of core elements including the UAV's location information, remaining battery power, mission completion status, and time information. The state transition probability is calculated by considering the joint probability of three independent factors: the probability of successful flight, the probability of completing the detection mission, and the probability of satisfying the constraints. Construct a comprehensive cost function, which includes a weighted sum of five dimensions: distance cost, time cost, energy cost, risk cost, and priority cost; Energy consumption constraints, time window constraints, flight safety constraints, and communication constraints are integrated as hard constraints.

[0011] Furthermore, the method for solving the optimization model includes: A three-layer decomposition optimization strategy is adopted. The first layer divides the region, dividing all anomalies into several sub-regions based on their spatial location and logical correlation. The second layer optimizes within each sub-region, independently solving for the locally optimal access order for each sub-region. The third layer optimizes the global sequence, treating each sub-region as a super node to construct a high-level constraint optimization model. The optimal strategy of the constrained Markov decision process optimization model is solved by using the value iteration algorithm, which calculates the optimal value function of each state in reverse from the terminal state after the task is completed. Based on the optimal strategy obtained from the solution, starting from the initial state of the UAV, the next abnormal point to be visited is selected in sequence according to the optimal strategy to generate a complete visit sequence.

[0012] Furthermore, the multi-sensor data cross-validation method includes: If an abnormal deviation is detected in the image brightness dimension, the reading change of the irradiance sensor at the same time is queried. If the irradiance sensor shows that the reading change is within the preset normal range, and the image brightness suddenly decreases above the preset threshold in a specific area, it is determined that the image is blocked by a flying object. If the image motion feature dimension identifies a non-linear, irregular motion trajectory, check whether the wind speed sensor displays corresponding airflow disturbances. If the wind speed sensor reading remains within the preset normal fluctuation range, and a biological flight motion pattern appears in the image, then it is determined that an actively flying organism has been detected. If a high-frequency texture pattern is detected in the image texture feature dimension, the texture region is simultaneously analyzed to determine whether it is accompanied by corresponding local brightness changes and motion trajectories, thereby identifying texture patterns with biological characteristics.

[0013] Furthermore, the calculation of the lead compensation parameter includes: The values ​​from wind speed and wind direction sensors are converted into three-dimensional wind speed vectors; the angle between the bird's flight direction and the wind direction is calculated; a correction model of wind force on bird flight speed is established, and the lateral offset caused by crosswinds is calculated; the corrected trajectory prediction results considering the influence of wind force are obtained.

[0014] According to another aspect of the present invention, an AI-powered intelligent operation and maintenance system for photovoltaic power plants based on image recognition and drone linkage is provided, comprising: The acquisition module collects images and meteorological data through a laser PTZ camera and NCU sensor, establishes a hardware clock synchronization mechanism to achieve timestamp calibration, stores data in pairs according to a unified time base, and forms a multi-dimensional data fusion matrix. The labeling module performs sliding window analysis on the fusion matrix, constructs a Kalman filter model to predict normal state parameters, and triggers anomaly labeling when the actual parameter deviation exceeds the dynamic threshold. False alarms are eliminated through multi-sensor cross-validation. The bird removal module predicts bird trajectories based on Kalman filtering when an abnormal event is marked as bird intrusion. It calculates lead compensation parameters by combining wind speed and wind direction, and initiates the bird removal process before the birds reach the photovoltaic modules using a predictive laser positioning algorithm. The detection module constructs a constrained Markov decision process optimization model when an abnormal event is marked as a device failure. It integrates the distribution of abnormal areas and energy consumption constraints to calculate the optimal access sequence to guide the UAV to perform inspection tasks.

[0015] Compared with existing technologies, this invention utilizes a multi-data acquisition system combining a laser PTZ camera and a sensor NCU to establish a clock synchronization mechanism for timestamp calibration, thereby improving the efficiency of multi-dimensional data acquisition and fusion. It introduces sliding window analysis and a Kalman filter model to dynamically predict normal state parameters of the photovoltaic power station, significantly improving the accuracy of anomaly detection and effectively reducing false alarm rates through multi-sensor cross-validation. When the anomaly is bird intrusion, trajectory prediction and laser positioning algorithms are used for precise deterrence, improving component protection capabilities. When the anomaly is equipment failure, an optimization model based on a constrained Markov decision process is used to intelligently plan inspection paths and task sequences, significantly improving UAV inspection efficiency and reducing energy consumption. This invention can significantly reduce maintenance costs while improving the accuracy, efficiency, and reliability of photovoltaic power station operation and maintenance, achieving a new breakthrough in intelligent operation and maintenance of photovoltaic power stations. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of an AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the meteorological data acquisition station equipment layout of the present invention; Figure 3 This is a schematic diagram showing the division of power plant areas and coordinate labels according to the present invention.

[0017] In the diagram: 1. Lightning rod; 2. Anemometer; 3. Irradiance meter; 4. Wind direction indicator; 5. Laser bird detector; 6. Camera; 7. Upper column; 8. Lower column; 9. Tracking bracket NCU; 10. Foundation. Detailed Implementation

[0018] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0019] As mentioned in the background section above, existing technologies have three main problems: First, they cannot take into account the real-time fusion and efficient analysis of multi-dimensional environmental and equipment data during data collection; second, existing anomaly detection algorithms are easily affected by environmental interference, resulting in prominent false alarms and missed alarms, and lack an accurate anomaly classification and processing mechanism; and third, drones have insufficient decision-making optimization capabilities in intelligent operation and maintenance task scheduling, making it difficult to effectively reduce inspection energy consumption or improve time efficiency.

[0020] Figure 1 This is a system block diagram of an AI-powered intelligent operation and maintenance system and method for photovoltaic power plants based on image recognition and drone linkage, according to an embodiment of the present invention. Figures 1 to 3 As shown, the AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone collaboration includes: S1: Images and meteorological data are collected through laser PTZ camera and sensor NCU, a hardware clock synchronization mechanism is established to realize timestamp calibration, and the data is stored in pairs according to a unified time base to form a multi-dimensional data fusion matrix.

[0021] A high-speed laser-integrated pan-tilt camera is installed on top of the meteorological data collection pole at the photovoltaic power station's transformer substation. This camera features 50x optical zoom and infrared night vision capabilities, allowing for 360-degree continuous rotation and an effective monitoring radius of up to 500 meters. The camera incorporates a high-precision image sensor, continuously acquiring high-definition image data from the photovoltaic power station at a frequency of 30 frames per second. The image resolution reaches 1920 x 1080 pixels, ensuring clear identification of distant bird activity and subtle changes on the surface of the components.

[0022] Meanwhile, a field sensor network control unit (NCU) is deployed in the middle of the meteorological data collection pole. This control unit serves as a data aggregation node, responsible for the unified management and coordination of data collection from various sensor devices. The NCU incorporates a high-precision quartz crystal oscillator, providing a stable local clock reference with an accuracy of one millionth of a second, thus providing a unified time reference for the entire data acquisition system.

[0023] Three types of meteorological sensors—an anemometer, wind direction meter, and irradiance meter—are installed at different heights on the pole. The anemometer uses a three-cup design, with a measurement range of 0 to 60 meters per second and an accuracy of ±0.1 meters per second, outputting a wind speed measurement once per second. The wind direction meter uses a wind vane design, with a measurement range of 0 to 360 degrees and an accuracy of ±3 degrees, monitoring wind direction changes in real time and outputting angle values. The irradiance meter uses a silicon photodiode sensor, measuring solar irradiance intensity from 0 to 2 kilowatts per square meter with an accuracy of ±5%, collecting irradiance intensity data once per second.

[0024] Furthermore, to establish a timestamp calibration mechanism for multi-source data, a dual calibration strategy using Network Time Protocol (NTP) and Global Positioning System (GPS) is adopted. The NCU control unit obtains the standard time signal from the NTP server via Ethernet connection, while simultaneously receiving precise time information from GPS satellites. The two time sources mutually verify each other, ensuring that the system clock maintains microsecond-level synchronization accuracy with Coordinated Universal Time (UTC). During data acquisition, the NCU automatically adds a high-precision timestamp based on a unified time base to each data point from all sensor devices. The timestamp format includes year, month, day, hour, minute, second, and millisecond information.

[0025] Based on the timestamp calibration mechanism, a multi-dimensional data fusion matrix is ​​constructed, specifically: Each image is converted to grayscale, and the average grayscale value of the entire image is calculated as the brightness feature value. An edge detection algorithm is used to count the number of edge pixels in the image as the texture feature value. Frame difference is used to detect moving regions and calculate the proportion of moving pixels to the total number of pixels as the motion feature value. After these three image feature values ​​are extracted, they are combined with the anemometer values, wind direction angle values, and irradiance intensity values ​​collected at the same time to form a six-element data vector. If the data is organized in five-minute time windows, all data vectors collected within each five-minute period are arranged in chronological order into a single row of a matrix. The six columns of the matrix correspond to image brightness features, image texture features, image motion features, wind speed values, wind direction angles, and irradiance intensity, respectively. Data is continuously collected for one hour, generating a twelve-row, six-column data fusion matrix, with each row representing the comprehensive data status within a five-minute time window. If the anemometer data is missing within a certain time window, the two most recent valid wind speed values ​​before and after the missing time are found, and their arithmetic mean is calculated and filled into the corresponding position in the matrix to ensure that there are no empty elements in the matrix, thus forming a complete time-series synchronized multidimensional data fusion matrix structure.

[0026] S2: Perform sliding window analysis on the fusion matrix, construct a Kalman filter model to predict normal state parameters, trigger an anomaly flag when the actual parameter deviation exceeds the dynamic threshold, and eliminate false alarms through multi-sensor cross-validation.

[0027] Sliding time window analysis is performed based on the acquired multidimensional data fusion matrix. First, a fixed-length time window is set to extract continuous historical data segments for analysis.

[0028] The sliding window length is set to three time points. The current analysis window contains a complete six-dimensional data vector at time T1, T2, and T3. Each vector contains parameter values ​​for six dimensions: image brightness features, image texture features, image motion features, wind speed, wind direction angle, and irradiance.

[0029] The window slides forward one time step, discarding the earliest data at time T1 and incorporating the new data at time T4, forming a new analysis window that includes times T2, T3, and T4. This continuous sliding method ensures that the analysis process can capture the dynamic trends of the data.

[0030] Furthermore, to construct a Kalman filter state estimation model within the sliding window to predict the system's normal state parameters at the next time step, a state transition matrix describing the system's dynamic behavior needs to be established. Based on data from three consecutive time steps within the window, the transition patterns of each dimension's parameters are estimated. The state transition coefficients are determined by calculating the numerical changes of each dimension between adjacent time steps. Specifically: Divide the brightness value at time T2 by the brightness value at time T1 to obtain the first transition ratio, and divide the brightness value at time T3 by the brightness value at time T2 to obtain the second transition ratio. Take the average of the two ratios as the state transition coefficient for the brightness dimension. The transition coefficients for other dimensions such as wind speed, image texture, image motion, wind direction, and irradiance are calculated in the same way. Then, the six transition coefficients are filled into the diagonal positions of a 6x6 state transition matrix. The off-diagonal elements of the matrix are set to zero, indicating that there is no cross-influence relationship between the dimensions.

[0031] Based on the constructed state transition matrix, the normal state parameters of the system at the next time step are predicted. The state transition matrix is ​​multiplied by the actual state vector at time T3, resulting in a prediction vector with six elements, representing the theoretical expected values ​​of each dimension's parameters at time T4. The predicted image brightness at time T4 is obtained by multiplying the brightness value at time T3 by the brightness transition coefficient, and the predicted wind speed is obtained by multiplying the wind speed value at time T3 by the wind speed transition coefficient. Predictions for other dimensions are calculated in the same way. Once all six predicted values ​​are obtained, a complete state prediction vector at time T4 is formed, representing the normal operating state at the next time step calculated based on the current system dynamic model.

[0032] After the actual sensor data acquisition is completed at time T4, the system compares the observed six-dimensional state vector with the six-dimensional state vector predicted by the Kalman filter model one by one. If the absolute difference between the actual acquired image brightness feature value and the predicted brightness value exceeds a preset brightness deviation threshold, an anomaly marker is triggered for that dimension. Similarly, if the deviation between the actual and predicted wind speed values ​​exceeds a wind speed deviation threshold, the wind speed dimension is also marked as an anomaly. If the deviations between the actual and predicted values ​​of other dimensions, such as image texture features, image motion features, wind direction angle, and irradiance, are all within their respective dynamic threshold ranges, these dimensions remain marked as normal. When the system detects that the deviation of any one or more of the six dimensions exceeds a threshold, an overall anomaly event marker is triggered, indicating that the current system state has deviated from the normal operating mode established based on historical data.

[0033] Furthermore, cross-validation based on multi-sensor data is used to eliminate false alarms caused by a single sensor failure. Specifically: When the system detects an abnormal deviation in the image brightness dimension, it immediately queries the specific reading of the irradiance sensor at the same moment to distinguish between bird obstruction and sensor malfunction. If the irradiance sensor shows a small fluctuation range of ±2% between the current and previous readings, but the image brightness suddenly decreases by more than 15% in a specific area of ​​the camera's field of view, the system determines that this local shadow change is caused by obstruction by a flying object rather than a change in the overall lighting environment, consistent with the obstruction characteristics of birds flying quickly across the lens. If the irradiance sensor also shows a corresponding decrease in light intensity, and the decrease is proportional to the change in image brightness, the system determines that it is a normal environmental change caused by cloud obstruction or weather changes, rather than an abnormal event. If the irradiance sensor reading remains normal, but the image brightness decreases uniformly across the entire field of view or irregular patchy dark areas appear, the system determines that the image sensor lens is contaminated, aging, or has an electronic circuit malfunction, and marks this anomaly as an equipment maintenance warning.

[0034] When the motion detection algorithm in the image motion feature dimension identifies non-linear, irregular motion trajectories in the central or upper region of the image, the system checks whether the wind speed sensor displays corresponding airflow disturbances. If the wind speed sensor reading remains within the normal fluctuation range of ±0.5 meters per second, and the wind direction sensor angle change is less than five degrees, but a clear wing-flapping motion pattern or zigzag flight trajectory appears in the image, the system determines that the detected motion signal originates from an actively flying organism. If the wind speed sensor shows strong winds exceeding three meters per second and wind direction changes exceeding ten degrees, and the image motion feature simultaneously displays a large-scale swaying or shaking pattern, the system determines it to be a normal environmental disturbance driven by wind. If both the wind speed and wind direction sensors display normal readings, but the image motion feature continuously displays repetitive shaking or regular swaying in a fixed area, the system determines that the camera bracket is loose or there is mechanical vibration interference, marking it as an equipment installation problem.

[0035] If the texture analysis algorithm for image texture features detects a high-frequency texture pattern that is significantly different from the background within the field of view, the system simultaneously analyzes whether the texture region is accompanied by corresponding local brightness changes and motion trajectories. If the texture complexity suddenly increases in a local area of ​​the image and a moving shadow effect appears simultaneously in that area, while the texture features of the surrounding background area remain stable, the system identifies the matching degree between the texture pattern with feather detail features and the bird's appearance features. If the texture complexity continuously increases at the image edge or at a fixed position without a corresponding motion trajectory, and the brightness change in that area exhibits a gradual rather than abrupt characteristic, the system determines that the image quality degradation is caused by dust accumulation, spider web coverage, or deposits on the lens surface. If the texture features show an increasing noise or blurring trend across the entire field of view, while other sensors display normal results, the system determines that the image sensor's electronic components are aging or the signal processing circuit is malfunctioning.

[0036] When the system detects the simultaneous presence of three visual features—irregular flight trajectories in image motion, partial occlusion and shadows in image brightness, and high-frequency feather-like textures in image texture—while the wind speed sensor reading remains calm or light, the wind direction sensor shows a stable direction, and the irradiance sensor shows uniform illumination—then the current detected event is confirmed as an active bird intrusion. If only a single image feature dimension is abnormal, while other dimensions and meteorological sensors are normal, the system determines it to be a device malfunction of the corresponding sensor. If the abnormal image features and abnormal meteorological parameters show a correlated change, the system determines it to be a normal environmental change rather than an abnormal event or device malfunction.

[0037] S3: When an abnormal event is marked as bird intrusion, the bird trajectory is predicted based on Kalman filtering, and the lead compensation parameters are calculated by combining wind speed and wind direction. The bird is driven away before it reaches the photovoltaic module by using a predictive laser positioning algorithm.

[0038] When the system identifies an abnormal event as a bird intrusion, the Kalman filter model first extracts the bird's position coordinates and trajectory information using continuous image data within the current sliding time window. By comparing image frames at three consecutive time points, it identifies the bird's positional changes in the image coordinate system and converts the pixel coordinates into actual spatial coordinates. If the bird is at time... , , The position coordinates are respectively , , ,in The axis represents the distance in the east-west direction. The axis represents the distance in the north-south direction. The axis represents the height above the ground, and all coordinates are in meters.

[0039] First, the instantaneous velocity vector of the bird is calculated using the position difference between adjacent time points, and the central difference method is used to improve the calculation accuracy. The formula is expressed as: , in, Indicates the first The position vector at time , These are the position vectors at three different times. This is the velocity vector.

[0040] Based on the velocity vector calculation results, the bird's acceleration vector is further calculated to capture changes in maneuverability and the curvature of its flight trajectory during flight. The acceleration calculation employs a second-order difference method, directly derived from the position data, and the formula is expressed as: , in, For time intervals.

[0041] Acceleration parameters reflect bird turning, climbing, or diving behaviors during flight, providing crucial information for predicting the curvature of future bird trajectories.

[0042] The Kalman filter uses calculated velocity and acceleration parameters to build a kinematic prediction model of bird flight, and predicts the bird's position at future moments using classical kinematic equations, expressed as: , in, This is the predicted future location vector of the bird. From the current moment The initial prediction timeframe.

[0043] After obtaining the basic bird movement prediction, real-time values ​​from wind speed and direction sensors are simultaneously read to analyze the impact of wind on bird flight paths. These scalar parameters are converted into three-dimensional wind speed vectors for vector calculations. Considering that birds are mainly affected by wind in the horizontal plane, the vertical wind speed component is set to zero. , in, For wind speed vectors, This refers to the scalar value of wind speed measured by the wind speed sensor. The zero-degree reference direction is due east, and the positive direction is counterclockwise.

[0044] Further calculations are made to determine the angle between the bird's flight direction and the wind direction to ascertain the wind's assist or drag effect on flight. The angle calculation utilizes the geometric relationship of the vector dot product, expressed by the formula: , The included angle parameter directly affects the extent to which wind force corrects the flight speed of birds. When the included angle is zero, the wind force acts entirely along the flight direction, while when the included angle is ninety degrees, the wind force mainly has a lateral effect.

[0045] Based on the calculated angle, a correction model for the effect of wind on bird flight speed is established. The corrected speed takes into account the boosting or dragging effect of wind, and the formula is expressed as: , in, This is the wind-corrected velocity vector of the bird. This is the original calculated bird velocity vector. This represents the wind force influence coefficient.

[0046] When the cosine of the angle is positive, it indicates flight with the wind, which increases the bird's effective flight speed. When the cosine is negative, it indicates flight against the wind, which decreases the flight speed.

[0047] Further considering the effect of crosswinds on the lateral deviation of bird flight paths, the trajectory deviation caused by the component of wind force perpendicular to the flight direction is calculated, as expressed by the formula: , in, The lateral offset vector caused by crosswinds. This is the crosswind influence coefficient. A horizontal unit vector perpendicular to the direction of bird flight.

[0048] The lateral offset is superimposed on the base trajectory prediction to obtain the complete trajectory considering the wind effect, expressed by the formula: , in, The final trajectory prediction location taking into account the influence of wind.

[0049] Based on wind-corrected trajectory prediction, the control parameters of the laser bird deterrent device are calculated. First, the estimated time for birds to reach the photovoltaic module surface is determined by solving for the intersection of the corrected trajectory and the photovoltaic panel plane. Let the installation height of the photovoltaic module be... The estimated time for birds to reach the surface of the photovoltaic module is: , in, The estimated time for birds to reach the surface of the photovoltaic modules. The installation height of the photovoltaic module above the ground is expressed in meters. For the current moment Bird's vertical height coordinates Velocity vector after wind correction The component in the vertical direction.

[0050] Further calculations were performed on the total response delay of the laser bird deterrent device, including the mechanical turning delay. Laser preheating delay And beam propagation delay. The formula is expressed as: , in, This refers to the total response delay time of the laser bird deterrent device. The response delay time of the laser mechanical steering system includes the action time of the horizontal and vertical steering mechanisms. This refers to the warm-up time of the laser from receiving a command to reaching a stable output power. The straight-line distance from the laser to the target interception point. Let be the speed at which light travels in a vacuum.

[0051] Since the beam propagation delay is relatively small, the main delays originate from the mechanical and thermal start-up processes. Therefore, based on delay calculations, the optimal start-up time of the laser is determined, expressed by the formula: , in, The optimal start-up time for the laser, starting from the current moment. Begin calculation. The estimated arrival time of birds at the photovoltaic modules, This represents the total system response delay time. This is to allow for a safety margin.

[0052] Furthermore, by calculating the angular parameters of the laser aiming at the interception point, the spatial coordinates of the interception point are first determined. The location of the interception point is determined by adding a delay time to the predicted location of the bird at the start time: , in, The three-dimensional spatial coordinates of the interception point. Indicates at time Wind-corrected trajectory prediction function; The actual interception time is the laser start-up time plus a system delay. The interception point coordinates are the ideal intersection of the laser beam and the bird's flight path.

[0053] Based on laser installation location Given the location of the interception point, calculate the horizontal turning angle of the laser, expressed by the formula: , in, This represents the laser's turning angle in the horizontal plane, with due east as the zero-degree reference. Here are the north-south coordinates of the interception point. Here are the north-south coordinates of the laser's installation location. The east-west coordinates of the interception point. The east-west coordinates of the laser's installation location.

[0054] Simultaneously, the vertical pitch angle is calculated to achieve precise aiming in three-dimensional space, expressed by the formula: , in, This represents the vertical pitch angle of the laser, with the horizontal plane as the zero-degree reference and upward as the positive angle. The vertical height coordinates of the interception point. This refers to the vertical height coordinates of the laser's installation location.

[0055] Finally, the laser intensity level is calculated based on the target distance and bird species, taking into account distance attenuation and the light sensitivity of different bird species. Laser intensity adjustment uses the inverse square law, expressed by the formula: , in, The adjusted laser output intensity level, The standard output intensity of the laser at a standard distance The intensity level at the location, For standard reference distance, This is the actual distance from the laser to the interception point. The light sensitivity correction coefficients for different bird species are as follows: 0.8 to 1.2 for large birds such as eagles, 1.0 to 1.5 for medium-sized birds such as pigeons, and 1.2 to 2.0 for small birds such as sparrows.

[0056] After calculating all parameters, the system sends control commands to the laser bird deterrent device, including start time, horizontal angle, vertical angle, and laser intensity level. For example... Figure 2As shown, after receiving a command, the laser bird deterrent device begins to execute preset mechanical actions, precisely activating laser emission at a calculated time point. The laser beam is directed at the location where the bird is about to pass through, following a pre-calculated angle. Because the system employs predictive control rather than a passive response mode, the laser beam can intervene to drive away birds before they even touch the photovoltaic module surface. By intercepting them in advance, it forces the birds to change their flight direction or abort their landing behavior, effectively avoiding the time lag problem of traditional monitoring systems where birds have already contacted or contaminated the photovoltaic equipment before initiating deterrent measures. This achieves proactive prevention and timely intervention against bird intrusion.

[0057] S4: When an abnormal event is marked as a device failure, construct a constrained Markov decision process optimization model, integrate the distribution of abnormal areas and energy consumption constraints, and calculate the optimal access sequence to guide the UAV to perform inspection tasks.

[0058] When the system detects a device malfunction, it first extracts information on all anomalies marked as malfunctioning devices from the fusion matrix at the current moment. This includes the three-dimensional spatial coordinates, device type, malfunction severity, and estimated detection time for each malfunctioning device. The system performs spatial analysis on these anomalies, calculating the straight-line distance and actual flight distance between any two points, identifying the spatial clustering characteristics and distribution density of the anomalies, and determining the feasible flight path for the UAV based on the physical layout of the photovoltaic array. Simultaneously, the system acquires current environmental parameters, including meteorological data such as wind speed and direction, temperature, humidity, and visibility, as well as the UAV's initial status information, including takeoff position, battery full charge capacity, and current battery level.

[0059] Based on the extracted anomaly information, the system constructs a state space for a discrete-time Markov decision process. Each state consists of five core elements: the UAV's three-dimensional position coordinates, the current remaining battery percentage, the set of anomaly identifiers that have been detected, the set of anomaly identifiers to be detected, and the timestamp of the current task execution. The system divides the entire inspection task's timeline into discrete decision moments, each corresponding to the moment when the UAV completes an anomaly detection and prepares to select the next target. The size of the state space is equal to the Cartesian product of all possible combinations of position, battery level, and task progress. The actual size of the state space is reduced by pre-excluding infeasible states where the battery is insufficient to reach any remaining anomalies.

[0060] Furthermore, a transition probability is calculated for each pair of adjacent states in the state space. This transition probability reflects the likelihood of the UAV successfully transitioning from its current state to the target state. The calculation of the transition probability considers the joint probability of three independent factors: the probability of successful flight, the probability of completing the detection task, and the probability of satisfying the time constraint. The probability of successful flight is calculated based on current wind speed conditions, flight distance, and UAV performance parameters; the probability decreases significantly when the wind speed exceeds the UAV's maximum wind resistance. The probability of completing the detection task is determined based on the detection complexity of target anomalies and historical equipment reliability data; the detection probability for complex faults is relatively low. The probability of satisfying the time constraint considers the possibility of the UAV completing the flight and detection tasks within a specified time window; transitions with a higher risk of timeout receive lower probability values. Multiplying these three probabilities yields the final state transition probability.

[0061] Based on the calculated state transition probabilities, a specific cost value is assigned to each executable flight action. The cost function comprehensively considers five dimensions: distance cost, time cost, energy cost, risk cost, and priority cost. Distance cost is proportional to the flight distance and includes a weighted sum of horizontal flight distance and vertical climb distance. Time cost is calculated based on the sum of flight time and hovering detection time, with flight time taking into account the UAV's acceleration / deceleration process and wind resistance. Energy cost is calculated through a detailed energy consumption model, including the accumulation of propulsion system energy consumption, detection equipment energy consumption, communication system energy consumption, and environmental adaptation energy consumption. Risk cost is determined according to the safety level of the flight path; paths passing through no-fly zones or communication blind spots incur higher risk costs. Priority cost is set based on the severity of anomalies and time urgency; higher-priority anomalies correspond to lower cost values. The five cost components are weighted and summed according to preset weights to obtain the comprehensive cost value.

[0062] Furthermore, the constructed state space, transition probability matrix, and action cost function are integrated into a complete Markov decision process optimization model. This model uses minimizing the expected total cost as its objective function, specifically expressed as the sum of the probability-weighted costs of all possible paths from the initial state to the final state. The decision variable is the optimal action chosen for each state, forming the policy function that maps states to optimal actions. The model adheres to Markov property constraints, meaning the choice of the next state depends only on the current state and the chosen action, and is independent of historical states. The model also includes state transition consistency constraints, ensuring that the probability transition matrix satisfies the basic properties of a random matrix: the sum of probabilities in each row equals 1 and all probability values ​​are non-negative.

[0063] Energy consumption constraints, time window constraints, flight safety constraints, and communication constraints are integrated as hard constraints into the basic Markov decision process optimization model, forming a constrained Markov decision process problem. The energy consumption constraint requires that the UAV's remaining battery power after any flight maneuver must be sufficient to safely return to the takeoff and landing point. The system calculates the energy consumption of the shortest path from the target anomaly point back to the base as a safety threshold. The time window constraint requires that critical anomalies be detected within a specified time. The system checks whether the earliest possible time to reach the target point from the current moment meets the time window requirement. The flight safety constraint prohibits the UAV from entering predefined danger zones or performing high-risk flights under adverse weather conditions. The system filters unsafe flight maneuvers based on real-time meteorological data and safety rules. The communication constraint requires the UAV to maintain a stable communication connection with the control center throughout the flight. The system eliminates flight paths within communication blind spots based on the signal coverage map. These constraints are implemented by setting infeasibility markers during state transitions; a state transition that violates any constraint is assigned a zero probability or an infinite cost.

[0064] Multi-objective optimization techniques are employed to balance conflicts between different optimization objectives. The main optimization objectives include: minimizing total flight time to improve inspection efficiency; minimizing total energy consumption to ensure all inspection tasks can be completed in a single flight; maximizing detection quality to ensure accurate fault identification; and minimizing flight risk to ensure the safety of the UAV and equipment. A weighted sum method is used to combine multiple objective functions into a single comprehensive objective function. The weight coefficients are dynamically adjusted based on the current task priority and system state: when a high-risk fault is detected, the system increases the weight of the time efficiency objective to accelerate response; when battery power drops to a warning level, the system increases the weight of the energy consumption optimization objective to ensure task completion; when weather conditions worsen, the system increases the weight of the flight safety objective to reduce accident risk; and when a critical equipment fault is detected, the system increases the weight of the detection quality objective to ensure diagnostic accuracy. Weight adjustment uses fuzzy logic rules, calculating the weight coefficients of each objective based on the membership function of the input parameters and updating the objective function expression of the constrained optimization model.

[0065] To address the state-space explosion problem caused by a large-scale distribution of outliers, a three-layer decomposition optimization strategy is implemented for the constrained Markov decision process (CPM) optimization model to reduce computational complexity and ensure solution quality. The first layer involves region partitioning, dividing all outliers into several sub-regions based on their spatial location and logical relationships. The number of outliers in each sub-region is controlled within a preset threshold, thus decomposing the original optimization model into multiple sub-models. The second layer performs sub-regional optimization, independently solving for the locally optimal access order in each sub-region's CPM sub-model using a small-scale dynamic programming algorithm to ensure computational efficiency. The third layer performs global sequence optimization, treating each sub-region as a supernode to construct a high-level constrained optimization model, optimizing the access order between sub-regions to obtain the globally optimal solution, while also considering local adjustments based on the connectivity relationships of outliers at sub-region boundaries.

[0066] A value iteration algorithm is employed to solve for the optimal policy in a constrained Markov decision process optimization model. Starting from the terminal state after task completion, the optimal value function for each state is calculated backwards. During the algorithm iteration, the expected reward of all possible actions satisfying the constraints is calculated for each state, and the action with the highest expected reward is selected as the optimal policy for that state. The value function is updated using a recursive form of the Bellman equation, adding the immediate reward to the discounted value of subsequent states while ensuring all constraints are satisfied. To accelerate convergence, a priority scanning technique is used, prioritizing the updating of states with significant value changes, and a convergence criterion is used to detect the algorithm's termination condition. When the state space is too large, an approximation iteration method is used, representing the value function through neural networks or linear functions while maintaining precise constraint satisfaction.

[0067] Based on the optimal strategy obtained from the solution, starting from the initial state of the UAV, the next abnormal point to be visited is selected sequentially according to the optimal strategy, generating a complete visit sequence. The generated visit sequence is fully verified to check its feasibility, including whether energy consumption constraints, time window constraints, and flight safety requirements are met. If a constraint violation is found, the process backtracks to the previous node in the decision tree to reselect a suboptimal action.

[0068] After the UAV begins executing the optimal access sequence, a real-time monitoring mechanism is established to track the task execution status, including the UAV's position and trajectory, battery level changes, detection task progress, and changes in environmental conditions. When the system detects an execution deviation or a new abnormal device, a dynamic replanning process is immediately initiated. The replanning process uses the current UAV state as the new starting state, treats the remaining unfinished anomalies as a new task set, reconstructs the constrained Markov decision process optimization model, and executes the complete solution process. To improve replanning efficiency, a rolling time-domain optimization method is adopted, recalculating only the optimal strategy within a finite future time window and using the value function from the previous solution as the initial estimate. When the detection results indicate that an anomaly does not require further processing, it is removed from the list of devices to be accessed, and the subsequent access sequence is adjusted accordingly.

[0069] This optimization method based on Markov decision processes can generate high-quality UAV inspection sequences under complex constraints, ensuring efficient completion of data collection and fault confirmation tasks for all abnormal areas within a single flight cycle.

[0070] In summary, the AI-powered intelligent operation and maintenance system and method for photovoltaic power plants based on image recognition and drone linkage, as described in this invention, are elucidated. It utilizes a multi-data acquisition system combining a laser PTZ camera and a sensor NCU to establish a clock synchronization mechanism for timestamp calibration, thereby improving the efficiency of multi-dimensional data acquisition and fusion. By introducing sliding window analysis and a Kalman filter model, it dynamically predicts the normal state parameters of the photovoltaic power plant, significantly improving the accuracy of anomaly detection. Furthermore, multi-sensor cross-validation effectively reduces the false alarm rate. When the anomaly is a bird intrusion, trajectory prediction and laser positioning algorithms are used for precise bird deterrence, enhancing component protection capabilities. When the anomaly is an equipment failure, an optimization model based on a constrained Markov decision process is employed to intelligently plan inspection paths and task sequences, significantly improving drone inspection efficiency and reducing energy consumption. This system can significantly reduce operation and maintenance costs while improving the accuracy, efficiency, and reliability of photovoltaic power plant operation and maintenance, achieving a new breakthrough in intelligent operation and maintenance of photovoltaic power plants.

Claims

1. A photovoltaic power plant AI-powered intelligent operation and maintenance method based on image recognition and drone collaboration, characterized in that, include: Images and meteorological data are collected by laser PTZ camera and sensor NCU, a hardware clock synchronization mechanism is established to realize timestamp calibration, and the data is stored in pairs according to a unified time base to form a multi-dimensional data fusion matrix. A sliding window analysis is performed on the fusion matrix to construct a Kalman filter model to predict normal state parameters. When the actual parameter deviation exceeds the dynamic threshold, an anomaly marker is triggered, and false alarms are eliminated through multi-sensor cross-validation. When an abnormal event is marked as bird intrusion, the bird trajectory is predicted based on Kalman filtering, and the lead compensation parameters are calculated by combining wind speed and wind direction. The predictive laser positioning algorithm is then used to drive the birds away before they reach the photovoltaic modules. When an abnormal event is marked as a device failure, a constrained Markov decision process optimization model is constructed. Taking into account the distribution of abnormal areas and energy consumption constraints, the optimal access sequence is calculated to guide the UAV to perform inspection tasks.

2. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage as described in claim 1, characterized in that, The method for constructing the multidimensional data fusion matrix includes: Perform grayscale processing on each image and extract image brightness feature values, texture feature values, and motion feature values; The extracted image feature values ​​are combined with the wind speed, wind direction, and irradiance values ​​at the same moment to form a multi-element data vector; Data is organized using preset time windows. All data vectors collected within each time window are arranged in chronological order to form a fusion matrix, with each column of the matrix corresponding to a different feature dimension.

3. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage as described in claim 2, characterized in that, The method for constructing the predictions of the Kalman filter model includes: Set a sliding window of a preset length, containing a complete multidimensional data vector at multiple consecutive time points; Based on data from consecutive moments within a sliding window, the state transition coefficients are determined by calculating the numerical changes of each dimension between adjacent moments. Construct a state transition matrix, and perform calculations based on this matrix and the actual state vector at the current time step to obtain the predicted values ​​of each dimension parameter at the next time step.

4. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage as described in claim 3, characterized in that, The predictive laser positioning algorithm includes: By comparing image frames at three consecutive time points, the positional changes of birds in the image coordinate system are identified, and pixel coordinates are converted into actual spatial coordinates. The instantaneous velocity and acceleration vectors of birds were calculated using the numerical difference method; Predict the future location of birds using kinematic equations.

5. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage according to claim 4, characterized in that, The calculation of control parameters for laser bird deterrence devices includes: Calculate the estimated time for birds to reach the surface of the photovoltaic module; calculate the total response delay time of the laser bird deterrent device; determine the optimal start-up time of the laser; calculate the horizontal turning angle and vertical pitch angle of the laser.

6. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage as described in claim 5, characterized in that, Constrained Markov decision process UAV inspection sequence optimization models include: Construct the state space of a discrete-time Markov decision process, where each state consists of core elements including the UAV's location information, remaining battery power, mission completion status, and time information. The state transition probability is calculated by considering the joint probability of three independent factors: the probability of successful flight, the probability of completing the detection mission, and the probability of satisfying the constraints. Construct a comprehensive cost function, which includes a weighted sum of five dimensions: distance cost, time cost, energy cost, risk cost, and priority cost; Energy consumption constraints, time window constraints, flight safety constraints, and communication constraints are integrated as hard constraints.

7. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage according to claim 6, characterized in that, The optimization model solution method includes: A three-layer decomposition optimization strategy is adopted. The first layer divides the region, dividing all anomalies into several sub-regions based on their spatial location and logical correlation. The second layer optimizes within each sub-region, independently solving for the locally optimal access order for each sub-region. The third layer optimizes the global sequence, treating each sub-region as a super node to construct a high-level constraint optimization model. The optimal strategy of the constrained Markov decision process optimization model is solved by using the value iteration algorithm, which calculates the optimal value function of each state in reverse from the terminal state after the task is completed. Based on the optimal strategy obtained from the solution, starting from the initial state of the UAV, the next abnormal point to be visited is selected in sequence according to the optimal strategy to generate a complete visit sequence.

8. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage as described in claim 7, characterized in that, The multi-sensor data cross-validation method includes: If an abnormal deviation is detected in the image brightness dimension, the reading change of the irradiance sensor at the same time is queried. If the irradiance sensor shows that the reading change is within the preset normal range, and the image brightness suddenly decreases above the preset threshold in a specific area, it is determined that the image is blocked by a flying object. If the image motion feature dimension identifies a non-linear, irregular motion trajectory, check whether the wind speed sensor displays corresponding airflow disturbances. If the wind speed sensor reading remains within the preset normal fluctuation range, and a biological flight motion pattern appears in the image, then it is determined that an actively flying organism has been detected. If a high-frequency texture pattern is detected in the image texture feature dimension, the texture region is simultaneously analyzed to determine whether it is accompanied by corresponding local brightness changes and motion trajectories, thereby identifying texture patterns with biological characteristics.

9. The AI-powered intelligent operation and maintenance method for photovoltaic power plants based on image recognition and drone linkage as described in claim 8, characterized in that, The calculation of the advance compensation parameter includes: The values ​​from wind speed and wind direction sensors are converted into three-dimensional wind speed vectors; the angle between the bird's flight direction and the wind direction is calculated; a correction model for the effect of wind on the bird's flight speed is established, and the lateral offset caused by crosswinds is calculated; the corrected trajectory prediction results considering the influence of wind are obtained.

10. A photovoltaic power plant AI intelligent operation and maintenance system based on image recognition and drone linkage, characterized in that: include: The acquisition module collects images and meteorological data through a laser PTZ camera and NCU sensor, establishes a hardware clock synchronization mechanism to achieve timestamp calibration, stores data in pairs according to a unified time base, and forms a multi-dimensional data fusion matrix. The labeling module performs sliding window analysis on the fusion matrix, constructs a Kalman filter model to predict normal state parameters, and triggers anomaly labeling when the actual parameter deviation exceeds the dynamic threshold. False alarms are eliminated through multi-sensor cross-validation. The bird removal module predicts bird trajectories based on Kalman filtering when an abnormal event is marked as bird intrusion. It calculates lead compensation parameters by combining wind speed and wind direction, and initiates the bird removal process before the birds reach the photovoltaic modules using a predictive laser positioning algorithm. The detection module constructs a constrained Markov decision process optimization model when an abnormal event is marked as a device failure. It integrates the distribution of abnormal areas and energy consumption constraints to calculate the optimal access sequence to guide the UAV to perform inspection tasks.

Images