New energy photovoltaic dynamic inspection method and system based on artificial intelligence

Abstract

The application provides a new energy photovoltaic dynamic inspection method and system based on artificial intelligence, and relates to the technical field of intelligent inspection of photovoltaic power stations.The application triggers inspection according to weather warnings, performance alarms or timing tasks, carries out initial path planning in combination with terrain, weather and historical data, and updates the path dynamically through an RRT* algorithm, collects multi-modal data including visible light images, infrared thermal imaging, EL detection data and positioning data during unmanned aerial vehicle inspection, integrates unmanned aerial vehicle data and ground sensor data to generate a unified fault feature matrix, uses a deep neural network to locate a defect area in real time, judge a defect type and divide a fault grade, and finally scores a photovoltaic system health degree according to the fault grade and analyzes a safe operation trend.The application realizes real-time fusion of multi-modal data and dynamic path planning, improves fault recognition accuracy and inspection efficiency, reduces artificial inspection cost and risk, and provides strong support for intelligent operation and maintenance of photovoltaic systems.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent inspection of photovoltaic power stations, and in particular to a new energy photovoltaic dynamic inspection method and system based on artificial intelligence. Background Art

[0002] With the rapid development of the photovoltaic power generation industry, the scale and number of photovoltaic power plants are constantly increasing, creating an increasingly urgent need for efficient and accurate inspection technology. Traditional methods for inspecting photovoltaic power plants primarily include manual inspections, static monitoring, and drone inspections. Traditional manual inspections are inefficient, with a single inspection of a 100MW power plant taking 5 days or more, and a missed inspection rate exceeding 15%. There are also risks associated with working at height. Static monitoring also presents efficiency and accuracy challenges, as fixed cameras or infrared thermal imagers cannot cover complex terrain and struggle to track component status changes in real time. Furthermore, fault identification in both of these methods often relies on manual analysis, resulting in long response times, averaging over 48 hours.

[0003] In traditional drone inspection methods, path planning for existing drone inspection missions is often based on pre-set fixed routes or simple rule-based planning. This makes it difficult to dynamically adjust and optimize based on real-time environmental information (such as terrain and weather conditions) and the status of the photovoltaic system. Furthermore, when encountering obstacles, the obstacle avoidance algorithms lack real-time performance and accuracy, making it impossible to quickly and efficiently avoid obstacles and replan the route. Existing fault identification algorithms are mostly based on traditional image processing techniques or simple machine learning models. They do not achieve real-time fusion of multimodal data and have limited accuracy in identifying the complex and diverse fault types of photovoltaic modules (such as cracks, hot spots, and package aging), making them difficult to meet the requirements of efficient and accurate inspections. Furthermore, these algorithms often lack the ability to classify fault levels, failing to provide a comprehensive and accurate basis for operation and maintenance decisions.

[0004] In summary, the existing photovoltaic inspection technology has obvious shortcomings in multimodal data fusion, dynamic path planning, fault identification accuracy, etc. There is an urgent need for a more intelligent and efficient photovoltaic dynamic inspection method to overcome the above defects and improve the operation and maintenance efficiency and reliability of the photovoltaic system. Summary of the Invention

[0005] The present invention aims to solve at least one of the above-mentioned technical problems existing in the prior art.

[0006] To this end, the first aspect of the present invention provides a new energy photovoltaic dynamic inspection method based on artificial intelligence.

[0007] The second aspect of the present invention provides a new energy photovoltaic dynamic inspection system based on artificial intelligence.

[0008] The present invention provides a new energy photovoltaic dynamic inspection method based on artificial intelligence, comprising:

[0009] Trigger corresponding inspection tasks based on weather warnings, photovoltaic system performance alarms or scheduled tasks;

[0010] Initial path planning is performed based on inspection tasks using terrain data, meteorological data, and historical flight data. A dynamic obstacle avoidance algorithm is used to avoid sudden obstacles along the path and update the path. During initial path planning, a reinforcement learning model is used to generate a flight trajectory covering all points of interest. The dynamic obstacle avoidance algorithm uses the RRT* algorithm.

[0011] The drone performs inspection tasks along the planned route; during the inspection process, it collects multimodal data of the photovoltaic system; the multimodal data includes visible light images, infrared thermal imaging, EL detection data, and positioning data;

[0012] Obtain data collected by ground sensors deployed in the photovoltaic system, integrate the data collected by the ground sensors and the data collected by drones, and generate a unified fault feature matrix;

[0013] Locating defective areas of the photovoltaic system using a real-time target detection algorithm based on a deep neural network according to the unified fault feature matrix, determining defect types of the defective areas using a deep convolutional neural network, and classifying fault levels of the defective areas;

[0014] The health of the PV system is scored according to the fault level of each fault area, and the health score is used to analyze the safe operation trend of the PV system.

[0015] The artificial intelligence-based dynamic inspection method for new energy photovoltaics according to the above technical solution of the present invention may also have the following additional technical features:

[0016] In the above technical solution, the types of inspection tasks include:

[0017] Scheduled inspection tasks, regular inspection tasks that start automatically at the set time;

[0018] Weather warning tasks are temporary inspection tasks triggered before or after monitoring of special weather conditions, such as sandstorms and hail.

[0019] Performance alarm task: a targeted inspection task triggered when abnormal power generation or component temperature in some areas of the PV system is detected;

[0020] In each inspection mission, the accident points found in previous inspection missions of the same type are obtained and marked as high-risk areas. The flight trajectory of this inspection mission needs to cover the high-risk areas.

[0021] In the above technical solution, the generation of a flight trajectory covering all points of interest using a reinforcement learning model includes:

[0022] Use the topographic data of the area where the photovoltaic system is located as the spatial benchmark and prioritize the inspection areas;

[0023] The artificial intelligence reinforcement learning model integrates risk weights and obstacle avoidance data to generate a global path; during the drone inspection flight, flight cost and value are estimated for benefit evaluation of route planning, and flight energy consumption estimation is introduced to ensure that the overall flight route is within the range.

[0024] In the above technical solution, the method for finding the optimal strategy parameters in the artificial intelligence reinforcement learning model is:

[0025]

[0026] in, The neural network parameters representing the next round of drone control action selection; Represents the function of finding the optimal strategy parameters; Indicates expected value; Indicates the degree of strategy change, that is, the probability ratio of the new and old strategies to select actions in the current state, represents the probability of the new action, represents the old action probability; represents the advantage function, which is used to evaluate the goodness of the current action relative to the average level; Represents a range-limiting function; represents the strategy update amplitude limit; t represents the time step; k represents the number of update rounds;

[0027] The advantage function is calculated as follows:

[0028]

[0029] in, Represents the action value function, which is used to evaluate the action in state Next action long-term benefits; Represents the state value function, used to evaluate the state average income; Indicates the power consumption weight.

[0030] In the above technical solution, the method of avoiding sudden obstacles on the path by using a dynamic obstacle avoidance algorithm includes:

[0031] Calculate the total path cost from the starting point to the new node, select the path with the minimum total cost, and dynamically balance obstacle avoidance and path length. The calculation method of the total path cost includes:

[0032]

[0033] in, represents the total path cost from the starting point to the new node; represents the cumulative cost from the starting point to the nearest explored node; represents the Euclidean distance from the most recently explored node to the new node; represents the risk weight coefficient, which is used to adjust the sensitivity of the path to the risk area; Indicates the risk value of the new node's location;

[0034] The risk value of the location of the new node is calculated based on historical fault data and real-time meteorological dynamics. The calculation method includes:

[0035]

[0036] in, represents the historical failure density at the location of the new node; Indicates the real-time wind speed influence coefficient.

[0037] In the above technical solution, the feature integration of the data collected by the ground sensors and the data collected by the drone to generate a unified fault feature matrix includes:

[0038] Interpolate and align ground sensor data based on the drone GPS timestamp;

[0039] The Transformer model is used to align the data collected by ground sensors and drones in time and space, mapping the image data and sensor data to a unified coordinate system to generate a unified fault feature matrix.

[0040] Among them, the Transformer model is used to perform scaled dot product attention analysis on the collected data; the expression of scaled dot product attention is:

[0041]

[0042] in, Represents the scaled dot product attention output result; represents the query matrix, represents the bond matrix, Represents the value matrix, using 、 、 They represent the characteristics of different modes of the photovoltaic system; represents the normalization function; represents the transposed matrix; represents the key vector dimension;

[0043] And through the multi-head attention mechanism, features of different dimensions are spliced ​​and integrated to obtain a unified fault feature matrix; the expression of the multi-head attention mechanism is:

[0044]

[0045] in, Represents the output of the multi-head attention mechanism; represents the splicing function; represents the output of the i-th attention head, ; Represents the output linear transformation matrix; in each attention head, different features are extracted from the input data through different linear transformation matrices.

[0046] In the above technical solution, the real-time target detection algorithm based on deep neural network adopts YOLOv10 algorithm;

[0047] In the YOLOv10 algorithm, the loss function of the YOLOv10 algorithm is constructed by three-stage collaborative optimization of classification loss, regression loss, and distribution focus loss; the loss function of the YOLOv10 algorithm is expressed as:

[0048]

[0049] in, represents the total loss of target detection; represents the classification loss weight; Represents the classification loss error; represents the regression loss weight; Represents the regression loss error; represents the distribution focusing loss weight; Represents the distributed focusing loss error.

[0050] In the above technical solution, the method for calculating the classification loss error includes:

[0051]

[0052] in, Represents the true category distribution, used to label the defect type of photovoltaic panels; Represents the predicted category probability distribution, that is, the confidence level of the fault type output by the model;

[0053] The calculation method of the regression loss error includes:

[0054]

[0055] in, Represents the intersection-over-union ratio, that is, the degree of overlap between the predicted box and the true box; Represents the Euclidean distance between the center point of the predicted box and the true box; Indicates the center point position of the prediction box; Indicates the center point position of the real frame; Represents the diagonal length of the smallest rectangle containing the predicted box and the true box; Represents the aspect ratio difference coefficient, which is used to measure the difference in the width and height ratio between the predicted box and the real box; represents the aspect ratio difference weight;

[0056] The calculation method of the distributed focusing loss error includes:

[0057]

[0058] in, Represents the probability distribution of bounding box coordinates, which is used to predict the length of hidden cracks in photovoltaic panels; Represents the true distribution value, that is, the discretized distribution of the true coordinates, which is used to mark data after Gaussian smoothing.

[0059] In the above technical solution, the deep convolutional neural network adopts the ResNet-50 network, and the ResNet-50 network includes four groups of residual blocks;

[0060] The first group of residual blocks is used to extract basic features of the photovoltaic panel, and the basic features of the photovoltaic panel include edge texture of the photovoltaic panel;

[0061] The second group of residual blocks is used to identify medium-scale defects in photovoltaic panels, including hidden cracks and stains;

[0062] The third group of residual blocks is used to detect small target defects, including solder joint detachment and cracks;

[0063] The fourth group of residual blocks is used for global feature fusion and classification decision.

[0064] The present invention also provides an artificial intelligence-based new energy photovoltaic dynamic inspection system, which is applied to the artificial intelligence-based new energy photovoltaic dynamic inspection method as described in any one of the above technical solutions, including:

[0065] Inspection task triggering module, triggering corresponding inspection tasks based on weather warnings, photovoltaic system performance alarms or scheduled tasks;

[0066] The dynamic path planning module performs initial path planning based on inspection tasks using terrain data, meteorological data, and historical flight data. It also uses a dynamic obstacle avoidance algorithm to avoid sudden obstacles along the path and update the path. During the initial path planning, a reinforcement learning model is used to generate a flight trajectory covering all points of interest. The dynamic obstacle avoidance algorithm uses the RRT* algorithm, enabling the drone to perform inspection tasks along the planned path.

[0067] The data acquisition module collects multimodal data of the photovoltaic system during the inspection process; the multimodal data includes visible light images, infrared thermal imaging, EL detection data and positioning data; and obtains the collected data of the ground sensors arranged in the photovoltaic system;

[0068] The edge data processing module integrates the features of the data collected by ground sensors and drones to generate a unified fault feature matrix;

[0069] An AI fault detection and classification module locates defective areas of the photovoltaic system using a real-time target detection algorithm based on a deep neural network according to the unified fault feature matrix, determines the defect type of the defective area using a deep convolutional neural network, and classifies the fault level of the defective area;

[0070] The fault handling module scores the health of the PV system according to the fault level of each fault area, uses the health score to analyze the safe operation trend of the PV system, and formulates a fault handling strategy.

[0071] In summary, due to the adoption of the above technical features, the beneficial effects of the present invention are:

[0072] This invention uses a reinforcement learning algorithm to optimize flight routes in real time. Combining meteorological data with component risk grading, it reduces ineffective inspection routes and improves efficiency by 20% compared to fixed routes. Through multimodal data fusion, this invention combines visible light, infrared thermal imaging, electroluminescence (EL), and electrical performance parameters for fault detection, achieving a fault detection accuracy exceeding 98% (e.g., 96.5% for hot spot detection and 98.2% for hidden crack detection). Using models such as YOLOv10, combined with knowledge distillation and lightweight design, it achieves real-time inference at the edge (latency ≤ 200ms) while maintaining high accuracy.

[0073] Specifically, this invention achieves intelligent operation and maintenance of photovoltaic power plants through multimodal perception, dynamic path planning, and edge cloud collaborative computing. Within the multimodal spatiotemporal alignment mechanism, a Transformer multi-head attention model is used to fuse visible light, infrared thermal imaging, and current sensor data to generate a 128×128×4-dimensional fault feature matrix, addressing cross-modal data heterogeneity. In the reinforcement learning-random sampling joint path planning, a PPO algorithm (reinforcement learning model) is used to generate paths covering high-risk areas. Combined with the RRT* dynamic obstacle avoidance algorithm (a random sampling-based path planning algorithm), this method achieves an inspection completion rate exceeding 98%. A lightweight federated learning architecture was constructed, and through knowledge distillation, the YOLOv10 defect area location model (confidence threshold > 0.9) and the ResNet-50 classification model were compressed to 300MB (latency ≤ 200ms). This allows for a single inspection of a 50MW power plant in ≤ 4 hours, a 9x improvement in efficiency compared to manual inspections. The final fault detection results achieved a 95.6% accuracy rate for detecting hidden cracks, with a missed detection rate of <2%. This invention overcomes the technical bottleneck of low accuracy and slow response of photovoltaic defect detection in complex environments. It has the characteristics of full automation, high robustness and evolution, and has been successfully applied to different scenes such as mountains and deserts.

[0074] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0076] Figure 1 This is a general flow chart of a new energy photovoltaic dynamic inspection method based on artificial intelligence according to an embodiment of the present invention;

[0077] Figure 2 This is a flowchart of the inspection task triggering, dynamic path planning, and data collection parts of the new energy photovoltaic dynamic inspection method based on artificial intelligence in one embodiment of the present invention;

[0078] Figure 3 This is a flow chart of the edge data processing and AI fault detection and classification parts of the artificial intelligence-based new energy photovoltaic dynamic inspection method according to one embodiment of the present invention;

[0079] Figure 4 This is a flowchart of the fault handling part of the new energy photovoltaic dynamic inspection method based on artificial intelligence in one embodiment of the present invention. DETAILED DESCRIPTION

[0080] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0081] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0082] Refer to the following Figures 1 to 4 To describe the new energy photovoltaic dynamic inspection method based on artificial intelligence provided according to some embodiments of the present invention.

[0083] Some embodiments of the present application provide a new energy photovoltaic dynamic inspection method based on artificial intelligence.

[0084] The first embodiment of the present invention proposes a new energy photovoltaic dynamic inspection method based on artificial intelligence, such as Figure 1 As shown, the following steps S1-S6 are included.

[0085] S1. Trigger corresponding inspection tasks based on weather warnings, photovoltaic system performance alarms or scheduled tasks.

[0086] like Figure 2 As shown, before the UAV starts flying, the type of inspection task should be determined. By distinguishing the task types, the important points of focus of this flight mission can be identified. According to the mission objectives, the tasks can be divided into two categories: meteorological-related tasks and system performance-related tasks. In some embodiments, the types of inspection tasks specifically include: scheduled inspection tasks, meteorological warning tasks, and performance warning tasks; among them, scheduled inspection tasks are Figure 2 The ordinary daily inspection tasks shown in the figure need to be triggered regularly under the premise that the system power generation performance has not degraded. The performance alarm task is a hardware fault inspection that is triggered when the system power generation performance has degraded.

[0087] The scheduled inspection task is a regular inspection task that is automatically started at a set time. In a specific embodiment, the scheduled inspection task needs to avoid the peak power generation of the photovoltaic system, for example, the regular inspection is automatically started at 3 am every day.

[0088] Weather warning missions are temporary inspection tasks triggered before or after monitoring of unusual weather conditions, such as sandstorms and hail. In one specific embodiment, an emergency inspection is triggered when the weather data interface receives weather forecast information indicating a sandstorm (PM2.5 > 300μg / m³) or hail warning. Note that inspections should be performed before and after unusual weather conditions to prevent damage to inspection equipment. This means that all drone inspections are suspended during extreme weather to prevent damage.

[0089] The performance alarm task is a targeted inspection task triggered when abnormal power generation or component temperature is detected in some areas of the photovoltaic system. In a specific embodiment, the system immediately initiates a targeted inspection when it detects a sudden drop of 10% in power generation in a certain area or abnormal component temperature (standard deviation > 5°C).

[0090] In each inspection mission, the accident points found in previous inspection missions of the same type are obtained and marked as high-risk areas. The flight trajectory of this inspection mission needs to cover the high-risk areas.

[0091] S2. Perform initial path planning based on the inspection mission based on terrain data, meteorological data, and historical flight data, and use a dynamic obstacle avoidance algorithm to avoid sudden obstacles on the path (such as flying birds and temporary buildings) and update the path. When performing initial path planning, a reinforcement learning model is used to generate a flight trajectory covering all points of interest. The dynamic obstacle avoidance algorithm uses the RRT* algorithm.

[0092] Continue reading Figure 2 It should be noted that terrain data typically refers to a three-dimensional map of the PV plant or other terrain data that characterizes the spatial distribution of the plant's surroundings, including terrain and obstacle coordinates. Based on the high-risk areas marked in different inspection tasks, the PV plant is divided into risk zones, which are then used to determine the inspection priority of each area within the inspection task. Areas with the highest potential for failure (such as marked high-risk areas) receive higher priority, while low-priority areas are considered unnecessary for inspection. Medium-priority areas are considered intermediate in importance.

[0093] Among them, meteorological data is used to determine whether the flight mission can be carried out, as well as priority inspection items. For example, takeoff is allowed when the wind speed is ≤12m / s, and hot spots are inspected first when the irradiance is >800W / m².

[0094] In some embodiments, generating a flight trajectory covering all points of interest using a reinforcement learning model includes:

[0095] Use the topographic data of the area where the photovoltaic system is located as the spatial benchmark and prioritize the inspection areas;

[0096] The artificial intelligence reinforcement learning model integrates risk weights and obstacle avoidance data to generate a global path; during the drone inspection flight, flight cost and value are estimated for benefit evaluation of route planning, and flight energy consumption estimation is introduced to ensure that the overall flight route is within the range.

[0097] In a specific embodiment, the method for finding the optimal strategy parameters in the artificial intelligence reinforcement learning model is:

[0098]

[0099] in, The neural network parameters representing the next round of drone action selection, such as steering, acceleration, etc. It represents the function of finding the optimal policy parameters, which is used to optimize the UAV path strategy, balance risk coverage and obstacle avoidance efficiency, and even obtain the maximum input parameters of the function. Represents the expected value, which is used to calculate the expectation of the state and action at the current time step and to evaluate the long-term benefits of the strategy; Indicates the degree of strategy change, that is, the probability ratio of the new and old strategies to select actions in the current state, represents the probability of the new action, represents the old action probability; represents the advantage function, which is used to evaluate the goodness of the current action relative to the average level; Represents a range-limiting function; It represents the strategy update range limit, which is used to limit the risk of strategy mutation, so that the PPO algorithm can generate efficient and safe inspection paths in complex environments (such as mountain photovoltaic power stations); t represents the time step; k represents the number of update rounds.

[0100] In this dynamic path planning method, the drone's flight trajectory is optimized based on real-time meteorological data (such as wind speed and irradiance) and historical fault heat maps. In path planning, argmax is used to find the optimal strategy parameter θ, which strikes an optimal balance between high-risk area coverage, obstacle avoidance efficiency, and flight time. For example, when a hot spot (temperature anomaly) appears in a certain area, the PPO algorithm prioritizes adjusting the path to cover the high-risk area.

[0101] The advantage function is calculated as follows:

[0102]

[0103] in, Represents the action value function, which is used to evaluate the action in state Next action long-term benefits; Represents the state value function, used to evaluate the state average income; Indicates the power consumption weight, where the total power is set to 100. This is used to measure the power consumption caused by flight distance to prevent over-range flight.

[0104] In some embodiments, avoiding sudden obstacles on the path using a dynamic obstacle avoidance algorithm includes:

[0105] Calculate the total path cost from the starting point to the new node, select the path with the minimum total cost, and dynamically balance obstacle avoidance and path length. The calculation method of the total path cost includes:

[0106]

[0107] in, represents the total path cost from the starting point to the new node; represents the cumulative cost from the starting point to the nearest explored node; represents the Euclidean distance from the most recently explored node to the new node; Represents the risk weight coefficient, which is used to adjust the sensitivity of the path to the risk area. The maximum value is 1 and the minimum value is 0. The value is proportional to the risk level. Indicates the risk value of the new node's location;

[0108] Specifically, the drone path is adjusted in real time during dynamic obstacle avoidance. For example, when a bird or temporary obstacle suddenly appears near a mountain power station, RRT* combines risk weights to generate a smooth trajectory with a low collision probability.

[0109] The risk value of the location of the new node is calculated based on historical fault data and real-time meteorological dynamics. The calculation method includes:

[0110]

[0111] in, represents the historical failure density at the location of the new node; Indicates the real-time wind speed influence coefficient.

[0112] The above model is trained using historical data. The final trained model can plan the inspection route in real time. The specific training method will not be described here.

[0113] In some embodiments, power management should also be performed during the path planning process, specifically by calculating the power required for return (reserving 20%), and automatically terminating the mission and returning if the limit is exceeded.

[0114] Specifically, the drone's dynamic path planning utilizes a multi-stage collaborative optimization strategy. Using a high-precision 3D map of the power plant as the spatial reference, a risk zoning module divides the inspection area into high, medium, and low priority zones (e.g., areas with frequent hot spots are marked as red, high priority). An artificial intelligence reinforcement learning (PPO) model integrates risk weights and obstacle avoidance data to generate a global path, increasing path coverage by 40% in high-risk areas. Simultaneously, a dynamic obstacle avoidance algorithm (RRT*) is activated to scan obstacle coordinates in real time. Path risk values ​​are calculated synchronously as RRT* nodes expand. A dual-threshold battery management mechanism (with a critical value of 15%) is implemented at the terminal. Low battery triggers a charging priority decision tree, ensuring a completion rate of >98% for a single inspection of a 50MW power plant.

[0115] S3. The drone performs the inspection task according to the planned route; during the inspection process, it collects multimodal data of the photovoltaic system; the multimodal data includes visible light images, infrared thermal imaging, EL detection data, and positioning data. At the same time, it obtains the data collected by the ground sensors deployed in the photovoltaic system;

[0116] In one specific embodiment, visible light imaging includes 4K resolution capture, with three images of each photovoltaic module taken at different angles (front view, 45° left angle, and 45° right angle). Infrared thermal imaging uses a FLIRT865 camera to record module surface temperature with an accuracy of ±0.5°C and a hot spot threshold of >20°C temperature difference. EL (electroluminescence) detection data is obtained using an EL detector to detect hidden cracks in photovoltaic modules (exposure time 200ms, current 5A). Positioning data requires RTK (real-time differential positioning) with an accuracy of ±1cm, and coordinates are synchronized with image metadata. Ground sensor data includes module temperature, current, and voltage data collected every 5 minutes and transmitted to an edge server via LoRa (long-range radio). Dust sensors also collect real-time dust accumulation data (>0.5mm triggers a cleaning alarm).

[0117] S4. Integrate the features of the data collected by ground sensors and the data collected by drones to generate a unified fault feature matrix.

[0118] like Figure 3 As shown in the figure, after acquiring the above-mentioned collected data, the integrated data is standardized, such as image denoising, sensor data alignment and outlier removal. Among them, in image denoising, non-local mean filtering is used to eliminate infrared image noise and retain edge details. In sensor data alignment, the GPS timestamp of the drone is used as the benchmark to interpolate and align the ground sensor data. Outlier removal is to remove unreasonable data that deviates significantly from the normal value. Figure 3 In this article, the edge refers to devices such as drones and hardware collection devices.

[0119] The Transformer model then maps the images (time series) and sensor data (spatial distribution) into a unified coordinate system. This standardizes the fault type, occurrence time, location, and risk level. In one specific embodiment, the resulting unified fault signature matrix is ​​a 128×128×4-dimensional matrix (with the dimensions representing temperature, infrared intensity, current deviation, and dust index, respectively).

[0120] In some embodiments, step S4 includes:

[0121] Interpolate and align ground sensor data based on the drone GPS timestamp;

[0122] The Transformer model is used to perform spatiotemporal alignment of data collected by ground sensors and drones, mapping the image data and sensor data to a unified coordinate system to generate a unified fault feature matrix. The Transformer model primarily performs scaled dot-product attention analysis (Scaled Dot-Product Attention) and multi-head attention mechanism analysis (Multi-Head Attention, MHA) on the collected data. Through separate computations and feature integration, it enables multi-perspective modeling of complex data. Publicly available data indicates that the Transformer model is useful for combining multimodal data such as photovoltaic panel cracks, hot spots, and electrical performance anomalies, adjusting path planning strategies through risk weighting, and supporting federated learning and edge deployment for adaptability to low-computing hardware.

[0123] In a specific embodiment, a Transformer model is used to perform scaled dot product attention analysis on the collected data; the expression of scaled dot product attention is:

[0124]

[0125] in, Represents the scaled dot product attention output result; represents the query matrix, represents the bond matrix, Represents the value matrix, using 、 、 Represent the characteristics of different modes of the photovoltaic system, 、 、 Through input data and learnable weight matrix The linear transformation is obtained to represent information from different perspectives; represents the normalization function; represents the transposed matrix; Indicates the dimension of the key vector. By default, it is the same as the dimension of the query matrix and the value matrix. It is used to scale the dot product result to prevent the gradient from vanishing (the dot product value is too large, causing the softmax gradient to approach 0).

[0126] In photovoltaic inspection, if the input data is multimodal data (such as visible light images, infrared thermal imaging, sensor current values), 、 、 It can represent the characteristics of different modalities respectively, fuse visible light (crack), infrared (temperature), and sensor (current) data to generate a unified fault feature matrix, and fuse multi-source information through the attention mechanism to improve fault detection accuracy.

[0127] This embodiment also uses the Transformer model to splice and integrate features of different dimensions through a multi-head attention mechanism to obtain a unified fault feature matrix. The expression of the multi-head attention mechanism is:

[0128]

[0129] in, Represents the output of the multi-head attention mechanism; represents the splicing function; represents the output of the i-th attention head, ; Represents the output linear transformation matrix; in each attention head, different features are extracted from the input data through different linear transformation matrices.

[0130] In a specific embodiment, the number of attention heads h is 8, and features of different subspaces are learned in parallel by multiple attention heads, for example Used to learn terrain obstacle avoidance paths (3D map data); Used to learn the impact of real-time wind speed on battery life (weather sensor); Used to learn historical fault heatmaps and optimize coverage priorities (time series database).

[0131] Among them, each attention head has a corresponding query transformation matrix , key transformation matrix Sum value transformation matrix , these transformation matrices are used to map the input data into different low-dimensional subspaces. For example, infrared thermal imaging data can be projected into different channels to capture features such as temperature gradient and hot spot distribution.

[0132] In a specific embodiment, the multi-head attention mechanism can optimize the task in the following ways: Multimodal data alignment: through Visible light images, infrared thermal imaging, and ground sensor data are integrated to generate a unified fault feature matrix. Dynamic path planning: Utilizes h heads of parallel computing to balance obstacle avoidance efficiency and risk coverage (e.g., coverage of high-risk areas is increased by 40%). Real-time reasoning at the edge: Through Integrate multi-dimensional features, adapt to lightweight model deployment, and achieve real-time fault detection (e.g., hidden crack location error < 5 pixels).

[0133] S5. Locate the defective area of ​​the photovoltaic system using a real-time target detection algorithm based on a deep neural network according to the unified fault feature matrix, determine the defect type of the defective area using a deep convolutional neural network, and classify the fault level of the defective area.

[0134] Continue reading Figure 3 In some embodiments, the deep neural network-based real-time object detection algorithm uses the YOLOv10 algorithm to locate the defect area, and the output result is the coordinates of the bounding box corresponding to the defect area. In a specific embodiment, the confidence threshold set by the YOLOv10 algorithm is greater than 0.9.

[0135] In the YOLOv10 algorithm, the loss function of the YOLOv10 algorithm is constructed through three-stage collaborative optimization of classification loss, regression loss, and distribution focus loss. Among them, based on classification loss, fault types can be accurately identified and multi-label output (such as hot spot + hidden crack composite fault) can be supported; based on regression loss, spatial relationships can be comprehensively considered to adapt to tilted / occluded target positioning; based on distribution focus loss, the ability to detect tiny defects can be improved through probabilistic modeling. In the new energy drone inspection, this loss function combined with edge computing optimization can achieve a single-flight 50MW power station full inspection time of ≤4 hours, which is significantly efficient compared to traditional manual inspection.

[0136] In a specific embodiment, the loss function of the YOLOv10 algorithm is expressed as:

[0137]

[0138] in, represents the total loss of target detection; Represents the classification loss weight, which can be 0.5. It is used to balance the importance of multi-category recognition. For example, the weight is reduced when the priority of stain detection is lower than that of hot spot detection. Represents the classification loss error; Represents the regression loss weight, which can be 0.05, to balance positioning accuracy and classification importance. Mountain power stations need to increase the weight appropriately due to the complex terrain. Represents the regression loss error; Represents the weight of the distribution focus loss, which is used to enhance the detection of small targets (such as micro-cracks <10 pixels) and reduce the missed detection rate of UAV high-altitude photography. It can be set to 0.5; Represents the distributed focusing loss error.

[0139] The calculation method of the classification loss error includes:

[0140]

[0141] in, Represents the true category distribution, used to mark the defect type of photovoltaic panels, such as hot spot = 1, stain = 0, and hidden crack = 0; Represents the predicted category probability distribution, that is, the confidence level of the fault type output by the model (needs to be processed by the Sigmoid function);

[0142] The calculation method of the regression loss error includes:

[0143]

[0144] in, Represents the intersection-over-union ratio, that is, the degree of overlap between the predicted box and the true box; Represents the Euclidean distance between the center point of the predicted box and the true box, which is used to accurately locate the crack position (error < 5 pixels); Indicates the center point position of the prediction box; Indicates the center point position of the real frame; Represents the diagonal length of the smallest rectangle containing the predicted box and the true box, which is used to measure the positioning accuracy of long-range targets (such as the edge area of ​​a large photovoltaic array); Represents the aspect ratio difference coefficient, which is used to measure the difference in the aspect ratio of the predicted box and the true box, and optimize the aspect ratio of the photovoltaic panel rectangular bounding box (typical aspect ratio is 1:2); represents the aspect ratio difference weight;

[0145] The calculation method of the distributed focusing loss error includes:

[0146]

[0147] in, Represents the probability distribution of bounding box coordinates, which is used to predict the length of hidden cracks in photovoltaic panels; Represents the true distribution value, that is, the discretized distribution of the true coordinates, which is used to mark the data after Gaussian smoothing (standard deviation σ = 1.5).

[0148] The deep convolutional neural network uses the ResNet-50 network to determine the defect type (hot spots, hidden cracks, stains, etc.).

[0149] The ResNet-50 network includes four groups of residual blocks;

[0150] The first group of residual blocks is used to extract basic features of the photovoltaic panel, and the basic features of the photovoltaic panel include edge texture of the photovoltaic panel;

[0151] The second group of residual blocks is used to identify medium-scale defects in photovoltaic panels, such as hidden cracks and stains;

[0152] The third group of residual blocks is used to detect small target defects, such as solder joint detachment and microcracks;

[0153] The fourth group of residual blocks is used for global feature fusion and classification decision.

[0154] In a specific embodiment, the structures of the four groups of residual blocks are shown in Table 1.

[0155] Table 1 Residual block structure of ResNet-50 network

[0156]

[0157] It should be noted that, in addition to the above features, the specific structure and training method of the YOLOv10 algorithm and ResNet-50 network are well known to those skilled in the art and will not be repeated here.

[0158] In some embodiments, YOLOv10 is used to locate defective areas on photovoltaic solar panels, and ResNet-50 is used to determine the defect type. Knowledge distillation is then used to compress the ResNet-50 model calculated by artificial intelligence into a MobileNet model, reducing the number of parameters to one-third and latency to ≤ 200ms. This model is compressed to 300MB after knowledge distillation and is compatible with NVIDIA Jetson AGX Xavier edge devices.

[0159] The detection results obtained by defect recognition based on the above ResNet-50 are compared with the detection results of VGG16 (an existing image recognition algorithm). The comparison results are shown in Table 2.

[0160] Table 2 Comparison of test results

[0161]

[0162] It can be seen that the use of ResNet-50 has greatly improved detection accuracy, inference speed and power consumption.

[0163] In some embodiments, when a fault type is detected, a fault level may be output synchronously to indicate the severity of the fault.

[0164] For example, the fault levels are divided into L1 (indicated by red, should be handled immediately), L2 (indicated by yellow, should be handled within 3 days), and L3 (indicated by green, can be observed).

[0165] Generate data analysis reports: including fault location (GPS coordinates), type, confidence level, and repair suggestions (such as component replacement).

[0166] S6. Score the health of the PV system according to the fault level of each fault area, and use the health score to analyze the safe operation trend of the PV system.

[0167] like Figure 4 As shown, step S6 is illustrated as the cloud-based decision-making and automated repair process. The data is stored separately: ground sensor data is stored in InfluxDB (a time series database), which supports queries by time range. The original images and annotations are stored in MinIO (a distributed object storage database), which retains 30 days of historical data.

[0168] As an extension, the Unity engine can be used to render a 3D photovoltaic power station model, and a thermal layer can be used to display the fault density (red indicates high density, blue indicates normal).

[0169] The health scoring algorithm is: weights L1=5, L2=3, L3=1.

[0170]

[0171] in, represents the health score, represents the number of faults with fault level Li, Indicates the weight corresponding to the fault level Li, when the fault level is L1, , when the fault level is L2, , when the fault level is L3, .

[0172] Scoring results can be pushed to the maintenance app (including navigation links) via the RabbitMQ message queue. When an L1 failure occurs, a text message is sent to the responsible person (if no response is received within 10 minutes, an escalation notification will be issued).

[0173] At the same time, based on the handling strategy, it can determine whether the fault requires manual repair. If the drone's self-healing action can be used to repair the fault, an autonomous robot is directly dispatched to perform the repair. The autonomous robot can perform cleaning, simple replacement, and regional power outages. For example, a cleaning robot can be triggered to perform targeted cleaning of components with stains greater than 30%. After the repair, the drone is used to capture high-definition images for comparison before and after the repair.

[0174] In some embodiments, to further improve the accuracy of each model, federated learning updates can be used. Each PV plant's local model uploads its gradient parameters to the cloud weekly (homomorphic encryption protects data privacy). The FedAvg algorithm is then used to aggregate and generate a global model in the cloud, which is then distributed to each edge node.

[0175] Other embodiments of the present invention provide an artificial intelligence-based new energy photovoltaic dynamic inspection system, which is applied to the artificial intelligence-based new energy photovoltaic dynamic inspection method as described in any of the above embodiments, including an inspection task trigger module, a dynamic path planning module, a data acquisition module, an edge data processing module, an AI fault detection and classification module, and a fault handling module.

[0176] The inspection task triggering module triggers the corresponding inspection task according to weather warnings, photovoltaic system performance alarms or scheduled tasks; the dynamic path planning module performs initial path planning based on the inspection task according to terrain data, meteorological data and historical flight data, and avoids sudden obstacles on the path through a dynamic obstacle avoidance algorithm and updates the path; wherein, when performing the initial path planning, a reinforcement learning model is used to generate a flight trajectory covering all points of interest; the dynamic obstacle avoidance algorithm adopts the RRT* algorithm; the UAV performs the inspection task according to the planned path; the data acquisition module collects multimodal data of the photovoltaic system during the inspection process; the multimodal data includes visible light images, infrared thermal imaging, EL detection data and fixed The system collects data from ground sensors arranged in the photovoltaic system and obtains the collected data. The edge data processing module integrates the features of the collected data from the ground sensors and the data collected by the drone to generate a unified fault feature matrix. The AI ​​fault detection and classification module locates the defective area of ​​the photovoltaic system according to the unified fault feature matrix using a real-time target detection algorithm based on a deep neural network, determines the defect type of the defective area using a deep convolutional neural network, and divides the fault level of the defective area. The fault handling module scores the health of the photovoltaic system according to the fault level of each fault area, uses the health score to analyze the safe operation trend of the photovoltaic system, and formulates a fault handling strategy.

[0177] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples.

[0178] Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A new energy photovoltaic dynamic inspection method based on artificial intelligence, characterized in that: include: Trigger corresponding inspection tasks based on weather warnings, photovoltaic system performance alarms or scheduled tasks; Initial path planning is performed based on inspection tasks using terrain data, meteorological data, and historical flight data. A dynamic obstacle avoidance algorithm is used to avoid sudden obstacles along the path and update the path. During initial path planning, a reinforcement learning model is used to generate a flight trajectory covering all points of interest. The dynamic obstacle avoidance algorithm uses the RRT* algorithm. The drone performs inspection tasks along the planned route; during the inspection process, it collects multimodal data of the photovoltaic system; the multimodal data includes visible light images, infrared thermal imaging, EL detection data, and positioning data; and obtains data collected by ground sensors deployed in the photovoltaic system; Integrate the features of data collected by ground sensors and drones to generate a unified fault feature matrix; Locating defective areas of the photovoltaic system using a real-time target detection algorithm based on a deep neural network according to the unified fault feature matrix, determining defect types of the defective areas using a deep convolutional neural network, and classifying fault levels of the defective areas; Score the health of the PV system based on the fault level of each fault area, and use the health score to analyze the safe operation trend of the PV system; The method of avoiding sudden obstacles on the path by using a dynamic obstacle avoidance algorithm includes: Calculate the total path cost from the starting point to the new node, select the path with the minimum total cost, and dynamically balance obstacle avoidance and path length. The calculation method of the total path cost includes: in, represents the total path cost from the starting point to the new node; represents the cumulative cost from the starting point to the most recently explored node; represents the Euclidean distance from the most recently explored node to the new node; represents the risk weight coefficient, which is used to adjust the sensitivity of the path to the risk area; Indicates the risk value of the new node's location; The risk value of the location of the new node is calculated based on historical fault data and real-time meteorological dynamics. The calculation method includes: in, represents the historical failure density at the location of the new node; Indicates the real-time wind speed influence coefficient.

2. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 1 is characterized in that: The types of inspection tasks include: Scheduled inspection tasks, regular inspection tasks that start automatically at the set time; Weather warning tasks are temporary inspection tasks triggered before or after monitoring of special weather conditions, such as sandstorms and hail. Performance alarm task: a targeted inspection task triggered when abnormal power generation or component temperature in some areas of the PV system is detected; In each inspection mission, the accident points found in previous inspection missions of the same type are obtained and marked as high-risk areas. The flight trajectory of this inspection mission needs to cover the high-risk areas.

3. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 1 is characterized in that: The use of the reinforcement learning model to generate a flight trajectory covering all points of interest includes: Use the topographic data of the area where the photovoltaic system is located as the spatial benchmark and prioritize the inspection areas; The artificial intelligence reinforcement learning model integrates risk weights and obstacle avoidance data to generate a global path; during the drone inspection flight, flight cost and value are estimated for benefit evaluation of route planning, and flight energy consumption estimation is introduced to ensure that the overall flight route is within the range.

4. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 3 is characterized in that: The method for finding the optimal strategy parameters in the artificial intelligence reinforcement learning model is: in, The neural network parameters representing the next round of drone control action selection; Represents the function of finding the optimal strategy parameters; Indicates expected value; Indicates the degree of strategy change, that is, the probability ratio of the new and old strategies to select actions in the current state, represents the probability of the new action, represents the old action probability; represents the advantage function, which is used to evaluate the goodness of the current action relative to the average level; Represents a range-limiting function; represents the strategy update amplitude limit; t represents the time step; k represents the number of update rounds; The advantage function is calculated as follows: in, Represents the action value function, which is used to evaluate the action in state Next action long-term benefits; Represents the state value function, used to evaluate the state average income; Indicates the power consumption weight.

5. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 1 is characterized in that: The feature integration of the data collected by the ground sensors and the data collected by the UAV is performed to generate a unified fault feature matrix, including: Interpolate and align ground sensor data based on the drone GPS timestamp; The Transformer model is used to align the data collected by ground sensors and drones in time and space, mapping the image data and sensor data to a unified coordinate system to generate a unified fault feature matrix. Among them, the Transformer model is used to perform scaled dot product attention analysis on the collected data; the expression of scaled dot product attention is: in, Represents the scaled dot product attention output result; represents the query matrix, represents the bond matrix, Represents the value matrix, using 、 、 They represent the characteristics of different modes of the photovoltaic system; represents the normalization function; represents the transposed matrix; represents the key vector dimension; And through the multi-head attention mechanism, features of different dimensions are spliced ​​and integrated to obtain a unified fault feature matrix; the expression of the multi-head attention mechanism is: in, Represents the output of the multi-head attention mechanism; represents the splicing function; represents the output of the i-th attention head, ; Represents the output linear transformation matrix; in each attention head, different features are extracted from the input data through different linear transformation matrices.

6. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 1 is characterized in that: The real-time target detection algorithm based on deep neural network adopts YOLOv10 algorithm; In the YOLOv10 algorithm, the loss function of the YOLOv10 algorithm is constructed by three-stage collaborative optimization of classification loss, regression loss, and distribution focus loss; the loss function of the YOLOv10 algorithm is expressed as: in, represents the total loss of target detection; represents the classification loss weight; Represents the classification loss error; represents the regression loss weight; Represents the regression loss error; represents the distribution focusing loss weight; Represents the distributed focusing loss error.

7. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 6 is characterized in that: The calculation method of the classification loss error includes: in, Represents the true category distribution, used to label the defect type of photovoltaic panels; Represents the predicted category probability distribution, that is, the confidence level of the fault type output by the model; The calculation method of the regression loss error includes: in, Represents the intersection-over-union ratio, that is, the degree of overlap between the predicted box and the true box; Represents the Euclidean distance between the center point of the predicted box and the true box; Indicates the center point position of the prediction box; Indicates the center point position of the real frame; Represents the diagonal length of the smallest rectangle containing the predicted box and the true box; Represents the aspect ratio difference coefficient, which is used to measure the difference in the width and height ratio between the predicted box and the real box; represents the aspect ratio difference weight; The calculation method of the distributed focusing loss error includes: in, Represents the probability distribution of bounding box coordinates, which is used to predict the length of hidden cracks in photovoltaic panels; Represents the true distribution value, that is, the discretized distribution of the true coordinates, which is used to mark data after Gaussian smoothing.

8. The artificial intelligence-based new energy photovoltaic dynamic inspection method according to claim 1 is characterized in that: The deep convolutional neural network adopts a ResNet-50 network, and the ResNet-50 network includes four groups of residual blocks; The first group of residual blocks is used to extract basic features of the photovoltaic panel, and the basic features of the photovoltaic panel include edge texture of the photovoltaic panel; The second group of residual blocks is used to identify medium-scale defects in photovoltaic panels, including hidden cracks and stains; The third group of residual blocks is used to detect small target defects, including solder joint detachment and cracks; The fourth group of residual blocks is used for global feature fusion and classification decision.

9. A new energy photovoltaic dynamic inspection system based on artificial intelligence, characterized in that: The artificial intelligence-based new energy photovoltaic dynamic inspection method applied to any one of claims 1 to 8 comprises: Inspection task triggering module, triggering corresponding inspection tasks based on weather warnings, photovoltaic system performance alarms or scheduled tasks; The dynamic path planning module performs initial path planning based on inspection tasks using terrain data, meteorological data, and historical flight data. It also uses a dynamic obstacle avoidance algorithm to avoid sudden obstacles along the path and update the path. During the initial path planning, a reinforcement learning model is used to generate a flight trajectory covering all points of interest. The dynamic obstacle avoidance algorithm uses the RRT* algorithm, enabling the drone to perform inspection tasks along the planned path. The data acquisition module collects multimodal data of the photovoltaic system during the inspection process; the multimodal data includes visible light images, infrared thermal imaging, EL detection data and positioning data; and obtains the collected data of the ground sensors arranged in the photovoltaic system; The edge data processing module integrates the features of the data collected by ground sensors and drones to generate a unified fault feature matrix; An AI fault detection and classification module locates defective areas of the photovoltaic system using a real-time target detection algorithm based on a deep neural network according to the unified fault feature matrix, determines the defect type of the defective area using a deep convolutional neural network, and classifies the fault level of the defective area; The fault handling module scores the health of the PV system according to the fault level of each fault area, uses the health score to analyze the safe operation trend of the PV system, and formulates a fault handling strategy.

Images