An image acquisition method deployed in a UAV to cope with a photovoltaic device

Abstract

This application relates to the technical field of unmanned aerial vehicle (UAV) inspection, and discloses an image acquisition method for photovoltaic (PV) equipment deployed on a UAV. The method includes: acquiring terrain and PV module layout data of the target PV power station, generating a global pre-planned flight path containing baseline flight altitude parameters; while the UAV flies along the path, constructing a local 3D point cloud model using LiDAR to adjust flight altitude and trajectory, and fine-tuning attitude and camera orientation; simultaneously acquiring visible light and infrared thermal imaging images and performing pixel-level spatial registration to obtain fused image data; using an onboard lightweight deep learning model to complete defect detection and classifying defects into three levels according to preset standards; summarizing and optimizing the detection results and data, then uploading them to a cloud platform for storage and analysis and updating the knowledge base; the cloud platform then distributes the updated model parameters to edge servers and UAV onboard equipment through federated learning. This application can improve the acquisition stability and detection capability of UAV PV image acquisition.

Description

Technical Field

[0001] This application relates to the technical field of drone inspection, and in particular to an image acquisition method for photovoltaic equipment deployed in a drone. Background Technology

[0002] In the inspection of photovoltaic power plants in complex terrains such as mountains and hills, automated image acquisition by drones has become the mainstream application method. Existing acquisition solutions mostly adopt fixed flight routes, which are poorly adaptable to terrain undulations and the actual layout of photovoltaic modules, and it is difficult to maintain a stable and compliant acquisition angle. Some solutions equipped with dual-light acquisition units also have problems with insufficient coordination between visible light and infrared data acquisition and low fusion matching accuracy, making it difficult to guarantee the quality of basic acquisition data.

[0003] Existing defect detection solutions largely rely on centralized cloud processing, with weak real-time detection and refined grading capabilities on the airborne end, failing to meet the refined operation and maintenance needs of photovoltaic power plants. Furthermore, the data transmission and model update modes are simplistic, resulting in high cloud computing load, low model iteration efficiency, and difficulty in continuously improving detection effectiveness over long-term inspections. Current UAV-based photovoltaic image acquisition methods suffer from technical problems related to insufficient acquisition stability and inadequate defect grading and detection capabilities. As can be seen from the above, there are still problems to be solved in how to improve the acquisition stability and detection capability of photovoltaic images acquired by UAVs. Summary of the Invention

[0004] To improve the stability and detection capabilities of photovoltaic image acquisition by UAVs, this application provides an image acquisition method for photovoltaic equipment deployed in UAVs.

[0005] Firstly, this application provides an image acquisition method for photovoltaic equipment deployed in a drone, employing the following technical solution: A method for image acquisition of photovoltaic (PV) equipment deployed in a drone involves pre-acquiring terrain data and PV module layout data of the target PV power station, generating a global pre-planned flight path covering the target PV power station. The global pre-planned flight path includes reference flight altitude parameters corresponding to the terrain of the PV power station, including: When controlling the UAV to fly along the pre-planned global flight path, the UAV uses an airborne lidar to scan the terrain and photovoltaic modules below in real time to construct a local three-dimensional point cloud model. The flight altitude and trajectory are adjusted based on the local three-dimensional point cloud model. At the same time, visual servo technology is used to identify the boundaries and normal vectors of the photovoltaic modules and to fine-tune the flight attitude and lens orientation so that the image acquisition lens and the surface of the photovoltaic modules maintain a preset acquisition angle. By simultaneously triggering the visible light and infrared thermal imaging lenses of the UAV, the photovoltaic module after the attitude is adjusted is simultaneously acquired, and visible light images and infrared thermal imaging images with the same field of view at the same time are obtained. The visible light images and infrared thermal imaging images are then spatially registered at the pixel level to obtain the registered fused image data. The fused image data is input into a lightweight deep learning model deployed on an airborne edge computing device to complete the defect detection of photovoltaic modules. Based on a preset quantitative judgment standard, the photovoltaic defects are divided into three levels, and the defect detection results and classification information are obtained. The defect detection results, classification information, and fused image data are transmitted to the regional edge server to complete data aggregation and model parameter optimization. Then, they are uploaded to the cloud platform to complete data storage analysis and knowledge base updates. The cloud platform uses federated learning to distribute the updated model parameters to the regional edge server and the airborne edge computing device of the drone.

[0006] Optionally, the step of adjusting the flight altitude and trajectory based on the local 3D point cloud model includes: The local three-dimensional point cloud model is divided into grids according to the horizontal width and vertical spacing of a single photovoltaic string in the target photovoltaic power station, resulting in several unit point cloud grids that correspond one-to-one with a single photovoltaic string. The point cloud data in each unit point cloud raster is separated by elevation clustering to obtain the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base. The mean elevation and maximum elevation of the upper cluster point cloud in each unit point cloud raster are extracted. Based on the baseline flight altitude of the global flight pre-planned path, and combined with the average elevation of all unit point cloud grids obtained from the current local 3D point cloud model, a continuous and smooth flight altitude envelope is fitted. At the same time, the lateral offset of the global flight pre-planned path is corrected according to the center coordinates of each unit point cloud grid, and a locally optimized flight trajectory adapted to the photovoltaic string arrangement in the current scanning area is generated. The drone is controlled to fly along a locally optimized flight trajectory, with its flight altitude following the flight altitude envelope throughout the entire flight. The difference between the flight altitude and the maximum elevation of the upper cluster point cloud within the corresponding unit point cloud grid is not less than a preset safe distance.

[0007] Optionally, the step of performing elevation clustering separation on the point cloud data within each unit point cloud raster to obtain the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base includes: Based on the physical size parameters and installation tilt angle parameters of the photovoltaic modules in the target photovoltaic power station obtained in advance, the effective value range of elevation clustering is set for the corresponding unit point cloud grid, and isolated invalid point cloud data with elevation exceeding the effective value range are removed. The median of the elevation distribution of the remaining valid point clouds within the unit point cloud grid is used as the initial segmentation threshold. The segmentation threshold is updated through iterative calculation. Point clouds with elevations higher than the updated segmentation threshold are classified as initial upper cluster point clouds, and point clouds with elevations lower than the updated segmentation threshold are classified as initial lower cluster point clouds. Plane fitting is performed on the initial upper cluster point cloud, and the deviation between the fitted plane tilt angle and the preset installation tilt angle of the photovoltaic string corresponding to the unit point cloud grid is verified. If the deviation exceeds the preset allowable range, the point cloud data corresponding to the deviation exceeding the preset allowable range is included in the initial lower cluster point cloud. Compare the difference between the initial upper cluster point cloud elevation mean of the current unit point cloud grid and the adjacent unit point cloud grid in the same row. If the difference exceeds the preset continuous threshold, iterate and update the segmentation threshold again and perform clustering separation and verification operations again until the difference meets the preset continuous threshold requirement. Finally, the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base are obtained.

[0008] Optionally, the step of simultaneously identifying the photovoltaic module boundary and normal vector using visual servoing technology, and fine-tuning the flight attitude and camera orientation, includes: After the UAV completes the rasterization of the local 3D point cloud model, it locks the region of interest in the image for visual servo recognition based on the spatial coordinate range of the corresponding unit point cloud raster, and performs photovoltaic module boundary recognition processing only on the image content within the locked region of interest. The image within the region of interest is processed by grayscale conversion and gradient enhancement. Combined with the pre-acquired physical size parameters and string arrangement rules of the photovoltaic modules, the complete closed boundary of the photovoltaic modules is extracted, and invalid recognition results with incomplete boundaries or size deviations exceeding the preset range are eliminated. Based on the complete closed boundary of the photovoltaic module, and combined with the point cloud coordinate data of the corresponding region in the local three-dimensional point cloud model, the surface plane equation of the photovoltaic module is obtained by fitting, and the surface normal vector of the photovoltaic module is calculated based on the surface plane equation. Using the surface normal vector of the photovoltaic module as a reference, the angular deviation between the optical axis of the image acquisition lens and the normal vector is calculated. Based on the angular deviation, the pitch and roll attitude of the UAV and the pitch and yaw orientation of the lens are adjusted in a split-axis decoupling manner until the angular deviation falls within the preset allowable range.

[0009] Optionally, the step of performing pixel-level spatial registration of the visible light image and the infrared thermal imaging image to obtain the registered fused image data includes: Using the complete closed boundary of the photovoltaic module extracted by visual servoing technology as the registration reference, the vertex coordinates corresponding to the complete closed boundary of the same photovoltaic module are located in the visible light image and the infrared thermal imaging image, respectively, as the same registration points of the two sets of images; By combining the three-dimensional spatial coordinate data of the photovoltaic module corresponding area in the local three-dimensional point cloud model, spatial coordinate mapping correction is performed on two sets of corresponding registration points to eliminate the field of view offset caused by the installation baseline difference between the visible light lens and the infrared thermal imaging lens. Based on the corrected registration points, a homography transformation matrix is ​​constructed. The pixel coordinates of the infrared thermal imaging image are mapped to the pixel coordinate system of the visible light image through the homography transformation matrix, thus completing the pixel-level alignment of the two images and obtaining the registered fused image data.

[0010] Optionally, the step of classifying photovoltaic defects into three levels according to a preset quantitative judgment standard to obtain defect detection results and classification information includes: After completing defect detection on the fused image data, six quantitative data items corresponding to the defects are extracted. The quantitative data items are defect type, geometric size, infrared thermal imaging temperature rise difference, distribution location of defects in photovoltaic modules, defect area ratio in a single photovoltaic module, and defect contiguousness of adjacent modules in the same string. Threshold ranges for six quantitative data items were set for the three defect levels: emergency, important, and general. At the same time, a weighting coefficient was set for the degree of impact of the defect on the power generation and safety of the photovoltaic module for each quantitative dimension. The weighting coefficients for defect type, infrared thermal imaging temperature rise difference, and the situation of defects in the same string are higher than the weighting coefficients of the other three dimensions. The six extracted quantitative data points are substituted into the corresponding threshold ranges to complete the matching. The comprehensive judgment score of the defect is calculated by combining the weighting coefficients. Based on the comprehensive judgment score, the photovoltaic defects are classified into the corresponding levels, and the defect detection results and classification information are obtained.

[0011] Optionally, before performing the defect level classification, the following steps are included: First, obtain the measured data of ambient temperature and solar irradiance at the time when the drone collects the corresponding image. At the same time, retrieve the pre-stored nominal power, service life, and installation tilt angle parameters corresponding to the photovoltaic module. Based on measured solar irradiance data, and combined with the photoelectric conversion characteristics of photovoltaic cells, the threshold range corresponding to the temperature rise difference in infrared thermal imaging is corrected. The correction magnitude is positively correlated with the difference between measured solar irradiance data and irradiance under standard test conditions. Based on measured ambient temperature data and the number of years the modules have been in service, the threshold range corresponding to the defect geometry and defect area ratio is corrected. The correction range is positively correlated with the number of years the modules have been in service. The corrected threshold range is compared and verified with the measured data of adjacent defect-free photovoltaic modules in the same string. If the verification deviation exceeds the preset range, the threshold range is calibrated a second time based on the measured data of adjacent defect-free modules. After the calibration is completed, the defect level classification operation is performed.

[0012] Optionally, after completing the defect level classification, the following steps are included: For photovoltaic modules with defects classified as emergency level, a second image acquisition is immediately triggered. During the second acquisition, the drone's flight position, altitude, and lens orientation are adjusted based on the unit point cloud grid data of the photovoltaic module with defects classified as emergency level, so that the lens optical axis coincides with the normal vector of the module surface. At the same time, the lens field of view is reduced and the image acquisition resolution is improved. The image acquisition, defect detection, and level classification operations are repeated. The fused image data, defect detection results, and classification information obtained from the secondary acquisition are set as the highest transmission priority and transmitted to the regional edge server first. Defect data of other levels are transmitted in sequence according to the preset order. After the regional edge server completes the annotation and verification of the emergency level defect data, it uses the annotated dataset for incremental training of a lightweight deep learning model. The trained and optimized model parameters are then distributed to the airborne edge computing device of the drone through the cloud platform.

[0013] Optionally, the cloud platform distributes the updated model parameters to the regional edge server and the onboard edge computing device of the drone through federated learning, including: Each regional edge server will divide the defect data that has been locally annotated and verified into corresponding sample subsets according to the defect level, and set the sample subset corresponding to the emergency level defects as the priority training sample set. The cloud platform distributes the initial parameters of the global lightweight deep learning model to all connected regional edge servers. The global lightweight deep learning model is the global benchmark model of the lightweight deep learning model. Each regional edge server completes the incremental training of its local model based on a local sample subset, with the priority training sample set as the core, and obtains the local model update parameters. Each regional edge server only uploads the local model update parameters to the cloud platform, without uploading the original fused image data. The cloud platform performs weighted aggregation processing on all uploaded local model update parameters to obtain the updated global model parameters. The cloud platform distributes the updated global model parameters to the edge servers in each region, and then the edge servers in each region synchronize the model parameters to the airborne edge computing devices of the corresponding drones under their jurisdiction, thus completing the iterative update of the model across the entire system.

[0014] Secondly, this application provides an image acquisition device for photovoltaic equipment deployed in a drone, employing the following technical solution: An image acquisition device deployed in a drone to deal with photovoltaic equipment pre-acquires terrain data and photovoltaic module layout data of the target photovoltaic power station, and generates a global pre-planned flight path covering the target photovoltaic power station. The global pre-planned flight path includes reference flight altitude parameters corresponding to the terrain of the photovoltaic power station, including: The global path and adaptive flight control module controls the UAV to fly along the pre-planned global flight path. It uses an airborne lidar to scan the terrain and photovoltaic modules below in real time to construct a local three-dimensional point cloud model. Based on the local three-dimensional point cloud model, it adjusts the flight altitude and trajectory. At the same time, it uses visual servo technology to identify the boundaries and normal vectors of the photovoltaic modules and fine-tunes the flight attitude and lens orientation to keep the image acquisition lens and the surface of the photovoltaic modules at a preset acquisition angle. The dual-light image synchronous acquisition and registration module synchronously triggers the visible light and infrared thermal imaging lenses of the UAV to synchronously acquire the photovoltaic module after the attitude is adjusted, and obtains visible light images and infrared thermal imaging images with the same field of view at the same time. The visible light images and infrared thermal imaging images are then spatially registered at the pixel level to obtain the registered fused image data. The photovoltaic defect detection and intelligent grading module integrates image data input into a lightweight deep learning model deployed on an airborne edge computing device to complete photovoltaic module defect detection. Based on a preset quantitative judgment standard, the photovoltaic defects are divided into three levels, and the defect detection results and grading information are obtained. The edge-cloud collaborative model update module transmits defect detection results, classification information, and fused image data to the regional edge server to complete data aggregation and model parameter optimization. Then, it uploads the data to the cloud platform to complete data storage analysis and knowledge base updates. The cloud platform uses federated learning to distribute the updated model parameters to the regional edge server and the airborne edge computing device of the drone.

[0015] In summary, this application includes at least one of the following beneficial technical effects: Based on a pre-generated global flight pre-planned path, a local 3D point cloud model is constructed in real time during flight using an airborne LiDAR. The model is then rasterized according to the arrangement size of a single photovoltaic string, and point cloud data of the top surface of the photovoltaic module and the terrain base are obtained through elevation clustering. This data is then fitted to the continuous flight altitude envelope that adapts to the changes in module elevation, while correcting lateral path offset. This achieves real-time adaptive adaptation of flight altitude and trajectory to terrain undulations and module arrangement, avoiding the problem of acquisition distance deviation. Simultaneously, the region of interest is locked by the point cloud raster range for visual recognition, accurately extracting the complete boundary and surface normal vector of the photovoltaic module. The drone's flight attitude and camera orientation are finely adjusted by split-axis decoupling, maintaining the preset acquisition angle throughout the entire process. Combined with dual-lens synchronous triggering and pixel-level spatial registration based on module boundaries, the problems of acquisition perspective deviation, image defocus, and mismatch between two light sources are completely solved.

[0016] By deploying a lightweight deep learning model on airborne edge computing equipment, real-time defect detection is achieved based on registered fused image data. Quantitative data from six core dimensions, including defect type, infrared temperature rise difference, and defect clustering, are combined with corresponding weighting coefficients to calculate a comprehensive defect judgment score, enabling three-level refined defect classification. Simultaneously, the classification threshold is adjusted by incorporating ambient temperature, solar irradiance, and the module's service life at the time of data acquisition, significantly reducing misjudgments and missed detections caused by environmental and operating conditions. For emergency-level defects, a secondary high-precision acquisition is triggered to further ensure the completeness of core safety defect detection. Furthermore, through a federated learning architecture between regional edge servers and the cloud platform, global model aggregation and optimization are completed without uploading original image data, ensuring data privacy. Parameter distribution enables continuous iteration of the entire system's detection model, steadily improving the defect detection capabilities of photovoltaic inspections and adapting to the refined operation and maintenance needs of large-scale, complex-terrain photovoltaic power plants. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an image acquisition method for photovoltaic equipment deployed in a drone, according to an exemplary embodiment.

[0018] Figure 2 This is a structural block diagram illustrating an image acquisition device deployed in a drone to deal with photovoltaic equipment, according to an exemplary embodiment. Detailed Implementation

[0019] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0020] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0021] This application discloses an image acquisition method for photovoltaic equipment deployed in a drone, referring to... Figure 1 The system first acquires terrain data and photovoltaic module layout data of the target photovoltaic power station, then generates a global pre-planned flight path covering the target photovoltaic power station. This global pre-planned flight path includes baseline flight altitude parameters that match the terrain of the target photovoltaic power station. Figure 1 ,include: When controlling the UAV to fly along the pre-planned global flight path, the S100 uses an airborne LiDAR to scan the terrain and photovoltaic modules below in real time, constructing a local 3D point cloud model. Based on the local 3D point cloud model, the flight altitude and trajectory are adjusted. At the same time, visual servo technology is used to identify the boundaries and normal vectors of the photovoltaic modules, and the flight attitude and lens orientation are finely adjusted to keep the image acquisition lens and the surface of the photovoltaic modules at a preset acquisition angle.

[0022] In this embodiment of the invention, the specific execution process of S100 is as follows: S101 executes global pre-planned path cruise and real-time LiDAR scanning operations. It controls the UAV to load a pre-generated global flight pre-planned path and perform cruise flight along the path. During the flight, the airborne LiDAR is activated simultaneously to perform real-time high-frequency line scanning of the power station terrain and photovoltaic modules directly below the flight path at a preset scanning frequency. This obtains the original 3D point cloud data corresponding to the current cruise area, ensuring that the scanning range completely covers all photovoltaic modules and terrain areas within the UAV's current field of view, with no scanning blind spots.

[0023] S102 performs local 3D modeling and dynamic flight trajectory adjustment operations. Based on the raw 3D point cloud data acquired in real time by the LiDAR, a local 3D point cloud model of the current cruise area of ​​the UAV is constructed online on the airborne terminal. Based on this local 3D point cloud model, the terrain elevation and the top height data of the photovoltaic module in the current area are calculated in real time. Based on this, the flight altitude and horizontal flight trajectory of the UAV are dynamically adjusted so that the vertical distance between the UAV and the surface of the photovoltaic module is always within the preset optimal acquisition range.

[0024] S103 performs photovoltaic module feature recognition and relative pose calculation operations. While the lidar is scanning and the trajectory is being macroscopically adjusted, the airborne vision servo system is started simultaneously. The vision acquisition unit acquires real-time images of the photovoltaic modules below. Based on the image feature recognition algorithm, the boundary contour and surface normal vector of the photovoltaic modules are extracted. The relative angle deviation and position deviation between the current image acquisition lens and the surface of the photovoltaic modules are calculated, and the deviation data is quantified and output in real time.

[0025] S104 executes closed-loop fine-tuning of acquisition attitude and optimal acquisition pose locking operation. Based on the angle and position deviation data calculated by the visual servo system, it fine-tunes the drone's flight pitch and roll attitude in real time, and simultaneously adjusts the gimbal orientation of the image acquisition lens to keep the optical axis of the image acquisition lens and the normal vector of the photovoltaic module surface at the preset optimal acquisition angle. This completes the closed-loop calibration of attitude before acquisition, locks the optimal acquisition pose, and ensures that the relative pose between the lens and the photovoltaic module surface meets the preset acquisition requirements before each image acquisition.

[0026] By using real-time LiDAR scanning and local 3D point cloud modeling during cruise along a globally pre-planned path, dynamic macroscopic adjustments to the drone's flight altitude and horizontal trajectory are achieved. This perfectly adapts to the local elevation changes of photovoltaic power stations in complex terrains such as mountains and hills, solving the problems of traditional fixed-altitude cruise modes being unable to adapt to terrain undulations and prone to insufficient acquisition resolution or collision risks. Simultaneously, visual servo technology enables real-time identification of photovoltaic module boundaries and normal vectors, completing fine-tuning of the acquisition attitude. From the beginning of acquisition, it ensures that the lens and the surface of the photovoltaic module are always at the optimal acquisition angle. This solves the problems of image distortion, incomplete module images, and inconsistent acquisition parameters of different modules caused by inconsistent module installation tilt angles and drone flight attitude fluctuations when using traditional fixed gimbal acquisition.

[0027] The S200 synchronously triggers the visible light and infrared thermal imaging lenses of the UAV to simultaneously acquire images of the photovoltaic modules after the attitude has been adjusted, obtaining visible light images and infrared thermal images with the same field of view at the same time. The visible light images and infrared thermal images are then spatially registered at the pixel level to obtain the registered fused image data.

[0028] In this embodiment of the invention, the specific execution process of S200 is as follows: S201 executes a dual-lens synchronous acquisition trigger operation. After the UAV completes the calibration of its flight attitude and lens orientation and enters the optimal acquisition range of the target photovoltaic module, it simultaneously activates the onboard visible light lens and infrared thermal imaging lens through a hardware synchronous trigger signal to simultaneously expose and acquire the same photovoltaic module. This ensures that the acquisition time of the two images is completely synchronized and the shooting field of view is completely matched, with no time difference or field of view misalignment.

[0029] S202 performs standardized preprocessing operations on the dual-light raw images. The standardized preprocessing operations are performed on the simultaneously acquired visible light raw images and infrared thermal imaging raw images, respectively. Specifically, the preprocessing operations include image denoising, lens distortion correction, invalid background area cropping, and resolution unification, to obtain standardized visible light images and infrared thermal imaging images, eliminating image blurring and edge distortion caused by inherent lens distortion, ambient light interference, and drone shaking.

[0030] S203 performs pixel-level spatial registration and fusion of dual-light images. Based on the pre-calibrated camera intrinsic and extrinsic parameters and lens installation fixed parameters of the dual lenses, pixel-level spatial registration is performed on the pre-processed visible light image and infrared thermal imaging image. The pixel coordinates of the two images are mapped to the same spatial coordinate system, so that each pixel point of the two images corresponds one-to-one, and the dual-light image fusion is completed to obtain the registered fused image data. This ensures that the fused image contains both the appearance and internal temperature distribution information of the photovoltaic module.

[0031] By synchronously triggering hardware to acquire visible light and infrared thermal imaging lenses simultaneously with the same field of view, the problems of time difference, field of view deviation, and image content misalignment caused by traditional dual-lens time-division acquisition can be solved. This enables full coverage acquisition of both appearance defects and internal temperature rise defects of photovoltaic modules, avoiding missed defect types caused by a single image source. At the same time, standardized preprocessing eliminates image interference factors, and pixel-level spatial registration achieves a one-to-one correspondence between pixels of the two images. This accurately integrates the appearance details of the visible light image with the temperature distribution information of the infrared thermal imaging image, allowing defect detection to combine appearance features and thermal features simultaneously. This significantly improves the accuracy of defect identification and the differentiation of defect types, further reducing the probability of missed or false defects.

[0032] The S300 integrates image data into a lightweight deep learning model deployed on an airborne edge computing device to complete photovoltaic module defect detection. Based on a preset quantitative judgment standard, the photovoltaic defects are divided into three levels, and the defect detection results and classification information are obtained.

[0033] In this embodiment of the invention, the specific execution process of S300 is as follows: S301 performs real-time defect detection on the airborne end. It inputs the registered and fused image data into a lightweight deep learning detection model pre-deployed in the UAV's airborne edge computing device in real time. The model identifies, classifies and locates photovoltaic module defects in the fused image in real time, and outputs core data such as defect type, component number, defect size and temperature rise. This ensures that the defect detection inference process is completed locally on the airborne end without relying on cloud data transmission.

[0034] S302 performs a defect risk quantification and grading operation. Based on the preset photovoltaic defect quantification judgment standard, combined with the defect type, size, temperature rise range, and impact on power generation efficiency data output by the model, the identified photovoltaic module defects are divided into three risk levels to complete the defect grading judgment. The three risk levels correspond to three maintenance priorities: emergency handling, key maintenance, and routine maintenance.

[0035] S303 performs a defect data structure binding operation, which associates and binds the defect type, location, size, and temperature rise data identified by defect detection with the corresponding defect classification information, original fused image data, and photovoltaic module number information one by one. A unique identifier matching the corresponding module is added to each set of defect data, generating a standardized and structured defect detection result data package to ensure that the data is bound to the corresponding photovoltaic module one by one and there is no data mismatch.

[0036] By using airborne edge computing devices for local model inference, unlike the traditional serial mode of "collecting data and then sending it back to the cloud for processing", real-time defect detection is achieved. Defect identification can be completed without waiting for image transmission, which can significantly shorten the defect discovery cycle. At the same time, a lightweight deep learning model adapted to airborne computing power is adopted to meet the real-time processing needs of UAVs during patrol while ensuring detection accuracy.

[0037] The S400 transmits defect detection results, grading information, and fused image data to the regional edge server for data aggregation and model parameter optimization. Then, it uploads the data to the cloud platform for data storage analysis and knowledge base updates. The cloud platform then distributes the updated model parameters to the regional edge server and the onboard edge computing device of the drone through federated learning.

[0038] In this embodiment of the invention, the specific execution process of S400 is as follows: S401 executes the regional edge server data aggregation operation. After the UAV completes the current patrol mission, it transmits the structured defect detection result data packet, classification information and corresponding fused image data to the regional edge server deployed in the corresponding photovoltaic power station through the wireless communication network. The regional edge server aggregates, cleans and standardizes the full amount of data uploaded by multiple UAVs and multiple batches of inspections within its jurisdiction, and completes the local unified management of the inspection data of the power station under its jurisdiction.

[0039] S402 executes incremental optimization of the regional edge model. Based on the aggregated full-volume defect detection data and the corresponding fused image annotation data, the regional edge server performs incremental training and parameter optimization on the lightweight deep learning model deployed on the airborne end, generating optimized local model parameters. At the same time, the aggregated full-volume standardized data and the optimized local model parameters are uploaded to the cloud platform to complete the uplink transmission of data and model parameters.

[0040] S403 performs cloud platform data management and knowledge base update operations. The cloud platform receives full data and local model parameters uploaded by edge servers in various regions, completes the archiving and storage of full inspection data, and performs multi-dimensional statistical analysis. At the same time, based on full defect data across regions, industries, and multiple power plants, it continuously updates the photovoltaic defect knowledge base, optimizes the defect quantification judgment criteria, and provides full data support for the optimization of photovoltaic power plant operation and maintenance strategies and design and installation schemes.

[0041] S404 executes a global model update and distribution operation based on federated learning. The cloud platform uses the federated learning framework to aggregate and update the global model based on the model optimization parameters uploaded by the edge servers in each region. Then, the updated global model parameters are distributed to the corresponding regional edge servers and the airborne edge computing devices of the drones under their jurisdiction, completing the full-link closed-loop iterative update of the model. At the same time, after the model parameters are distributed, the temporary cache data corresponding to this inspection task is cleared, completing the full-process closed loop of this inspection operation.

[0042] Through a three-tiered data flow architecture from the airborne terminal to the regional edge and the cloud, hierarchical management and efficient processing of inspection data can be achieved. The regional edge server, as an intermediate node, can not only achieve local aggregation and rapid viewing of data from the power plants under its jurisdiction, meeting the needs of immediate handling of urgent defects on site, but also reduce the bandwidth and data processing pressure on the cloud. At the same time, through incremental model optimization at the regional edge, the model can be adapted to different scenarios based on the on-site inspection data of the same power plant, which can solve the problem of insufficient generalization ability of general models in specific power plant scenarios. In addition, the global aggregation, update and distribution of the model can be completed through the federated learning framework, without the need to centrally upload the original production data of each power plant to the cloud. This can not only ensure the privacy and security of the production data of each power plant and meet the relevant data compliance requirements, but also achieve global optimization of the model based on multi-scenario data across the industry. Finally, a closed loop of the entire process of collection, detection, optimization and distribution can be formed, which can solve the problem that traditional models cannot be optimized in real time once deployed and the detection accuracy cannot be continuously improved, and can achieve continuous iterative upgrades of model capabilities.

[0043] Based on the solutions in the embodiments of this application, and combined with the mass production scenario of drone inspection of large-scale mountain photovoltaic power stations, an example is given. Drone inspection of photovoltaic power stations is a core link in realizing the full life cycle operation and maintenance of photovoltaic modules and ensuring the power generation efficiency of the power station. As photovoltaic power stations develop on a large scale in complex terrain areas such as mountains and hills, the photovoltaic modules are scattered, the terrain has large undulations and differences, and the environmental conditions are complex. This places extremely stringent requirements on the image acquisition accuracy, defect recognition real-time performance, complex terrain adaptability, and model generalization ability of drone inspection. In particular, for large-scale mountain photovoltaic power stations of megawatt level and above, the inspection coverage is wide and the number of modules reaches hundreds of thousands, making the dual requirements of inspection efficiency and defect detection accuracy even more prominent.

[0044] Traditional photovoltaic drone inspection solutions generally employ a pre-planned fixed-altitude cruising path combined with a fixed gimbal orientation for image acquisition. During operation, they can only perform fixed actions according to the preset path, failing to adapt to terrain undulations and deviations in component installation tilt angles. This easily leads to problems such as image distortion, inconsistent acquisition angles, and loss of defect details. Some solutions use a model that transmits images back to the cloud for defect detection after inspection, which not only puts pressure on data transmission bandwidth and causes delays in defect detection, but also fails to provide real-time response to abnormal situations during inspection. Other solutions use a model that is centrally trained in the cloud and then uniformly deployed, which cannot adapt to the scene characteristics of different power plants, has insufficient model generalization ability, and requires uploading original production data from the power plant for centralized training, posing data privacy and compliance risks. At the same time, traditional solutions cannot achieve a closed-loop process of inspection, detection, and model optimization. The model detection accuracy cannot be continuously improved with the accumulation of inspection data, making it difficult to meet the mass production requirements of large-scale, high-precision inspection of photovoltaic power plants in large and complex terrains.

[0045] After using the solution in the embodiments of the present invention, a pre-planning step is first performed to obtain high-precision terrain data and photovoltaic module layout data of the target photovoltaic power station in advance, generate a global flight pre-planning path covering the entire power station, and embed a reference flight altitude parameter that matches the terrain of the power station to provide a basic cruise framework for the entire inspection process.

[0046] After pre-planning is completed, the system automatically enters step S100, controlling the UAV to cruise along the pre-planned global flight path. During flight, the onboard LiDAR scans the terrain and photovoltaic modules below in real time, constructing a local 3D point cloud model and dynamically adjusting the flight altitude and horizontal trajectory to adapt to elevation changes in complex terrain. Simultaneously, visual servo technology identifies the boundaries and normal vectors of the photovoltaic modules in real time, fine-tuning the UAV's flight attitude and the orientation of the lens gimbal to ensure that the lens and the surface of the photovoltaic modules always maintain the preset optimal acquisition angle and lock the optimal acquisition pose. Real-time closed-loop calibration of the acquisition attitude can be achieved without manual intervention, thereby ensuring the quality and consistency of the acquired images.

[0047] After the optimal acquisition pose is locked in step S100, the system automatically enters step S200. The visible light and infrared thermal imaging lenses are activated simultaneously via hardware synchronous trigger signals to acquire dual-light images with the same field of view at the same time. The acquired dual-light images are then standardized and preprocessed, and pixel-level spatial registration and fusion are performed to obtain fused image data that simultaneously contains information on the appearance and temperature distribution of the components. The entire acquisition and fusion process is completed synchronously with the UAV cruise, without adding any extra operation time.

[0048] After the fused image data is generated, it automatically enters the S300 step, where the fused image data is input in real time into a lightweight deep learning model deployed on the airborne edge computing device. The real-time identification, location and classification of photovoltaic module defects are completed locally on the airborne end. Then, based on the preset quantitative judgment standard, the three-level risk classification of defects is completed. Finally, the defect data, classification information and fused image data are structurally bound together to generate a standardized defect detection result data package, realizing a real-time inspection mode of "collecting, detecting and outputting results on the same time", without waiting for the entire inspection process to be completed to obtain defect detection results.

[0049] Finally, the S400 step is executed. After the UAV completes the patrol mission, it transmits the defect detection result data packet and fused image data to the regional edge server to complete the aggregation of all inspection data and incremental optimization of the model. Then, the standardized data and local model parameters are uploaded to the cloud platform. The cloud platform completes the storage and analysis of all data and updates the photovoltaic defect knowledge base. At the same time, it completes the aggregation and update of the global model through the federated learning framework. Then, the optimized model parameters are distributed to the regional edge server and the UAV's onboard edge computing device to complete the full-link closed-loop iteration of the model. At the same time, the temporary cached data of this inspection is cleared to prepare for the next inspection mission.

[0050] This solution significantly improves the image acquisition accuracy and consistency of photovoltaic power plants in complex terrain without increasing the inspection cycle of a single power plant. It achieves full coverage acquisition and real-time detection of external and internal defects of photovoltaic modules. At the same time, through the collaboration of a three-level architecture from the airborne end to the regional edge end and the cloud, it realizes efficient flow of inspection data and closed-loop iteration of the model throughout the entire process. Real-time attitude calibration solves the problems of traditional solutions being unable to adapt to complex terrain and unstable image quality. Real-time airborne edge detection solves the problems of delayed defect detection and high data transmission pressure. Federated learning achieves dual protection of global model optimization and data privacy and security. It perfectly solves the technical problems of low acquisition accuracy, poor real-time defect detection, insufficient model generalization ability, high data compliance risk, and difficulty in balancing mass production efficiency and detection accuracy in traditional photovoltaic drone inspection solutions. It fully meets the mass production requirements of large-scale, high-precision, and high-efficiency inspection of photovoltaic power plants in complex terrain.

[0051] In this embodiment of the application, the step of adjusting the flight altitude and trajectory based on a local 3D point cloud model specifically includes: Precise division of unit point cloud grids: First, obtain the local 3D point cloud model constructed in real time by the airborne lidar. Using the horizontal width and vertical spacing of a single photovoltaic string in the target photovoltaic power station as the fixed grid size, the local 3D point cloud model is divided into equal-sized grids, resulting in several unit point cloud grids. Each unit point cloud grid corresponds one-to-one with a single photovoltaic string on site, completing the splitting of the overall point cloud data by photovoltaic string, avoiding mutual interference between point cloud data of different strings.

[0052] Point cloud elevation clustering and feature extraction: For all point cloud data within each divided unit point cloud raster, clustering is performed according to elevation values ​​to separate two sets of point clouds with significantly different elevation features. The upper cluster of point clouds with higher elevation values ​​corresponds to the top surface of the photovoltaic module, while the lower cluster of point clouds with lower elevation values ​​corresponds to the terrain base below the photovoltaic module. After separation, the mean and maximum elevation values ​​of the upper cluster of point clouds within each unit point cloud raster are extracted as the core quantitative basis for flight altitude and trajectory adjustment.

[0053] Locally optimized flight trajectory generation adapted to the array arrangement: Based on the baseline flight altitude of the pre-planned global flight path, and combined with the average elevation of the upper cluster point cloud of all unit point cloud grids divided by the current local 3D point cloud model, a continuous and smooth flight altitude envelope is fitted to avoid abrupt changes in flight altitude; at the same time, based on the center coordinates of each unit point cloud grid, the lateral offset of the global flight path is corrected to ensure that the flight trajectory is precisely aligned with the center position of the photovoltaic array, and finally, a locally optimized flight trajectory that is fully adapted to the actual array arrangement of photovoltaic arrays in the current scanning area is generated.

[0054] Trajectory following and safety threshold control: The drone is controlled to perform cruise flight along the generated locally optimized flight trajectory. During the flight, the drone's real-time flight altitude changes synchronously with the fitted flight altitude envelope. At the same time, the flight altitude threshold is strictly controlled to ensure that the difference between the drone's real-time flight altitude and the maximum elevation of the upper cluster point cloud in the corresponding unit point cloud grid is never less than the preset safety distance, thus avoiding the risk of collision between the drone and photovoltaic modules from the execution level.

[0055] By segmenting the gridded point cloud by photovoltaic strings and accurately extracting the height features of the top surface of the modules through elevation clustering, it is possible to achieve smooth adaptation between the flight altitude and the actual elevation of the strings, and precise alignment between the flight trajectory and the string arrangement. Furthermore, flight safety is ensured through hard threshold control. This can solve the problem of mismatch between the globally pre-planned path and the actual arrangement and elevation of the photovoltaic strings in complex terrain, and can significantly improve the safety of inspection flights and the consistency of the quality of acquired images.

[0056] In this embodiment of the application, the step of performing elevation clustering separation on the point cloud data within each unit point cloud grid to obtain the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base specifically includes: Valid elevation interval setting and invalid point cloud initial screening: Based on the physical dimensions and installation tilt angle parameters of the photovoltaic modules in the target photovoltaic power station obtained in advance, the effective value interval for elevation clustering is defined in advance for the corresponding unit point cloud grid. This interval is a reasonable elevation range that matches the photovoltaic modules and the terrain base at that location. Then, isolated and scattered invalid point cloud data with elevations exceeding the effective interval are removed from the unit point cloud grid, thus completing the preliminary cleaning of interference data.

[0057] Iterative threshold calculation and initial clustering: The median of the elevation distribution of the remaining valid point clouds within the unit point cloud grid is used as the initial segmentation threshold. The segmentation threshold is continuously updated through iterative calculation until the threshold converges and stabilizes. Then, the valid point clouds within the grid are initially divided using the updated final segmentation threshold as the boundary. Point clouds with elevations higher than the segmentation threshold are classified as initial upper cluster point clouds, and point clouds with elevations lower than the segmentation threshold are classified as initial lower cluster point clouds, thus completing the initial clustering and splitting.

[0058] Plane fitting verification and data correction based on installation tilt angle: Perform plane fitting on the divided initial upper cluster point cloud, calculate the actual tilt angle of the fitted plane, and then compare the deviation of this tilt angle with the preset installation tilt angle of the photovoltaic string corresponding to the unit point cloud grid; if the deviation exceeds the preset allowable range, it is determined that the point cloud corresponding to the deviation does not belong to the valid data of the top surface of the photovoltaic module, and it is assigned to the initial lower cluster point cloud to complete the error correction of the initial clustering result and remove the interference point cloud that does not belong to the top surface of the module.

[0059] Based on the continuity verification of adjacent grids and the final clustering output: compare the difference between the current unit point cloud grid and the initial upper cluster point cloud elevation mean of the adjacent unit point cloud grids in the same row. The installation height of the photovoltaic strings in the same row has natural continuity, and the difference will not exceed a reasonable range under normal conditions. If the difference exceeds the preset continuity threshold, it is determined that there is a deviation in the current clustering result, and the process returns to the step of iteratively updating the segmentation threshold, and performs clustering separation and tilt angle verification again until the difference meets the preset continuity threshold requirement. After all verifications pass, the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base are finally locked.

[0060] By employing a four-pronged closed-loop control system—invalid point cloud initial screening, iterative threshold clustering, installation tilt angle fitting verification, and adjacent grid continuity verification—various interference items in the point cloud data are precisely eliminated. This addresses the issues of insufficient separation accuracy of point clouds between the top surface of photovoltaic modules and the terrain substrate, as well as large deviations in clustering results under complex terrain conditions.

[0061] In this embodiment of the application, the step of simultaneously identifying the photovoltaic module boundary and normal vector using visual servoing technology, and fine-tuning the flight attitude and lens orientation, includes: With the visual recognition area defined, after the UAV completes the point cloud rasterization, it directly uses the spatial coordinate range of the corresponding unit point cloud raster to delineate the region of interest in the image for visual servo recognition. Only the component boundary recognition is performed on this local area, and the entire image is no longer processed.

[0062] Image enhancement and effective boundary screening: First, the image of the region of interest is processed by grayscale and gradient enhancement to make the edge features of the photovoltaic module more prominent. Then, combined with the known physical size and arrangement rules of the module, the complete closed boundary of the module is extracted, and the incomplete or obviously mismatched boundary results are directly eliminated.

[0063] The surface plane equation of the photovoltaic module is fitted and the normal vector is calculated. Based on the extracted complete closed boundary and the three-dimensional point cloud coordinate data of the same area, the surface plane equation of the photovoltaic module is fitted. Then, the normal vector of the module surface is calculated through the plane equation, which serves as the reference for attitude calibration.

[0064] The attitude and lens orientation are adjusted by split-axis decoupling. The angular deviation between the optical axis and the normal vector of the acquisition lens is calculated using the normal vector of the component surface as the standard. Then, the adjustment amount is calculated separately according to different axis systems such as pitch, roll, and yaw, and the attitude and lens orientation of the UAV are corrected separately until the angular deviation falls within the allowable range.

[0065] By using region-focused recognition, precise boundary extraction, and fine-tuned attitude adjustment, the lens optical axis is always properly aligned with the surface of the photovoltaic module, effectively avoiding image distortion caused by acquisition angle deviation.

[0066] In this embodiment of the application, the step of performing pixel-level spatial registration of a visible light image and an infrared thermal imaging image to obtain registered fused image data specifically includes: The system uses the complete closed boundary of the photovoltaic module, which has been extracted and verified in the visual servoing process, as the registration benchmark. The system locates the vertex coordinates corresponding to the complete closed boundary of the same photovoltaic module in the simultaneously acquired visible light image and infrared thermal image. The corresponding vertices in the two sets of images are used as the same registration points for the dual-image registration, which avoids the problems of general feature point matching being easily affected by environmental interference and invalid registration points.

[0067] Field of view offset spatial correction combines the three-dimensional spatial coordinate data of the corresponding area of ​​the photovoltaic module in the local three-dimensional point cloud model to perform spatial coordinate mapping correction on two sets of corresponding registration points. This offsets the field of view offset caused by the baseline difference and different field of view angles between the visible light lens and the infrared thermal imaging lens due to the installation position, and corrects the spatial misalignment problem of the corresponding registration points.

[0068] Homography transformation and pixel-level alignment: Based on the corrected registration points, a homography transformation matrix adapted to the mapping relationship between the two images is constructed. This matrix maps all pixel coordinates of the infrared thermal imaging image to the pixel coordinate system of the visible light image, achieving pixel-level precise alignment of the two images and finally obtaining the registered fused image data.

[0069] The above registration process uses the verified and effective component closure boundary as a benchmark, and combines three-dimensional spatial coordinate correction of dual-lens field of view offset to achieve high-precision pixel-level alignment of visible light and infrared thermal imaging images, which can solve the registration misalignment problem caused by dual-lens field of view difference.

[0070] In this embodiment of the application, the step of classifying photovoltaic defects into three levels according to a preset quantitative judgment standard to obtain defect detection results and classification information specifically includes: After the lightweight deep learning model completes the defect detection, localization and classification of fused image data, it does not simply output the defect type, but further extracts six quantitative data directly related to the defect risk level from the fused image (including visible light appearance features and infrared thermal features) to ensure the scientific nature and traceability of the classification judgment. Specifically, the criteria are as follows: First, defect type: clearly identifying the specific category of defects, such as hot spots, microcracks, broken glass, dirt, and junction box abnormalities, as different types of defects have significantly different degrees of harm to the module. Second, geometric dimensions: accurately measuring the length, width, and area of ​​defects to quantify their physical size. Third, infrared thermal imaging temperature rise difference: calculating the temperature difference between the defect area and the normal operating area of ​​the module through infrared thermal imaging image comparison, intuitively reflecting the severity of the defect (e.g., the temperature rise difference of hot spot defects is directly related to the risk of fire). Fourth, the distribution location of defects within the photovoltaic module: distinguishing between defects located in the central area, edge area, and near the junction box, as defects in different locations have different impacts on the module's power generation efficiency and structural stability. Fifth, the defect area ratio within a single photovoltaic module: calculating the ratio of the total area of ​​all defects on a single module to the total area of ​​the module, reflecting the degree of damage to a single module. Sixth, the defect contiguous situation of adjacent modules within the same string: statistically analyzing whether adjacent modules within the same photovoltaic string have the same or different types of defects, the number and range of contiguous defects, as contiguous defects can easily lead to a decrease in the overall power generation efficiency of the string and even spread safety hazards.

[0071] To achieve accurate quantitative judgment of defect levels, a comprehensive grading system is established in advance. Firstly, for the three defect levels—urgent, important, and general—clear and detailed threshold ranges are set for the six quantitative data items mentioned above. Each quantitative data item for each level has a clear value range, ensuring that defects of different severity can be accurately distinguished. For example, the temperature rise difference threshold is set higher for hot spot defects in the urgent level, while it is set lower for dirt defects in the general level. Regarding the defect area ratio, the threshold set for the urgent level is much higher than that for the important and general levels, clearly defining the boundaries of defect severity for different levels. Secondly, considering the impact of each quantitative dimension on the photovoltaic module's power generation efficiency and operational safety, differentiated weighting coefficients are set for each quantitative dimension. The core principle is "the greater the impact, the higher the weight." Among them, the three dimensions of defect type (which directly determines the nature of the defect's harm, such as glass breakage being a structural defect and hot spots being a safety defect, with harm far greater than ordinary dirt), infrared thermal imaging temperature rise difference (which directly reflects the activity level of the defect; the higher the temperature rise, the greater the safety hazard and power generation loss), and defect cascading of adjacent modules within the same string (cascading defects are prone to triggering chain reactions, leading to overall string failure and expanding the scope of harm) have the most significant impact on the safe operation of modules and power generation efficiency. Therefore, the weighting coefficients set are significantly higher than those of the three dimensions of defect geometry, defect distribution location, and defect area ratio of a single module, ensuring that core risk factors dominate the classification judgment and avoiding secondary factors from interfering with the rationality of the classification results.

[0072] After data extraction and parameter configuration are completed for comprehensive quantitative judgment and precise defect level classification, the core execution stage of graded judgment begins. A dual judgment mode of "threshold matching + weighted calculation" is adopted to ensure the accuracy and consistency of the grading results. First, the six quantitative data points extracted from a single defect are substituted into the threshold ranges corresponding to the three levels of emergency, importance, and general, and matched one by one to preliminarily determine the possible level range to which the defect belongs. Then, combined with preset weighting coefficients for each quantitative dimension, the matching results corresponding to each quantitative data point (if it meets a certain level threshold, the corresponding score is taken; if it does not meet, the corresponding deduction value is taken) are weighted and summed to calculate the comprehensive judgment score for the defect. The comprehensive score directly reflects the overall risk level of the defect. Finally, based on the range of the comprehensive judgment score, the photovoltaic defect is precisely classified into the corresponding level—if the comprehensive score reaches the emergency level threshold, it is judged as an emergency defect requiring immediate action; if it reaches the importance level threshold but does not reach the emergency level threshold, it is judged as an important defect requiring priority action; if it reaches the general level threshold but does not reach the importance level threshold, it is judged as a general defect and handled according to the regular operation and maintenance process. After the grading is completed, the detailed information of defect detection (defect type, location, size, etc.) and the grading results are integrated simultaneously to form complete defect detection results and grading information, ensuring that the grading of each defect has clear data support and a traceable judgment process.

[0073] By constructing a complete grading system of "multi-dimensional quantitative extraction + differentiated weighted judgment + precise threshold matching", the limitations of traditional defect grading, which is "based on only a single dimension, with vague judgment standards and chaotic priorities", are broken. This system not only achieves precise quantification and scientific grading of photovoltaic defect risks, but also highlights the priority of defects related to safety hazards and major power generation losses through the weighted configuration of core risk dimensions. This provides power plant operation and maintenance personnel with clear and actionable handling guidelines, effectively guiding them to prioritize the handling of urgent and important defects, avoid major safety hazards, reduce power generation losses, and avoid the waste of operation and maintenance resources caused by the over-handling of routine defects, thus ensuring the safe, stable and efficient operation of photovoltaic power plants.

[0074] In this embodiment of the application, before performing defect level classification, the specific steps include: Before initiating the defect level classification operation, the acquisition and retrieval of classification benchmark parameters are completed by acquiring two types of core benchmark parameters: one type is real-time operating condition data, which accurately collects the measured values ​​of on-site ambient temperature and solar irradiance at the time corresponding to the image captured by the drone. These two parameters directly determine the real-time operating status of the photovoltaic module and are the core influencing factors for infrared temperature rise detection; the other type is the inherent attribute parameters of the module, which retrieve the nominal power, service life, and installation tilt angle of the photovoltaic module from the pre-stored database to clarify the basic performance benchmark and aging degradation of the module.

[0075] Using measured solar irradiance as the core benchmark and combining it with the standard photoelectric conversion characteristics of photovoltaic cells, the graded threshold range corresponding to the temperature rise difference in infrared thermal imaging is dynamically corrected. The correction magnitude is positively correlated with the difference between the measured irradiance and the irradiance under standard test conditions (1000W / ㎡). This eliminates the judgment bias caused by fluctuations in the base temperature rise of the module under different light intensities, avoiding the missed detection of serious defects due to insufficient temperature rise values ​​in low light environments, or the misjudgment of normal operating conditions due to excessive base temperature rise in strong light environments.

[0076] Based on measured ambient temperature and the service life of the modules, the grading threshold ranges corresponding to defect geometry and defect area percentage are corrected, with the correction magnitude positively correlated with the service life of the modules. The core is to adapt to the natural performance degradation of modules as they grow with service time, reasonably adjust the threshold boundaries for older modules, and avoid misjudging normal aging wear as defects. At the same time, considering the impact of ambient temperature on the operating state of the modules, the judgment criteria for size-related defects are corrected to eliminate detection errors caused by environmental temperature changes.

[0077] After completing the above threshold correction, the corrected threshold range is compared and verified with the measured benchmark data of adjacent defect-free photovoltaic modules under the same operating conditions in the same string. If the verification deviation exceeds the preset reasonable range, the threshold range is calibrated a second time based on the measured data of adjacent defect-free modules to completely eliminate the benchmark deviation caused by module installation differences and line losses in the same string. After calibration, the subsequent defect level classification operation is performed.

[0078] By combining real-time on-site operating conditions and dynamic threshold correction of component attributes, along with cross-validation of components in the same string under the same operating conditions, the interference of environmental fluctuations, component aging, and differences in operating conditions on defect classification is completely eliminated, ensuring the accuracy and consistency of defect classification results for components in different scenarios and with different service life.

[0079] In this embodiment of the application, after completing the defect level classification, the specific steps include: After completing the defect classification of the entire panel, a secondary verification and data acquisition process is immediately triggered for photovoltaic modules with defects classified as emergency. During the secondary acquisition, the unit point cloud grid data corresponding to the module is used as a precise benchmark. The flight position, altitude, and lens gimbal orientation of the drone are fine-tuned to ensure that the lens optical axis is completely aligned with the normal vector of the module surface. At the same time, the lens field of view is reduced and the image acquisition resolution is increased to perform close-range, high-precision shooting of the target module. Subsequently, the entire process of simultaneous dual-light image acquisition, registration and fusion, defect detection, and classification is repeated to complete the verification and validation of emergency defects, thereby avoiding the risk of misjudgment.

[0080] The fused image data obtained from the secondary acquisition, the verified defect detection results, and the classification information are set as the highest transmission priority and transmitted to the regional edge server first. This ensures that data on major security risks are delivered to the on-site operation and maintenance end as soon as possible, guaranteeing the timeliness of emergency response. Other important and general defect data are transmitted in the preset conventional order, without occupying the transmission bandwidth of emergency data, thus avoiding delays in the handling of emergency defects.

[0081] After receiving emergency-level defect data, the regional edge server first completes manual annotation and compliance verification. Then, the annotated emergency defect-specific dataset is used for incremental training of the lightweight deep learning model on the airborne end, which specifically enhances the model's feature recognition capability for high-risk emergency defects. The optimized model parameters are then distributed to the airborne edge computing device of the drone through the cloud platform to complete the targeted iterative upgrade of the model.

[0082] By employing high-precision secondary verification of urgent defects, high-priority data transmission, and incremental training on a dedicated dataset, the system achieves rapid verification and prioritized handling of major safety hazards in photovoltaic power plants. It also specifically enhances the model's accuracy in identifying high-risk defects, forming a complete closed loop from hazard discovery to verification and confirmation, rapid handling, and model optimization. This ensures the safe operation of the power plant while continuously improving the detection reliability of the inspection model. In this application, the cloud platform distributes the updated model parameters to the regional edge server and the onboard edge computing device of the drone through federated learning, specifically including: Local sample hierarchical division and priority training set setting: Each regional edge server first divides the full set of defect data that has been labeled and verified locally into corresponding sample subsets according to defect level. At the same time, the sample subset corresponding to the emergency level defects is set as the priority training sample set, which clarifies the core priority of local training and ensures that the features of high-risk defects are learned in the training.

[0083] The cloud platform distributes the initial parameters of the global lightweight deep learning model to all connected regional edge servers. This global model serves as the unified benchmark model for all lightweight models on the UAV's airborne end. Each regional edge server uses its pre-defined sample subset as the core to complete the incremental training of its local model, ultimately obtaining updated parameters for the local model that are adapted to the characteristics of the local power plant scenario.

[0084] Local parameter upload and cloud-based weighted aggregation update: Each regional edge server only uploads the locally trained model update parameters to the cloud platform, without uploading the original fused image data and original defect data locally, thus protecting the privacy of production data of each power station; after receiving the local model update parameters uploaded by all regional edge servers, the cloud platform performs weighted aggregation processing to finally generate the updated global model parameters.

[0085] The global model parameters are distributed and updated synchronously across the entire system. The cloud platform distributes the updated global model parameters to all connected regional edge servers. Each regional edge server then synchronizes the new model parameters to all UAV-borne edge computing devices within its jurisdiction, ultimately completing the unified iterative update of the model for all terminals across the entire system.

[0086] By adopting a model of local training, parameter aggregation, and global distribution, the system achieves global optimization of the model and targeted enhancement of the ability to identify high-risk defects without uploading the original production data of each power station and while fully ensuring data privacy and compliance. At the same time, it completes synchronous iteration of all terminal models in the entire system, which can solve the problems of high data compliance risk and insufficient scenario generalization ability of traditional centralized model training.

[0087] This application discloses an image acquisition device deployed in a drone to acquire images of photovoltaic equipment, referring to... Figure 2 The system first acquires terrain data and photovoltaic module layout data of the target photovoltaic power station, and then generates a global pre-planned flight path covering the target photovoltaic power station. The global pre-planned flight path includes the reference flight altitude parameters corresponding to the terrain of the photovoltaic power station, including: The global path and adaptive flight control module 001 controls the UAV to fly along the pre-planned global flight path. It uses an airborne lidar to scan the terrain and photovoltaic modules below in real time, constructs a local 3D point cloud model, and adjusts the flight altitude and trajectory based on the local 3D point cloud model. At the same time, it uses visual servo technology to identify the boundaries and normal vectors of the photovoltaic modules, and fine-tunes the flight attitude and lens orientation to keep the image acquisition lens and the surface of the photovoltaic modules at a preset acquisition angle. The dual-light image synchronous acquisition and registration module 002 synchronously triggers the visible light and infrared thermal imaging lenses of the UAV to synchronously acquire the photovoltaic module after the attitude is adjusted, and obtains visible light images and infrared thermal imaging images with the same field of view at the same time. The visible light images and infrared thermal imaging images are then spatially registered at the pixel level to obtain the registered fused image data. The photovoltaic defect detection and intelligent grading module 003 inputs fused image data into a lightweight deep learning model deployed on an airborne edge computing device to complete the photovoltaic module defect detection. Based on a preset quantitative judgment standard, the photovoltaic defects are divided into three levels, and the defect detection results and grading information are obtained. The edge-cloud collaborative model update module 004 transmits defect detection results, grading information, and fused image data to the regional edge server to complete data aggregation and model parameter optimization. Then, it uploads the data to the cloud platform to complete data storage analysis and knowledge base updates. The cloud platform uses federated learning to distribute the updated model parameters to the regional edge server and the airborne edge computing device of the drone.

[0088] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for image acquisition of photovoltaic equipment deployed in a drone, characterized in that, The terrain data and photovoltaic module layout data of the target photovoltaic power station are acquired in advance to generate a global pre-planned flight path covering the target photovoltaic power station. The global pre-planned flight path includes the reference flight altitude parameters corresponding to the terrain of the photovoltaic power station, including: When controlling the UAV to fly along the pre-planned global flight path, the UAV uses an airborne lidar to scan the terrain and photovoltaic modules below in real time to construct a local three-dimensional point cloud model. The flight altitude and trajectory are adjusted based on the local three-dimensional point cloud model. At the same time, visual servo technology is used to identify the boundaries and normal vectors of the photovoltaic modules and to fine-tune the flight attitude and lens orientation so that the image acquisition lens and the surface of the photovoltaic modules maintain a preset acquisition angle. By simultaneously triggering the visible light and infrared thermal imaging lenses of the UAV, the photovoltaic module after the attitude is adjusted is simultaneously acquired, and visible light images and infrared thermal imaging images with the same field of view at the same time are obtained. The visible light images and infrared thermal imaging images are then spatially registered at the pixel level to obtain the registered fused image data. The fused image data is input into a lightweight deep learning model deployed on an airborne edge computing device to complete the defect detection of photovoltaic modules. Based on a preset quantitative judgment standard, the photovoltaic defects are divided into three levels, and the defect detection results and classification information are obtained. The defect detection results, classification information, and fused image data are transmitted to the regional edge server to complete data aggregation and model parameter optimization. Then, they are uploaded to the cloud platform to complete data storage analysis and knowledge base updates. The cloud platform uses federated learning to distribute the updated model parameters to the regional edge server and the airborne edge computing device of the drone.

2. The image acquisition method for photovoltaic equipment deployed in a drone according to claim 1, characterized in that, The steps of adjusting flight altitude and trajectory based on the local 3D point cloud model include: The local three-dimensional point cloud model is divided into grids according to the horizontal width and vertical spacing of a single photovoltaic string in the target photovoltaic power station, resulting in several unit point cloud grids that correspond one-to-one with a single photovoltaic string. The point cloud data in each unit point cloud raster is separated by elevation clustering to obtain the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base. The mean elevation and maximum elevation of the upper cluster point cloud in each unit point cloud raster are extracted. Based on the baseline flight altitude of the global flight pre-planned path, and combined with the average elevation of all unit point cloud grids obtained from the current local 3D point cloud model, a continuous and smooth flight altitude envelope is fitted. At the same time, the lateral offset of the global flight pre-planned path is corrected according to the center coordinates of each unit point cloud grid, and a locally optimized flight trajectory adapted to the photovoltaic string arrangement in the current scanning area is generated. The drone is controlled to fly along a locally optimized flight trajectory, with its flight altitude following the flight altitude envelope throughout the entire flight. The difference between the flight altitude and the maximum elevation of the upper cluster point cloud within the corresponding unit point cloud grid is not less than a preset safe distance. 3.The image acquisition method for coping with photovoltaic equipment deployed in a UAV of claim 2, wherein, The step of performing elevation clustering separation on the point cloud data within each unit point cloud raster to obtain the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base includes: Based on the physical size parameters and installation tilt angle parameters of the photovoltaic modules in the target photovoltaic power station obtained in advance, the effective value range of elevation clustering is set for the corresponding unit point cloud grid, and isolated invalid point cloud data with elevation exceeding the effective value range are removed. The median of the elevation distribution of the remaining valid point clouds within the unit point cloud grid is used as the initial segmentation threshold. The segmentation threshold is updated through iterative calculation. Point clouds with elevations higher than the updated segmentation threshold are classified as initial upper cluster point clouds, and point clouds with elevations lower than the updated segmentation threshold are classified as initial lower cluster point clouds. Plane fitting is performed on the initial upper cluster point cloud, and the deviation between the fitted plane tilt angle and the preset installation tilt angle of the photovoltaic string corresponding to the unit point cloud grid is verified. If the deviation exceeds the preset allowable range, the point cloud data corresponding to the deviation exceeding the preset allowable range is included in the initial lower cluster point cloud. Compare the difference between the initial upper cluster point cloud elevation mean of the current unit point cloud grid and the adjacent unit point cloud grid in the same row. If the difference exceeds the preset continuous threshold, iterate and update the segmentation threshold again and perform clustering separation and verification operations again until the difference meets the preset continuous threshold requirement. Finally, the upper cluster point cloud of the corresponding photovoltaic module top surface and the lower cluster point cloud of the corresponding terrain base are obtained. 4.The image acquisition method for coping with photovoltaic equipment deployed in a UAV according to claim 2, wherein, The steps of simultaneously identifying the boundaries and normals of photovoltaic modules using visual servoing technology, and fine-tuning the flight attitude and camera orientation, include: After the UAV completes the rasterization of the local 3D point cloud model, it locks the region of interest in the image for visual servo recognition based on the spatial coordinate range of the corresponding unit point cloud raster, and performs photovoltaic module boundary recognition processing only on the image content within the locked region of interest. The image within the region of interest is processed by grayscale conversion and gradient enhancement. Combined with the pre-acquired physical size parameters and string arrangement rules of the photovoltaic modules, the complete closed boundary of the photovoltaic modules is extracted, and invalid recognition results with incomplete boundaries or size deviations exceeding the preset range are eliminated. Based on the complete closed boundary of the photovoltaic module, and combined with the point cloud coordinate data of the corresponding region in the local three-dimensional point cloud model, the surface plane equation of the photovoltaic module is obtained by fitting, and the surface normal vector of the photovoltaic module is calculated based on the surface plane equation. Using the surface normal vector of the photovoltaic module as a reference, the angular deviation between the optical axis of the image acquisition lens and the normal vector is calculated. Based on the angular deviation, the pitch and roll attitude of the UAV and the pitch and yaw orientation of the lens are adjusted in a split-axis decoupling manner until the angular deviation falls within the preset allowable range.

5. The method of claim 4, wherein the UAV is deployed to capture images of the photovoltaic installation. The step of performing pixel-level spatial registration between the visible light image and the infrared thermal imaging image to obtain the registered fused image data includes: Using the complete closed boundary of the photovoltaic module extracted by visual servoing technology as the registration reference, the vertex coordinates corresponding to the complete closed boundary of the same photovoltaic module are located in the visible light image and the infrared thermal imaging image, respectively, as the same registration points of the two sets of images; By combining the three-dimensional spatial coordinate data of the photovoltaic module corresponding area in the local three-dimensional point cloud model, spatial coordinate mapping correction is performed on two sets of corresponding registration points to eliminate the field of view offset caused by the installation baseline difference between the visible light lens and the infrared thermal imaging lens. Based on the corrected registration points, a homography transformation matrix is ​​constructed. The pixel coordinates of the infrared thermal imaging image are mapped to the pixel coordinate system of the visible light image through the homography transformation matrix, thus completing the pixel-level alignment of the two images and obtaining the registered fused image data. 6.The image acquisition method for coping with photovoltaic equipment deployed in a UAV according to claim 1, wherein, The step of classifying photovoltaic defects into three levels according to a preset quantitative judgment standard and obtaining defect detection results and classification information includes: After completing defect detection on the fused image data, six quantitative data items corresponding to the defects are extracted. The quantitative data items are defect type, geometric size, infrared thermal imaging temperature rise difference, distribution location of defects in photovoltaic modules, defect area ratio in a single photovoltaic module, and defect contiguousness of adjacent modules in the same string. Threshold ranges for six quantitative data items were set for the three defect levels: emergency, important, and general. At the same time, a weighting coefficient was set for the degree of impact of the defect on the power generation and safety of the photovoltaic module for each quantitative dimension. The weighting coefficients for defect type, infrared thermal imaging temperature rise difference, and the situation of defects in the same string are higher than the weighting coefficients of the other three dimensions. The six extracted quantitative data points are substituted into the corresponding threshold ranges to complete the matching. The comprehensive judgment score of the defect is calculated by combining the weighting coefficients. Based on the comprehensive judgment score, the photovoltaic defects are classified into the corresponding levels, and the defect detection results and classification information are obtained.

7. The method of claim 6, wherein the UAV is deployed to capture images of the photovoltaic installation. Before performing the defect level classification, the following is included: First, obtain the measured data of ambient temperature and solar irradiance at the time when the drone collects the corresponding image. At the same time, retrieve the pre-stored nominal power, service life, and installation tilt angle parameters corresponding to the photovoltaic module. Based on measured solar irradiance data, and combined with the photoelectric conversion characteristics of photovoltaic cells, the threshold range corresponding to the temperature rise difference in infrared thermal imaging is corrected. The correction magnitude is positively correlated with the difference between measured solar irradiance data and irradiance under standard test conditions. Based on measured ambient temperature data and the number of years the modules have been in service, the threshold range corresponding to the defect geometry and defect area ratio is corrected. The correction range is positively correlated with the number of years the modules have been in service. The corrected threshold range is compared and verified with the measured data of adjacent defect-free photovoltaic modules in the same string. If the verification deviation exceeds the preset range, the threshold range is calibrated a second time based on the measured data of adjacent defect-free modules. After the calibration is completed, the defect level classification operation is performed. 8.The image acquisition method for coping with photovoltaic equipment deployed in a UAV according to claim 7, wherein, After completing the defect level classification, the following is included: For photovoltaic modules with defects classified as emergency level, a second image acquisition is immediately triggered. During the second acquisition, the drone's flight position, altitude, and lens orientation are adjusted based on the unit point cloud grid data of the photovoltaic module with defects classified as emergency level, so that the lens optical axis coincides with the normal vector of the module surface. At the same time, the lens field of view is reduced and the image acquisition resolution is improved. The image acquisition, defect detection, and level classification operations are repeated. The fused image data, defect detection results, and classification information obtained from the secondary acquisition are set as the highest transmission priority and transmitted to the regional edge server first. Defect data of other levels are transmitted in sequence according to the preset order. After the regional edge server completes the annotation and verification of the emergency level defect data, it uses the annotated dataset for incremental training of a lightweight deep learning model. The trained and optimized model parameters are then distributed to the airborne edge computing device of the drone through the cloud platform. 9.The image acquisition method for coping with photovoltaic equipment deployed in a UAV of claim 8, wherein, The cloud platform uses federated learning to distribute updated model parameters to regional edge servers and onboard edge computing devices of drones, including: Each regional edge server will divide the defect data that has been locally annotated and verified into corresponding sample subsets according to the defect level, and set the sample subset corresponding to the emergency level defects as the priority training sample set. The cloud platform distributes the initial parameters of the global lightweight deep learning model to all connected regional edge servers. The global lightweight deep learning model is the global benchmark model of the lightweight deep learning model. Each regional edge server completes the incremental training of its local model based on a local sample subset, with the priority training sample set as the core, and obtains the local model update parameters. Each regional edge server only uploads the local model update parameters to the cloud platform, without uploading the original fused image data. The cloud platform performs weighted aggregation processing on all uploaded local model update parameters to obtain the updated global model parameters. The cloud platform distributes the updated global model parameters to the edge servers in each region, and then the edge servers in each region synchronize the model parameters to the airborne edge computing devices of the corresponding drones under their jurisdiction, thus completing the iterative update of the model across the entire system.

10. An image acquisition device deployed in an unmanned aerial vehicle to address a photovoltaic installation, characterized in that, The terrain data and photovoltaic module layout data of the target photovoltaic power station are acquired in advance to generate a global pre-planned flight path covering the target photovoltaic power station. The global pre-planned flight path includes the reference flight altitude parameters corresponding to the terrain of the photovoltaic power station, including: The global path and adaptive flight control module controls the UAV to fly along the pre-planned global flight path. It uses an airborne lidar to scan the terrain and photovoltaic modules below in real time to construct a local three-dimensional point cloud model. Based on the local three-dimensional point cloud model, it adjusts the flight altitude and trajectory. At the same time, it uses visual servo technology to identify the boundaries and normal vectors of the photovoltaic modules and fine-tunes the flight attitude and lens orientation to keep the image acquisition lens and the surface of the photovoltaic modules at a preset acquisition angle. The dual-light image synchronous acquisition and registration module synchronously triggers the visible light and infrared thermal imaging lenses of the UAV to synchronously acquire the photovoltaic module after the attitude is adjusted, and obtains visible light images and infrared thermal imaging images with the same field of view at the same time. The visible light images and infrared thermal imaging images are then spatially registered at the pixel level to obtain the registered fused image data. The photovoltaic defect detection and intelligent grading module integrates image data input into a lightweight deep learning model deployed on an airborne edge computing device to complete photovoltaic module defect detection. Based on a preset quantitative judgment standard, the photovoltaic defects are divided into three levels, and the defect detection results and grading information are obtained. The edge-cloud collaborative model updating module transmits the defect detection result, the grading information and the fusion image data to the regional edge server to complete data aggregation and model parameter optimization, and then uploads to the cloud platform to complete data storage analysis and knowledge base updating. The cloud platform updates the model parameters and sends them to the regional edge server and the airborne edge computing device of the unmanned aerial vehicle through federated learning.

Images