High-precision real-time photovoltaic panel multi-source image end-to-end defect detection method and system

Abstract

The invention discloses a high-precision real-time photovoltaic panel multi-source image end-to-end defect detection method and system, and relates to the technical field of photovoltaic detection and computer vision. The objective of the invention is to solve the problems of poor single-mode adaptability, insufficient multi-source fusion, poor real-time performance and no closed-loop optimization in the prior art. According to the method, infrared and visible light features are extracted through a double-branch network containing a GRB module and an SGB module, after fusion is conducted through an FB module, defect detection is completed through a PANet and a YOLO detection head with an attention module, and an optimized closed loop is formed by combining edge deployment testing and feedback adjustment. The system comprises modules such as a double-branch encoder and the like, and the performance is improved by adopting a lightweight network architecture and a reasoning optimization technology. On an edge computing platform, the time consumption of single-frame reasoning is less than or equal to 35ms, the defect mAP reaches more than 93%, the false alarm rate in a strong light environment is less than or equal to 4%, the method is suitable for automatic inspection of a large-scale photovoltaic power station, and the operation and maintenance cost can be reduced.

Description

Technical Field

[0001] This invention relates to the fields of photovoltaic power plant operation and maintenance and computer vision technology. Specifically, it relates to a method and system for intelligent detection of photovoltaic panel defects by integrating visible light and infrared images, which is particularly suitable for real-time and accurate automated inspection by an edge computing platform mounted on a drone. Background Technology

[0002] With the rapid development of the photovoltaic power generation industry, the scale of photovoltaic power plants is expanding daily, and the deployment environments are becoming increasingly complex (such as deserts, mountains, and rooftops). Traditional inspection methods relying on manual visual inspection or handheld equipment suffer from low efficiency, high costs, poor inspection consistency, and safety hazards, and can no longer meet the operation and maintenance needs of large-scale power plants. Therefore, automated inspection technology based on drone platforms has become the mainstream development direction in the industry.

[0003] Machine vision technology plays a central role in automated defect detection. Currently, it mainly relies on two types of image data: visible light images and infrared thermal imaging images. Visible light images can clearly present the surface condition of photovoltaic panels, such as cracks, dust accumulation, bird droppings, etc., with rich texture details, but they cannot detect internal defects in the cells (such as hot spots) or bypass diode failures. Infrared thermal imaging images, on the other hand, can effectively identify internal defects related to thermal characteristics, such as hot spots, multiple spots, streaks, and no loads, by capturing abnormal temperature distribution of the panel. However, its spatial resolution is low, it lacks texture information, and it is difficult to accurately classify and locate defects.

[0004] Existing technical solutions are mostly based on single image sources for detection, which has obvious limitations. Some methods attempt to fuse the detection results of two modalities at the decision layer, but fail to achieve deep fusion at the feature level, limiting the complementary advantages and making them susceptible to false detections. Other studies attempt feature fusion, but the network structure is complex and computationally intensive, making it difficult to implement real-time inference on edge devices with limited computing power, and thus failing to meet the stringent timeliness requirements of inspection operations.

[0005] Furthermore, the operating environment of photovoltaic power plants is complex and variable (such as strong midday sunlight, cloudy days, and dust storms), posing challenges to the generalization ability and robustness of a single model in different scenarios. Currently, there is a lack of an end-to-end detection solution that can operate in real time at the edge and adaptively optimize based on on-site detection feedback. This has become a key technical bottleneck restricting further improvement in the level of intelligent operation and maintenance of photovoltaic systems. Summary of the Invention

[0006] To address the problems of incomplete defect detection, poor model real-time performance, insufficient environmental adaptability, and lack of continuous optimization capabilities in the aforementioned background technologies, this invention provides a high-precision, real-time, multi-source image end-to-end defect detection method and system for photovoltaic panels. This invention achieves efficient complementary fusion of visible light and infrared images at the feature layer through innovative network structure design, and constructs a complete technical closed loop from cloud training to edge deployment, and then to continuous optimization based on feedback.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a high-precision, real-time, multi-source image end-to-end defect detection method for photovoltaic panels, comprising the following steps: Step S1: Extract the input infrared image features and visible light image features respectively using the infrared feature extraction backbone network based on GRB and SGB modules and the visible light feature extraction backbone network. Step S2: Input the extracted infrared image features and visible light image features into the feature fusion network based on the FB module to perform multi-scale feature fusion and obtain the fused multi-scale feature map; Step S3: Input the multi-scale feature map into the feature decoder based on the path aggregation network for feature enhancement and aggregation, and input the multi-scale feature pyramids obtained after aggregation into the corresponding YOLO detection heads for defect localization and classification; wherein, a lightweight attention module is set in front of the YOLO detection head; Step S4: Use the registered visible-infrared photovoltaic panel defect dataset to train the model containing the feature extraction backbone network, feature fusion network, feature decoder and detection head to obtain the trained detection model; Step S5: Deploy the trained detection model on an edge computing platform; perform defect detection under different photovoltaic scenarios and environmental conditions, collect the detection results and corresponding multi-source image data, form a scene performance dataset, and upload it to the cloud model optimization center; Step S6: Evaluate the scene performance dataset. When at least one of the following indicators—detection accuracy, false positive rate, or false negative rate—fails to reach a preset performance threshold in a specific scene, trigger the model optimization process. The model optimization process includes: S6.1 Analyze the scene data where the performance is substandard and determine the optimization strategy; if the performance degradation is caused by specific environmental interference, prioritize adjusting the preprocessing algorithm parameters or the detection head confidence threshold; if it is caused by new scene features that have not been fully learned, initiate model parameter updates. S6.2 Use the relevant data in the scenario performance dataset as supplementary training samples and send them back to the training end; use the currently deployed model as initialization, use the supplementary training samples to fine-tune all or part of the model parameters, or combine them with the original training set for incremental training; S6.3 evaluates the optimized model on the validation set. Once the performance is confirmed to meet the standards, a new model file is generated and updated to the edge computing platform, replacing the original detection model.

[0008] Secondly, the present invention provides a high-precision, real-time, multi-source image end-to-end defect detection system for photovoltaic panels to implement the above-mentioned method, comprising: The feature encoding module includes a parallel infrared feature extraction subnetwork and a visible light feature extraction subnetwork. Each subnetwork contains a GRB module and an SGB module, which are used to extract the infrared features and visible light features of the input image, respectively. The feature fusion module, which is built on the FB module and connected to the feature encoding module, is used to receive and fuse the infrared features and visible light features, and output a multi-scale fused feature map. The feature decoding and detection module includes a path aggregation network and a decoupled YOLO detection head. The input of the path aggregation network is connected to the feature fusion module for enhancing and aggregating multi-scale fused feature maps. The YOLO detection head is connected to the output of the path aggregation network for defect localization and classification of the aggregated features. A lightweight attention module is provided in front of the YOLO detection head. The model training module is used to train the overall model consisting of the feature encoding module, feature fusion module, and feature decoding and detection module using the registered visible-infrared photovoltaic panel defect dataset. The edge deployment and inference module is used to optimize and deploy the trained model to the edge computing platform, and to perform real-time defect detection tasks under different photovoltaic scenarios and environmental conditions. The data collection and performance monitoring module, deployed at the edge, is used to collect multi-source image data, detection results and environmental parameters during the inference process, and calculate real-time performance indicators to form a scene performance dataset. A cloud-based model optimization center, communicatively connected to the edge deployment and inference module and the data collection and performance monitoring module, is used to receive and store scene performance datasets; the cloud-based model optimization center includes: The performance evaluation unit is used to determine whether the performance in each scenario meets the standards. The optimization decision unit analyzes the reasons and generates optimization strategies when performance fails to meet the standards. These strategies include adjusting edge-side algorithm parameters or initiating model updates. The parameter update unit, when the strategy is model update, calls the model training module to fine-tune or incrementally train the latest model as the baseline using relevant data in the scene performance dataset. The model distribution unit distributes the optimized model or parameter configuration file to the edge deployment and inference module to complete the model update.

[0009] Compared with the prior art, the beneficial technical effects of the present invention are: Through the collaborative design of GRB-SGB-FB, a deep fusion of visible light texture details and infrared thermal anomaly information is achieved at the feature level, enabling simultaneous and accurate detection of surface defects (bird droppings, dust, damage) and internal defects (hot spots, multiple spots, etc.), overcoming the limitations of single-modal detection.

[0010] The network backbone adopts a lightweight design, and key modules balance performance and efficiency. Combined with edge-side model optimization technology (such as TensorRT quantization), the entire system can achieve high real-time performance of ≤35ms single-frame inference time on resource-constrained platforms such as Jetson Nano, meeting the high-speed inspection requirements of UAVs.

[0011] The innovative SGB edge enhancement module and FB fusion strategy improve the model's robustness to disturbances such as illumination changes and local occlusion. Especially in desert high-light environments, the combination with a reflection suppression preprocessing algorithm can effectively reduce false alarms.

[0012] The greatest advantage of this invention lies in constructing a closed-loop system of "edge perception - cloud optimization - dynamic update". The model is no longer a static entity, but can adaptively adjust and optimize through feedback loops based on new scenarios and problems encountered in actual inspections, significantly improving the practicality of long-term deployment and the stability of detection performance throughout its lifecycle.

[0013] From image input to defect classification and localization output, everything is done by a single model, without the need for complex post-processing or decision fusion steps. The process is simple and conducive to engineering and large-scale deployment. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall network structure of the multi-source image fusion detection model provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the internal structure of the GRB module in an embodiment of the present invention; Figure 3 This is a schematic diagram of the internal structure of the FB module in an embodiment of the present invention; Figure 4 This is a schematic diagram of the internal structure of the SGB module in an embodiment of the present invention; Figure 5A schematic diagram of the system workflow and architecture provided for embodiments of the present invention. Detailed Implementation

[0015] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0016] I. Specific Implementation Process of Defect Detection Methods 1. Dual-modal data acquisition and preprocessing Data acquisition equipment: Employs a Hikvision DS-2TD2617B-6 / PA dual-light camera module to simultaneously acquire visible light images (2 megapixels, 1920×1080 resolution) and infrared images (160×120 pixels, temperature sensitivity ≤0.05℃); paired with a BH1750 light sensor to collect ambient light intensity in real time (measurement range 0-65535lx, accuracy ±20%). Figure 5 As shown.

[0017] Data preprocessing: Illumination data collected by a light sensor is used to calculate the reflectance coefficient based on an adaptive dynamic threshold segmentation reflectance suppression algorithm. The formula "corrected pixel value = original pixel value × (1 - reflectance coefficient) + infrared grayscale value × reflectance coefficient" is used to correct the visible light image pixel values, eliminating strong light reflection interference. Figure 5 The core processing flow of the "preprocessing module" in China.

[0018] Dataset Construction: Dual-modal images of desert photovoltaic power stations, rooftop photovoltaic power stations, and mountain photovoltaic power stations were collected. Spatial registration was achieved using the SIFT feature matching algorithm, and the dataset was divided into training, validation, and test sets in a 7:2:1 ratio. The dataset contains 12,000 pairs, covering seven types of defects: bird droppings, dust, panel damage (visible light annotation), hot spots, multiple spots, streaks, and no load (infrared annotation). The annotation format is VOC, and the LabelImg 1.8.6 tool was used for annotation. Specific scenes are defined by environmental parameter labels, such as light intensity > 80,000 lx and weather = sandstorm, light intensity <10000lx and weather = cloudy, temperature> Temperatures of 35°C and humidity of <20%, etc.

[0019] 2. Implementation of dual-branch feature encoding Network Architecture: Two parallel and independent lightweight MobileNetV3 convolutional networks are constructed, serving as infrared feature extraction sub-networks and visible light feature extraction sub-networks, respectively. The network depth configuration is "1×1 convolutional layer → 3×3 convolutional layer → ReLU activation function → GRB module → 3×3 convolutional layer → GRB module", as shown below. Figure 1 As shown.

[0020] Core module configuration: Each sub-network embeds two GRB modules, and each GRB module integrates one SGB module; for example... Figure 2 As shown, the main path in the GRB module uses three layers of densely connected convolutions (output channels are 32, 64, and 128 respectively), while the residual path passes through the SGB module; as... Figure 4 As shown, the SGB module uses 3×3 horizontal and vertical Scharr operators to extract gradient features, and the two feature paths are fused element-wise. The input image is uniformly adjusted to 640×640 resolution, and the output is a 256-channel deep feature map, as shown. Figure 5 The output logic of the "feature encoding module".

[0021] 3. Implementation of multi-scale feature fusion Fusion module configuration: A multi-scale feature fusion network is composed of two cascaded FB modules; for example... Figure 3 As shown, the FB module includes a feature splicing unit, a 1×1 initial fusion convolutional layer (output channels 128), a semantic fusion branch (two consecutive 3×3 convolutional layers), an edge fusion branch (integrated SGB module), and an additive fusion unit.

[0022] Fusion Process: The 256-channel features output from the dual branches are merged by channels through a concatenation unit, and then reduced to 128 channels by a 1×1 convolutional layer to complete the initial fusion. Next, contextual information is fused through semantic branches, and detailed features are enhanced through edge branches. Finally, a 64-channel multi-scale fused feature map is output through an addition unit, as shown below. Figure 1 As shown, and in line with Figure 5 The core function of the "feature fusion module" in China.

[0023] 4. Feature Decoding and Defect Detection Implementation Feature aggregation: The 64-channel fused feature map is input into the PANet path aggregation network. High-level semantic information is passed through a top-down upsampling path, and low-level spatial details are fused using a bottom-up downsampling path, generating feature pyramids at three scales: 64, 128, and 256 channels. Figure 1 As shown.

[0024] Defect Detection: The feature pyramid first inputs a lightweight SE attention module, which generates channel weights through global average pooling and fully connected layers to enhance the feature response of small target defects; then it inputs a YOLOv8 detection head (decoupled structure). The classification branch uses the cross-entropy loss function to predict the probability of the defect category, and the regression branch uses the CIoU loss function to predict the bounding box coordinates and confidence, thus completing defect localization and classification. Figure 5 The output flow of the "Feature Decoding and Detection Module" in the middle.

[0025] 5. Model Training and Closed-Loop Optimization Implementation Training parameter configuration: Training was conducted using PyTorch 1.13.0 + cu117 framework, with a batch size of 16. The AdamW optimizer was used (initial learning rate 1e-3), and the learning rate was dynamically adjusted using a cosine annealing strategy. Training lasted for 300 epochs. Data augmentation was performed during training: random rotation (0-10 degrees), hue jitter (2%), saturation jitter (50%), brightness jitter (30%), horizontal flip (probability 20%), image translation (±5%), and multi-scale scaling (0.8-1.2x). Training was stopped when the validation set mAP reached 92%.

[0026] Edge Deployment: After the trained model is optimized and quantized with INT8 using the TensorRT 8.0 toolchain, it is deployed on the NVIDIA Jetson Nano edge computing platform (4GB RAM, Ubuntu 20.04 LTS operating system). The model communicates with dual-light cameras via a USB 3.0 interface and receives light sensor data via an I2C interface, such as... Figure 5 The deployment logic and hardware connection relationships of the "edge computing platform" in China.

[0027] Closed-loop optimization: Tested under conditions such as strong midday sunlight, sandstorms, and cloudy days at a desert power station, by... Figure 5 The "Data Collection and Monitoring Module" collects metrics such as detection accuracy (mAP), false alarm rate, and single-frame inference time. Specific scenarios are defined by environmental parameter labels (e.g., light intensity > 80000 lx and weather = dust storm, light intensity < 5000 lx and weather = nighttime, wind speed > 5 m / s and weather = dust storm, etc.). For each scenario, corresponding performance thresholds are preset (e.g., mAP ≥ 88% and false alarm rate ≤ 6% for strong light and dust storm scenarios, mAP ≥ 85% and false negative rate ≤ 4% for nighttime scenarios). If the performance of a certain scenario fails to meet the standards, the module analyzes the bottleneck in locating erroneous samples (e.g., deviation in correcting reflective areas, false negatives of small targets), generates parameter adjustment instructions (e.g., adjusting the dynamic threshold of reflectivity, increasing or decreasing the number of GRB / FB modules). The model parameters are adjusted according to the instructions, and the model is retrained until the performance meets the standards. Figure 5The link from "cloud model optimization center → distribution of updated model / parameters" pushes the optimized model to the edge computing platform to replace the original model, ensuring that the single-frame inference time is still ≤35ms.

[0028] In this incremental training, an experience replay strategy is adopted. In one optional embodiment, the original training set and the newly collected scene performance data samples are mixed in a 1:1 ratio. Core samples covering all defect categories and evenly distributed across all scenes are selected from the original training set and combined with the supplementary samples from the new scenes to form a new training set. The model is fine-tuned for 50 epochs using 80% of the learning rate of the original training hyperparameters to ensure that the model adapts to the features of the new scene while retaining its original generalization ability.

[0029] II. Hardware and Software Integration Implementation of the Defect Detection System 1. Hardware System Integration Core hardware components: The edge computing platform uses an NVIDIA Jetson Nano (4GB RAM, supports CUDA 11.7); the dual-light camera assembly is a Hikvision DS-2TD2617B-6 / PA, supporting simultaneous 30fps frame rate acquisition; the light sensor is a BH1750; the power supply module uses a 12V / 5A DC power supply, providing unified power for the edge computing platform and peripherals, such as... Figure 5 The hardware components and signal flow are shown.

[0030] Hardware connection method: The dual-light camera connects to the Jetson Nano via a USB 3.0 interface to ensure high-speed image data transmission; the light sensor communicates with the edge computing platform via an I2C interface to transmit light intensity data; all hardware is fixed to a drone inspection bracket or a ground inspection robot to ensure a stable acquisition angle (3-5 meters vertical distance from the photovoltaic panel), such as... Figure 5 The connection relationship between the hardware and the preprocessing module and feature encoding module.

[0031] 2. Software System Configuration and Integration Basic software environment: The operating system is Ubuntu 20.04 LTS for Jetson, the deep learning framework is PyTorch 1.13.0+cu117, the inference optimization tool is TensorRT 8.0.1, the image acquisition and processing library is OpenCV 4.5.5, and the programming language is Python 3.8.10.

[0032] Module integration logic: Connecting various software modules using Python scripts to achieve... Figure 5The system automates the entire process from "image acquisition → preprocessing → feature encoding → feature fusion → decoding detection → data collection → performance evaluation → optimization and update". After the system starts, it collects environmental parameters such as light intensity, weather type, temperature, and humidity in real time, and automatically adds environmental parameter labels to the detection data to define specific scenarios (such as light intensity > 80000 lx and weather = dust, temperature > 40℃ and weather = sunny, etc.). At the same time, it displays the detection results (defect category, location, confidence level) and the corresponding performance indicators of the scenario. It supports manually setting the performance thresholds for each scenario and exporting the detection report (including defect statistics, environmental parameter labels, and performance data).

[0033] 3. System Operation and Testing Verification Operational Process: After system startup, dual-light cameras simultaneously acquire visible light and infrared images of the photovoltaic panel, while a light sensor simultaneously acquires ambient light data. The preprocessing module corrects glare issues in the visible light images, which are then sequentially passed to the feature encoding, fusion, and decoding detection modules to output defect detection results. The data collection and monitoring module simultaneously records the detection results, original images, and environmental parameters, uploading them to the cloud-based model optimization center, fully adhering to... Figure 5 The diagram shows the system workflow and architecture.

[0034] Test and verification scenarios: Three typical specific scenarios were defined by environmental parameter labels: "Light intensity > 80000 lx + Weather = Dust storm + Temperature 35-45℃" (desert strong light and dust storm scenario), "Light intensity 5000-20000 lx + Weather = Cloudy + Humidity 60-80%" (rooftop cloudy scenario), and "Light intensity < 10000 lx + Weather = Night + Temperature 0-15℃" (mountainous night scenario). After testing, the system achieved an average detection accuracy (mAP) of 93% for various defects in each scenario, a false alarm rate of 4% in the desert strong light and dust storm scenario, and a single frame inference time of 32ms, meeting the performance indicators set by this invention.

[0035] III. Verification of the Effects of the Examples This embodiment verifies the technical effect through comparative experiments, and the results are as follows: Detection accuracy: Compared with single-modal detection models (visible light YOLOv8 or infrared YOLOv8), the dual-modal fusion scheme of this invention improves mAP by more than 9%, and the recall rate of small target defects (such as fine scratches and small bird droppings) is improved by 15%.

[0036] Real-time performance: After TensorRT quantization optimization, the inference time per frame is ≤35ms, which is more than 40% faster than the traditional multi-source fusion model (inference time is about 60ms), meeting the real-time requirements of UAV inspection.

[0037] Environmental adaptability: In complex and specific scenarios such as "light intensity > 80000 lx and weather = sandstorm" and "light intensity < 5000 lx and weather = nighttime", the false alarm rate is reduced by 30% compared with the scheme without reflection suppression and closed-loop optimization, and the detection robustness is significantly improved.

[0038] Intelligence level: It realizes end-to-end closed-loop optimization, and can complete model parameter adjustment and performance iteration without manual intervention. It is more than 50 times more efficient than manual inspection and significantly reduces operation and maintenance costs.

[0039] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A high-precision, real-time, multi-source image end-to-end defect detection method for photovoltaic panels, characterized in that, Includes the following steps: Step S1: Extract the input infrared image features and visible light image features respectively using the infrared feature extraction backbone network based on GRB and SGB modules and the visible light feature extraction backbone network. Step S2: Input the extracted infrared image features and visible light image features into the feature fusion network based on the FB module to perform multi-scale feature fusion and obtain the fused multi-scale feature map; Step S3: Input the multi-scale feature map into the feature decoder based on the path aggregation network for feature enhancement and aggregation, and input the multi-scale feature pyramids obtained after aggregation into the corresponding YOLO detection heads for defect localization and classification; wherein, a lightweight attention module is set in front of the YOLO detection head; Step S4: Use the registered visible-infrared photovoltaic panel defect dataset to train the model containing the feature extraction backbone network, feature fusion network, feature decoder and detection head to obtain the trained detection model; Step S5: Deploy the trained detection model on an edge computing platform; perform defect detection under different photovoltaic scenarios and environmental conditions, collect the detection results and corresponding multi-source image data, form a scene performance dataset, and upload it to the cloud model optimization center; Step S6: Evaluate the scene performance dataset. When at least one of the following indicators—detection accuracy, false positive rate, or false negative rate—fails to reach a preset performance threshold in a specific scene, trigger the model optimization process. The model optimization process includes: S6.1 Analyze the scene data where the performance is substandard and determine the optimization strategy; if the performance degradation is caused by specific environmental interference, prioritize adjusting the preprocessing algorithm parameters or the detection head confidence threshold; if it is caused by new scene features that have not been fully learned, initiate model parameter updates. S6.2 Use the relevant data in the scenario performance dataset as supplementary training samples and send them back to the training end; use the currently deployed model as initialization, use the supplementary training samples to fine-tune all or part of the model parameters, or combine them with the original training set for incremental training; S6.3 evaluates the optimized model on the validation set. Once the performance is confirmed to meet the standards, a new model file is generated and updated to the edge computing platform, replacing the original detection model.

2. The high-precision real-time multi-source image end-to-end defect detection method for photovoltaic panels according to claim 1, characterized in that, In step S1, the GRB module adopts a dense connection structure and integrates the SGB module into its residual path; the SGB module uses horizontal and vertical Scharr operators to convolve the input feature map respectively to extract and fuse the edge gradient features in the horizontal and vertical directions.

3. The high-precision real-time multi-source image end-to-end defect detection method for photovoltaic panels according to claim 1, characterized in that, In step S2, the execution process of the FB module includes: stitching and initially fusing the infrared and visible light feature maps of the same level, and then processing them in parallel through a semantic fusion branch and an edge fusion branch, wherein the edge fusion branch integrates an SGB module, and finally adding the output features of the two branches to obtain the fused features of that level.

4. The high-precision real-time multi-source image end-to-end defect detection method for photovoltaic panels according to claim 1, characterized in that, In step S3, the lightweight attention module is either an SE attention module or a CBAM module; the YOLO detection head adopts a decoupled structure with independent classification and regression branches.

5. The high-precision real-time multi-source image end-to-end defect detection method for photovoltaic panels according to claim 1, characterized in that, In step S4, the data augmentation methods used during model training include at least one of the following: random rotation, color perturbation, multi-scale scaling, horizontal random flipping, and image translation.

6. The high-precision real-time multi-source image end-to-end defect detection method for photovoltaic panels according to claim 1, characterized in that, In step S5, when performing defect detection, the input visible light image is preprocessed using a reflection suppression algorithm. The algorithm identifies reflective areas based on adaptive dynamic threshold segmentation and corrects the pixel value according to the formula "corrected pixel value = original pixel value × (1 - reflection coefficient) + infrared gray value × reflection coefficient". The reflection coefficient is dynamically calculated from the light sensor data. In step S6.1, adjusting the preprocessing algorithm parameters includes dynamically adjusting the calculation parameters or mapping curve of the reflection coefficient.

7. The high-precision real-time multi-source image end-to-end defect detection method for photovoltaic panels according to claim 1, characterized in that, The preset performance thresholds include at least one of the following: average accuracy threshold, false alarm rate threshold, and false negative rate threshold.

8. A high-precision, real-time, multi-source image end-to-end defect detection system for photovoltaic panels, characterized in that, The system for implementing the method as described in any one of claims 1-7 includes: The feature encoding module includes a parallel infrared feature extraction subnetwork and a visible light feature extraction subnetwork. Each subnetwork contains a GRB module and an SGB module, which are used to extract the infrared features and visible light features of the input image, respectively. The feature fusion module, which is built on the FB module and connected to the feature encoding module, is used to receive and fuse the infrared features and visible light features, and output a multi-scale fused feature map. The feature decoding and detection module includes a path aggregation network and a decoupled YOLO detection head. The input of the path aggregation network is connected to the feature fusion module for enhancing and aggregating multi-scale fused feature maps. The YOLO detection head is connected to the output of the path aggregation network for defect localization and classification of the aggregated features. A lightweight attention module is provided in front of the YOLO detection head. The model training module is used to train the overall model consisting of the feature encoding module, feature fusion module, and feature decoding and detection module using the registered visible-infrared photovoltaic panel defect dataset. The edge deployment and inference module is used to optimize and deploy the trained model to the edge computing platform, and to perform real-time defect detection tasks under different photovoltaic scenarios and environmental conditions. The data collection and performance monitoring module, deployed at the edge, is used to collect multi-source image data, detection results and environmental parameters during the inference process, and calculate real-time performance indicators to form a scene performance dataset. A cloud-based model optimization center, communicatively connected to the edge deployment and inference module and the data collection and performance monitoring module, is used to receive and store scene performance datasets; the cloud-based model optimization center includes: The performance evaluation unit is used to determine whether the performance in each scenario meets the standards. The optimization decision unit analyzes the reasons and generates optimization strategies when performance fails to meet the standards. These strategies include adjusting edge-side algorithm parameters or initiating model updates. The parameter update unit, when the strategy is model update, calls the model training module to fine-tune or incrementally train the latest model as the baseline using relevant data in the scene performance dataset. The model distribution unit distributes the optimized model or parameter configuration file to the edge deployment and inference module to complete the model update.

9. The high-precision real-time multi-source image end-to-end defect detection system for photovoltaic panels according to claim 8, characterized in that, The GRB module in the feature encoding module has a main path containing dense connections and a residual path integrating the SGB module; the SGB module has built-in horizontal and vertical Scharr operator convolution kernels.

10. The high-precision real-time multi-source image end-to-end defect detection system for photovoltaic panels according to claim 8, characterized in that, The FB module in the feature fusion module includes: a splicing and preliminary fusion unit, and a semantic fusion branch and an edge fusion branch connected in parallel thereafter, wherein the edge fusion branch includes an SGB module.

11. The high-precision real-time multi-source image end-to-end defect detection system for photovoltaic panels according to claim 8, characterized in that, The path aggregation network in the feature decoding and detection module is used to achieve top-down semantic transmission and bottom-up detail fusion; the YOLO detection head contains independent classification convolutional branches and regression convolutional branches.

12. The high-precision real-time multi-source image end-to-end defect detection system for photovoltaic panels according to claim 8, characterized in that, The edge deployment and inference module also includes a preprocessing unit, which integrates a reflection suppression algorithm. If the optimization strategy generated by the optimization decision unit is to adjust the edge-side algorithm parameters, the adjusted reflection coefficient calculation parameters or detection confidence threshold are sent to the preprocessing unit or detection module for hot updating through the model distribution unit.

Images