A photovoltaic panel contamination detection method, system, computer device and storage medium

Abstract

The application provides a photovoltaic panel pollution detection method, system, computer equipment and storage medium, and belongs to the technical field of photovoltaic module pollution detection. The method comprises the following steps: in the training stage, a UAV and a ground feature extraction network are constructed, and through comparison learning of a positive sample pair (the same pollution area) and a negative sample pair (different pollution areas), two models are forced to output similar feature vectors for the same pollution, so that cross-view feature space alignment is realized. In the application stage, the Dropout layer of the ground model is activated, the uncertainty of each pixel is estimated through multiple random forward propagation, the fusion weight of the UAV and the ground view is dynamically adjusted according to the uncertainty during fusion, and the higher the uncertainty, the lower the view weight; finally, the proportion of pollution pixels is counted, the pollution type weight is combined, and a cleanliness index is output. The application significantly improves the detection accuracy and model universality under complex illumination and seasonal changes, and provides a reliable quantitative evaluation basis for photovoltaic cleaning.

Description

Technical Field

[0001] This invention belongs to the field of photovoltaic module pollution detection technology, specifically relating to a photovoltaic panel pollution detection method, system, computer equipment, and storage medium. Background Technology

[0002] Current photovoltaic panel contamination detection technologies typically combine drone aerial photography with ground inspection. Drones acquire macroscopic images of the photovoltaic array through high-altitude aerial photography, using target detection networks (such as YOLOv4) to identify panel locations and significantly contaminated areas. Ground equipment acquires high-resolution images of the panel surface through close-up photography, using semantic segmentation networks (such as SegFormer) to perform pixel-level fine segmentation of the contaminated areas. Finally, the detection results from both perspectives are aligned and weighted through spatial mapping. This type of method has been applied in actual power plants and can, to a certain extent, take into account both macroscopic distribution and microscopic details.

[0003] However, in existing technologies, UAV detection models and ground-based detection models are trained independently, and their mathematical representations (feature vectors) of contaminated areas are completely independent. This leads to significant differences in the feature representations of the same contaminated area in the two models, causing information to cancel each other out rather than reinforce each other during weighted fusion, thus limiting the fusion accuracy. Furthermore, the ambient light conditions at photovoltaic power plants vary significantly with seasons and weather. Under conditions of cloudy / rainy spring, strong sunlight in summer, oblique sunlight in autumn, and low-angle sunlight in winter, the imaging characteristics of the same contaminated area differ greatly from different viewpoints. The two independently trained models, each adapting to its own training data distribution, cannot maintain stable cross-viewpoint consistency under different lighting conditions, resulting in poor model versatility and significant fluctuations in detection accuracy across different seasons or weather conditions, severely impacting the reliability of year-round cleaning decisions. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method, system, computer equipment, and storage medium for detecting contamination in photovoltaic panels.

[0005] To achieve the above objectives, the present invention provides a method for detecting contamination in photovoltaic panels, comprising: Aerial images of the photovoltaic modules to be cleaned by drones and images taken by unmanned vehicles are collected, and the photovoltaic modules to be cleaned are divided into multiple work units. Based on the contaminated area in the preset unmanned vehicle image, a first drone aerial image belonging to the same work unit as the preset unmanned vehicle image and a second drone aerial image belonging to a different work unit are matched.

[0006] The preset images captured by the unmanned vehicle, the first aerial image captured by the first UAV, and the second aerial image captured by the second UAV are respectively input into the semantic segmentation network, the first target detection network, and the second target detection network, and the outputs the unmanned vehicle view pollution mask, the first UAV view pollution candidate box, and the second UAV view pollution candidate box; the unmanned vehicle view pollution mask represents the pixel-level pollution segmentation result in the unmanned vehicle image.

[0007] The contamination mask and contamination candidate boxes are mapped to a unified spatial coordinate system. The spatial similarity of the unmanned vehicle's view contamination mask with the first UAV's view contamination candidate boxes and the second UAV's view contamination candidate boxes is calculated to obtain a first similarity and a second similarity. Based on the first similarity and the second similarity, a joint loss function is constructed. The parameters of the semantic segmentation network, the first object detection network, and the second object detection network are simultaneously iteratively optimized through backpropagation until the first similarity of the joint loss function is maximized and the second similarity is minimized, thus obtaining the trained semantic segmentation network and the first object detection network.

[0008] By fusing the contamination mask from the perspective of the unmanned vehicle output by the trained semantic segmentation network and the contamination candidate box from the perspective of the drone output by the first target detection network, the contamination detection result of the photovoltaic panel to be cleaned is obtained.

[0009] Preferably, the semantic segmentation network adopts the encoder structure of the SegFormer semantic segmentation network to output the unmanned vehicle view pollution mask; the first object detection network and the second object detection network both adopt the backbone network structure of the YOLOv4 object detection network to output the first UAV view pollution candidate box and the second UAV view pollution candidate box; the second object detection network is only used as a negative sample generator during the training phase and does not participate in inference during the application phase.

[0010] Preferably, both the first similarity and the second similarity are cosine similarities, calculated using the following formula: ; Where A is a one-dimensional vector obtained by flattening the unmanned vehicle's view pollution mask, and B is a one-dimensional vector obtained by converting the drone's view pollution candidate box into a binary mask and flattening it.

[0011] Preferably, the joint loss function is: ; in, For the perspective of autonomous vehicles, a contamination mask. Contaminated candidate boxes from the first-person drone perspective. To contaminate candidate boxes from the second drone's perspective, sim( ) is the similarity function, and τ is the temperature coefficient.

[0012] Preferably, the semantic segmentation network also outputs an uncertainty measure for each pixel in the unmanned vehicle's view pollution mask; the step of fusing the unmanned vehicle's view pollution mask output by the trained semantic segmentation network and the drone's view pollution candidate boxes output by the first target detection network to obtain the pollution detection result of the photovoltaic panel to be cleaned specifically includes: transforming the unmanned vehicle's view pollution mask and uncertainty measure to the local coordinate system of the photovoltaic panel through spatial mapping; mapping the drone's view pollution candidate boxes to the local coordinate system of the same panel; dynamically adjusting the fusion weight of the drone's view pollution candidate boxes according to the uncertainty measure to generate a fused pollution distribution map; calculating the pollution coverage ratio according to the fused pollution distribution map, and outputting the pollution detection result of the photovoltaic panel to be cleaned.

[0013] Preferably, the pollution coverage ratio is calculated based on the fused pollution distribution map, and the pollution detection result of the photovoltaic panel to be cleaned is output. Specifically, this includes: according to the fused pollution distribution map, counting the number of polluted pixels in the panel, obtaining the total number of effective pixels of the panel, and dividing the number of polluted pixels by the total number of effective pixels to obtain the pollution coverage ratio. Based on the pollution mask from the perspective of the autonomous vehicle, the number of pixels for each type of pollution in the panel is obtained. The number of pixels for each type of pollution is divided by the total number of effective pixels in the panel and then multiplied by the preset weight corresponding to that type of pollution. The calculation results of all categories are then added together to obtain the weighted pollution intensity. The weighted pollution intensity is subtracted from the value 1 to obtain the photovoltaic surface cleanliness index CISS. The fused pollution distribution map, pollution coverage ratio, and photovoltaic surface cleanliness index (CISS) are output as the pollution detection and cleanliness assessment results of the target panel.

[0014] Preferably, the calculation of the uncertainty measure specifically includes: obtaining N classification results for each pixel of the image captured by the unmanned vehicle, and calculating the variance of the N classification results as the uncertainty measure of that pixel.

[0015] The present invention also provides a photovoltaic panel contamination detection system, comprising: The data acquisition module is used to acquire aerial images of the photovoltaic modules to be cleaned by drones and images taken by unmanned vehicles, and to divide the photovoltaic modules to be cleaned into multiple work units; based on the polluted area in the preset unmanned vehicle image, it matches a first drone aerial image belonging to the same work unit as the preset unmanned vehicle image, and a second drone aerial image belonging to a different work unit from the preset unmanned vehicle image.

[0016] The training module is used to input preset images captured by an unmanned vehicle, aerial images captured by a first UAV, and aerial images captured by a second UAV into a semantic segmentation network, a first object detection network, and a second object detection network, respectively, and output an unmanned vehicle view pollution mask, a first UAV view pollution candidate box, and a second UAV view pollution candidate box. The unmanned vehicle view pollution mask represents the pixel-level pollution segmentation result in the unmanned vehicle image. The pollution mask and pollution candidate boxes are mapped to a unified spatial coordinate system, and the spatial similarity of the unmanned vehicle view pollution mask with the first UAV view pollution candidate box and the second UAV view pollution candidate box is calculated to obtain a first similarity and a second similarity. Based on the first similarity and the second similarity, a joint loss function is constructed. The parameters of the semantic segmentation network, the first object detection network, and the second object detection network are simultaneously optimized iteratively through backpropagation until the first similarity of the joint loss function is maximized and the second similarity is minimized, thus obtaining the trained semantic segmentation network and the first object detection network.

[0017] The detection module is used to fuse the pollution mask from the perspective of the unmanned vehicle output by the trained semantic segmentation network and the pollution candidate box from the perspective of the drone output by the first target detection network to obtain the pollution detection result of the photovoltaic panel to be cleaned.

[0018] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the photovoltaic panel contamination detection method.

[0019] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any of the steps in the photovoltaic panel contamination detection method.

[0020] The photovoltaic panel pollution detection method provided by this invention has the following beneficial effects: This invention uses three models for joint training and introduces contrastive loss. Its core advantage lies in forcibly aligning the feature representations of the same polluted area by the UAV and ground models. This invention constructs positive sample pairs (same polluted area) and negative sample pairs (different polluted areas), and uses the contrastive loss function to force the UAV model and ground model to output similar feature vectors for the same pollution and different feature vectors for different pollutions, thereby achieving alignment of the feature space across perspectives. This mechanism solves the problem of inconsistent outputs between the two models, allowing the information from the two perspectives to enhance each other rather than cancel each other out during fusion, significantly improving fusion accuracy. Simultaneously, because the aligned features are more robust to environmental changes such as illumination and seasons, the consistency of model detection under different weather conditions is guaranteed, effectively solving the shortcomings of existing technologies such as poor universality and fluctuations in detection accuracy across seasons. Attached Figure Description

[0021] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a photovoltaic panel contamination detection method according to an embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0024] This invention proposes a photovoltaic panel contamination detection and cleanliness assessment method based on air-ground multi-view collaboration, applicable to heterogeneous unmanned systems composed of UAVs and unmanned vehicles. The method uses array-level macroscopic images acquired by UAVs and panel-level close-up images acquired by unmanned vehicles as input. Through target detection, pixel-level semantic segmentation, air-ground spatial alignment, and multi-view fusion, it ultimately outputs a contamination distribution map and a uniformly quantified cleanliness index for each photovoltaic panel.

[0025] Based on this, the present invention provides a photovoltaic panel contamination detection method, specifically a photovoltaic panel contamination detection method based on contrastive learning and uncertainty perception, such as... Figure 1 As shown, it includes the following steps: S1. Synchronous acquisition of multi-source aerial and ground images and establishment of data association. The drones conduct aerial inspections of the photovoltaic arrays along a preset inspection route, acquiring high-altitude images covering the entire row or multiple rows of photovoltaic modules; the unmanned vehicles travel along the module arrangement direction within the photovoltaic array channel, taking close-up photos of the corresponding panel surfaces to acquire high-resolution close-up images.

[0026] To ensure that data from different platforms can be correctly matched in subsequent processing, this invention performs unified time calibration on the image acquisition process of UAVs and unmanned vehicles and records their respective pose information. By establishing the correlation between UAV images, ground images, and specific photovoltaic panels through timestamps and pose information, it provides basic data support for subsequent air-to-ground spatial alignment.

[0027] During the synchronous acquisition of multi-source images from both air and ground, the image sequence acquired by the UAV (UAV aerial images) is represented as follows: ; in, This represents the image of the drone in the i-th frame. Indicates the corresponding collection timestamp. The pose transformation matrix of the UAV relative to the power station reference coordinate system at that moment is represented by u, where u is the UAV's viewpoint and g is the ground unmanned vehicle's viewpoint. The i-th frame or j-th frame represents the frame index in the image sequence.

[0028] Similarly, the sequence of close-up images collected by the autonomous vehicle (images taken by the autonomous vehicle) is represented as: ; in, This represents the ground image of the j-th frame. Indicates the corresponding collection timestamp. This represents the pose transformation matrix of the unmanned vehicle relative to the power station's reference coordinate system at that moment.

[0029] To establish the correspondence between UAV images and ground images, this invention uses time synchronization constraints as the initial association criterion to construct a set of air-to-ground image pairs: ; in, The threshold for allowable time synchronization error is defined. Based on this, the spatial consistency of candidate image pairs is further filtered using the pose information of the two platforms. Let the spatial position of the photovoltaic panel in the power station reference coordinate system be... When both the UAV's view frustum and the ground camera's view frustum cover the panel area, the image pair is retained as the effective air-ground image observation pair for the panel.

[0030] By combining the aforementioned temporal consistency constraints and spatial observability constraints, a multi-source image association set for the same photovoltaic panel is established, providing a reliable data input foundation for subsequent panel-level detection, segmentation, and fusion.

[0031] S2. Joint detection of photovoltaic panels and significantly polluted areas from a drone perspective based on YOLOv4. After completing the aerial and ground image acquisition and panel-level data association in S1, this invention uses the aerial images acquired by the UAV as the array-level global perception input to automatically locate the photovoltaic panel units and simultaneously screen out significantly polluted areas with obvious appearance differences, providing spatial constraints for subsequent panel-level trimming and fine analysis.

[0032] Specifically, for each frame of drone image obtained in S1 The image is input into a pre-trained YOLOv4 object detection network. The network takes the entire image as input and performs regression and classification of photovoltaic panel targets through a multi-scale feature pyramid structure.

[0033] The detection results of the YOLOv4 network on this image are represented as a set of bounding boxes: ; in, In the image The number of photovoltaic panels detected.

[0034] For any detected photovoltaic panel target, its bounding box is uniformly represented as: ; in, This indicates the center position of the panel area in the UAV image coordinate system. and These represent the width and height of the panel area in the image, respectively; i refers to the i-th frame, and k refers to the k-th photovoltaic panel in a certain frame of the drone image.

[0035] Based on the above test results, a spatial index structure of all photovoltaic panels in the array can be established from the perspective of the drone.

[0036] While completing panel localization, this invention introduces a significant contamination target category into the category output branch of YOLOv4, enabling the network to output candidate boxes of large contamination areas with obvious appearance differences in the same forward inference process.

[0037] images of drones The detection results for significantly polluted areas are also represented as a set of candidate boxes: ; in, This indicates the number of candidate boxes with significant drone view contamination detected in the image.

[0038] Any significantly contaminated candidate box is represented as: ; Where i refers to the i-th frame, and m refers to the m-th salient drone view contamination candidate box in a certain frame of drone image.

[0039] The aforementioned contamination candidate boxes are only used to help describe the coarse spatial distribution of contamination in the array and are not considered as the final fine-grained contamination segmentation result. To associate significant contamination candidate information with specific photovoltaic panel areas, this invention further calculates the spatial overlap between each contamination candidate box and the panel detection box.

[0040] For a certain panel frame With a certain pollution candidate box The overlap ratio between the two is defined as: ; When the overlap ratio exceeds a preset threshold, the candidate contamination box from the drone's perspective is considered to belong to a significantly contaminated area of ​​that panel region. Therefore, a corresponding set of significantly contaminated candidate boxes from the drone's perspective can be constructed for each panel region.

[0041] ; in, This is the threshold for determining overlap.

[0042] Through the above process, the present invention simultaneously obtains, from the perspective of a drone: firstly, the precise spatial position of each photovoltaic panel in the image. The first is the direct input for panel region clipping and geometric normalization in subsequent S3; the second is the set of significant UAV-view contamination candidate boxes corresponding to each photovoltaic panel. As a subsequent step in S6, this serves as the basis for constructing contamination confidence information from the perspective of drones. Spatial basis.

[0043] This step simultaneously completes panel localization and coarse screening of significant contamination through a single forward detection process. While ensuring array-level real-time performance, it provides reliable spatial constraints and complementary information sources for subsequent fine semantic segmentation based on a ground-view perspective.

[0044] S3, Panel Area Cropping and Geometric Normalization from Drone View In S2, drone images have already been obtained. The detection frames corresponding to each photovoltaic panel in the middle. .

[0045] This step uses the detection result as a spatial constraint to perform panel-level cropping on the UAV image and establish a unified local panel representation. Specifically, for UAV images... Any panel detection box in:

[0046] ; By cropping the corresponding rectangular region from the original image, we obtain the drone-view sub-image of this panel: ; Where e represents the e-th sub-image.

[0047] Because changes in the drone's flight altitude and attitude can lead to differences in scale and shape between different panel sub-images, this invention further performs geometric normalization on the cropped panel sub-images to ensure a unified spatial representation for different panels in subsequent fusion processing. Through... Applying a two-dimensional affine transformation to map it to a unified local coordinate plane of the panel yields a normalized UAV panel image:

[0048] ; in, This represents the projection mapping from the UAV image coordinate system to the local panel coordinate system. This represents the geometric transformation operator from the drone's viewpoint to the local coordinate system of the panel.

[0049] The output of this step This serves as the basis for subsequent air-to-ground spatial alignment in S5 and the construction of drone-perspective contamination confidence in S6.

[0050] S4. SegFormer-based semantic segmentation of near-field ground contamination For the close-up ground images obtained in S1 The input is fed into the trained SegFormer semantic segmentation network to perform pixel-level classification of contaminated areas on the panel surface.

[0051] For any pixel position p The network outputs the probability distribution of which the pixel belongs to each category: ; in, , The categories indicate the type: category 0 represents a clean area, and the other categories represent different types of contamination.

[0052] Based on the maximum a posteriori principle, the pollution mask for the autonomous vehicle's view from the ground perspective is obtained: Where g represents the driverless vehicle's perspective.

[0053] Meanwhile, to provide continuous confidence information for subsequent multi-view fusion, this invention defines the comprehensive probability that a pixel belongs to a contaminated area under the ground view as: .

[0054] This step outputs a segmentation map in the ground image coordinate system. and the corresponding pollution probability map .

[0055] This invention constructs a contrastive learning framework during the training phase to achieve feature space alignment between the UAV perspective and the ground perspective.

[0056] First, based on the contaminated areas in a preset unmanned vehicle (UAV) image, a first UAV aerial image belonging to the same work unit as the preset UAV image and a second UAV aerial image belonging to a different work unit are matched. The preset UAV image, the first UAV aerial image, and the second UAV aerial image are then input into a semantic segmentation network, a first object detection network, and a second object detection network, respectively, outputting a UAV-view contamination mask, a first UAV-view contamination candidate box, and a second UAV-view contamination candidate box. The contamination mask and contamination candidate boxes are mapped to a unified spatial coordinate system, and spatial similarity is calculated between the UAV-view contamination mask and the first and second UAV-view contamination candidate boxes to obtain a first similarity and a second similarity.

[0057] Construct the contrastive loss function: ; in, For the perspective of autonomous vehicles, a contamination mask. Contaminated candidate boxes from the first-person drone perspective. To contaminate candidate boxes from the second drone's perspective, sim( ) is the similarity function, and τ is the temperature coefficient.

[0058] The similarity scores are all cosine similarity, calculated using the following formula: ; Where A is a one-dimensional vector obtained by flattening the unmanned vehicle's view contamination mask, and B is a one-dimensional vector obtained by converting the drone's view contamination candidate box into a binary mask and flattening it. The unmanned vehicle's view contamination mask is flattened into vector A. The first drone's view contamination candidate box is mapped to a binary mask in a unified spatial coordinate system and flattened into vector B. The second drone's view contamination candidate box is similarly processed into vector B-. The cosine similarity between vectors A and B is calculated as the first similarity, and the cosine similarity between vectors A and B- is calculated as the second similarity.

[0059] The second object detection network is used only as a negative sample generator during the training phase and does not participate in inference during the application phase. During training, each batch constructs 32 pairs of positive samples (images of unmanned vehicles and drones from the same work unit) and 32 pairs of negative samples (images of unmanned vehicles and drones from different work units), with a temperature coefficient τ of 0.07; all three networks use the same learning rate of 1×10⁻⁶. -4The Adam optimizer is used for end-to-end joint training until the joint loss function converges. The parameters of the semantic segmentation network, the first object detection network, and the second object detection network are simultaneously and iteratively optimized through backpropagation until the first similarity of the joint loss function is maximized and the second similarity is minimized, thus obtaining the trained semantic segmentation network and the first object detection network.

[0060] S5. Panel-level spatial alignment and coordinate mapping from an open-ground perspective. Since the UAV panel images in S3 are uniformly represented in the panel local coordinate system, while the ground segmentation results in S4 are still in the ground camera coordinate system, it is necessary to establish a mapping relationship between the ground images and the panel coordinate system.

[0061] For ground image regions belonging to the same panel, construct a geometric transformation from the ground image coordinate system to the panel's local coordinate system: ; in, Represents the pixel coordinates in the ground image. This represents the pixel coordinates in the local coordinate system of the panel. This is the mapping operator from the ground viewpoint to the panel plane, where the superscript p represents the local coordinate system of the panel.

[0062] This mapping projects the ground segmentation results and the probability map onto the panel coordinate system: ; in, This represents the projection mapping from the ground image coordinate system to the local panel coordinate system.

[0063] This achieves pixel-level alignment between the drone's viewpoint and the ground viewpoint within the same local coordinate system on the same panel.

[0064] S6, Multi-view pollution information fusion between air and ground In the local coordinate system of the panel, the UAV view information and the ground view information of the same panel are fused.

[0065] First, the set of salient drone view pollution candidate boxes corresponding to this panel in S2 is... Mapping to the panel coordinate system, construct a contamination confidence map from the drone's perspective: ; in, This represents the set of regions after mapping the contamination candidate boxes to the panel coordinate system.

[0066] This invention maintains the activation of the Dropout layer in the semantic segmentation network during the inference phase. Images captured by an autonomous vehicle are input into the semantic segmentation network to perform multiple random forward propagations, obtaining N classification results for each pixel. The variance of these N classification results is then calculated as a measure of the uncertainty of that pixel. .

[0067] Subsequently, for any pixel position P in the panel coordinate system, a weighted fusion method is used to construct the fusion contamination probability: ; in, This represents the pollution probability mapped from the ground view. Believe in pollution from the perspective of drones. These are weighting coefficients set based on visibility and occlusion status. Due to the uncertainty of the ground perspective, The uncertainty of the drone's perspective (can be set to a fixed value).

[0068] Finally, the fusion segmentation result is obtained based on the fusion probability: ; In this context, 1 represents a contaminated area, 0 represents a clean area, and merge represents fusion.

[0069] S7. Calculation of Pollution Coverage Ratio For a given panel, count the number of contaminated pixels in the fusion and segmentation results within the panel coordinate system. Let the total number of valid pixels for this panel be... The number of contaminated pixels is:

[0070] ; in, For indicator functions, the subscript pol indicates pollution.

[0071] The pollution coverage ratio of the panel is defined as follows: ; The subscript "all" indicates the total number of valid entries.

[0072] S8. Construction of a cleanliness index based on pollution type weighting Based on the fusion and segmentation results, we further utilize the pollution category information provided by the pollution mask from the perspective of the autonomous vehicle for weighted modeling.

[0073] For the Pollution is defined in terms of the number of pixels in the panel as follows: ; Where c represents the pollution category, c=1,2,3,…,C.

[0074] Let the corresponding weights be... The weighted pollution intensity is then defined as: ; The final definition of the photovoltaic surface cleanliness index is: ; S9. Result Output and Panel-Level Data Encapsulation The pollution distribution map obtained for each photovoltaic panel in the panel coordinate system. The contamination coverage ratio (R) and the cleanliness index (CISS) are the final detection and evaluation results output for this panel, and are bound and packaged with the corresponding panel number as standard input data for subsequent cleaning control methods. This step only completes the output of contamination detection and quantitative evaluation results, and does not involve cleaning path planning or control strategy generation.

[0075] To address the shortcomings of existing technologies (such as Chinese patent CN120411091B), this invention proposes a method for detecting contamination and assessing cleanliness of open-air and ground-based multi-view photovoltaic panels based on YOLOv4 and SegFormer, aiming to solve the following technical problems: First, it addresses the challenge of simultaneously capturing both macroscopic pollution distribution and microscopic pollution details. By integrating aerial images from drones and close-up images from unmanned vehicles, and establishing a unified mapping mechanism from detection results to the spatial index of photovoltaic modules, it achieves an integrated representation of pollution information at the array and module levels.

[0076] Second, it addresses the issue of missed or false detections of transparent and mixed contaminants in complex lighting environments and under highly reflective backgrounds. By introducing the SegFormer semantic segmentation network based on an attention mechanism, pixel-level fine segmentation of contaminated areas is performed, improving the algorithm's robustness to complex optical environments and mixed contamination scenarios.

[0077] Third, it addresses the issue that pollution assessment results are primarily qualitative and cannot directly guide cleaning strategies. By constructing a module-level cleanliness index (CISS), it achieves continuous quantitative output of the pollution level of each photovoltaic module, providing direct input for subsequent cleaning path planning and cleaning intensity adjustment.

[0078] Fourth, it addresses the issue that single-platform detection results are insufficient to support air-ground collaborative operation decisions. By establishing a multi-perspective fusion mechanism oriented towards the component layer, it enables joint modeling of UAV and unmanned vehicle detection information within a unified evaluation framework.

[0079] Fifth, this invention addresses the inconsistency in feature spaces between UAV models and ground-based models. Existing technologies train two models independently, resulting in significantly different feature vectors for the same contaminated area, causing information to cancel each other out during fusion. This invention uses contrastive learning to force the two models to output similar feature vectors for the same contaminated area, achieving cross-view feature space alignment.

[0080] Sixth, this invention addresses the lack of uncertainty awareness during the fusion process. Existing fusion methods use fixed weights, which cannot be dynamically adjusted based on the reliability of the model's own predictions. This invention estimates the uncertainty of each pixel through multiple random forward propagations. During fusion, the weight of the viewpoint is reduced for areas with high uncertainty, avoiding unreliable predictions that contaminate the fusion results. Existing fusion methods use fixed weights, which cannot be dynamically adjusted based on the uncertainty of the model's own predictions for each viewpoint. In areas where the model is prone to errors, such as contaminated boundaries and reflective areas, fixed-weight fusion may lead to false positives or false negatives, thus affecting the reliability of subsequent cleaning decisions and causing waste of water resources and energy.

[0081] This invention also has the following advantages: I. Cross-view feature alignment to achieve mutual information enhancement In existing technologies, UAV detection models and ground segmentation models are trained independently, resulting in significant differences in the feature vectors output for the same contaminated area. During weighted fusion, this leads to information cancellation rather than enhancement. This invention constructs positive and negative sample pairs and introduces a contrastive loss function, forcing the two models to output similar feature vectors for the same contaminated area, thus achieving cross-view feature space alignment. During fusion, the information from the two perspectives can mutually enhance each other, significantly improving fusion accuracy.

[0082] II. Uncertainty perception fusion enhances robustness in complex scenarios Existing fusion methods use fixed weights, which cannot be dynamically adjusted based on the reliability of the model's predictions. This invention maintains the Dropout layer activation during the inference phase and estimates the uncertainty of each pixel through multiple random forward propagations. During fusion, it dynamically reduces the fusion weight of regions with high uncertainty (such as contaminated boundaries or reflective areas). This mechanism effectively avoids the model making incorrect judgments in uncertain regions, thus contaminating the fusion result and significantly improving detection robustness under complex lighting and reflective conditions.

[0083] Third, it has strong cross-seasonal applicability and ensures consistent testing throughout the year. The ambient light conditions at photovoltaic power plants vary significantly with the seasons and weather. Under conditions of cloudy and rainy weather in spring, strong sunlight in summer, oblique sunlight in autumn, and low-angle sunlight in winter, the imaging characteristics of the same polluted area differ greatly from different viewing angles. This invention uses comparative learning to make two models more adaptable to changes in light conditions. The aligned feature space remains stable under different lighting conditions, solving the problem of significant fluctuations in detection accuracy across seasons in existing technologies and ensuring the consistency of detection results throughout the year.

[0084] IV. Output a quantitative cleanliness index to support intelligent cleaning decisions. Existing technologies mostly output "whether contamination exists" or coarse-grained category labels, which are difficult to directly support intelligent decisions such as adjusting cleaning intensity and optimizing cleaning sequence. This invention, based on a fused contamination distribution map, statistically analyzes the proportion of contaminated pixels and constructs a weighted contamination intensity by combining contamination type weights, ultimately outputting a photovoltaic surface cleanliness index (CISS). This index is a continuous value between 0 and 1, which can be directly used as the basis for adjusting cleaning parameters and support the generation of differentiated cleaning strategies.

[0085] V. Collaborative Optimization of Training and Reasoning This invention achieves feature space alignment through contrastive learning during the training phase and adaptive fusion through uncertainty awareness during the inference phase. The two phases are synergistically optimized and mutually supportive. The alignment during training provides a stable feature foundation for the inference phase, while uncertainty awareness in the inference phase compensates for boundary regions that are difficult to cover during training. Together, they form a complete technology chain from model training to practical application.

[0086] In summary, this invention effectively solves the problems of inconsistent feature space, low fusion accuracy, and poor cross-seasonal versatility in existing technologies by combining cross-view feature alignment and uncertainty perception fusion. It significantly improves the accuracy, robustness, and versatility of photovoltaic panel contamination detection, and provides a reliable quantitative evaluation basis for intelligent cleaning of photovoltaic power plants.

[0087] Based on the same inventive concept, the present invention also provides a photovoltaic panel contamination detection system, comprising: The data acquisition module is used to acquire aerial images of the photovoltaic modules to be cleaned by drones and images taken by unmanned vehicles, and to divide the photovoltaic modules to be cleaned into multiple work units; based on the polluted area in the preset unmanned vehicle image, it matches a first drone aerial image belonging to the same work unit as the preset unmanned vehicle image, and a second drone aerial image belonging to a different work unit from the preset unmanned vehicle image.

[0088] The training module is used to input preset images captured by an unmanned vehicle, aerial images captured by a first UAV, and aerial images captured by a second UAV into a semantic segmentation network, a first object detection network, and a second object detection network, respectively, and output an unmanned vehicle view pollution mask, a first UAV view pollution candidate box, and a second UAV view pollution candidate box. The unmanned vehicle view pollution mask represents the pixel-level pollution segmentation result in the unmanned vehicle image. The pollution mask and pollution candidate boxes are mapped to a unified spatial coordinate system, and the spatial similarity of the unmanned vehicle view pollution mask with the first UAV view pollution candidate box and the second UAV view pollution candidate box is calculated to obtain a first similarity and a second similarity. Based on the first similarity and the second similarity, a joint loss function is constructed. The parameters of the semantic segmentation network, the first object detection network, and the second object detection network are simultaneously optimized iteratively through backpropagation until the first similarity of the joint loss function is maximized and the second similarity is minimized, thus obtaining the trained semantic segmentation network and the first object detection network.

[0089] The detection module is used to fuse the pollution mask from the perspective of the unmanned vehicle output by the trained semantic segmentation network and the pollution candidate box from the perspective of the drone output by the first target detection network to obtain the pollution detection result of the photovoltaic panel to be cleaned.

[0090] This invention also provides a computer device. At the hardware level, this computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the photovoltaic panel contamination detection method provided above.

[0091] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the photovoltaic panel contamination detection method provided above.

[0092] Specific limitations regarding the computational system for photovoltaic panel contamination detection methods can be found in the limitations of photovoltaic panel contamination detection methods described above, and will not be repeated here. Each module in the aforementioned photovoltaic panel contamination detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0093] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for detecting contamination in photovoltaic panels, characterized in that, Includes the following steps: Aerial images of the photovoltaic modules to be cleaned by drones and images taken by unmanned vehicles are collected, and the photovoltaic modules to be cleaned are divided into multiple work units. Based on the polluted area in the preset unmanned vehicle image, a first drone aerial image belonging to the same work unit as the preset unmanned vehicle image and a second drone aerial image belonging to a different work unit are matched. The preset images captured by the unmanned vehicle, the first aerial image captured by the drone, and the second aerial image captured by the drone are respectively input into the semantic segmentation network, the first target detection network, and the second target detection network, and the outputs are the unmanned vehicle view pollution mask, the first drone view pollution candidate box, and the second drone view pollution candidate box; the unmanned vehicle view pollution mask represents the pixel-level pollution segmentation result in the unmanned vehicle image. The contamination mask and contamination candidate boxes are mapped to a unified spatial coordinate system. The spatial similarity of the unmanned vehicle's view contamination mask with the first and second UAV view contamination candidate boxes is calculated to obtain a first similarity and a second similarity. Based on the first and second similarities, a joint loss function is constructed. The parameters of the semantic segmentation network, the first object detection network, and the second object detection network are simultaneously iteratively optimized through backpropagation until the first similarity of the joint loss function is maximized and the second similarity is minimized, thus obtaining the trained semantic segmentation network and the first object detection network. The contamination mask output by the trained semantic segmentation network and the UAV view contamination candidate boxes output by the first object detection network are fused to obtain the contamination detection result of the photovoltaic panel to be cleaned.

2. The method for detecting contamination in photovoltaic panels according to claim 1, characterized in that, The semantic segmentation network adopts the encoder structure of the SegFormer semantic segmentation network and outputs a contamination mask from the perspective of the unmanned vehicle; the first object detection network and the second object detection network both adopt the backbone network structure of the YOLOv4 object detection network and output the first and second UAV perspective contamination candidate boxes; the second object detection network is only used as a negative sample generator during the training phase and does not participate in inference during the application phase.

3. The method for detecting contamination in photovoltaic panels according to claim 1, characterized in that, Both the first and second similarities are cosine similarities, calculated using the following formula: ； Where A is a one-dimensional vector obtained by flattening the unmanned vehicle's view pollution mask, and B is a one-dimensional vector obtained by converting the drone's view pollution candidate box into a binary mask and flattening it.

4. The photovoltaic panel contamination detection method according to claim 1, characterized in that, The joint loss function is: ； in, For the perspective of autonomous vehicles, a contamination mask. Contaminated candidate boxes from the first-person drone perspective. To contaminate candidate boxes from the second drone's perspective, sim( ) is the similarity function, and τ is the temperature coefficient.

5. The method for detecting contamination in photovoltaic panels according to claim 1, characterized in that, The semantic segmentation network also outputs an uncertainty measure for each pixel in the unmanned vehicle's view pollution mask. The process of fusing the unmanned vehicle's view pollution mask output by the trained semantic segmentation network with the drone's view pollution candidate boxes output by the first target detection network to obtain the pollution detection result for the photovoltaic panel to be cleaned specifically includes: transforming the unmanned vehicle's view pollution mask and uncertainty measure to the local coordinate system of the photovoltaic panel through spatial mapping; mapping the drone's view pollution candidate boxes to the local coordinate system of the same panel; dynamically adjusting the fusion weights of the drone's view pollution candidate boxes according to the uncertainty measure to generate a fused pollution distribution map; statistically analyzing the pollution coverage ratio based on the fused pollution distribution map, and outputting the pollution detection result for the photovoltaic panel to be cleaned.

6. The method for detecting contamination in photovoltaic panels according to claim 5, characterized in that, Based on the fused pollution distribution map, the pollution coverage ratio is calculated, and the pollution detection results of the photovoltaic panel to be cleaned are output. Specifically, this includes: based on the fused pollution distribution map, counting the number of polluted pixels in the panel, obtaining the total number of effective pixels of the panel, and dividing the number of polluted pixels by the total number of effective pixels to obtain the pollution coverage ratio. Based on the pollution mask from the perspective of the autonomous vehicle, the number of pixels for each type of pollution in the panel is obtained. The number of pixels for each type of pollution is divided by the total number of effective pixels in the panel and then multiplied by the preset weight corresponding to that type of pollution. The calculation results of all categories are then added together to obtain the weighted pollution intensity. The weighted pollution intensity is subtracted from the value 1 to obtain the photovoltaic surface cleanliness index CISS. The fused pollution distribution map, pollution coverage ratio, and photovoltaic surface cleanliness index (CISS) are output as the pollution detection and cleanliness assessment results of the target panel.

7. The photovoltaic panel contamination detection method according to claim 5, characterized in that, The calculation of the uncertainty measure specifically includes: obtaining N classification results for each pixel of the image captured by the unmanned vehicle, and calculating the variance of the N classification results as the uncertainty measure of that pixel.

8. A photovoltaic panel pollution detection system, characterized in that, include: The data acquisition module is used to collect aerial images of the photovoltaic modules to be cleaned by drones and images taken by unmanned vehicles, and to divide the photovoltaic modules to be cleaned into multiple work units; Based on the polluted area in the preset unmanned vehicle image, a first drone aerial image belonging to the same work unit as the preset unmanned vehicle image is matched with a second drone aerial image belonging to a different work unit than the preset unmanned vehicle image. The training module is used to input preset images captured by an unmanned vehicle, aerial images captured by a first UAV, and aerial images captured by a second UAV into a semantic segmentation network, a first object detection network, and a second object detection network, respectively, and output an unmanned vehicle view pollution mask, a first UAV view pollution candidate box, and a second UAV view pollution candidate box. The unmanned vehicle view pollution mask represents the pixel-level pollution segmentation result in the unmanned vehicle image. The pollution mask and pollution candidate boxes are mapped to a unified spatial coordinate system, and the spatial similarity of the unmanned vehicle view pollution mask with the first UAV view pollution candidate box and the second UAV view pollution candidate box is calculated to obtain a first similarity and a second similarity. Based on the first similarity and the second similarity, a joint loss function is constructed. The parameters of the semantic segmentation network, the first object detection network, and the second object detection network are simultaneously iteratively optimized through backpropagation until the first similarity of the joint loss function is maximized and the second similarity is minimized, thus obtaining the trained semantic segmentation network and the first object detection network. The detection module is used to fuse the pollution mask from the perspective of the unmanned vehicle output by the trained semantic segmentation network and the pollution candidate box from the perspective of the drone output by the first target detection network to obtain the pollution detection result of the photovoltaic panel to be cleaned.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 7.

Images