Photovoltaic sand and dust identification method based on color space fusion and lightweight learning

Abstract

The invention relates to a photovoltaic sand and dust identification method based on color space fusion and lightweight learning, and the method comprises the following steps: S1, obtaining image data of a photovoltaic module, detecting a module region in an image, and obtaining a mask image of the module region; s2, extracting an ROI image of the photovoltaic module, and performing geometric correction processing; s3, converting the ROI image from an RGB color space to an HSV color space; s4, respectively carrying out threshold setting on the H, S and V three-channel images based on the HSV color space; s5, fusing threshold setting results of the H, S and V channels, generating a final sand and dust pollution mask pattern, and completing sand and dust region identification; and S6, calculating the sand-dust covering proportion of the polluted area on the surface of the component, and dividing pollution grades according to the sand-dust covering proportion. According to the invention, the dust area on the surface of the photovoltaic module can be accurately extracted under different illumination conditions, standardized quantitative evaluation of the pollution degree is realized, and the method is suitable for intelligent cleaning management and remote operation and maintenance scheduling scenes of a photovoltaic system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and intelligent operation and maintenance of photovoltaic systems, and particularly relates to a photovoltaic dust identification method based on color space fusion and light learning. BACKGROUND

[0002] With the large-scale application of photovoltaic power generation systems in desert, gobi and other arid and semi-arid areas, the accumulation of dust on the surface of photovoltaic modules has become an important factor affecting the power generation efficiency and equipment life. Dust deposition can cause light shading, electrical performance degradation and local hot spot effect, thereby causing system power output attenuation.

[0003] At present, the detection methods for surface contamination of photovoltaic modules mainly include infrared imaging, electrical performance parameter analysis and image processing. Among them, the identification method based on image processing gradually becomes the mainstream direction of research and application due to its non-contact, low cost and strong adaptability. However, the traditional image segmentation method generally relies on low-level visual features such as gray distribution and edge detection, and has poor adaptability to complex lighting, strong reflection and fuzzy pollution boundary in actual scenes. Although the image recognition method based on deep learning has strong feature extraction capability, its model structure is complex, highly dependent on training samples, and has weak interpretability, which limits its deployment and application in resource-constrained scenarios. In addition, the existing methods are mostly based on single color channel or fixed threshold for dust area segmentation, lack of multi-channel information fusion and dynamic adjustment strategy, and are difficult to accurately reflect the spatial distribution characteristics of the contaminated area. At the same time, in terms of quantification of contamination degree, there is still a lack of standardized evaluation mechanism, which is not conducive to providing effective support for subsequent cleaning scheduling and intelligent operation and maintenance system.

[0004] Therefore, the existing technology still has problems such as insufficient recognition accuracy, poor adaptability, lack of pollution level quantification, and the like, and there is an urgent need for an image recognition and pollution evaluation method suitable for variable environments, light structure and strong deployability. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a photovoltaic dust identification method based on color space fusion and light learning suitable for variable environments, light structure and strong deployability.

[0006] To solve the above problems, the photovoltaic dust identification method based on color space fusion and light learning provided by the present application comprises the following steps:

[0007] S1 obtains image data of a photovoltaic module and detects a module area in the image to obtain a mask image of the module area;

[0008] S2 extracts an ROI image of the photovoltaic module based on the module mask image and performs geometric correction processing;

[0009] S3 converts the ROI image from the RGB color space to the HSV color space for sand and dust area feature analysis in hue, saturation, and brightness dimensions;

[0010] S4 sets thresholds for the H, S, and V three-channel images of the HSV color space respectively to identify areas that may be contaminated by sand and dust;

[0011] S5 fuses the threshold setting results of the H, S, and V channels to generate a final sand and dust contamination mask and complete sand and dust area identification;

[0012] S6 calculates the sand and dust coverage ratio of the contaminated area on the component surface based on the sand and dust contamination mask and divides the contamination level accordingly.

[0013] The image data of the photovoltaic component in step S1 is obtained by the following method: collected by an industrial camera fixedly installed on an indoor support in a laboratory environment, or collected by a camera installed on a drone, a patrol robot, or a fixed monitoring point; the image type is a standard RGB image; the standard RGB image shooting angles include vertical shooting, oblique shooting, and lateral shooting, and the shooting environment covers different time periods and natural light conditions.

[0014] The mask image of the component area in step S1 is obtained by the following method: the collected standard RGB image is input into a YOLOv11 semantic segmentation model trained by transfer learning to locate and segment the photovoltaic component in the image, and a mask image of the component area is obtained; the YOLOv11 semantic segmentation model is based on a pre-trained model and is obtained by transfer learning training using an image dataset obtained under actual working conditions; the dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1.

[0015] The ROI image of the photovoltaic component in step S2 is extracted by the following method: the Canny edge detection algorithm is used on the component mask image to extract the edge contour of the component area, and a minimum enclosing quadrilateral fitting operation is performed based on the extracted edge contour, and the area surrounded by the minimum enclosing quadrilateral is taken as the ROI image of the photovoltaic component.

[0016] The geometric correction in step S2 is processed by the following method: the coordinates of the four corner points of the component boundary are obtained based on the ROI image of the photovoltaic component, a homography matrix is constructed, and a perspective transformation is performed on the ROI image of the photovoltaic component.

[0017] The thresholds of the H, S, and V three-channel images of the HSV color space in step S4 are set by the following methods respectively:

[0018] ①For the H channel of the HSV color space, a threshold setting method based on the frequency histogram statistics of the images in the dataset is adopted, and the threshold interval of the H channel is determined as [2°, 40°];

[0019] ②For the S channel in the HSV color space, a threshold setting method based on multi-dimensional statistical features of images and a LightGBM regression model is adopted to realize adaptive threshold prediction; the multi-dimensional image features include 20 kinds of basic statistical features, histogram structure features, wavelet energy features, channel coupling features, edge and gradient structure features, and global coupling structure features;

[0020] ③For the V channel of the HSV color space, a threshold setting method of heuristic peak-valley analysis is adopted:

[0021] ⅰThe normalized frequency histogram of the V channel image is calculated, and Gaussian smoothing is performed thereon;

[0022] ⅱIdentification of the main peak of the dust: identify the main peak position P r in the brightness region [140, 250]; if no local peak satisfying the significance requirement is detected in the brightness region [140, 250], the position of the maximum value of the smoothed histogram H s (v) in this interval is taken as the main peak position; the valley bottom position between the nearest left peak and the main peak in the left region of P r is found and set as v min , if there is no effective left peak, a fixed offset is estimated; the position where the brightness value frequency is first lower than 1% of the global maximum value frequency in the right region of P r is found and set as v max , if the condition is not met, a bottom width is set;

[0023] ⅲDetermination of the heuristic peak-valley analysis and adaptive threshold range.

[0024] The dust region identification method in step S5 refers to a joint judgment of each pixel point in the image based on the threshold setting results of each channel in the HSV color space, and a logical "and" operation of the threshold judgment results of the three channels is adopted to realize accurate extraction of the dust region.

[0025] The dust region identification method refers to performing threshold judgment on each pixel point in the image in the H channel, the S channel and the V channel, if the H channel value of the pixel is within the set hue range [2°, 40°], and the S channel value is within the saturation threshold interval obtained by the LightGBM model prediction, and the V channel value is within the brightness threshold interval obtained by the normalized frequency histogram analysis of the current image, the pixel is marked as a dust region and assigned a value of 255 in the mask image; otherwise, it is marked as a background region and assigned a value of 0 in the mask image.

[0026] The sand dust coverage ratio of the contaminated area in the step S6 on the surface of the component refers to that according to the sand dust contamination mask obtained in the step S5, all pixels in the image are traversed point by point, the number N of sand dust region pixels is counted dust , that is, the number of pixel points with a pixel value of 255 in the mask, and the number N of pixels outside the sand dust region is counted panel , that is, the number of pixel points with a pixel value of 0 in the mask; and then the sand dust coverage ratio is calculated as follows:

[0027]

[0028] A photovoltaic sand dust identification system based on color space fusion and light learning for the above method, characterized in that: the system is composed of an image acquisition and component region detection module (1), an ROI extraction and geometric correction module (2), a color space conversion module (3), a channel feature extraction and threshold setting module (4), a mask generation and contaminated area identification module (5) and a contaminated area statistics and grade division module (6) connected in turn and connected with a computer terminal respectively; the image acquisition and component region detection module (1) is connected with an industrial camera fixedly installed on an indoor support, or connected with a camera provided on an unmanned aerial vehicle, a patrol robot or a fixed monitoring point.

[0029] Compared with the prior art, the present application has the following advantages:

[0030] 1. The present application converts the RGB color space into the HSV color space, and extracts the features of the H, S and V three color channels respectively, effectively dealing with the sand dust identification problem in harsh environments such as strong reflection and complex light changes, compared with the traditional single channel threshold segmentation method, the present application can stably identify the sand dust region, avoiding the identification precision fluctuation caused by light change or reflection interference in actual scene.

[0031] 2. The present application overcomes the missed detection problem of single color dimension method in processing boundary fuzzy or color change region by fusing multi-channel feature and dynamic threshold setting strategy, significantly improves the identification precision and region integrity of sand dust region. The experimental results show that the F1_score of the present application can reach more than 98%, which proves that the identification effect of the present application in complex environment is excellent.

[0032] 3. The present application first introduces the LightGBM model for dynamically predicting the threshold interval of the S channel, combines with the image statistical features, improves the adaptability of the segmentation algorithm under different environmental conditions; at the same time, through the SHAP explainability tool, the decision-making process of the LightGBM model can be explained, which improves the transparency of the system and the trust of the industrial application.

[0033] 4、The application provides an automatic pollution coverage calculation method based on a sand area mask and a ROI area, and according to a set pollution level standard, the pollution degree can be accurately divided into multiple levels such as mild, moderate and severe, thereby providing a reliable basis for subsequent intelligent cleaning scheduling and maintenance decision-making.

[0034] 5、The YOLOv11 semantic segmentation model and the LightGBM model are combined, and through comparative experiments, it is verified that the smaller version of the YOLOv11 semantic segmentation model can efficiently run under the condition of ensuring similar accuracy, and is suitable for different computing resources and hardware platforms.

[0035] 6、The application can accurately extract the sand area on the surface of the photovoltaic module under different light conditions, and realize standardized quantitative evaluation of the pollution degree, has the characteristics of light structure, strong interpretability, easy edge deployment and the like, and is suitable for intelligent cleaning management and remote operation and maintenance scheduling scenes of the photovoltaic system.

[0036] 7、The application does not depend on a large number of training samples, and can adapt to various photovoltaic module types and installation environments, has strong expansibility and universality, can meet the long-term monitoring and pollution management needs in the intelligent operation and maintenance system of large-scale photovoltaic power stations, and has high practicability and generalizability.

[0037] 8、The application can be widely deployed in unmanned aerial vehicles, edge computing terminals and fixed monitoring equipment and various industrial environments to meet the needs of remote monitoring and intelligent operation and maintenance. BRIEF DESCRIPTION OF DRAWINGS

[0038] The specific embodiments of the application will be further described in detail below with reference to the accompanying drawings.

[0039] Figure 1 The flowchart of the application.

[0040] Figure 2 The H channel frequency histogram distribution diagram of the sand dust affected area under five typical photovoltaic panel scenes in the application.

[0041] Figure 3 The frequency histogram distribution diagram of the sand dust area and the background area in the S channel under five typical photovoltaic panel scenes in the application.

[0042] Figure 4 The SHAP feature contribution scatter plot analysis combination diagram of the LightGBM submodel for predicting the upper and lower limits of the S channel threshold in the application. Wherein: (a) is a submodel analysis diagram for predicting the upper limit; (b) is a submodel analysis diagram for predicting the lower limit.

[0043] Figure 5The frequency histogram distribution diagram of the dust region and the background region in the V channel under the five typical photovoltaic panel scenes in the application is shown.

[0044] Figure 6 The three-channel joint threshold judgment logic diagram in the application is shown.

[0045] Figure 7 The dust region segmentation mask diagram and the dust region segmentation effect comparison diagram in the application are shown.

[0046] Figure 8 The segmentation visualization comparison result diagram of the three segmentation methods on the typical dust pollution image in the application is shown.

[0047] Figure 9 The system connection diagram in the application is shown.

[0048] In the figure: 1- image acquisition and component region detection module; 2- ROI extraction and geometric correction module; 3- color space conversion module; 4- channel feature extraction and threshold setting module; 5- mask generation and pollution area identification module; 6- pollution area statistics and grade division module. DETAILED DESCRIPTION

[0049] As shown in Figure 1 , a photovoltaic dust identification method based on color space fusion and lightweight learning includes the following steps:

[0050] S1 obtains image data of a photovoltaic component and detects a component region in the image to obtain a mask image of the component region. The specific process is as follows:

[0051] (1) In a laboratory environment, industrial cameras fixedly installed on indoor supports are used to collect image data of photovoltaic components. The image collection simulates the shooting angle on the surface of an actual photovoltaic component, including vertical, inclined, and lateral directions. The shooting environment covers different time periods and natural light conditions. By adjusting the pitch angle of the camera and the direction of the light source, typical natural light and shielding scenes are reconstructed for training and verification of the dust identification algorithm. The image type is a standard RGB image.

[0052] The image acquisition method of the application can also be used to obtain images from actual photovoltaic power stations through cameras installed on unmanned aerial vehicles, inspection robots, or fixed monitoring points to meet the application requirements of remote dust detection of photovoltaic panels.

[0053] (2) The collected standard RGB image is input into the YOLOv11 semantic segmentation model trained based on transfer learning to locate and segment the photovoltaic component in the image, and the corresponding mask image of the component region is output.

[0054] The YOLOv11 semantic segmentation model adopts a yolo11x-seg.pt pre-training model in the YOLOv11 semantic segmentation framework as a base model; the model has the ability to simultaneously perform target detection and semantic segmentation, can accurately identify the boundaries of photovoltaic components in an image, and generate a pixel-level segmentation mask; the above pre-training model is retrained through transfer learning to adapt to specific application scenarios and image features.

[0055] In the transfer learning process, the training data set used is obtained by collecting in a laboratory environment, a total of 410 RGB images; the collected images cover different shooting angles, lighting conditions and pollution levels to ensure that the model has good generalization ability; the data set is manually annotated using the Labelme image annotation tool, mainly frame-by-frame accurately outlining the photovoltaic cell boundary profile on the surface of the photovoltaic panel to generate the corresponding mask image.

[0056] To train the model, the data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1 to improve the segmentation accuracy and generalization ability of the model in real application scenarios; the input image size of the model is set to 1280x1280, the total number of training rounds (Epoch) is 200 rounds, the initial learning rate is 0.001, the batch size is set to 6, and the remaining training parameters are set according to the official recommended configuration of YOLOv11.

[0057] In actual application, in addition to yolo11x-seg.pt, other semantic segmentation pre-training models in the YOLOv11 series (such as yolo11n-seg.pt) can also be selected according to deployment requirements and computing resources, or models with semantic segmentation capabilities such as YOLOv8 can be used instead to complete the detection and mask generation task of photovoltaic components, with good replaceability and adaptability.

[0058] S2 extracts the ROI (region of interest) image of the photovoltaic component based on the component mask image and performs geometric correction processing.

[0059] Edge detection is performed on the component mask image generated in step S1, and the Canny edge detection algorithm is used to extract the edge profile of the component region. The Canny edge detection algorithm is a commonly used image processing algorithm that can effectively detect significant edges in an image and provide clear boundary information for subsequent quadrilateral fitting.

[0060] Although the YOLOv11 semantic segmentation model can provide relatively accurate segmentation results when detecting the region of the photovoltaic module, the edges of the mask output by the model still have fine burrs and irregular contours, which will affect the effective implementation of subsequent geometric correction. Therefore, further contour extraction processing is required for the mask image to ensure that the main contour of the module region is retained. The contour extraction process extracts the outer contour of the module region by calling the findContours function of OpenCV. In order to ensure accurate extraction of the boundary of the photovoltaic module, the present application further performs a minimum circumscribed quadrilateral fitting operation on the extracted main contour. Specifically, the approxPolyDP function of OpenCV is used to obtain a minimum circumscribed rectangle through polygon approximation fitting according to the contour information by adjusting the epsilon parameter. Finally, the region enclosed by the obtained minimum circumscribed quadrilateral is taken as the ROI image of the photovoltaic module. The ROI image not only effectively removes the background noise in the image, but also provides accurate area support for subsequent image geometric correction and pollution area identification. In the geometric correction, the coordinates of the four corner points of the minimum circumscribed quadrilateral are used as the basis for subsequent perspective transformation and standardization.

[0061] After completing the ROI extraction of the photovoltaic module, in order to eliminate the tilt angle distortion in the image shooting process and realize the standardization of subsequent pollution coverage calculation, the present application performs a geometric correction process on the ROI image of the photovoltaic module. This process uses perspective transformation to convert the tilt angle in the image to a front view image, thereby eliminating the distortion caused by the deviation of the shooting angle, realizing view angle correction and geometric standardization, unifying the arrangement direction of the module, reducing distortion interference, providing stable input for subsequent color channel analysis, and ensuring the accuracy of subsequent analysis.

[0062] The correction process calculates the homography matrix (Homography Matrix) H between the original image and the target standard image based on the four corner point coordinates of the module, and completes the space mapping through the homogeneous coordinate transformation formula.

[0063] The specific transformation relationship is as follows:

[0064]

[0065] Where (x, y) is the coordinate of a pixel point in the original image; (x', y') is the transformed coordinate; w is the homogeneous scale factor of the original point; w' is the homogeneous scale factor of the transformed point; H is a 3x3 homography matrix.

[0066] To ensure that the transformed image has consistent measurement characteristics, the present application introduces the area conservation criterion, that is, to maintain the consistency of the pixel area in the background area of the photovoltaic module. The area conversion formula is as follows:

[0067]

[0068] Wherein: S and S' represent the pixel area before and after transformation respectively; |det(H)| is the determinant of the homography matrix. The relationship is used for subsequent standardization calculation of pollution coverage, ensuring the physical consistency and robustness of area conversion.

[0069] It should be noted that although the present application adopts perspective transformation to correct the inclined view image, other image correction methods (such as bilinear transformation, affine transformation, etc.) can also be selected and replaced according to different actual application scenarios and needs to adapt to different photovoltaic power station environments.

[0070] S3 converts the ROI image from the RGB color space to the HSV color space, so as to perform dust area feature analysis on the hue, saturation and brightness dimensions.

[0071] The corrected image is converted from the RGB space to the HSV color space using an image processing library such as OpenCV, and three channel images of H, S and V are extracted respectively for subsequent mask generation and multi-channel fusion, where H∈[0, 180°], S∈[0, 255], and V∈[0, 255].

[0072] The formula used for the theoretical calculation of RGB to HSV is as follows:

[0073]

[0074] V = max(R, G, B)

[0075] Wherein: X = min(R, G, B), and min(R, G, B) represents the minimum color component value in the three individual pixel values of R, G and B, max(R, G, B) represents the maximum color component value in the three individual pixel values of R, G and B, and the remaining symbol meanings are consistent with the standard definition of HSV.

[0076] The image processing of the present application is based on the OpenCV library, and for adaptation to the OpenCV library, the theoretical values are mapped as follows:

[0077] H: Theoretical range [0, 360°], compressed to [0, 180°] in OpenCV (i.e. H OpenCV = H 理论 / 2);

[0078] S: Theoretical range [0, 1], scaled to [0, 255] in OpenCV (i.e. S OpenCV = S 理论 × 255);

[0079] V: Theoretical range [0, 1], scaled to [0, 255] in OpenCV (i.e. VOpenCV = V 理论 × 255);

[0080] The final OpenCV output H∈[0,180°], S∈[0,255], V∈[0,255].

[0081] S4 Based on the H, S, V three-channel image of HSV color space, threshold setting is performed respectively to identify the area where dust pollution may exist.

[0082] ①For the H channel in the HSV color space, a threshold setting method based on the frequency histogram statistical results of the images in the data set is adopted. By statistically analyzing and cross-validating the H channel frequency histograms of the dust areas in five typical photovoltaic scene categories, it is found that the H channel distribution of the dust area under different illumination conditions has high consistency, and the distribution shows strong concentration. Specifically, the main peak value of the dust area stably distributes in the lower hue range (i.e. between 2° and 40° of the H channel), and there is a significant distinguishing characteristic between the distribution of the background area. Therefore, in the H channel processing process, a fixed threshold strategy is adopted, and the threshold interval of the H channel is uniformly set as [2°, 40°], which is used to screen the pixel points that may belong to the dust area. This threshold interval covers the main distribution area of the dust hue under various typical photovoltaic panel shooting scenes.

[0083] The typical scenes include: (1) normal vertical angle shooting, medium light intensity. (2) 45° oblique shooting, medium light intensity. (3) 45° oblique shooting, strong direct light. (4) 45° oblique shooting, weak diffuse light. (5) mixed light conditions with shadows and mirror reflections.

[0084] Based on the distribution statistics of these scenes, the present application determines a fixed hue threshold interval [2°, 40°], which covers the main hue distribution range of the dust area under different illumination conditions. It should be particularly noted that this threshold interval is not set empirically, but is derived from the distribution statistics and cross-validation analysis of the dust area in the data set, and has strong universality and robustness. In order to enhance the stability of the algorithm in complex lighting environments and improve the fault tolerance of the segmentation results, the present application leaves a certain margin in the upper and lower limits of the threshold interval to cope with the boundary transition part between the dust area and the background, so that even in scenes with large changes in light intensity, the algorithm can still stably identify the dust area.

[0085] The selection of this threshold interval has strong robustness and can adapt to dust recognition under different illumination conditions. Through experimental verification, setting this threshold interval can effectively eliminate misidentification caused by factors such as light changes and different viewing angles. In addition, the setting of the threshold interval can also balance the discrimination between the dust area and the background area, improving the segmentation accuracy.

[0086] Figure 2 Figure 1 is a schematic diagram of the H channel frequency histogram distribution of the sand dust affected area in five typical photovoltaic panel scenarios in the present application. The figure shows the frequency density distribution of the sand dust area in the H channel under different light conditions. It can be seen from the figure that the distribution of the sand dust area in each scenario is concentrated in the interval [2°, 40°], which proves the effectiveness and universality of the threshold setting.

[0087] ②For the S channel in the HSV color space, a threshold setting method based on image multi-dimensional statistical features and LightGBM regression model is adopted to realize adaptive threshold prediction. Specifically, the S channel reflects the saturation information of the pixels in the image, which can effectively capture the difference in color properties between the sand dust area and the background area. Under different light conditions, the distribution of the sand dust area presents high consistency. However, due to the influence of factors such as light change and sand dust particle density, there is a certain overlap phenomenon in the frequency histogram distribution of the S channel, especially in complex scenarios, it is difficult for the fixed threshold method to stably distinguish sand dust and background. Therefore, in order to improve the robustness and adaptability of the threshold setting, the present application adopts the LightGBM model to automatically predict the dynamic threshold upper and lower limits by training 20 statistical features extracted from the S channel image.

[0088] Figure 3 Figure 2 is a schematic diagram of the frequency histogram distribution of the sand dust area and the background area in the S channel in five typical photovoltaic panel scenarios in the present application. Based on the statistical analysis of the S channel image frequency histogram of the five typical photovoltaic panel scenarios ("normal vertical angle shooting, medium light intensity", "tilted 45° shooting, medium light intensity", "tilted 45° shooting, strong direct light", "tilted 45° shooting, weak diffuse light", "mixed light conditions with shadow and mirror reflection") in the data set, it can be found that the distribution of the sand dust area and the background area is mainly concentrated in the low saturation range (0~150). However, the frequency histogram curve of the sand dust area and the background area in the image presents an overlapping trend in multiple scenarios, especially in scenario 5 (mixed light conditions with shadow and mirror reflection), the overlap is more serious, making it difficult for the fixed threshold to effectively distinguish. Therefore, the present application trains the LightGBM model to automatically predict the threshold upper and lower limits of the S channel according to the statistical features of the image.

[0089] In order to realize adaptive threshold modeling, the present application first extracts 20 multi-dimensional image features including basic statistical features, histogram structure features, wavelet energy features, channel coupling features, edge and gradient structure features, and global coupling structure features from the data set. Among them:

[0090] Basic statistical features: mean, median, standard deviation, skewness, kurtosis and energy of saturation.

[0091] Histogram structure features: main peak position, main peak height, 10% quantile, 90% quantile, full width at half maximum (FWHM) and histogram entropy.

[0092] Wavelet energy features: first-level approximation energy, second-level detail energy, third-level detail energy and third-level detail standard deviation.

[0093] Channel coupling features: mutual information between S channel and V channel.

[0094] Edge and gradient structure features: mean of luminance gradient and standard deviation of luminance gradient.

[0095] Global coupling structure features: linear correlation coefficient between S channel and V channel.

[0096] Then, these features are input into the LightGBM regression model for training, and the model output is the optimal S channel upper and lower threshold interval corresponding to each image. Unlike the traditional empirical threshold setting method, this method can adaptively set the threshold according to the actual features of each image, improving the segmentation accuracy and robustness.

[0097] During the model training process, the present application uses a manually annotated dust image dataset, which contains calibrated optimal S channel upper and lower thresholds. The dataset is divided into training set, validation set and test set in the ratio of 8:1:1. To ensure the generalization ability and robustness of the model, a 5-fold cross-validation strategy is adopted, and the Bayesian optimization is used based on the Optuna framework to automatically adjust the hyperparameters. During the training process, the key training parameters include:

[0098] Leaf node number 8-256, tree depth 3-10, row sampling ratio 0.6-1.0, column sampling ratio 0.6-1.0, L1 and L2 regularization parameters 1e-4-10.0, learning rate norm 1e-3-0.1, number of iterations 200-1200, and early stopping mechanism is set (if the performance of the validation set does not improve for 20 consecutive rounds, the training is terminated).

[0099] Figure 4 SHAP feature contribution scatter plot analysis combined graph of the LightGBM sub-model in the present application for predicting the upper and lower limits of the S channel threshold, wherein 4(a) is an analysis schematic diagram of the sub-model for predicting the upper limit, and 4(b) is an analysis schematic diagram of the sub-model for predicting the lower limit. In order to further analyze the global explainability of the LightGBM model, the present application uses the SHAP (SHapley Additive exPlanations) method to analyze the feature contribution. In Figure 4In (a), the model's prediction of the upper limit of the S-channel threshold mainly relies on features such as the 90th percentile, standard deviation, third-level detail energy, and mutual information between the S-channel and V-channel, which show weak correlation in traditional linear analysis but have significant contributions to the model's prediction in SHAP analysis. In Figure 4 In (b), the prediction of the lower threshold relies more on the synergy of saturation statistics (such as standard deviation, skewness, kurtosis) and gradient-based features, further demonstrating that the LightGBM model can effectively mine complex nonlinear interactions between features.

[0100] ③The method for setting the threshold of the V-channel in the HSV color space is:

[0101] First, analyze the brightness characteristics of the V-channel image. In the identification of dust area images, the brightness information of the V-channel plays an auxiliary role in distinguishing dust areas from background areas. Experiments show that the brightness frequency histogram of the dust area is significantly shifted to the right compared to the background area, and the main peak position is clear, but its distribution is easily affected by the exposure of the shot, and there is some instability.

[0102] Figure 5 The frequency histogram distribution of the V-channel in the five typical photovoltaic panel scenarios in the present application is shown in the figure. Figure 5 The distribution characteristics of the dust area and the background area in the V-channel frequency histogram under five typical photovoltaic panel scenarios ("normal vertical angle shooting, medium light intensity", "tilted 45° shooting, medium light intensity", "tilted 45° shooting, strong direct light", "tilted 45° shooting, weak diffuse light", "mixed light conditions with shadows and specular reflection") in the data set are shown, providing ideas for subsequent V-channel threshold setting methods.

[0103] Therefore, the present application proposes a threshold setting method based on heuristic peak-valley analysis, aiming to improve the robustness and stability of the V-channel segmentation process. The specific steps are as follows:

[0104] ⅰConstruction and Gaussian smoothing of the normalized brightness frequency histogram. Calculate the normalized frequency histogram of the V-channel image and perform Gaussian smoothing to remove high-frequency noise and short-term fluctuations.

[0105] The mathematical expression of Gaussian smoothing is:

[0106]

[0107] where: H s (v) is the normalized frequency histogram of Gaussian smoothing; H(v+k) is the histogram value at the neighborhood position near the brightness value v; G(k;σ) is the Gaussian kernel function; k is the relative offset; σ is the standard deviation of Gaussian distribution.

[0108] ii. Recognition of the main peak of dust. Since the pixel values in the range of [140, 250] contain the main brightness distribution of the dust area, the main peak of the dust area can be effectively highlighted after smoothing. Therefore, in the Gaussian smoothed normalized frequency histogram, the main peak position of the dust in the brightness interval [140, 250] is found, which represents the brightness characteristics of the dust area in the V channel and provides a reference for the subsequent threshold setting. That is, the search range of the main peak is limited to V∈[140, 250], so as to exclude the low-frequency interference of the background area. The derivative extreme detection method based on Gaussian smoothing is used to dynamically locate the main peak in the interval [140, 250]. The main peak is usually the highest peak in the interval, and its calculation process can be represented as:

[0109]

[0110] wherein: P r is the main peak position; p is the local maximum point of H s (v); and P is the global peak set.

[0111] The specific method for identifying the main peak is as follows: the main peak position P r is identified in the brightness region [140, 250]; if no local peak that meets the significance requirement is detected in the brightness region [140, 250], the maximum position of the smoothed histogram H s (v) in the interval is taken as the main peak position; the valley bottom position between the nearest left peak and the main peak in the left region of P r is found and set as v min , if there is no effective left peak, a fixed offset (such as a -40 brightness value margin) is estimated; the position where the brightness value frequency is first lower than 1% of the global maximum value frequency in the right region of P r is found and set as v max , if the condition is not met, a bottom width (such as +50 brightness value margin) is set.

[0112] iii. Heuristic peak valley analysis and determination of adaptive threshold range. For the identified main peak, further combined with the peak valley position between the left peak and the main peak and the right side low frequency attenuation characteristics, the adaptive threshold range of the V channel is determined according to these characteristics. In actual application, this method determines the upper limit of the threshold interval by analyzing the right side low frequency attenuation characteristics of the main peak, and determines the lower limit of the threshold interval by analyzing the valley bottom position between the nearest left peak and the main peak.

[0113] On the basis of determining the main peak, the threshold lower limit is determined according to the valley bottom between the left peak and the main peak, and its calculation process can be represented as:

[0114]

[0115] wherein: v min is the V channel threshold lower limit; is the left peak valley position; Δ is the margin parameter; P L is the main peak left side peak set.

[0116] The threshold upper limit is determined according to the main peak right side low frequency cutoff point, and the calculation process can be represented as:

[0117]

[0118] wherein: v max is the V channel threshold upper limit, and η is the frequency threshold.

[0119] According to the above analysis, the segmentation threshold range of the V channel is adaptively determined. This method can automatically adjust the threshold according to the image characteristics under different light conditions, avoiding the misjudgment problem caused by fixed threshold setting. In this way, the brightness information of the dust area can be stably extracted, and the recognition limitation of other channels (such as S channel) under low contrast or low saturation conditions can be effectively compensated.

[0120] S5 fuses the threshold setting results of H channel, S channel and V channel, generates the final dust pollution mask image, and completes the dust area recognition.

[0121] The generation method of dust pollution mask image is based on the threshold setting results of each channel in HSV color space, and through joint judgment of each pixel point in the image, the logical "and" operation of three channel threshold judgment results is adopted, so as to realize the accurate extraction of dust area. Specifically, for each pixel point in the image, threshold judgment is performed on H channel, S channel and V channel respectively:

[0122] H channel: if the H channel value of the pixel is within the set hue range [2°, 40°], the pixel may belong to the dust area;

[0123] S channel: if the S channel value of the pixel is within the saturation threshold interval obtained by prediction through the LightGBM model, the dust area is further screened;

[0124] V channel: if the V channel value of the pixel is within the brightness threshold interval obtained based on the normalized frequency histogram analysis of the current image, the pixel is finally confirmed as the dust area.

[0125] If the above three conditions are met, the pixel is marked as the dust area and assigned a value of 255 in the mask image; otherwise, it is marked as the background area and assigned a value of 0 in the mask image.

[0126] Figure 6Fig. 1 is a schematic diagram of the three-channel joint threshold judgment logic in the present application, which shows the joint operation process after the threshold judgment of the H channel, S channel and V channel respectively. Through this diagram, it can be clearly seen that only when the pixel meets the conditions in the threshold range of the three channels at the same time, it will be judged as a sand and dust area, thereby avoiding the misjudgment caused by a single channel.

[0127] The method avoids the superposition of channel mask errors in the traditional method through three-channel threshold judgment, thereby improving the accuracy of sand and dust area extraction. At the same time, the method emphasizes the physical properties of the three channels, thereby enhancing the physical interpretability of the segmentation result and fitting the actual response law of sand and dust in the spectrum. In addition, the method is simple and efficient in operation, avoids complex post-processing, is suitable for real-time operation on resource-limited edge platforms, and has the advantages of low power consumption and efficient processing.

[0128] In addition, in order to further improve the quality of the sand and dust mask graph, the present application can perform morphological filtering (such as opening operation, closing operation) on the generated sand and dust mask graph to remove noise points and smooth the boundary. If morphological filtering processing is needed, appropriate kernel size and filtering times can be selected for optimization to improve the boundary smoothness and accuracy of the mask graph.

[0129] Figure 7 Fig. 2 is a comparison diagram of the sand and dust area segmentation mask graph and the sand and dust area segmentation effect in the present application. The sand and dust area segmentation mask graph generated by the method of the present application is clearly shown in the figure, and it can be seen that the boundary and details of the sand and dust area are accurately segmented. Compared with the segmentation effect of the traditional method, the method of the present application can more accurately capture the details of the sand and dust area, avoid misjudgment caused by changes in illumination or noise interference, and has more accurate and stable segmentation effect.

[0130] S6 calculates the dust coverage ratio of the pollution area on the component surface based on the sand and dust pollution mask graph, and divides the pollution level accordingly.

[0131] According to the sand and dust pollution mask graph extracted in step S5, all pixels in the image are traversed point by point, and the number of sand and dust area pixels N dust , i.e. the number of pixel points with a pixel value of 255 in the mask. At the same time, according to the sand and dust pollution mask graph extracted in step S5, all pixels in the image are traversed point by point, and the number of pixels outside the sand and dust area N panel , i.e. the number of pixel points with a pixel value of 0 in the mask. Then the dust coverage ratio (Dust Coverage Ratio, DCR) is calculated as follows:

[0132]

[0133] The ratio reflects the actual shielding ratio of the dust in the component detection area, which is a direct quantitative basis for pollution level evaluation.

[0134] Finally, according to the preset photovoltaic panel pollution level standard (see Table 1), the dust coverage ratio is mapped to the corresponding level label.

[0135] Table 1: Photovoltaic panel pollution level division

[0136] Rank DCR interval Description 0 rank 0％ No pollution 1 rank 0% < DCR < 5% Light pollution, no impact on power generation 2 rank 5% < DCR < 10% Moderate pollution, recommended cleaning 3 rank 10% < DCR < 15% Severe pollution, significant decrease in power generation 4 rank DCR>15% Extremely heavy pollution, should be cleaned and maintained immediately

[0137] It should be noted that in order to verify the effectiveness and superiority of the method of the present application, two representative image segmentation methods are selected as the baseline for comparison experiments: one is the traditional Otsu adaptive threshold method based on gray information, and the other is the U-Net deep learning model with end-to-end semantic segmentation capability. The Otsu adaptive threshold method is based on gray image global segmentation, which automatically determines the threshold value based on the maximum inter-class variance principle, without the need for a training process. The U-Net deep learning segmentation model uses an encoder-decoder architecture, is trained on the same training dataset, uses a cross-entropy loss function, an Adam optimizer (initial learning rate 1e-4), a Batch Size of 1, and is trained for 100 rounds. The model with the best performance on the validation set is selected for testing. The dataset used in all comparison experiments is the 410 images obtained by the present application in the laboratory, which are randomly divided into training, validation and test sets in the ratio of 8:1:1. The image annotation work is done using the Labelme annotation tool to ensure the accuracy and consistency of the dataset. During the training process, all algorithm models used for comparison (including the present application) are not pre-trained.

[0138] IoU (Intersection over Union), Precision (precision), Recall (recall) and F1_score (F1 score) are classical evaluation indicators commonly used in image segmentation tasks, which are used to accurately measure the degree of fit between algorithm segmentation results and true conditions. IoU is the ratio of the intersection to the union between the predicted results and the true results, which can intuitively reflect the proportion of the overlapping area; Precision focuses on the proportion of true positives in the predicted positive results, reflecting the accuracy of the predicted positive results; Recall focuses on the proportion of correctly predicted positives in the true positives, reflecting the ability to capture positive targets; F1_score is the harmonic mean of Precision and Recall, which comprehensively considers the performance of the two and balances the accurate prediction and comprehensive capture ability of positive results. The formulas of the indicators are as follows:

[0139]

[0140] Wherein: TP, FP and FN represent true positive (predicted as positive and actually positive), false positive (predicted as positive but actually negative) and false negative (predicted as negative but actually positive) respectively. IoU is used to measure the similarity of the overlap of prediction and true result, Precision and Recall reflect the recognition effect of positive targets from different dimensions, and F1_score gives a balanced evaluation. The higher the values of IoU and F1_score, the better the performance of the algorithm in the corresponding dimension, and the more accurate and comprehensive the image segmentation task can be achieved.

[0141] The comparative experiment results are shown in Table 2.

[0142] Table 2: Index evaluation results of each method on sand dust area segmentation task (%)

[0143] Method IoU Precision Recall F1_Score Otsu adaptive threshold method 53 53 100 69 U-Net 93 98 95 95 The method of the application 96 99 97 98

[0144] From Table 2, it can be seen that the Otsu threshold method performs poorly in all indicators except Recall, with IoU and Precision only 53%, and F1_Score only reaching 69%, reflecting its insufficient recognition ability in complex background conditions; the U-Net segmentation method is slightly inferior to the method of the present application in all indicators. In contrast, the method of the present application achieves the highest value in the three evaluation indicators of IoU, Precision and F1_Score, which are IoU = 96%, Precision = 99%, F1_Score = 98%, although the performance in the Recall indicator is slightly lower, which is mainly due to the more conservative segmentation strategy under the condition of multi-channel joint, thereby reducing the detection rate of redundant areas, resulting in some real areas being missed, which is actually a more accurate performance of the method of the present application in segmenting dust, further verifying the advantages of the method of the present application in accuracy, robustness and generalization.

[0145] Figure 8 The visual comparison results of the three segmentation methods in the present application on the typical sand dust pollution image are shown in the schematic diagram. Obviously, compared with other baseline methods, the method of the present application is more accurate in perceiving the boundary of the sand dust area and the background details such as the photovoltaic panel, and the final sand dust detection result is highly consistent with the actual distribution. This is due to the feature fusion module based on HSV multi-channel joint segmentation in the present application, which strengthens the sensitivity to the difference between the sand dust edge and the complex background through cross-channel feature cooperation and dynamic threshold adaptation, and accurately distinguishes the sand dust and interference texture.

[0146] A photovoltaic dust identification system based on color space fusion and lightweight learning for the above method, such as Figure 9The system is composed of an image acquisition and component region detection module 1, an ROI extraction and geometric correction module 2, a color space conversion module 3, a channel feature extraction and threshold setting module 4, a mask generation and pollution area identification module 5, and a pollution area statistics and grade division module 6 connected in sequence and connected with a computer terminal respectively; the image acquisition and component region detection module 1 is connected with an industrial camera fixedly installed on an indoor support, or connected with a camera provided on an unmanned aerial vehicle, a patrol robot or a fixed monitoring point.

[0147] The image acquisition and component region detection module 1 is responsible for acquiring the RGB image of the photovoltaic module and performing standardization processing on the image and detecting the photovoltaic module region based on the acquired RGB image, the image type is a standard RGB image, the shooting angle can include vertical shooting, shooting at an inclined angle or lateral shooting to adapt to the image acquisition requirements under different light intensities and installation conditions, and the detection of the photovoltaic module region adopts a YOLOv11 semantic segmentation model trained through transfer learning to locate the photovoltaic module, thereby providing a basis for subsequent ROI extraction and pollution detection.

[0148] The ROI extraction and geometric correction module 2 extracts a region of interest (ROI) based on the detection result of the component region, after detecting the photovoltaic module region, uses a Canny edge detection algorithm to perform edge detection on the image, then uses a minimum circumscribed quadrilateral fitting algorithm to determine the boundary of the component, acquires the coordinates of the four corner points of the component boundary, constructs a homography matrix, performs perspective transformation on the photovoltaic module ROI image, realizes view angle correction and geometric standardization, unifies the arrangement direction of the component, reduces distortion interference, and provides stable input for subsequent color channel analysis.

[0149] The color space conversion module converts the corrected image from the RGB space to the HSV color space using image processing libraries such as OpenCV, and extracts three channel images of H, S and V respectively, which are used for subsequent mask generation and multi-channel fusion, where H∈[0, 180°], S∈[0, 255], and V∈[0, 255].

[0150] The channel feature extraction and threshold setting module 4 performs feature extraction and threshold setting on each color channel (H, S, V) respectively, where the H channel adopts a fixed threshold strategy, sets the hue range to [2°, 40°], the S channel extracts 20 image features based on the basic statistical features, histogram structural features, wavelet energy features, channel coupling features, edge and gradient structural features, and global coupling structural features of the image, then inputs these features into a lightweight machine learning model LightGBM to predict the dynamic threshold interval under the current lighting environment, and the V channel extracts the brightness threshold range through brightness normalization frequency histogram analysis.

[0151] The mask generation and pollution area identification module 5 generates a dust pollution mask image by performing a three-channel joint threshold judgment on each pixel point in the image. The three-channel joint threshold judgment is based on a pixel-level logical AND operation of the H, S, and V three channels. The pixel points that simultaneously satisfy the three threshold ranges are determined as the dust area. In the generated mask image, the pixel value of the dust area is 255, and the rest is the background area with a pixel value of 0. In addition, the module can also optionally perform a morphological filtering operation to optimize the mask boundary.

[0152] The pollution area statistics and grade division module 6 is used to perform area statistics on the dust area in the mask image, calculate the pixel proportion of the dust area in the entire photovoltaic module area as the pollution coverage rate, and output a pollution grade label such as no pollution, light pollution, moderate pollution, heavy pollution, or extremely heavy pollution according to the pollution coverage rate and a preset pollution grade division standard.

Claims

1. A photovoltaic dust identification method based on color space fusion and light learning, comprising the following steps: S1. Obtain image data of a photovoltaic module and detect a module region in the image to obtain a mask image of the module region; S2. Extract an ROI image of the photovoltaic module based on the module mask image and perform geometric correction processing; S3. Convert the ROI image from an RGB color space to an HSV color space to analyze dust region features in hue, saturation, and brightness dimensions; S4. Set thresholds for H, S, and V three-channel images of the HSV color space respectively to identify regions that may be contaminated by dust; S5. Fuse threshold setting results of the H, S, and V channels to generate a final dust contamination mask and complete dust region identification; S6. Calculate a dust coverage ratio of a contaminated region on a module surface based on the dust contamination mask and divide the contaminated region into a contamination level accordingly.

2. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The image data of the photovoltaic module in step S1 is obtained by the following method: collected by an industrial camera fixedly installed on an indoor support in a laboratory environment, or collected by a camera installed on a UAV, a patrol robot, or a fixed monitoring point; the image type is a standard RGB image; the standard RGB image shooting angles include vertical shooting, oblique shooting, and lateral shooting, and the shooting environment covers different time periods and natural light conditions.

3. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The mask image of the module region in step S1 is obtained by the following method: the collected standard RGB image is input into a YOLOv11 semantic segmentation model trained by transfer learning to locate and segment the photovoltaic module in the image to obtain the mask image of the module region; the YOLOv11 semantic segmentation model is based on a pre-trained model and is obtained by transfer learning training using an image dataset obtained by shooting under actual working conditions; the dataset is divided into a training set, a validation set, and a test set according to a ratio of 8:1:

1.

4. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The ROI image of the photovoltaic module in step S2 is extracted by the following method: a Canny edge detection algorithm is used to extract an edge contour of the module region from the module mask image, and a minimum enclosing quadrilateral fitting operation is performed based on the extracted edge contour, and a region surrounded by the minimum enclosing quadrilateral is taken as the ROI image of the photovoltaic module.

5. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The geometric correction in step S2 is processed by the following method: four corner point coordinates of a module boundary are obtained based on the ROI image of the photovoltaic module, a homography matrix is constructed, and a perspective transformation is performed on the ROI image of the photovoltaic module.

6. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The thresholds of the H, S, and V three-channel images of the HSV color space in step S4 are set by the following methods respectively: ① For the H channel of the HSV color space, a threshold setting method based on frequency histogram statistical results of images in the dataset is used to determine that the threshold interval of the H channel is [2°, 40°]; ②For the S channel in the HSV color space, an adaptive threshold setting method based on image-based multi-dimensional statistical features and a LightGBM regression model is adopted; the multi-dimensional image features include 20 kinds of features, including basic statistical features, histogram structure features, wavelet energy features, channel coupling features, edge and gradient structure features, and global coupling structure features; ③For the V channel in the HSV color space, a threshold setting method based on heuristic peak-valley analysis is adopted: ⅰThe normalized frequency histogram of the V channel image is calculated, and Gaussian smoothing is performed; ⅱIdentification of the main peak of dust: identify the main peak position P in the brightness region [140, 250] r ; if no local peak satisfying the significance requirement is detected in the brightness region [140, 250], the position of the maximum of the smoothed histogram H s (v) in this interval is taken as the main peak position; in the P r left region, find the position of the valley between the nearest left peak and the main peak, set as v min , if there is no valid left peak, then fall back to a fixed offset estimation; in the P r right region, find the position where the frequency of the brightness value first falls below 1% of the global maximum value frequency, set as v max , if the condition is not met, set a bottom width; ⅲHeuristic peak-valley analysis and determination of adaptive threshold range.

7. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The sand and dust area recognition method in step S5 refers to a joint judgment of each pixel in the image based on the threshold setting results of each channel in the HSV color space, and a logical "and" operation of the threshold judgment results of the three channels is adopted to realize accurate extraction of the sand and dust area.

8. The photovoltaic dust identification method based on color space fusion and light learning according to claim 7, characterized in that: The sand and dust area recognition method refers to performing threshold judgment on each pixel in the image in the H channel, the S channel and the V channel, if the H channel value of the pixel is within the set hue range [2°, 40°], and the S channel value is within the saturation threshold interval obtained by the LightGBM model prediction, and the V channel value is within the brightness threshold interval obtained by the normalized frequency histogram analysis of the current image, the pixel is marked as a sand and dust area and assigned a value of 255 in the mask image; otherwise, it is marked as a background area and assigned a value of 0 in the mask image.

9. The photovoltaic dust identification method based on color space fusion and light learning according to claim 1, characterized in that: The sand dust coverage ratio of the contaminated area in step S6 on the surface of the component refers to that according to the sand dust contaminated mask diagram extracted in step S5, all pixels in the image are traversed point by point, the number N of sand dust area pixels is counted dust , that is, the number of pixel points with a pixel value of 255 in the mask, and the number N of pixels outside the sand dust area is counted panel , that is, the number of pixel points with a pixel value of 0 in the mask; and then the sand dust coverage ratio is calculated as follows:

10. A color space fusion and light weight learning based photovoltaic dust identification system for the method of any of claims 1 to 8, characterized in that: The system is composed of an image acquisition and component area detection module (1), an ROI extraction and geometric correction module (2), a color space conversion module (3), a channel feature extraction and threshold setting module (4), a mask generation and pollution area recognition module (5), and a pollution area statistics and grade division module (6) connected in sequence and connected with a computer terminal; the image acquisition and component area detection module (1) is connected with an industrial camera fixedly installed on an indoor support, or connected with a camera provided on an unmanned aerial vehicle, a patrol robot or a fixed monitoring point.

Images