An Adaptive Cleaning Method for Photovoltaic Panels Based on Multispectral Image Recognition

Abstract

This invention relates to the field of image recognition technology and discloses an adaptive cleaning method for photovoltaic panels based on multispectral image recognition. The method includes: acquiring visible light reflection images, near-infrared reflection images, and thermal infrared temperature images of the same photovoltaic module with a temporal correspondence; determining the effective area of ​​the module and dividing it into local windows through spatial correspondence processing; generating local periodic response intensity and local attenuation based on the periodic response of cell boundaries and grid lines in the reflection images, and generating a local isotopic temperature rise response by combining it with the thermal infrared temperature image; further forming an isotopic coupling index to characterize the correspondence between the periodic response decrease and temperature rise within the same local window; verifying conventional dirt candidates to obtain dirt labels and dirt levels; and then generating a cleaning task and supporting post-cleaning verification. This invention is used for dirt identification and cleaning maintenance of photovoltaic power plant modules.

Description

Technical Field

[0001] This invention belongs to the field of image recognition technology, specifically relating to an adaptive cleaning method for photovoltaic panels based on multispectral image recognition. Background Technology

[0002] During photovoltaic power plant inspections, maintenance personnel typically use visible light cameras, infrared devices, drones, or tracked inspection devices to acquire images of the module surfaces to identify dust, mud, bird droppings, and shading. In the actual environment, factors such as sunlight angle, cloud shadows, support structure obstruction, glass reflection, module aging, electrical hotspots, and thermal imager calibration discrepancies can all alter image brightness, temperature readings, and local texture. Relying solely on a single image channel, fixed thresholds, or standard classification output can lead to the system misclassifying shadows, reflections, or electrical anomalies as dirt, and may also result in insufficient response to thin dust, edge dirt, and localized patches. For large-scale power plants, misjudgments can lead to ineffective cleaning, missed cleaning, and increased workload for verification, while also impacting the operating window of cleaning equipment, water consumption, and track or robot scheduling. Therefore, the inspection system needs to integrate reflective and temperature images from the same module within the same inspection period and convert the identification results into actionable cleaning tasks. Summary of the Invention

[0003] The present invention aims to provide an adaptive cleaning method for photovoltaic panels based on multispectral image recognition, which solves the technical problems mentioned in the background art.

[0004] To achieve the above-mentioned objectives, the present invention provides the following technical solution.

[0005] An adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition includes: Acquire visible light reflectance images, near-infrared reflectance images, and thermal infrared temperature images of the same photovoltaic module with a time-corresponding relationship; The visible light reflection image, the near-infrared reflection image, and the thermal infrared temperature image are subjected to spatial correspondence processing to make the same photovoltaic module correspond to the same effective area of ​​the module in the three types of images, the effective area of ​​the module is determined, and the effective area of ​​the module is divided into multiple local windows; The period offset is determined based on the inherent orthogonal periodic structure of the photovoltaic module formed by the cell boundaries and grid lines in the effective region of the module, and the local orthogonal periodic structure intensity is obtained based on the period offset. The local structural attenuation is generated based on the comparison between the strength of the local orthogonal periodic structure and the median reference inside the component. Based on the thermal infrared temperature image, a local isotopic temperature rise response corresponding to the same local window is generated; Based on the local structural attenuation amount and the local isotopic temperature rise response within the same local window, a cell orthogonal periodic structure attenuation-temperature rise isotopic coupling characteristic is generated to characterize the spatial isotopic relationship between the two. Based on at least one of the three types of images, conventional dirt candidates are obtained. The structural consistency of the conventional dirt candidates is verified by utilizing the attenuation-temperature rise co-coupling feature of the orthogonal periodic structure of the battery cell to obtain dirt labels. The dirt level is determined based on the dirt markings, and a cleaning task is generated based on the dirt level.

[0006] Compared with the prior art, the present invention has the following substantial features and significant progress: This invention uses visible light reflection images, near-infrared reflection images, and thermal infrared temperature images for the inspection and identification of the same photovoltaic module. It combines the periodic response changes of cell boundaries and grid lines on the module with the temperature rise response at the same location for dirt detection, which can reduce misjudgments of shadows, reflections, and thermal anomalies caused by single brightness, single temperature, or ordinary classification output. At the same time, dirt levels and cleaning tasks are formed based on dirt markings, so that the inspection results can directly serve the scheduling of cleaning equipment. After cleaning, the results are verified by re-acquiring images, thereby improving the availability of dirt identification results in on-site operation and maintenance and the targeting of cleaning triggers. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the functional modules and processing mechanism of the present invention. Detailed Implementation

[0008] The following description is provided in conjunction with the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the invention; those skilled in the art can make equivalent substitutions or combinations for the specific implementations.

[0009] Figure 1 In this process, the photovoltaic module 100 is the object to be detected and cleaned. The multispectral image acquisition module 200, the spatial correspondence processing module 300, the periodic response processing module 400, the temperature rise response processing module 500, the coupling verification module 600, the dirt level determination module 700, and the cleaning task execution and verification module 800 work together in sequence according to the data flow. After cleaning is completed, the module returns to the multispectral image acquisition module 200 through the cleaning task execution and verification module 800 to form a verification closed loop.

[0010] This embodiment is applicable to on-site inspection and cleaning maintenance of crystalline silicon photovoltaic modules. The photovoltaic module 100 typically has a rectangular outer frame and includes regularly arranged rectangular solar cells inside. The boundaries of the solar cells and the grid lines form an approximately orthogonal and approximately periodic structure on the plane of the photovoltaic module 100.

[0011] Figure 1 The photovoltaic module 100 is connected to the multispectral image acquisition module 200 via an image acquisition relationship; the multispectral image acquisition module 200 outputs visible light reflection image, near-infrared reflection image, and thermal infrared temperature image to the spatial correspondence processing module 300; the spatial correspondence processing module 300 outputs the effective area and local window of the module to the periodic response processing module 400 and the temperature rise response processing module 500, respectively; the periodic response processing module 400 outputs the local structural attenuation to the coupling verification module 600; the temperature rise response processing module 500 outputs the local isotopic temperature rise response to the coupling verification module 600; the coupling verification module 600 outputs a dirt mark to the dirt level determination module 700; the dirt level determination module 700 outputs the dirt level to the cleaning task execution and verification module 800; the cleaning task execution and verification module 800 sends back the trigger relationship of the re-acquisition image after cleaning to the multispectral image acquisition module 200.

[0012] Specifically, when relying solely on brightness changes, temperature increases, or ordinary classification output, local shadows, specular reflections, electrical thermal anomalies, and thin layers of dust may exhibit similar grayscale, spectral, or temperature responses in the image, leading to instability in conventional dirt candidates. Therefore, this embodiment simultaneously uses visible light reflectance images, near-infrared reflectance images, and thermal infrared temperature images, and correlates changes in the inherent orthogonal periodic structure of the photovoltaic module formed by cell boundaries and grid lines with temperature changes at the same location.

[0013] The modules involved in this embodiment can be implemented by one or more image processing units, industrial controllers, embedded processors, or servers. The multispectral image acquisition module 200 can correspond to a combination of a visible light camera, a near-infrared camera, and a thermal imager, or it can correspond to a sensor combination mounted on a drone, a track-mounted inspection device, a fixed bracket, or a cleaning robot. The cleaning task execution and verification module 800 can call upon existing cleaning equipment in the power plant, such as roller brushes, scrapers, sprayers, or air blowers, to complete the cleaning action.

[0014] Example 1: See Figure 1 The multispectral image acquisition module 200 acquires visible light reflectance images, near-infrared reflectance images, and thermal infrared temperature images of the same photovoltaic module 100. A time correspondence means that the three types of images correspond to the same photovoltaic module 100, and the acquisition time is sufficient to represent the same inspection status. In one embodiment, the three types of images are acquired through hardware synchronous triggering; in another embodiment, the three types of images are acquired through software timestamp matching.

[0015] Visible light reflectance image is denoted as Near-infrared reflectance image is denoted as The thermal infrared temperature image is denoted as .in, and The image coordinates are represented. The visible light reflection image is used to reflect the reflection state of the surface of the photovoltaic module 100, the cell boundaries, and the grid lines in the visible light band; the near-infrared reflection image is used to reflect the reflection state of the surface of the photovoltaic module 100, the cell boundaries, and the grid lines in the near-infrared band; the thermal infrared temperature image is used to reflect the temperature distribution of the surface of the photovoltaic module 100 or the radiation temperature matrix output by the thermal imager.

[0016] When acquiring three types of images, the multispectral image acquisition module 200 simultaneously records the component number, acquisition time, acquisition location, and camera orientation of the photovoltaic module 100. The component number is used to associate the identification results with the specific photovoltaic module 100 in subsequent steps; the acquisition time is used to determine whether the three types of images satisfy a temporal correspondence; and the acquisition location and camera orientation are used to assist the spatial correspondence processing module 300 in determining the position of the same photovoltaic module 100 in the three types of images.

[0017] To ensure the stability of subsequent processing, the multispectral image acquisition module 200 can remove images that are severely out of focus, overexposed, have missing frames, or contain large areas of occlusion after acquisition. This removal can be achieved through routine sharpness evaluation, brightness range checking, and effective pixel ratio checking. If any of the three types of images of the same photovoltaic module 100 is removed, the photovoltaic module 100 will not be included in the structural consistency verification in this inspection, and a re-acquisition record will be generated.

[0018] In practical applications, if the visible light reflectance image, near-infrared reflectance image, and thermal infrared temperature image have different resolutions, the multispectral image acquisition module 200 does not directly determine dirt from the three types of images. Instead, it sends the three types of images to the spatial correspondence processing module 300 for corresponding processing. The spatial correspondence processing module 300 uses the effective area coordinate system of the component after perspective unfolding as a unified coordinate system. Data from the three types of images are mapped to this unified coordinate system before participating in local window calculations.

[0019] Example 2: The spatial correspondence processing module 300 receives visible light reflectance images, near-infrared reflectance images, and thermal infrared temperature images output by the multispectral image acquisition module 200. The goal of spatial correspondence processing is to ensure that the same photovoltaic module 100 corresponds to the same effective area of ​​the module in the three types of images, and to divide the effective area of ​​the module into multiple local windows.

[0020] In one implementation, the spatial correspondence processing module 300 detects the rectangular boundary of the photovoltaic module 100 in the visible light reflection image. The rectangular boundary can be obtained through conventional edge detection, line detection, corner detection, template matching, or search range constraints based on an existing component ledger. After determining the rectangular boundary of the photovoltaic module 100, the spatial correspondence processing module 300 performs perspective unfolding on the area to be unfolded corresponding to the photovoltaic module 100.

[0021] The area after perspective is unfolded is defined as the effective area of ​​the component, and the background area outside the component's border is removed. The effective area of ​​the component is denoted as... In the coordinate system of the effective area of ​​the component after perspective unfolding, the effective area mask of the component is denoted as... When pixel coordinates When it belongs to the valid area of ​​the component, Select 1 if the value is 1, otherwise select 0.

[0022]

[0023] in, Indicates the valid area of ​​the component. Indicates the effective area mask of the component. This represents the image coordinates after perspective unfolding. The effective area of ​​the component only represents the pixel range and is no longer used to represent the local window statistical domain.

[0024] The spatial mapping processing module 300 maps the effective area of ​​the module to the near-infrared reflection image and the thermal infrared temperature image, so that the visible light reflection image, near-infrared reflection image, and thermal infrared temperature image correspond to the same effective area of ​​the module. This mapping can be achieved through camera calibration parameters, affine transformation, perspective transformation, or registration based on the boundary of the photovoltaic module 100.

[0025] The spatial processing module 300 then divides the effective area of ​​the component into multiple local windows. These local windows are denoted as... The set of local windows within the effective area of ​​a component is denoted as... , is used to represent all valid local windows within the effective area of ​​a component that participate in statistics and judgment.

[0026]

[0027] in, This represents a collection of local windows within the effective area of ​​a component. Indicates the first A local window, Indicates the first The percentage of effective pixels within the component's valid area within a local window. This represents the effective pixel ratio threshold. The effective pixel ratio threshold is used to exclude local windows located at the boundary of the component's effective area that have insufficient effective pixels.

[0028]

[0029] in, Represents a local window The effective pixel ratio, Represents a local window The number of pixels that belong to the valid area of ​​the component. Represents a local window The total number of pixels. The effective pixel ratio threshold can be a conventional value not less than 0.8; when the power plant already has image quality rules requiring a higher ratio, those image quality rules shall prevail.

[0030] The size of the local window is determined based on the period offset. If the period offset is... and The lengths of the local window in the two orthogonal directions are denoted as follows: and The local window size satisfies the following formula to ensure that each effective local window at least covers an area that can reflect periodic repetition.

[0031]

[0032] in, and These represent local windows. The length in two orthogonal directions and These represent the pixel lengths of the two period offsets. The above sizing rules utilize only the period offsets already read from the inherent orthogonal periodic structure of the photovoltaic module.

[0033] Example 3: The periodic response processing module 400 receives the effective region of the module and a set of local windows within the effective region from the spatial correspondence processing module 300, and determines the period offset based on the inherent orthogonal periodic structure of the photovoltaic module formed by the cell boundaries and grid lines within the effective region of the module. The period offset is read from the periodic structure of the photovoltaic module 100 itself in the visible light reflection image.

[0034] In one embodiment, the periodic response processing module 400 performs two-dimensional autocorrelation processing on the visible light reflection image within the effective area of ​​the module to obtain similarity responses along different directions within the effective area of ​​the module. Since the cell boundaries and grid lines form an approximately orthogonal and approximately periodic structure, a principal periodic response will appear along the two orthogonal directions after the two-dimensional autocorrelation processing.

[0035] In another implementation, the periodic response processing module 400 performs spectral processing on the visible light reflection image within the effective area of ​​the module, extracts the dominant frequency components corresponding to two orthogonal directions, and then determines the period offset based on the spatial period corresponding to the dominant frequency components. Both two-dimensional autocorrelation processing and spectral processing are conventional image processing methods, and their function is to read the period offset from the inherent orthogonal periodic structure of the photovoltaic module.

[0036] Periodic offset includes those obtained along two orthogonal directions and When selecting the period offset, the period response processing module 400 excludes responses at zero offset and selects the non-zero main period response with the largest amplitude in both the long side and short side directions of the module. When multiple candidate main period responses exist in the same direction, the candidate main period response that corresponds to the boundary of adjacent cells or the grid spacing and has the most repetitions within the effective area of ​​the module is selected.

[0037] To avoid mistaking localized dirt or shading for period shift, the period response processing module 400 statistically analyzes the main period response of the entire or larger area within the effective region of the module, rather than using only the response of a single local window. If localized dirt exists within the effective region of the module, it is typically an irregular patch, thin layer, or strip, and will not stably exhibit the same orthogonal period as the cell boundaries and grid lines throughout the entire effective region of the module. Therefore, it will not replace the inherent orthogonal periodic structure of the photovoltaic module as the main source of period shift.

[0038] Example 4: After determining the period offset, the periodic response processing module 400 reads the window-normalized autocorrelation values ​​corresponding to the period offset from the visible light reflectance image and the near-infrared reflectance image within each local window, and determines the intensity of the local orthogonal periodic structure based on the window-normalized autocorrelation values. This step corresponds to... Figure 1 The mid-space corresponding processing module 300 outputs the data stream of the component's effective area and local window to the periodic response processing module 400.

[0039] For bands The window-normalized autocorrelation value is denoted as .in, , Indicates the wavelength band of the visible light reflection image. Indicates the spectral band of the near-infrared reflectance image. Indicates a partial window. This indicates the periodic offset. The window-normalized autocorrelation value is used to reflect the degree of structural repetition at the periodic offset within the local window.

[0040]

[0041] in, Indicates band Window-normalized autocorrelation value, Represents a local window offset according to period Translation of the local window The set of overlapping pixels between them Represents the pixel coordinates in the overlapping pixel set. Indicates band In pixel coordinates Image values ​​at that location, Indicates band Image value at the offset pixel coordinates express The mean value within the overlapping pixel set express The mean value within the set of overlapping pixels.

[0042] When the denominator of the normalized autocorrelation formula for the window is zero, the periodic response processing module 400 sets the corresponding normalized autocorrelation value of the window to 0 and records the local window as a low-structure-information local window. This processing avoids invalid values ​​caused by a zero denominator, and at the same time avoids mistaking a local window lacking structural information for one with a high local orthogonal periodic structure strength.

[0043]

[0044] in, Indicates band The intensity of local orthogonal periodic structures, Represents a local window In period offset The window-normalized autocorrelation value at that location, Represents a local window In period offset The normalized autocorrelation value at the window location. This formula, by averaging the normalized autocorrelation values ​​of the window in two orthogonal directions, represents the visibility of the inherent orthogonal periodic structure of the photovoltaic module formed by the cell boundaries and grid lines within a local window.

[0045] When the cell boundaries and grid lines are clearly visible in a local window, the window-normalized autocorrelation value is higher at the period offset, and the local orthogonal periodic structure strength is correspondingly higher. When the local window is covered by surface dirt, the local visibility of the cell boundaries and grid lines below in the visible light reflectance and near-infrared reflectance images decreases, the window-normalized autocorrelation value decreases at the period offset, and the local orthogonal periodic structure strength decreases accordingly.

[0046] Example 5: The periodic response processing module 400 establishes an internal median reference for the intensity of local orthogonal periodic structures within the effective area of ​​the module, and compares the intensity of local orthogonal periodic structures in each local window with the internal median reference to obtain the amount of local structure attenuation that represents the decrease in visibility of cell boundaries and grid lines within the corresponding local window.

[0047]

[0048] in, Indicates band The internal median reference of the component. This represents a collection of local windows within the effective area of ​​a component. This refers to a local window within the set of local windows that are active within the component's region. Represents a local window The intensity of local orthogonal periodic structures. Using a set of local windows within the component's effective region for statistical analysis avoids mixing the pixel range of the component's effective region with the statistical domain of the local windows.

[0049]

[0050] in, Indicates the amount of local structural attenuation. This represents the intensity of local orthogonal periodic structures obtained from visible light reflectance images. This represents the intensity of local orthogonal periodic structures obtained from near-infrared reflectance images. This represents the internal median reference of the component corresponding to the visible light reflection image. This represents the internal median reference of the component corresponding to the near-infrared reflectance image. This represents the normalized numerical stability constant of the structure. The normalized numerical stability constant of the structure is a dimensionless positive number, and its order of magnitude is smaller than that of the intensity of a normal locally orthogonal periodic structure, in order to avoid the denominator being zero.

[0051] The structural normalized numerical stability constant is used only for calculations related to local structural attenuation and dimensionless characteristic normalization, and not for denominators with temperature dimensions. By this limitation, local structural attenuation is considered dimensionless.

[0052] When a local window maintains normal structural visibility and Approaching zero, the local structural attenuation is near zero. When local windows are covered by surface contaminants, causing a decrease in the visibility of cell boundaries and grid lines,... Less than The local structural attenuation increases. When the structural response within a local window is higher than the median reference within the component, the local structural attenuation can be negative.

[0053] Example 6: The temperature rise response processing module 500 receives the effective region of the component and a set of local windows within the effective region from the spatial correspondence processing module 300, and generates a local isotopic temperature rise response by combining it with the thermal infrared temperature image. This step corresponds to... Figure 1 The intermediate space corresponding processing module 300 outputs the data stream of the component's effective area and local window to the temperature rise response processing module 500.

[0054]

[0055] in, This represents the median temperature of the effective region of the component. This represents a collection of local windows within the effective area of ​​a component. This refers to a local window within the set of local windows that are active within the component's region. Represents a local window The average or median temperature in the thermal infrared temperature image. The median temperature of the effective area of ​​the module is obtained from the current acquisition status of the same photovoltaic module 100, and can adapt to different ambient temperatures, different sunshine conditions and different inspection times.

[0056]

[0057] in, This indicates the absolute deviation of the median temperature. This indicates the calculation of the absolute median deviation. Represents a local window The average or median temperature in a thermal infrared temperature image. The median absolute deviation of the temperature uses the same temperature units as the thermal infrared temperature image.

[0058]

[0059] in, This indicates the local isotopic temperature rise response. Represents a local window The average or median temperature in a thermal infrared temperature image. This represents the median temperature of the effective region of the component. This indicates the absolute deviation of the median temperature. This represents the normalized numerical stability constant for temperature. The normalized numerical stability constant for temperature uses the same temperature unit as the thermal infrared temperature image and is used only in the temperature normalization denominator, not in the dimensionless characteristic denominator.

[0060] Local isotopic temperature rise response through The positive temperature rise relative to the median temperature of the effective region of the module is retained, while the temperature difference below the median temperature of the effective region of the module is not considered as a fouling enhancement factor. Since both the numerator and denominator are in the dimension of temperature, the local isotopic temperature rise response is a dimensionless quantity.

[0061] The temperature rise response processing module 500 outputs the local isotopic temperature rise response to the coupling verification module 600. The local isotopic temperature rise response itself is not directly equivalent to dirt, because electrical thermal anomalies may also cause temperature rise; this result needs to be used together with the local structural attenuation amount within the same local window to form the attenuation-temperature rise isotopic coupling characteristics of the orthogonal periodic structure of the solar cell.

[0062] Example 7: The coupling verification module 600 receives the local structural attenuation amount output by the periodic response processing module 400 and the local isotopic temperature rise response output by the temperature rise response processing module 500. Based on the local structural attenuation amount and local isotopic temperature rise response within the same local window, the coupling verification module 600 generates a cell orthogonal periodic structural attenuation-temperature rise isotopic coupling characteristic to characterize the spatial isotopic relationship between the two.

[0063]

[0064] in, This indicates the in-situ coupling characteristics of the degradation-temperature rise in the orthogonal periodic structure of the solar cell. Indicates the amount of local structural attenuation. This represents the local isotopic temperature rise response. The local structural attenuation is dimensionless, and the local isotopic temperature rise response is also dimensionless. Therefore, the attenuation-temperature rise isotopic coupling characteristic of the orthogonal periodic structure of the solar cell is dimensionless.

[0065] The attenuation-temperature rise in-situ coupling characteristic of the orthogonal periodic structure of a solar cell requires that the local structural attenuation and the local in-situ temperature rise response correspond to the same local window. If a local window only has a temperature rise but insufficient local structural attenuation, the attenuation-temperature rise in-situ coupling characteristic of the orthogonal periodic structure of the solar cell will not increase significantly; similarly, if a local window only has a brightness change but insufficient local in-situ temperature rise response, the attenuation-temperature rise in-situ coupling characteristic of the orthogonal periodic structure of the solar cell will not increase significantly.

[0066] This design corresponds to the imaging mechanism of dirt on the surface of the photovoltaic module 100. The dirt is located on the outer surface of the cover glass of the photovoltaic module 100. Its irregular patches, thin layers or stripes can reduce the local visibility of the underlying cell boundaries and grid lines in visible light reflection images and near-infrared reflection images. At the same time, it may change the absorption and heat dissipation conditions, causing a relative temperature rise in the same location in the thermal infrared temperature image.

[0067] Example 8: The coupled verification module 600 obtains conventional dirt candidates based on at least one of three image types. These conventional dirt candidates can be obtained through threshold segmentation, difference detection after brightness normalization, spectral difference calculation, texture features, or the output of a general classifier. These methods are all conventional image processing or machine learning techniques, and their purpose is to provide candidate regions to be verified.

[0068] The confidence level of the conventional dirt candidate is denoted as The confidence level for routine dirt indicates the effectiveness of routine treatment methods on localized areas. The initial assessment strength of a region belonging to dirt. The confidence level of regular dirt can be obtained from threshold distance, classifier probability, or rule scoring, but in subsequent structural consistency verification, it is uniformly referred to as the confidence level of regular dirt.

[0069] Structural consistency verification uses a high-confidence threshold, a low-confidence threshold, a characteristic threshold, a structural attenuation threshold, a periodic structure retention threshold, and a temperature rise response threshold. The high-confidence threshold is denoted as... The low confidence threshold is denoted as The feature threshold is denoted as The structural attenuation threshold is denoted as The periodic structure preservation threshold is denoted as The temperature rise response threshold is denoted as .

[0070] The high-confidence threshold, low-confidence threshold, and feature threshold were determined through validation using conventionally labeled samples. These samples included local windows labeled as dirt, local windows labeled as non-dirt thermal anomaly candidates, and local windows labeled as shadow or reflection candidates. The low-confidence and high-confidence thresholds met the following criteria: In the calibration sample set, the high-confidence threshold and low-confidence threshold are determined by the sample distribution of the confidence level of conventional dirt, ensuring that the false positive rate of conventional dirt candidates does not exceed the preset upper limit of the false positive rate and the false negative rate does not exceed the preset upper limit of the false negative rate. The feature threshold is selected from the sample distribution of the attenuation-temperature rise isotopic coupling characteristics of the orthogonal periodic structure of the solar cell, ensuring that the false positive rate of fuzzy conventional dirt candidates after structural consistency verification does not exceed the preset upper limit of the false positive rate and the false negative rate does not exceed the preset upper limit of the false negative rate. The high-confidence threshold, low-confidence threshold, and feature threshold are used consistently in the same power plant, the same type of photovoltaic module 100, or the same inspection batch.

[0071]

[0072] in, Indicates dirt marking, Represents a local window It was identified as dirt. Represents a local window Not identified as dirt. For regular dirt candidates with a confidence level higher than the high confidence threshold, the coupling verification module 600 generates a dirt label; for regular dirt candidates with a confidence level between the low confidence threshold and the high confidence threshold, a dirt label is generated when the decay-temperature rise isotopic coupling feature of the orthogonal periodic structure of the solar cell reaches the feature threshold.

[0073] The condition for local structural attenuation is defined as follows: When a conventional fouling candidate exhibits a local isotopic temperature rise response and satisfies... However, it was not satisfied. At this time, the coupling verification module 600 marks the regular fouling candidate as a non-fouling thermal anomaly candidate. This process is used to distinguish areas where there is only a temperature rise but the visibility of cell boundaries and grid lines does not decrease accordingly.

[0074] The preset periodic structure strength condition is defined as the periodic structure retention ratio of the local window not being lower than the periodic structure retention threshold. The periodic structure retention ratio is denoted as... .

[0075]

[0076] in, This indicates that the periodic structure maintains a ratio. and Represents a local window The intensity of local orthogonal periodic structures, and Indicates the median reference within the component. This represents the structure-normalized numerical stability constant.

[0077] When a regular dirt candidate has a brightness change and meets the following conditions When a candidate for conventional dirt meets the preset periodic structure strength condition, the coupling verification module 600 marks the candidate for conventional dirt as a shadow candidate or a reflective candidate. This process is used to distinguish regions where only brightness changes but the inherent orthogonal periodic structure of the photovoltaic module is still maintained.

[0078] When the same local window simultaneously meets multiple anomaly labeling conditions, the processing priority is as follows: first, perform image quality removal; after the image quality is qualified, first determine whether the confidence level of regular dirt is higher than the high confidence threshold; if it is not higher than the high confidence threshold but belongs to the fuzzy regular dirt candidate, perform the battery cell orthogonal periodic structure decay-temperature rise co-position coupling feature verification; if no dirt label is generated, then determine the non-dirt thermal anomaly candidate; finally, determine the shadow candidate or reflection candidate.

[0079] Example 9: The dirt level determination module 700 receives the dirt markers output by the coupling verification module 600 and determines the dirt level based on the dirt markers. This step corresponds to... Figure 1 The intermediate coupling verification module 600 outputs a data stream of dirt markings to the dirt level determination module 700.

[0080] In one implementation, the fouling level determination module 700 first normalizes the decay-temperature rise in-situ coupling characteristics of the orthogonal periodic structure of the battery cell to obtain normalized coupling characteristics. The normalized coupling characteristics are denoted as... The reference scale for coupling feature normalization is denoted as... The 95th percentile value is determined by the 95th percentile of the attenuation-temperature rise isotopic coupling characteristic of the orthogonal periodic structure of the cells within the local window set of the effective area of ​​the same photovoltaic module 100. When determining the 95th percentile value, the dirt level determination module 700 arranges the attenuation-temperature rise isotopic coupling characteristics of the orthogonal periodic structure of the cells within the local window set of the effective area of ​​the same photovoltaic module 100 in ascending order of value, and takes the value located at the 95th percentile as the reference scale for normalizing the coupling characteristic; when the 95th percentile falls between two adjacent ranked values, the larger ranked value is taken as the 95th percentile value. The selection rule for the 95th percentile value remains unchanged within the same power plant, the same model of photovoltaic module 100, or the same inspection batch; when the 95th percentile value is zero, the maximum value of the attenuation-temperature rise isotopic coupling characteristic of the orthogonal periodic structure of the cells within the photovoltaic module 100 is used; when the maximum value is still zero, the normalized coupling characteristic is set to zero.

[0081]

[0082] in, This represents the normalized coupling characteristics. This indicates the in-situ coupling characteristics of the degradation-temperature rise in the orthogonal periodic structure of the solar cell. This represents the reference scale for normalizing coupling features. This represents the normalized numerical stability constant for coupling characteristics. The normalized numerical stability constant for coupling characteristics is a dimensionless positive number and is used only for the dimensionless normalization of the degradation-temperature rise co-location coupling characteristics of orthogonal periodic structures in solar cells.

[0083] The fouling severity determination module 700 determines the feature-weighted fouling severity based on fouling labels and normalized coupled features. The feature-weighted fouling severity is used to give higher weight to fouling regions that simultaneously exhibit significant local structural attenuation and local isotopic temperature rise response in the severity statistics.

[0084]

[0085] in, Indicates the first Characteristic weighted dirt level of 100 photovoltaic modules. Indicates the first The set of local windows within the effective area of ​​the photovoltaic module 100. Indicates the first The number of effective windows for a 100-cell photovoltaic module. Indicates dirt marking, This represents the normalized coupling characteristics. This represents the weight. The weight is a dimensionless, non-negative number, and its value satisfies the following conditions: When setting weights, the dirt level determination module 700... Within a certain range, the value that maximizes the consistency rate of the verification samples is selected; if multiple values ​​correspond to the same consistency rate, the smallest value is selected. The consistency rate represents the proportion of samples whose dirt level, determined by feature-weighted dirt degree and dirt coverage, is the same as the manually confirmed dirt level. The determined weights are used consistently within the same power plant, the same type of photovoltaic module 100, or the same inspection batch.

[0086]

[0087] in, Indicates the first The dirt coverage of a 100-cell photovoltaic module. Indicates dirt marking, Indicates the first The set of local windows within the effective area of ​​the photovoltaic module 100. Indicates the first The effective window count of a 100-cell photovoltaic module. Fouling coverage reflects the extent of the fouling area, while feature-weighted fouling severity reflects both the extent of the fouling area and the severity under the combined effect of normalized coupling features.

[0088] The fouling level is determined based on a combination of feature-weighted fouling severity and fouling coverage. Let the threshold values ​​for low and high levels corresponding to fouling coverage be respectively... and The low-level and high-level thresholds corresponding to the feature-weighted dirtiness are respectively... and And satisfy as well as The low-level threshold, high-level threshold, low-level threshold, and high-level threshold corresponding to the feature-weighted fouling degree are pre-stored by the power plant operation and maintenance rules, and the level consistency rate is ensured to meet the preset level consistency rate requirements through regular verification samples. These thresholds are used consistently within the same power plant, the same type of photovoltaic module (100), or the same inspection batch. When the fouling coverage rate and feature-weighted fouling degree yield different levels, the higher of the two corresponding levels is taken as the fouling level.

[0089] The cleaning task execution and verification module 800 receives the dirt level output by the dirt level determination module 700 and generates a cleaning task based on the dirt level. The cleaning task may include the photovoltaic module 100's module number, module location, dirt level, cleaning sequence, and preset cleaning mode. The sequencing, path planning, and specific cleaning actions of the cleaning tasks can be implemented by calling the conventional functions of existing cleaning equipment in the power plant.

[0090] Example 10: After generating a cleaning task based on the level of dirt, the cleaning task execution and verification module 800 performs the cleaning according to the task. During cleaning, the cleaning equipment can perform roller brushing, scraping, spraying, or air blowing according to a preset cleaning mode, and record the process status, cleaning completion mark, and fault information. This step is a downstream execution link of the recognition result, and the image recognition processing logic of this embodiment is not limited to specific cleaning organizations.

[0091] After cleaning is completed, the cleaning task execution and verification module 800 triggers the multispectral image acquisition module 200 to reacquire visible light reflection image, near-infrared reflection image and thermal infrared temperature image.

[0092] During the post-cleaning verification process, the spatial correspondence processing module 300 re-executes the spatial correspondence processing, the periodic response processing module 400 redetermines the periodic offset, the local orthogonal periodic structure strength, and the local structure attenuation, the temperature rise response processing module 500 regenerates the local isotopic temperature rise response, the coupling verification module 600 regenerates the attenuation-temperature rise isotopic coupling characteristics of the cell orthogonal periodic structure and re-executes the structural consistency verification, and the dirt level determination module 700 redetermines the dirt level.

[0093] The preset cleaning judgment conditions are pre-stored by the power plant operation and maintenance rules and are used consistently within the same power plant, the same type of photovoltaic module 100, or the same inspection batch. The preset cleaning judgment conditions include: the dirt level still reaching the cleaning trigger level; the dirt coverage rate still reaching the cleaning trigger threshold; the feature-weighted dirt degree still reaching the cleaning trigger threshold; or the dirt level reduction before and after cleaning is insufficient. The cleaning trigger level, the cleaning trigger threshold corresponding to the dirt coverage rate, the cleaning trigger threshold corresponding to the feature-weighted dirt degree, and the judgment range for insufficient dirt level reduction before and after cleaning are all read before the cleaning task is generated. When any of the above conditions is met, the cleaning task execution and review module 800 generates a prompt for re-cleaning or a manual review prompt; when none of the above conditions are met, the cleaning task execution and review module 800 saves the cleaning effect record.

[0094] The "Clean Again" prompt triggers the cleaning equipment to perform cleaning again, while the "Manual Review" prompts maintenance personnel to check for potentially stubborn dirt, equipment malfunctions, or abnormal heat sources. Cleaning effectiveness records can include dirt markings before and after cleaning, dirt coverage, feature-weighted dirt intensity, dirt level, and cleaning task execution results.

[0095] For example, at the start of a complete inspection cycle, the system reads the arrangement information, module number, allowable cleaning time, and inspection path of the photovoltaic modules 100 in the power station, and activates the multispectral image acquisition module 200. After the photovoltaic modules 100 enter the field of view, the multispectral image acquisition module 200 acquires visible light reflection images, near-infrared reflection images, and thermal infrared temperature images of the same photovoltaic module 100 with a time correspondence.

[0096] Subsequently, the spatial correspondence processing module 300 performs spatial correspondence processing on the visible light reflection image, near-infrared reflection image, and thermal infrared temperature image. The spatial correspondence processing module 300 detects the rectangular boundary of the photovoltaic module 100 in the visible light reflection image, performs perspective unfolding on the area to be unfolded corresponding to the photovoltaic module 100, determines the unfolded area as the effective area of ​​the module, and maps the effective area of ​​the module to the near-infrared reflection image and the thermal infrared temperature image.

[0097] The spatial correspondence processing module 300 divides the effective region of the component into multiple local windows, forming a set of local windows within the effective region of the component. The periodic response processing module 400 performs two-dimensional autocorrelation processing or spectral processing on the visible light reflectance image within the effective region of the component to determine the period shift. The periodic response processing module 400 further reads the window-normalized autocorrelation values ​​corresponding to the period shift in the visible light reflectance image and the near-infrared reflectance image within each local window, and determines the intensity of the local orthogonal periodic structure.

[0098] The periodic response processing module 400 establishes an internal median reference within a set of local windows in the effective region of the component, and compares the local orthogonal periodic structure strength of each local window with the internal median reference to obtain the local structure attenuation. The temperature rise response processing module 500 determines the median temperature and median absolute temperature deviation of the effective region of the component based on the thermal infrared temperature image, and generates a local isotopic temperature rise response based on the difference between the temperature of each local window and the median temperature of the effective region of the component, as well as the median absolute temperature deviation.

[0099] The coupling verification module 600 receives the local structural attenuation and local isotopic temperature rise response, and generates attenuation-temperature rise isotopic coupling features of the orthogonal periodic structure of the solar cell. Simultaneously, the coupling verification module 600 obtains conventional fouling candidates based on at least one of three types of images, and uses the attenuation-temperature rise isotopic coupling features of the orthogonal periodic structure of the solar cell to perform structural consistency verification on the conventional fouling candidates.

[0100] Local windows that pass the structural consistency check are marked with dirt tags. Local windows that fail the structural consistency check but show temperature rise can be marked as non-dirt thermal anomaly candidates. Local windows that show brightness changes and meet preset periodic structural strength conditions can be marked as shadow candidates or reflection candidates. The dirt level determination module 700 determines the feature-weighted dirt degree based on dirt tags and normalized coupled features, determines the dirt coverage rate based on the proportion of dirt tags in the effective area of ​​the component, and determines the dirt level based on the feature-weighted dirt degree and the dirt coverage rate.

[0101] The cleaning task execution and verification module 800 generates a cleaning task based on the level of dirt and performs cleaning according to the task. After cleaning is completed, the cleaning task execution and verification module 800 triggers the multispectral image acquisition module 200 to reacquire the three types of images and repeats the above processing flow to complete the post-cleaning verification.

[0102] As an optional implementation, the surface of the photovoltaic module 100 is free of obvious dirt, and the cell boundaries and grid lines remain stably visible in both visible light and near-infrared reflectance images. The period offset determined by the period response processing module 400 corresponds to the inherent orthogonal periodic structure of the photovoltaic module, with the local orthogonal periodic structure strength close to the median reference within the module, and the local structure attenuation close to zero or negative. The local isotopic temperature rise response obtained by the temperature rise response processing module 500 is also at a low level. The cell orthogonal periodic structure attenuation-temperature rise isotopic coupling characteristics generated by the coupling verification module 600 are insufficient to support the generation of dirt markers from fuzzy conventional dirt candidates, and the dirt level determined by the dirt level determination module 700 does not trigger a cleaning task.

[0103] As an optional implementation, the photovoltaic module 100 has a thin layer of dust or mottled dirt on its surface. Since the thin layer of dust reduces the local visibility of the underlying cell boundaries and grid lines in visible and near-infrared reflectance images, the periodic response processing module 400 obtains a higher local structural attenuation within the corresponding local window. Simultaneously, due to changes in absorption and heat dissipation conditions, the corresponding local window exhibits a local isotopic temperature rise response relative to the median temperature of the effective region of the module in the thermal infrared temperature image. The coupling verification module 600 obtains the cell orthogonal periodic structure attenuation-temperature rise isotopic coupling characteristics that reach a feature threshold within the same local window, thereby generating dirt labels for conventional dirt candidates falling between the low-confidence threshold and the high-confidence threshold.

[0104] As an optional implementation, the surface or surrounding structure of the photovoltaic module 100 causes changes in the brightness of certain local windows, and conventional processing methods may yield conventional dirt candidates. However, if the cell boundaries and grid lines within this local window still maintain a periodic structure in both the visible light reflection image and the near-infrared reflection image, and the periodic structure retention ratio is not lower than the periodic structure retention threshold, then the conventional dirt candidate satisfies a preset periodic structure intensity condition. The coupling verification module 600 marks this conventional dirt candidate as a shadow candidate or a reflective candidate, without generating a dirt label.

[0105] As an optional implementation, if a local window in the photovoltaic module 100 shows a temperature rise in the thermal infrared temperature image, conventional processing methods may yield conventional fouling candidates. If the local isotopic temperature rise response of this local window is not lower than the temperature rise response threshold, but the local structural attenuation does not reach the structural attenuation threshold, it indicates that the area exhibits a temperature rise but not a decrease in the visibility of the corresponding periodic structure. The coupling verification module 600 marks this conventional fouling candidate as a non-fouling thermal anomaly candidate to prompt subsequent maintenance attention, but not as a direct trigger for fouling cleaning.

[0106] This implementation can be executed by a processor installed in an inspection device, cleaning robot, edge computing device, or power plant server. The processor can read program instructions from the memory to complete the functions of the multispectral image acquisition module 200, spatial correspondence processing module 300, periodic response processing module 400, temperature rise response processing module 500, coupling verification module 600, dirt level determination module 700, and cleaning task execution and verification module 800.

[0107] Visible light reflection images, near-infrared reflection images, and thermal infrared temperature images can be stored as image matrices or temperature matrices. The effective region of a component can be stored as a mask or coordinate range. Local windows can be stored as window indices and coordinate boundaries. The set of local windows within the effective region of a component can be stored using window indices and coordinate boundaries.

[0108] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. An adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition, characterized in that, include: Acquire visible light reflectance images, near-infrared reflectance images, and thermal infrared temperature images of the same photovoltaic module with a time-corresponding relationship; The visible light reflection image, the near-infrared reflection image, and the thermal infrared temperature image are subjected to spatial correspondence processing to make the same photovoltaic module correspond to the same effective area of ​​the module in the three types of images, the effective area of ​​the module is determined, and the effective area of ​​the module is divided into multiple local windows; The period offset is determined based on the inherent orthogonal periodic structure of the photovoltaic module formed by the cell boundaries and grid lines in the effective region of the module, and the local orthogonal periodic structure intensity is obtained based on the period offset. The local structural attenuation is generated based on the comparison between the strength of the local orthogonal periodic structure and the median reference inside the component. Based on the thermal infrared temperature image, a local isotopic temperature rise response corresponding to the same local window is generated; Based on the local structural attenuation amount and the local isotopic temperature rise response within the same local window, a cell orthogonal periodic structure attenuation-temperature rise isotopic coupling characteristic is generated to characterize the spatial isotopic relationship between the two. Based on at least one of the three types of images, conventional dirt candidates are obtained. The structural consistency of the conventional dirt candidates is verified by utilizing the attenuation-temperature rise co-coupling feature of the orthogonal periodic structure of the battery cell to obtain dirt labels. The dirt level is determined based on the dirt markings, and a cleaning task is generated based on the dirt level.

2. The adaptive cleaning method for photovoltaic panels based on multispectral image recognition according to claim 1, characterized in that, The spatial correspondence processing includes: The rectangular boundary of the photovoltaic module is detected in the visible light reflection image. The area to be unfolded corresponding to the photovoltaic module is unfolded through perspective. The unfolded area is determined as the effective area of ​​the module. The effective area of ​​the module is mapped to the near-infrared reflection image and the thermal infrared temperature image, so that the visible light reflection image, the near-infrared reflection image and the thermal infrared temperature image correspond to the same effective area of ​​the module.

3. The adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition according to claim 1, characterized in that, Determining the period offset includes: Two-dimensional autocorrelation processing or spectral processing is performed on the visible light reflection image within the effective area of ​​the component to select the principal periodic response that appears along two orthogonal directions, and the period offset is determined based on the principal periodic response so that the period offset originates from the inherent orthogonal periodic structure of the photovoltaic module.

4. The adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition according to claim 1, characterized in that, Obtaining the strength of the local orthogonal periodic structure includes: Within each local window, the window-normalized autocorrelation values ​​of the visible light reflection image and the near-infrared reflection image corresponding to the periodic offset are read, and the intensity of the local orthogonal periodic structure is determined based on the window-normalized autocorrelation values.

5. The adaptive cleaning method for photovoltaic panels based on multispectral image recognition according to claim 1, characterized in that, Generating the local structural attenuation includes: An internal median reference for the intensity of the local orthogonal periodic structure is established within the effective region of the component, and the intensity of the local orthogonal periodic structure of each local window is compared with the internal median reference to obtain the amount of local structure attenuation representing the decrease in visibility of cell boundaries and grid lines within the corresponding local window.

6. The adaptive cleaning method for photovoltaic panels based on multispectral image recognition according to claim 1, characterized in that, Generating the local isotopic temperature rise response includes: Based on the thermal infrared temperature image, the median temperature and median absolute temperature deviation of the effective region of the component are determined, and the local isotopic temperature rise response is generated according to the difference between the temperature of each local window and the median temperature of the effective region of the component, as well as the median absolute temperature deviation.

7. The adaptive cleaning method for photovoltaic panels based on multispectral image recognition according to claim 1, characterized in that, The structural consistency verification of the conventional fouling candidates using the attenuation-temperature rise in-situ coupling characteristics of the orthogonal periodic structure of the solar cell includes: Obtain the confidence level of the conventional dirt candidates; For conventional dirt candidates whose confidence level is higher than the high confidence threshold, the dirt label is generated; For conventional dirt candidates whose confidence level is between the low confidence threshold and the high confidence threshold, the dirt marker is generated when the decay-temperature rise isotopic coupling feature of the orthogonal periodic structure of the solar cell reaches the feature threshold.

8. The adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition according to claim 7, characterized in that, The structural consistency check also includes: The conventional fouling candidate that has the local isotopic temperature rise response but does not meet the local structural attenuation condition required to generate the fouling label is labeled as a non-fouling thermal anomaly candidate. The conventional dirt candidates that exhibit brightness variations and meet the preset periodic structure strength condition are marked as shadow candidates or reflective candidates. The preset periodic structure strength condition is that the intensity of the local orthogonal periodic structure has not decreased to the level required to generate the dirt marker.

9. The adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition according to claim 1, characterized in that, Determining the level of dirt includes: The attenuation-temperature rise co-location characteristics of the orthogonal periodic structure of the battery cell are normalized to obtain normalized coupling characteristics. The feature-weighted dirt level is determined based on the dirt markers and the normalized coupling features; The dirt coverage rate is determined based on the proportion of the dirt markers within the effective area of ​​the component; The dirt level is determined based on the weighted dirt level of the aforementioned characteristics and the dirt coverage.

10. The adaptive cleaning method for photovoltaic panel dirt based on multispectral image recognition according to claim 1, characterized in that, After generating a cleaning task based on the aforementioned level of dirt, the process also includes: Perform cleaning according to the cleaning task described; After cleaning is completed, the visible light reflection image, the near-infrared reflection image, and the thermal infrared temperature image are reacquired. Regenerate the attenuation-temperature rise co-coupling characteristics of the orthogonal periodic structure of the battery cell, re-execute the structural consistency check, and redetermine the fouling level; When the redefined level of dirt meets the preset cleaning criteria, a prompt for further cleaning or a prompt for manual review is generated.

Images