A method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging

Abstract

This invention discloses a real-time evaluation method for photovoltaic panel defects and performance based on multispectral imaging, belonging to the field of computer processing technology. The method includes the following steps: S01, emitting multispectral light including visible light, near-infrared, and thermal infrared bands to the photovoltaic panel, and simultaneously receiving reflection and radiation signals based on the multispectral light to obtain a multispectral image dataset; wherein, the dataset includes visible light images, photoluminescence images in the near-infrared band, and thermal infrared images; S02, performing registration and fusion processing on the multispectral image dataset to generate a registered multispectral fused image. This invention, through multispectral information fusion and intelligent analysis, achieves simultaneous diagnosis of internal and external defects of photovoltaic panels and quantitative evaluation of power generation performance, significantly improving detection efficiency and maintenance accuracy.

Description

Technical Field

[0001] This invention relates to the field of computer processing technology, and in particular to a method for real-time evaluation of defects and performance of photovoltaic panels based on multispectral imaging. Background Technology

[0002] As the global energy structure shifts towards a green and low-carbon model, solar photovoltaic (PV) power generation, as a clean and renewable energy source, has seen its installed capacity grow rapidly. Ensuring the long-term stable operation and efficient power generation of large-scale PV power plants has become a focus of industry attention. As the core energy conversion unit of a PV power generation system, the health of the PV panel directly determines the power generation efficiency, safety, and return on investment of the entire power plant. However, during manufacturing, transportation, installation, and operation for over 25 years, PV panels are inevitably subject to mechanical stress, environmental corrosion, and electrical stress, leading to various defects.

[0003] These defects can be mainly classified into the following categories: 1) Manufacturing and structural defects, such as microcracks in solar cells, broken grids, and poor welding; 2) Performance degradation defects, such as potential-induced degradation (PID) and snail trails; 3) Operational malfunction defects, such as hot spots and bypass diode failures. These defects not only cause an immediate decrease in the output power of photovoltaic panels, but some defects (such as severe hot spots) can even cause fires, posing a serious threat to the safety of power plants.

[0004] Currently, the detection of defects and performance of photovoltaic panels mainly relies on the following traditional technologies: Electroluminescence (EL) inspection: This technology requires applying a forward bias voltage to the photovoltaic panel under darkroom conditions to make it emit near-infrared light, and then capturing the emission image with a high-sensitivity camera. Defect areas emit less light due to the increased number of recombination centers, appearing as dark areas in the image. EL inspection is very sensitive to internal defects such as microcracks and broken grids, but it requires shutdown and power-off operation, making it impossible to perform online inspections of operating power plants. Furthermore, the equipment is expensive and the inspection efficiency is extremely low, making it unsuitable for rapid inspections of large-scale power plants.

[0005] Infrared thermal imaging (IRT) inspection: This technology uses a thermal imager to non-contactly measure the temperature distribution of photovoltaic panels during operation. Defective areas (such as hot spots) generate abnormal heat due to increased local series resistance, appearing as bright spots in the thermal image. IRT can achieve online inspection, but its detection effectiveness is easily affected by external factors such as ambient temperature, wind speed, and light intensity. Furthermore, it is not sensitive to defects that do not produce significant thermal effects (such as certain PID defects or reduced parallel resistance), making it difficult to comprehensively assess all defect types.

[0006] IV Characteristic Curve Test: By measuring the current-voltage characteristic curve of a photovoltaic panel, key performance parameters such as maximum output power, fill factor, open-circuit voltage, and short-circuit current can be accurately calculated. This method can accurately assess the overall electrical performance of a photovoltaic panel, but it cannot locate the specific defects and their types. Furthermore, the testing process is cumbersome and time-consuming, making it impossible to achieve rapid, large-scale surveys.

[0007] Visual inspection: Relying on the experience of maintenance personnel to observe the surface of photovoltaic panels, it can only detect obvious macroscopic defects such as damage, discoloration, and dust accumulation. It is powerless to detect internal and microscopic defects, is highly subjective, and has a high rate of missed detection.

[0008] In summary, existing detection technologies have significant limitations: some, like EL, require offline shutdown and cannot be performed in real time; some, like IRT, provide only a single dimension of information and cannot be comprehensive; and some, like IV curve testing, cannot pinpoint locations and cannot be precise. All of these limitations make it difficult to meet the urgent needs of modern large-scale photovoltaic power plants for efficient, comprehensive, online, automated, and quantitative operation and maintenance.

[0009] In recent years, multispectral imaging technology has been widely used in agriculture, environmental monitoring, and military fields due to its ability to simultaneously acquire image information of targets in multiple spectral bands. This technology, by integrating information from different physical mechanisms, makes it possible to achieve more comprehensive condition assessments. A few researchers have begun to attempt to combine multispectral or visible light with thermal infrared light for photovoltaic detection, but existing methods mostly remain at the stage of simple image fusion and manual interpretation, or are only used for qualitative defect identification, lacking a complete real-time analysis system capable of achieving automatic defect identification and quantitative performance assessment. In particular, it has failed to fully utilize the photoluminescence characteristics in the near-infrared band to detect internal defects, and has also failed to establish a deep correlation model between multispectral features and electrical performance parameters. Summary of the Invention

[0010] To address the aforementioned technical problems, the present invention provides a method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging, the method comprising the following steps: S01. Emit multispectral light including visible light, near-infrared light and thermal infrared light to the photovoltaic panel, and simultaneously receive reflection and radiation signals based on the multispectral light to obtain a multispectral image dataset. The dataset includes visible light images, photoluminescence images in the near-infrared band, and thermal infrared images; S02. Perform registration and fusion processing on the multispectral image dataset to generate a registered multispectral fused image; S03. Analyze the multispectral fusion image using a pre-trained qualification recognition neural network to determine whether the image belongs to a qualified photovoltaic panel image. If qualified, proceed to the next step. S04. The qualified photovoltaic panel image is identified using a target detection neural network, and the first region corresponding to the defective part and the second region corresponding to the performance evaluation part are located respectively. S05. Based on the first region and the second region, extract image features and calculate the defect and performance difference index; S06. Generate a visualization report containing the multispectral fusion image, the labeling information of the first region and the second region, and the defect and performance difference indicators.

[0011] Preferably, the target detection neural network includes a first sub-neural network and a second sub-neural network connected in parallel. The first sub-neural network processes the photoluminescence image to identify defect features and outputs the first region, while the second sub-neural network processes the visible light image or thermal infrared image to identify performance-related regions and outputs the second region.

[0012] Preferably, in step S04, when the target detection neural network is used to identify the qualified photovoltaic panel image, the first region and the second region are marked with a bright outline, and the numerical value and type description of the defects and performance differences are dynamically displayed next to the image or in an overlay information box. Simultaneously, temperature distribution is rendered in the thermal infrared image region using color mapping, and abnormal light intensity intervals are marked in the photoluminescence image region.

[0013] As a preferred option, it also includes: Extract the first attribute information of the first region and the second attribute information of the second region respectively. The first attribute information and the second attribute information include at least one of the following: region area, average gray level, average photoluminescence intensity, extreme value of thermal infrared temperature, and texture feature parameters. The first attribute information and the second attribute information are compared with a preset threshold to generate and display the attribute comparison analysis results.

[0014] Preferably, the selection of the first region and the second region satisfies at least one of the following conditions: The first region and the second region are located within the same photovoltaic panel cell unit; There are no interfering features caused by shadows, occlusions, or reflections in the first and second regions; The image grayscale distribution in the first region and the second region is uniform, and the grayscale variance of both regions is less than a set threshold. The first region appears as a low-luminescence-intensity region in the photoluminescence image, while the second region appears as a region with intact structure or normal temperature in the visible light or thermal infrared image.

[0015] Preferably, the uniformity of the image grayscale distribution is determined by calculating whether the variance of the pixel grayscale values ​​within the region is less than a preset threshold; or... By analyzing the pixel grayscale histogram within the region, it can be determined whether the main distribution is concentrated within the preset grayscale range. The preset threshold or grayscale range is obtained based on statistics from a large number of normal photovoltaic panel images.

[0016] Preferably, the defects and performance differences include the photoluminescence intensity ratio and predicted electrical performance parameters, including: Calculate the average first photoluminescence intensity of the first region in the near-infrared band and the average second photoluminescence intensity of the second region in the near-infrared band; The photoluminescence intensity ratio is obtained based on the ratio of the first photoluminescence intensity to the second photoluminescence intensity. The photoluminescence intensity ratio and the temperature features extracted from the thermal infrared image are input into a pre-trained electrical performance correlation model, which outputs the predicted electrical performance parameters of the photovoltaic panel. The electrical performance parameters include at least one of the following: fill factor, series resistance, and maximum output power.

[0017] Preferably, the first photoluminescence intensity and the second photoluminescence intensity are the arithmetic mean or median value of the photoluminescence intensity of all pixels in the corresponding region, respectively. The electrical performance correlation model is a regression model based on neural networks or support vector machines. It is trained using historical data and can establish a nonlinear mapping relationship between multispectral features and measured electrical parameters.

[0018] The present invention has at least the following beneficial effects: By integrating multispectral information from near-infrared photoluminescence, visible light, and thermal infrared, the limitations of traditional single detection methods are overcome. Near-infrared photoluminescence is extremely sensitive to "invisible" defects such as internal microcracks and PID (partial detachment defects), while visible light and thermal infrared are adept at identifying surface anomalies and hot spot faults, respectively. This enables the system to simultaneously and comprehensively diagnose internal and external defects of photovoltaic panels, significantly improving the depth and breadth of detection.

[0019] An automated analysis pipeline was constructed by employing a parallel neural network for qualification recognition and a neural network for target detection. This pipeline can quickly screen qualified components and accurately locate defects and performance evaluation areas, greatly reducing the time and subjective errors of manual interpretation. This makes it ideal for integration with mobile platforms such as drones to achieve efficient, real-time inspection of large-scale photovoltaic power plants, effectively reducing operation and maintenance costs.

[0020] By calculating quantitative indicators such as photoluminescence intensity ratio and utilizing a pre-trained electrical performance correlation model, key electrical parameters such as fill factor and maximum power can be directly predicted. This enables maintenance personnel to intuitively and quantitatively grasp the specific impact of defects on power generation performance, providing direct and reliable data support for precise maintenance, asset assessment, and power generation prediction, thus achieving a significant value enhancement. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging, provided in Embodiment 1 of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0025] Example 1

[0026] This embodiment provides a method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging. The method includes the following steps: Figure 1 As shown: S01. Emit multispectral light including visible light, near-infrared light and thermal infrared light to the photovoltaic panel, and simultaneously receive the reflection and radiation signals based on the multispectral light to obtain a multispectral image dataset. The dataset includes visible light images, photoluminescence images in the near-infrared band, and thermal infrared images.

[0027] Specifically, multispectral imaging technology is used to simultaneously acquire image data of photovoltaic panels in three key bands: visible light, near-infrared photoluminescence, and thermal infrared. This constructs an information set that comprehensively reflects the multidimensional state of the panel, encompassing its structural integrity, internal carrier characteristics, and thermal performance. The scientific significance lies in the fact that images in a single band can only reveal one aspect of the photovoltaic panel's characteristics, exhibiting inherent blind spots. For example, visible light images are adept at capturing surface anomalies such as fragments, discoloration, and dust accumulation; near-infrared photoluminescence images are extremely sensitive to defects within the silicon wafer, such as microcracks, impurities, and potential-induced decay (PID), which exacerbate carrier recombination, as these defects significantly quench photoluminescence intensity; while thermal infrared images can directly locate localized overheating areas (hot spots) caused by increased series resistance or bypass diode failure. By simultaneously acquiring images of these three complementary physical mechanisms, an indispensable data foundation is laid for achieving comprehensive and accurate diagnosis from the outside in, and from structure to function.

[0028] In practical operation, the "light" emitted by the system to the photovoltaic panel consists of two parts: firstly, using sunlight or a standard light source as the excitation source for visible light and thermal infrared radiation; secondly, to obtain high-quality daytime photoluminescence signals, a near-infrared laser or LED array with a specific wavelength (commonly 808nm or 830nm) is usually required as the excitation source. During imaging, the visible light camera captures the reflected light from the surface of the photovoltaic panel; the near-infrared camera, under the aforementioned excitation light, specifically captures the longer-wavelength fluorescence (i.e., photoluminescence) generated after the silicon material is excited; and the thermal infrared camera directly receives the thermal radiation generated by the photovoltaic panel itself due to heat generation during operation. To achieve pixel-level precise correspondence of the three spectral bands, the key lies in "synchronization" technology. This requires hardware trigger signals to ensure that the three cameras are exposed at the same time, supplemented by precise mechanical structure design or post-processing image registration algorithms to eliminate positional deviations caused by different viewing angles and optical paths, ultimately forming a spatially strictly aligned multispectral image dataset.

[0029] S02. Perform registration and fusion processing on the multispectral image dataset to generate a registered multispectral fused image.

[0030] Specifically, image data from different spectral bands (visible light, near-infrared, and thermal infrared) that exhibit spatial deviations are integrated into a unified image that is precisely aligned at the pixel level and provides complementary information. The fundamental significance of this step lies in solving the inherent "information silo" problem in multi-sensor imaging. Because different cameras are physically installed independently, even with synchronous triggering, images of the same scene acquired by them will have subtle differences in viewpoint, position, and rotation angle. Without registration, directly analyzing the same "position" from images from different cameras is essentially comparing different physical points, which can lead to feature extraction errors, distorted regional contrast, and ultimately render defect identification and performance evaluation inaccurate.

[0031] Specifically, ORB (Oriented Fast and Rotated BRIEF) or SIFT (Scale-Invariant Feature Transform) feature points are extracted from the information-rich visible light image, and corresponding feature points are found in the near-infrared and thermal infrared images. Then, an optimal perspective transformation matrix (homography matrix) is robustly estimated using the RANSAC (Random Sample Consensus) algorithm. This matrix is ​​then used to perform geometric transformations (such as rotation, scaling, and translation) on the near-infrared and thermal infrared images, achieving sub-pixel-level precise spatial alignment with the visible light reference image. Following this, in the image fusion stage, a weighted fusion or principal component analysis (PCA) fusion strategy is applied to the registered multi-channel data. For example, a fusion vector [V, N, T] can be generated for each pixel, where V, N, and T represent the intensity or temperature value of that point in the visible light, near-infrared, and thermal infrared images, respectively. This processing not only generates standardized inputs that can be directly processed by subsequent neural networks, but more importantly, it links the visible surface features (V), the luminescence information (N) reflecting the recombination state of internal charge carriers, and the thermal distribution (T) indicating energy loss in the same coordinate frame, laying a unique and real data foundation for subsequent comprehensive analysis of the morphology, causes, and impact on electrical performance of defects.

[0032] S03. Analyze the multispectral fusion image using a pre-trained qualification recognition neural network to determine whether the image belongs to a qualified photovoltaic panel image. If qualified, proceed to the next step of processing.

[0033] Specifically, a large-scale, high-quality training dataset is constructed. This dataset should contain tens of thousands of multispectral fusion images of photovoltaic panels, manually labeled by experts into two categories: "qualified" and "unqualified." "Qualified" samples refer to images that are clear, interference-free, and can be used to accurately determine the health status of photovoltaic panels; while "unqualified" samples need to broadly cover various aforementioned anomalies, such as blurriness, overexposure, shadow occlusion, and foreign object intrusion, to ensure that the model can fully learn the characteristics of various quality defects. In terms of model selection, convolutional neural networks (such as EfficientNet, ResNet, or MobileNet optimized for mobile devices) pre-trained on large datasets like ImageNet are typically used as the basic architecture, leveraging their powerful feature extraction capabilities. Transfer learning is performed on specific datasets, adjusting and retraining the fully connected layers at the network's ends to adapt it to binary classification tasks. Data augmentation techniques (such as random rotation, brightness adjustment, and adding simulated noise) are used during training to improve the model's robustness. Ultimately, the deployed model can perform forward propagation on the input image and output a confidence score between 0 and 1. By setting a reasonable threshold (e.g., 0.85), it can automatically determine whether the image is "qualified" and allowed to proceed to the next stage, or determine whether it is "unqualified" and execute a preset strategy (such as triggering a re-shoot or marking it for manual review). This achieves intelligent gatekeeping of the data stream, ensuring the efficient and reliable operation of the entire evaluation system.

[0034] S04. Use a target detection neural network to identify qualified photovoltaic panel images and locate the first region corresponding to the defective part and the second region corresponding to the performance evaluation part, respectively. Furthermore, the target detection neural network includes a first sub-neural network and a second sub-neural network connected in parallel. The first sub-neural network is specifically designed to process photoluminescence images to identify defect features and output a first region. The second sub-neural network processes visible light images or thermal infrared images to identify performance-related regions and output a second region.

[0035] Furthermore, in the specific processing steps: Extract the first attribute information of the first region and the second attribute information of the second region respectively. The first attribute information and the second attribute information include at least one of the following: region area, average gray level, average photoluminescence intensity, extreme value of thermal infrared temperature, and texture feature parameters. The first attribute information and the second attribute information are compared with preset thresholds to generate attribute comparison analysis results and display them.

[0036] Specifically, the architecture employing a parallel first and second sub-neural network aims to fully utilize the unique physical characteristics of different spectral data sources to achieve precise identification tailored to specific needs. Near-infrared photoluminescence images can keenly reflect the internal recombination characteristics of semiconductor materials, exhibiting extremely high detection sensitivity for internal defects such as microcracks and potential-induced decay (PID) that do not involve significant temperature rises or surface deformation. Therefore, a first sub-neural network specifically designed for processing these images (such as a semantic segmentation network based on U-Net) can be trained to accurately capture subtle anomalies in luminescence intensity, thereby precisely outputting the defect region (first region). Visible light images contain rich surface texture and structural information, while thermal infrared images directly characterize the operating temperature field of the component; together, they constitute an ideal reference for evaluating the "health" status of the component. Therefore, the second sub-neural network (such as an object detection network based on YOLO or Faster R-CNN) is responsible for identifying structurally intact, temperature-normal regions from these modalities as a benchmark for performance evaluation (second region). This parallel and division-of-labor architecture avoids feature interference that may occur when different modal information is fused in the early stages, enabling the two sub-networks to achieve optimal performance in their respective areas of expertise, and ultimately laying a solid foundation for subsequent quantitative comparative analysis through region pairing.

[0037] Finally, multiple attribute information from the first and second regions is extracted and thresholds are compared to achieve quantitative diagnosis and objective severity classification of defects. This step is the core of moving qualitative identification towards quantitative assessment. Simply locating the defect area is insufficient; the question of "how severe is this defect?" must also be answered. By extracting attributes such as area, average grayscale, average photoluminescence intensity, extreme thermal infrared temperature, and texture features from the two regions respectively, the system can characterize the defect from multiple dimensions, including geometric, spectral, and thermal aspects. Subsequently, these attributes are compared with preset thresholds based on extensive historical data and experimental verification, automatically generating objective comparative analysis results. For example, when the area of ​​the first region is greater than threshold A and its average photoluminescence intensity is lower than threshold B, the system can automatically classify the defect as a "severe microcrack"; when the extreme temperature exceeds threshold C, it is marked as a "high-risk hotspot." This automated and quantitative analysis process effectively eliminates the subjectivity and inconsistency of manual interpretation, providing a reliable data-driven basis for generating accurate maintenance recommendations and optimizing power plant operation and maintenance strategies.

[0038] S05. Based on the first and second regions, extract image features and calculate defect and performance difference indicators; Furthermore, the selection of the first and second regions must satisfy at least one of the following conditions: The first and second regions are located within the same photovoltaic panel cell unit; There are no interfering features caused by shadows, occlusions, or reflections in the first and second regions; The image grayscale distribution in the first and second regions is uniform, and the grayscale variance of both regions is less than the set threshold. The first region appears as a low-luminescence-intensity area in the photoluminescence image, while the second region appears as a structurally intact or temperature-normal area in the visible light or thermal infrared image.

[0039] It should be noted that the uniformity of image grayscale distribution mentioned above is determined by calculating whether the variance of pixel grayscale values ​​within a region is less than a preset threshold; or, By analyzing the pixel grayscale histogram within the region, it can be determined whether the main distribution is concentrated within the preset grayscale range. The preset threshold or grayscale range is obtained based on statistics from a large number of normal photovoltaic panel images.

[0040] Secondly, defects and performance differences include the photoluminescence intensity ratio and predicted electrical performance parameters, including: Calculate the average first photoluminescence intensity of the first region in the near-infrared band and the average second photoluminescence intensity of the second region in the near-infrared band; The photoluminescence intensity ratio is obtained based on the ratio of the first photoluminescence intensity to the second photoluminescence intensity. The photoluminescence intensity ratio and temperature features extracted from thermal infrared images are input into a pre-trained electrical performance correlation model, which outputs predicted electrical performance parameters of the photovoltaic panel. The electrical performance parameters include at least one of the following: fill factor, series resistance, and maximum output power.

[0041] Furthermore, the first photoluminescence intensity and the second photoluminescence intensity are respectively the arithmetic mean or median value of the photoluminescence intensity of all pixels in the corresponding region; The electrical performance correlation model is a regression model based on neural networks or support vector machines. It is trained using historical data and can establish a nonlinear mapping relationship between multispectral features and measured electrical parameters.

[0042] Specifically, the two regions are located within the same photovoltaic panel cell. This arrangement aims to eliminate interference caused by inherent performance differences between different cells. A photovoltaic panel consists of multiple cells connected in series or parallel. Due to subtle differences in manufacturing processes, even on the same photovoltaic panel, the initial electrical performance and optical characteristics of different cells may fluctuate normally. If the defective region and the reference region are taken from different cells, the observed differences are likely to stem partly from these inherent inter-cell differences rather than purely from the defect, leading to distorted evaluation. To achieve this, during the image recognition stage, image segmentation algorithms (such as edge detection or thresholding methods) are used to identify the boundaries of individual cells, ensuring that the subsequently selected regions are strictly confined to the same cell.

[0043] Secondly, the requirement that the region be free of interfering features such as shadows, occlusions, or reflections is crucial for ensuring the authenticity of the acquired image signal. Shadows (such as clouds, building obstructions, or occlusions between components) significantly alter local lighting conditions, affecting photoluminescence intensity and apparent temperature; while reflections may obscure true surface features or generate false defect signals. These interferences introduce substantial measurement errors. In practice, multispectral comprehensive analysis is needed to identify interference: for example, a dark area appearing in a visible light image, if it does not show a corresponding temperature rise in a thermal infrared image and has a uniform photoluminescence intensity distribution in a near-infrared image, can be identified as a shadow rather than a defect. The algorithm automatically eliminates candidate regions containing such interfering features.

[0044] The requirement for uniform grayscale distribution in an image aims to ensure that the selected region is a typical and representative homogeneous area. A region with uniform grayscale distribution typically indicates that the material is consistent, the structure is intact, and there are no other types of defects or anomalies. This ensures that the extracted feature values ​​(such as average photoluminescence intensity) can stably represent the characteristics of the region. The implementation methods include two quantitative criteria: first, calculating the statistical variance, i.e., calculating the variance of the grayscale values ​​of all pixels within the candidate region and comparing it with a preset threshold obtained based on a large number of normal samples (e.g., the 95th percentile of the grayscale variance in normal regions); values ​​less than the threshold are considered uniform. Second, analyzing the grayscale histogram to observe whether its distribution is concentrated within a narrow, preset grayscale range. Both methods are objective and quantifiable standards, avoiding the arbitrariness of subjective judgment.

[0045] Specifically, in calculating the defect and performance difference index, using the photoluminescence intensity ratio as the core indicator has clear physical significance. Photoluminescence intensity is directly related to the minority carrier lifetime in semiconductor materials. The presence of defects acts as recombination centers, significantly reducing carrier lifetime and thus weakening photoluminescence intensity. Therefore, the photoluminescence intensity ratio (R_pl) of the defect region (first region) to the normal region (second region) can directly and sensitively reflect the degree of material performance degradation caused by defects. In implementation, the arithmetic mean or median of the photoluminescence intensity of all pixels in both regions is first calculated (the median is not sensitive to outliers and is more robust in the presence of noise), and then the ratio is calculated.

[0046] However, optical indicators alone cannot directly assess the impact on power generation. Therefore, a key advancement in this approach is to input both optical and thermal indicators (temperature features extracted from thermal infrared images, such as maximum temperature difference) into a pre-trained electrical performance correlation model. This model (e.g., a regression model based on neural networks or support vector machines) is trained using historical data to establish a nonlinear mapping relationship from "observable multispectral features" to "electrical performance parameters that are difficult to measure directly" (such as fill factor, series resistance, and maximum output power). For example, the training data may contain tens of thousands of samples, each including the multispectral features of a photovoltaic panel and its measured electrical parameters under standard test conditions. After learning this complex relationship, the model can quickly predict key electrical parameters on-site based solely on multispectral images, thus achieving a leap from qualitative judgment of "whether there are defects" to quantitative assessment of "how much performance is lost," providing the most direct and valuable data support for operation and maintenance decisions.

[0047] S06. Generate a visualization report containing a multispectral fusion image, labeling information for the first and second regions, and indicators of defects and performance differences.

[0048] Specifically, in the above embodiment, when using a target detection neural network to identify qualified photovoltaic panel images, the first and second regions are marked with a bright outline, and the numerical values ​​and type descriptions of defects and performance differences are dynamically displayed next to the image or in an overlaid information box. Simultaneously, temperature distribution is rendered in the thermal infrared image region using color mapping, and abnormal light intensity intervals are marked in the photoluminescence image region.

[0049] The introduction of visualization technologies such as highlighted outline marking, dynamic information boxes, and color mapping during the identification process is significant in transforming complex multispectral data and algorithm analysis results into intuitive and easily understandable decision-making information. The aim of this design is to greatly improve the system's human-computer interaction efficiency and the speed of on-site interpretation by maintenance personnel. By marking the first and second regions on the fused image with highlighted outlines (e.g., red for defects, green for normal), maintenance personnel can instantly pinpoint the problem location. Dynamically displayed numerical values ​​and type descriptions provide immediate quantitative diagnostic conclusions without the need for manual interpretation of the original image data. Simultaneously, color mapping is used to render temperature distribution in the thermal infrared region (e.g., a gradient from blue to red indicates low to high temperatures), and abnormal light intensity ranges are marked on the photoluminescence image (e.g., warm and cool hues represent light intensity). Essentially, this transforms invisible or difficult-to-observe physical quantities (temperature, carrier recombination rate) into visually discernible image features.

[0050] Example 2

[0051] This invention provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the following steps: Multispectral light, including visible light, near-infrared light, and thermal infrared light, is emitted into a photovoltaic panel, and reflection and radiation signals based on the multispectral light are received simultaneously to obtain a multispectral image dataset. The dataset includes visible light images, photoluminescence images in the near-infrared band, and thermal infrared images; The multispectral image dataset is registered and fused to generate a registered multispectral fused image. The pre-trained qualification recognition neural network is used to analyze the multispectral fusion image to determine whether the image belongs to the qualified photovoltaic panel image. If it is qualified, it proceeds to the next step of processing. The target detection neural network is used to identify qualified photovoltaic panel images, and the first region corresponding to the defect location and the second region corresponding to the performance evaluation location are located respectively. Based on the first and second regions, image features are extracted and defect and performance difference indices are calculated. Generates a visualization report that includes a multispectral fusion image, labeling information for the first and second regions, and indicators of defects and performance differences.

[0052] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0053] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0054] Example 3

[0055] This invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program segment, and the at least one instruction or the at least one program segment is loaded and executed by the processor to implement the following steps: Multispectral light, including visible light, near-infrared light, and thermal infrared light, is emitted into a photovoltaic panel, and reflection and radiation signals based on the multispectral light are received simultaneously to obtain a multispectral image dataset. The dataset includes visible light images, photoluminescence images in the near-infrared band, and thermal infrared images; The multispectral image dataset is registered and fused to generate a registered multispectral fused image. The pre-trained qualification recognition neural network is used to analyze the multispectral fusion image to determine whether the image belongs to the qualified photovoltaic panel image. If it is qualified, it proceeds to the next step of processing. The target detection neural network is used to identify qualified photovoltaic panel images, and the first region corresponding to the defect location and the second region corresponding to the performance evaluation location are located respectively. Based on the first and second regions, image features are extracted and defect and performance difference indices are calculated. Generates a visualization report that includes a multispectral fusion image, labeling information for the first and second regions, and indicators of defects and performance differences.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging, characterized in that, The method includes the following steps: S01. Emit multispectral light including visible light, near-infrared light and thermal infrared light to the photovoltaic panel, and simultaneously receive reflection and radiation signals based on the multispectral light to obtain a multispectral image dataset. The dataset includes visible light images, photoluminescence images in the near-infrared band, and thermal infrared images; S02. Perform registration and fusion processing on the multispectral image dataset to generate a registered multispectral fused image; S03. Analyze the multispectral fusion image using a pre-trained qualification recognition neural network to determine whether the image belongs to a qualified photovoltaic panel image. If qualified, proceed to the next step. S04. The qualified photovoltaic panel image is identified using a target detection neural network, and the first region corresponding to the defective part and the second region corresponding to the performance evaluation part are located respectively. S05. Based on the first region and the second region, extract image features and calculate the defect and performance difference index; S06. Generate a visualization report containing the multispectral fusion image, the labeling information of the first region and the second region, and the defect and performance difference indicators.

2. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 1, characterized in that, The target detection neural network includes a first sub-neural network and a second sub-neural network connected in parallel. The first sub-neural network processes the photoluminescence image to identify defect features and outputs the first region. The second sub-neural network processes the visible light image or thermal infrared image to identify performance-related regions and outputs the second region.

3. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 1, characterized in that, In step S04, when the target detection neural network is used to identify the qualified photovoltaic panel image, the first region and the second region are marked with a bright outline, and the numerical value and type description of the defects and performance differences are dynamically displayed next to the image or in an overlaid information box. Simultaneously, temperature distribution is rendered in the thermal infrared image region using color mapping, and abnormal light intensity intervals are marked in the photoluminescence image region.

4. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 1, characterized in that, Also includes: Extract the first attribute information of the first region and the second attribute information of the second region respectively. The first attribute information and the second attribute information include at least one of the following: region area, average gray level, average photoluminescence intensity, extreme value of thermal infrared temperature, and texture feature parameters. The first attribute information and the second attribute information are compared with a preset threshold to generate and display the attribute comparison analysis results.

5. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 1, characterized in that, The selection of the first region and the second region satisfies at least one of the following conditions: The first region and the second region are located within the same photovoltaic panel cell unit; There are no interfering features caused by shadows, occlusions, or reflections in the first and second regions; The image grayscale distribution in the first region and the second region is uniform, and the grayscale variance of both regions is less than a set threshold. The first region appears as a low-luminescence-intensity region in the photoluminescence image, while the second region appears as a region with intact structure or normal temperature in the visible light or thermal infrared image.

6. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 5, characterized in that, The uniformity of image grayscale distribution is determined by calculating whether the variance of pixel grayscale values ​​within a region is less than a preset threshold. or, By analyzing the pixel grayscale histogram within the region, it can be determined whether the main distribution is concentrated within the preset grayscale range. The preset threshold or grayscale range is obtained based on statistics from a large number of normal photovoltaic panel images.

7. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 1, characterized in that, The defects and performance differences include the photoluminescence intensity ratio and predicted electrical performance parameters, including: Calculate the average first photoluminescence intensity of the first region in the near-infrared band and the average second photoluminescence intensity of the second region in the near-infrared band; The photoluminescence intensity ratio is obtained based on the ratio of the first photoluminescence intensity to the second photoluminescence intensity. The photoluminescence intensity ratio and the temperature features extracted from the thermal infrared image are input into a pre-trained electrical performance correlation model, which outputs the predicted electrical performance parameters of the photovoltaic panel. The electrical performance parameters include at least one of the following: fill factor, series resistance, and maximum output power.

8. The method for real-time evaluation of photovoltaic panel defects and performance based on multispectral imaging according to claim 7, characterized in that, The first photoluminescence intensity and the second photoluminescence intensity are respectively the arithmetic mean or median value of the photoluminescence intensity of all pixels in the corresponding region; The electrical performance correlation model is a regression model based on neural networks or support vector machines. It is trained using historical data and can establish a nonlinear mapping relationship between multispectral features and measured electrical parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the visual localization method based on local variance and posterior probability classifier as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the visual localization method based on a local variance and posterior probability classifier as described in any one of claims 1 to 8.

Images