Photovoltaic panel fault detection method and system based on power generation comparison and large model

Abstract

This invention proposes a photovoltaic (PV) panel fault detection method and system based on power generation comparison and a large-scale model. It relates to the fields of PV power generation technology and intelligent operation and maintenance technology, addressing the problems of existing technologies, such as incomplete fault identification using single detection methods, low accuracy, insufficient real-time performance, difficulty in achieving early warning and precise fault location, and inability to effectively distinguish different fault types. This method collects multi-source data from PV panel arrays, including power generation data, electrical parameters, meteorological environmental data, optical image data, and infrared image data. Based on the comparison of power generation of various PV panel arrays under the same operating conditions, suspected faulty PV panel arrays are selected. The optical image data, infrared image data, and electrical parameters corresponding to the suspected faulty PV panel arrays are input into a pre-trained multimodal large-scale model for fault diagnosis, obtaining the fault diagnosis results. This invention solves the problems of existing technologies, achieving accurate detection and diagnosis of PV faults.

Description

Technical Field

[0001] This invention belongs to the fields of photovoltaic power generation technology and intelligent operation and maintenance technology, and in particular to a photovoltaic panel fault detection method and system based on power generation comparison and large model. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Driven by the "dual carbon" goals, photovoltaic (PV) power generation has developed rapidly and achieved large-scale grid connection, with power plant installed capacity continuing to grow. As the core component of a PV system, the reliability of PV panels directly affects power generation efficiency and operational safety. Due to long-term exposure to the complex outdoor environment, PV panels are susceptible to environmental influences, aging, and damage, leading to defects such as hot spots, microcracks, breakage, grid breaks, diode / junction box failures, and PID effects. These defects not only reduce power generation efficiency but can also, in severe cases, cause fires, resulting in significant losses and safety hazards.

[0004] Currently, mainstream photovoltaic (PV) panel fault detection technologies can be divided into three categories: The first category is based on electrical parameters, which infers faults by monitoring changes in parameters such as current, voltage, and power. This method has low deployment costs and can perform continuous monitoring, but it can only determine whether a fault has occurred, not accurately pinpoint the fault type and location. It is also susceptible to environmental interference and has weak ability to identify latent faults. The second category is based on image recognition, which uses visible light, infrared, or electroluminescent images to identify physical defects on the PV panel surface through computer vision. This allows for intuitive fault location, but it relies on manual or drone data collection, resulting in high costs, long cycles, poor real-time performance, and an inability to identify latent electrical faults. It is also prone to false detections due to shading. The third category is based on data models, which determines faults by comparing actual power generation with predicted values. This enables automated continuous monitoring, but it is highly dependent on the accuracy of the prediction model, easily affected by environmental factors, and cannot distinguish fault types, making it difficult to provide accurate guidance for operation and maintenance.

[0005] While multimodal large language model methods integrate multi-source data, they require model inference on all components, resulting in high computational redundancy and poor real-time performance. In addition, existing methods are mostly post-event detections, lacking early warning; single detection methods are difficult to integrate multi-source data, resulting in low identification rates for complex and latent faults; and most systems adopt a centralized cloud processing mode with high latency, which cannot meet the refined, low-cost, and high-real-time operation and maintenance needs of large-scale power plants. Summary of the Invention

[0006] To overcome the shortcomings of the existing technologies, this invention provides a photovoltaic panel fault detection method and system based on power generation comparison and a large model. By using power generation comparison for initial fault screening and a large model for precise detection through multimodal data fusion, the method significantly reduces model computational redundancy and system operating costs while ensuring detection accuracy. This solves the problems of incomplete identification, low accuracy, insufficient real-time performance, and inability to accurately locate and distinguish fault types in existing photovoltaic panel fault detection technologies.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: The first aspect of this invention provides a photovoltaic panel fault detection method based on power generation comparison and a large model, comprising: Collect multi-source data from photovoltaic panels, including power generation data, electrical parameters, meteorological environmental data, optical image data, and infrared image data. Based on multi-source data, and by comparing the power generation of each photovoltaic panel group under the same operating conditions, suspected faulty photovoltaic panel groups were screened out. The optical image data, infrared image data and electrical parameters of the suspected faulty photovoltaic panel group are input into the pre-trained multimodal large model for fault diagnosis, and the fault diagnosis results are obtained. In the multimodal large model, the visual encoder and the temporal encoder are connected in parallel and then connected to the multimodal fusion module and the prediction head in sequence. The visual encoder extracts visual features, and the temporal encoder extracts temporal fluctuation features. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used for fault classification and fault location.

[0008] As one implementation method, the electrical parameters include at least the real-time current, voltage, and power data of the photovoltaic panel group; the meteorological environmental data include at least irradiance, ambient temperature, humidity, wind speed, and precipitation status.

[0009] As one implementation method, the criteria for determining the same operating conditions are: within the same time period, the difference in irradiance in the area where the photovoltaic panel group is located is less than or equal to the preset irradiance threshold, the difference in ambient temperature is less than or equal to the preset temperature threshold, the wind speed is less than or equal to the preset wind speed threshold, and there is no precipitation.

[0010] As one implementation method, based on multi-source data and comparing the power generation of various photovoltaic panel groups under the same operating conditions, suspected faulty photovoltaic panel groups are screened out. The specific process is as follows: Based on meteorological and environmental data and according to the criteria for determining the same working conditions, multiple photovoltaic panel groups under the same weather conditions were selected and formed into a comparative analysis group. Within a preset time period, calculate and compare the cumulative power generation of each photovoltaic panel group within the group; Calculate and compare the statistical characteristic values ​​of cumulative power generation within the group; Based on statistical feature values, dynamic anomaly thresholds are set; Photovoltaic panel groups whose statistical characteristic values ​​are lower than the dynamic anomaly threshold are marked as suspected faulty photovoltaic panel groups.

[0011] As one implementation method, the optical image data, infrared image data, and electrical parameters corresponding to the suspected faulty photovoltaic panel group are input into a pre-trained multimodal large model for fault diagnosis. The specific process is as follows: The optical and infrared image data corresponding to the suspected faulty photovoltaic panel group are input into the visual encoder for feature extraction to obtain visual features, namely surface physical defect features and temperature anomaly features. The electrical parameters corresponding to the suspected faulty photovoltaic panel group are input into the time encoder for feature extraction to obtain the time fluctuation characteristics. By using the cross-attention mechanism of the multimodal fusion module, visual features and temporal fluctuation features are fused across modally to obtain multimodal fused features; The multimodal fusion features are input into the prediction head, and the fault category and fault area coordinates are output by the fault classification head and the fault location head, respectively.

[0012] As one implementation method, optical and infrared image data corresponding to suspected faulty photovoltaic panels are input into a visual encoder for feature extraction. The visual encoder employs a dual-branch lightweight ViT structure. The specific process is as follows: The surface physical defect features of the optical image data are extracted through the first branch, including the morphology and location features of the surface physical defects; The second branch extracts temperature anomaly features from infrared image data, including at least hot spots, local overheating, and abnormal temperature gradients. Surface physical defects and temperature anomalies constitute visual features.

[0013] As one implementation method, cross-modal fusion of visual features and temporal fluctuation features is performed through the cross-attention mechanism of the multimodal fusion module. The specific process is as follows: Visual features and temporal fluctuation features are input into the multimodal fusion module; Based on visual features and temporal fluctuation features, the proportion of the feature difference between suspected faulty photovoltaic panels and other normal photovoltaic panels in the total daily variation is calculated and used as the fluctuation proportion of the corresponding feature. Based on the fluctuation ratio, the query, key and value of the attention mechanism are adaptively allocated, and the feature with the highest difference is taken as the query; Calculate the similarity between the query and the key, and between the query and the value, and then score the feature with the highest similarity to obtain the similarity score. The similarity scores are scaled and normalized to obtain the attention weights; Features with low similarity are used as value vectors, and attention weights are multiplied and fused with the value vectors to obtain multimodal fused features.

[0014] As one implementation method, the multimodal large model training adopts a two-stage training strategy, which includes: the first stage, training using unlabeled photovoltaic data to enable the model to learn the general characteristics of photovoltaic multi-source data; and the second stage, fine-tuning using multimodal data with fault category labels and fault region mask labels to optimize the multimodal large model.

[0015] As one implementation method, it also includes preprocessing the multi-source data of the photovoltaic panel group, specifically including: cleaning the operating data, removing outliers, time alignment and normalization; and denoising, cropping and scaling the optical and infrared images.

[0016] As one implementation method, it also includes generating operation and maintenance reports and graded maintenance recommendations based on the fault diagnosis results output by the multimodal large model.

[0017] A second aspect of the present invention provides a photovoltaic panel fault detection system based on power generation comparison and a large model, comprising: The data acquisition module is used to collect multi-source data from the photovoltaic panel group, including power generation data, electrical parameters, meteorological environmental data, optical image data, and infrared image data. The anomaly screening module is used to screen out suspected faulty photovoltaic panel groups based on multi-source data and the comparison results of the power generation of each photovoltaic panel group under the same operating conditions. The intelligent diagnostic module is used to input the optical image data, infrared image data and electrical parameters of the suspected faulty photovoltaic panel group into the pre-trained multimodal large model for fault diagnosis and to obtain the fault diagnosis results. In the multimodal large model, the visual encoder and the temporal encoder are connected in parallel and then connected to the multimodal fusion module and the prediction head in sequence. The visual encoder extracts visual features, and the temporal encoder extracts temporal fluctuation features. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used for fault classification and fault location.

[0018] The above one or more technical solutions have the following beneficial effects: In this embodiment, the present invention adopts a two-stage architecture combining rapid screening based on power generation comparison and precise diagnosis using a multimodal large model. By filtering out most normal photovoltaic panels through low-cost power generation comparison, deep inference of the multimodal large model is triggered only for a small number of suspected faulty photovoltaic panels. Compared with the prior art, which requires model inference for all photovoltaic modules, this significantly reduces computational redundancy and system operating costs, and solves the problem of poor real-time performance.

[0019] In this embodiment, the multimodal large model design includes a visual encoder, a temporal encoder, a multimodal fusion module employing a cross-attention mechanism, and a dual-branch prediction head. This enables feature extraction and feature-level fusion prediction, jointly analyzing surface physical defect features from optical images, temperature anomaly features from infrared images, and temporal fluctuation features of electrical parameters. Specifically, the visual encoder uses a dual-branch lightweight ViT structure to extract physical defect and temperature anomaly features respectively; the temporal encoder extracts temporal fluctuation features of electrical parameters; the multimodal fusion module uses a cross-attention mechanism for weighted fusion; and the prediction head includes a fault classification head and a fault location head, which output the probability distribution of fault categories and the coordinate bounding box of the fault region, respectively, enabling precise fault location and fault type differentiation. This design overcomes the limitations of single detection methods, simultaneously identifying surface physical defects, temperature anomalies, and latent electrical faults, significantly reducing the false negative and false positive rates, and greatly improving the ability to identify composite and latent faults. It solves the problem that existing technologies cannot effectively distinguish between different fault types.

[0020] In this embodiment, preliminary fault screening based on power generation comparison is implemented. Specifically, photovoltaic panel groups under the same weather conditions are selected for comparative analysis. The characteristic value of the cumulative power generation within each group is calculated, and a dynamic anomaly threshold is set. Photovoltaic panel groups whose cumulative power generation deviates from the threshold are marked as suspected faults. This mechanism can identify anomalies and issue warnings before a fault causes significant power generation loss, allowing maintenance personnel sufficient time to handle the situation, effectively preventing the fault from escalating, and reducing power generation loss and safety risks.

[0021] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0023] Figure 1 This is a flowchart of the photovoltaic panel fault detection method based on power generation comparison and large model in this embodiment 1; Figure 2 This is a flowchart of the multimodal large model fault diagnosis process in Embodiment 1. Figure 3 This is a schematic diagram of the cross-attention mechanism of the multimodal fusion module in this embodiment. Figure 4 This is a schematic diagram of the photovoltaic panel fault detection system architecture based on power generation comparison and large model in this embodiment 2. Detailed Implementation

[0024] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0026] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0027] Example 1 This embodiment discloses a photovoltaic panel fault detection method based on power generation comparison and large model.

[0028] To more clearly illustrate this embodiment, the photovoltaic panel fault detection process based on power generation comparison and a large model can be specifically described as follows: A photovoltaic panel fault detection method based on power generation comparison and large-scale model includes: S1. Collect multi-source data from the photovoltaic panel group, including power generation data, electrical parameters, meteorological environment data, optical image data, and infrared image data; S2. Based on multi-source data, and according to the comparison results of the power generation of each photovoltaic panel group under the same operating conditions, suspected faulty photovoltaic panel groups are screened out. S3. Input the optical image data, infrared image data and electrical parameters of the suspected faulty photovoltaic panel group into the pre-trained multimodal large model for fault diagnosis and obtain the fault diagnosis results. In the multimodal large model, the visual encoder and the temporal encoder are connected in parallel and then connected to the multimodal fusion module and the prediction head in sequence. The visual encoder extracts visual features, and the temporal encoder extracts temporal fluctuation features. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used for fault classification and fault location.

[0029] Existing photovoltaic panel fault detection technologies suffer from several drawbacks, including incomplete fault identification, low accuracy, insufficient real-time performance, high deployment and maintenance costs, difficulty in achieving early warning and precise fault location, and inability to effectively distinguish between different fault types. This embodiment proposes a photovoltaic panel fault detection method based on power generation comparison and a large model, and designs a photovoltaic panel fault prediction model that includes a visual encoder, a time encoder, a multimodal fusion module, and a prediction head.

[0030] like Figure 1As shown, in step S1, multi-source data of the photovoltaic panel group is collected, including power generation data, electrical parameters, meteorological environment data, optical image data and infrared image data.

[0031] S101. Collect multi-source data from the photovoltaic panel group, including operational data, optical image data, and infrared image data.

[0032] (1) Real-time acquisition of the operating data of each photovoltaic panel group in the photovoltaic array, including power generation data, electrical parameters and meteorological environment data.

[0033] Multi-source data acquisition equipment is deployed at the photovoltaic power station site, including current sensors, voltage sensors, power acquisition modules, weather stations (irradiance sensors, temperature and humidity sensors, etc.), visible light cameras, and infrared thermal imagers, to complete the real-time acquisition of all-dimensional data of the photovoltaic panels.

[0034] 1) Collect electrical parameters and power generation data.

[0035] A current sensor, a voltage sensor, and a power acquisition module are deployed at the combiner box of each photovoltaic panel string to collect the current, voltage, and power data of the string in real time. The acquisition interval is set to 10 minutes, and the daily power generation data is calculated synchronously.

[0036] Among them, the electrical parameters include at least the real-time current, voltage, and power data of the photovoltaic panel group, with a collection interval of no more than 15 minutes.

[0037] 2) Collect meteorological and environmental data.

[0038] One meteorological station is deployed for every 500 mu (approximately 33 hectares) of the photovoltaic power station area to collect data on irradiance, ambient temperature, relative humidity, wind speed, and precipitation status, with a collection interval of 5 minutes.

[0039] The meteorological environmental data includes at least irradiance, ambient temperature, humidity, wind speed, and precipitation status. The preset time period is daily, weekly, or monthly, which can be flexibly configured according to the scale of the photovoltaic power station and its operation and maintenance needs.

[0040] (2) Simultaneously collect optical image data and infrared image data of photovoltaic panels.

[0041] Fixed visible light cameras and infrared thermal imagers are deployed in the photovoltaic array inspection channel. Each device covers 20 photovoltaic panel strings, and the acquisition cycle is set to 30 minutes. The resolution of the visible light camera is set to 1920×1080, and the temperature measurement range of the infrared thermal imager is set to -20℃~150℃ with a temperature measurement accuracy of ±0.5℃.

[0042] S102. Preprocess the multi-source data of the photovoltaic panel group.

[0043] In this example, the specific process is as follows: (1) Clean the collected running data, remove outliers, align the time, and normalize it.

[0044] 1) Data cleaning and outlier removal.

[0045] The 3σ criterion is used to eliminate outliers caused by sensor malfunctions and transmission interference. Missing values ​​are filled by linear interpolation of adjacent data in the same group at the same time.

[0046] 2) Time alignment processing.

[0047] Multi-source data with different collection frequencies and different collection terminals are uniformly mapped to a 10-minute granular time axis to ensure the consistency of all data in the time dimension.

[0048] 3) Normalization process.

[0049] For numerical data such as power generation, electrical parameters, and meteorological data, the min-max normalization method is used to map them to the [0,1] interval to eliminate the influence of different dimensions on subsequent analysis.

[0050] (2) Denoise, crop, and scale the optical and infrared images.

[0051] Following the steps outlined above, by deploying current sensors, voltage sensors, power sensors, irradiance sensors, temperature and humidity sensors, visible light cameras, and infrared thermal imagers, real-time data is collected on the photovoltaic panel array's power generation, electrical parameters (current, voltage, power), meteorological environmental data (irradiance, temperature, humidity, wind speed, precipitation status), and optical and infrared images. Preprocessing of the multi-source data eliminates differences in time scale and dimensions, providing a high-quality, aligned, and standardized data foundation for subsequent power generation comparison screening and multimodal model diagnosis.

[0052] like Figure 1 As shown, in step S2, based on multi-source data and the comparison results of the power generation of each photovoltaic panel group under the same operating conditions, suspected faulty photovoltaic panel groups are screened out.

[0053] The specific process is as follows: (1) Based on meteorological environmental data and according to the judgment criteria of the same working conditions, multiple photovoltaic panel groups under the same weather conditions were selected to form a comparative analysis group.

[0054] The criteria for determining the same operating conditions are: within the same time period, the difference in irradiance in the area where the photovoltaic panel is located is less than or equal to the preset irradiance threshold, the difference in ambient temperature is less than or equal to the preset temperature threshold, the wind speed is less than or equal to the preset wind speed threshold, and there is no precipitation.

[0055] In this embodiment, the preset irradiance threshold is 10% of the total irradiance, the preset temperature threshold is 5℃, and the preset wind speed threshold is 3m / s.

[0056] The criteria for determining the same weather conditions are as follows: within the same time period of the day, the difference in irradiance in the area where the photovoltaic panel group is located is ≤10%, the difference in ambient temperature is ≤5℃, there is no precipitation, and the wind speed is ≤3m / s. Photovoltaic panel groups that meet the above conditions are classified into the same comparative analysis group.

[0057] (2) Within the preset time period, calculate and compare the cumulative power generation of each photovoltaic panel group in the group.

[0058] The preset time period is daily, weekly, or monthly, which can be flexibly configured according to the scale of the photovoltaic power station and operation and maintenance needs.

[0059] In this example, the preset time period is daily, and the daily power generation data is calculated cumulatively among photovoltaic panel groups that meet the same weather conditions.

[0060] (3) Calculate and compare the statistical characteristic values ​​of the cumulative power generation within the group.

[0061] The statistical characteristics include the mean and standard deviation of the cumulative power generation of all photovoltaic panel groups within the comparative analysis group.

[0062] Calculate the mean μ and standard deviation σ of the cumulative power generation of all photovoltaic panels in the group.

[0063] (4) Set dynamic anomaly thresholds based on statistical feature values.

[0064] Based on statistical characteristic values, a dynamic anomaly threshold is set using the following formula: ; Where k is an adjustable coefficient.

[0065] (5) Photovoltaic panel groups whose statistical characteristic values ​​are lower than the dynamic anomaly threshold are marked as suspected faulty photovoltaic panel groups.

[0066] When the cumulative power generation of a photovoltaic panel group is lower than the threshold, the photovoltaic panel group is marked as a suspected faulty photovoltaic panel group. At the same time, the multi-source data corresponding to the group string is packaged and uploaded to the cloud server, triggering the subsequent accurate diagnosis process, which can significantly reduce the amount of computation for subsequent model inference.

[0067] If the cumulative power generation of a photovoltaic panel group does not fall below the threshold, multi-source data will continue to be collected in real time.

[0068] Through the above steps, the dynamic anomaly threshold judgment mechanism for power generation under the same operating conditions can identify anomalies and issue warnings in the early stage of a fault (before the fault causes significant power generation loss and safety hazards). This solves the problem of post-event detection in existing technologies, provides sufficient time for operation and maintenance personnel to handle the situation, effectively avoids the escalation of the fault, and reduces the power generation loss and safety risks of photovoltaic power plants.

[0069] like Figure 1 As shown, in step S3, the optical image data, infrared image data and electrical parameters corresponding to the suspected faulty photovoltaic panel group are input into the pre-trained multimodal large model for fault diagnosis, and the fault diagnosis results are obtained.

[0070] S301. Construct a large multimodal model and train it.

[0071] (1) Construct a multimodal large model.

[0072] The large multimodal model adopts a Transformer-based encoder architecture, including a visual encoder, a temporal encoder, a multimodal fusion module, and a prediction head. The visual encoder and the temporal encoder are connected in parallel and then sequentially connected to the multimodal fusion module and the prediction head.

[0073] The visual encoder is a dual-branch lightweight ViT structure that extracts surface physical defect features from optical images and temperature anomaly features from infrared images, respectively. The temporal encoder is a Transformer temporal coding layer that extracts time-series fluctuation features of electrical parameters. The multimodal fusion module uses a cross-attention mechanism to weight and fuse surface physical defect features, temperature anomaly features, and time-series fluctuation features to generate multimodal fusion features. The prediction head includes a fault classification head and a fault location head, which output fault category labels and coordinate boxes of fault areas, respectively.

[0074] (2) Train the multimodal large model.

[0075] The training process for a multimodal large model specifically includes: 1) Construct a multimodal dataset of photovoltaic faults.

[0076] The dataset samples include optical images, infrared images, time-series data of electrical parameters, and corresponding fault category labels and fault area mask annotations.

[0077] 2) A two-stage training strategy is adopted.

[0078] In the first stage, self-supervised pre-training is carried out using large-scale unlabeled photovoltaic operation data to learn the general feature representation of photovoltaic multi-source data.

[0079] The specific steps of the training process are as follows: Step S302. Large-scale unlabeled photovoltaic operation data is input into the multimodal large model. Visual features are extracted by a visual encoder, and temporal fluctuation features are extracted by a temporal encoder. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used to classify and locate faults, and output the fault category label and the coordinate box of the fault area.

[0080] In the second stage, supervised fine-tuning is performed using the labeled photovoltaic fault multimodal dataset. Simultaneously, data augmentation techniques are employed to expand the training samples and optimize the model's ability to identify complex and latent faults. The specific steps of the training process are as described in step S302.

[0081] 3) A multi-task learning framework is adopted to simultaneously optimize the loss function of the two tasks of fault classification and fault location.

[0082] The loss functions for the two tasks are cross-entropy loss and concurrency loss, respectively.

[0083] The cross-entropy loss function is expressed as follows: ; in, Y ij These are real category labels. F ij It is a predicted classification label, that is, a fault category label.

[0084] And the loss function is calculated as follows: ; Where A∩B represents the intersection area of ​​the coordinate frames A and B of the fault region, and A∪B represents the union area of ​​the coordinate frames A and B of the fault region.

[0085] After the above steps, a trained large model is obtained, which enables the multimodal large model to accurately identify various fault types such as hot spots, microcracks, breakage, grid breakage, diode faults, junction box faults, and PID effects. It can also effectively distinguish between compound faults and latent faults, and output accurate fault category probability distributions and fault area coordinate boxes.

[0086] like Figure 2 As shown in step S302, the optical image data, infrared image data, and electrical parameters corresponding to the suspected faulty photovoltaic panel group are input into the pre-trained multimodal large model for fault diagnosis.

[0087] For the suspected faulty photovoltaic panel groups marked in step S2, optical image data, infrared image data, and electrical parameter time-series data for the corresponding time period are retrieved. The retrieved multi-source data is input into a pre-trained multimodal large-scale model. The multimodal large-scale model, through a feature-level fusion strategy, jointly analyzes the surface physical defect features of the optical images, the temperature anomaly features of the infrared images, and the time-series fluctuation features of the electrical parameters, outputting diagnostic results including fault type, fault location, and fault severity. The specific process is as follows: (1) Input the optical and infrared image data corresponding to the suspected faulty photovoltaic panel group into the visual encoder for feature extraction to obtain visual features, namely surface physical defect features and temperature anomaly features.

[0088] The optical and infrared image data corresponding to the suspected faulty photovoltaic panel group are input into the visual encoder for feature extraction. The visual encoder adopts a dual-branch lightweight ViT structure. The specific process is as follows: 1) Extract surface physical defect features from optical image data through the first branch, including the morphology and location features of surface physical defects.

[0089] A lightweight dual-branch ViT structure is adopted, with two branches processing optical and infrared images respectively. The first branch, the optical image branch, extracts the morphological and location features of surface physical defects such as cracks, dirt, broken cells, and broken glass.

[0090] 2) Extract temperature anomaly features from infrared image data through the second branch, including at least hot spots, local overheating, and abnormal temperature gradients.

[0091] The second branch, namely the infrared image branch, extracts features such as hot spots, local overheating, and abnormal temperature gradients.

[0092] Surface physical defect features and temperature anomaly features constitute visual features, and a visual feature vector is output.

[0093] (2) Input the electrical parameters corresponding to the suspected faulty photovoltaic panel group into the timing encoder for feature extraction to obtain the timing fluctuation characteristics.

[0094] The timing encoder is responsible for processing the time series data of electrical parameters, extracting electrical fault characteristics such as sudden changes, drifts, mismatches, and nonlinear fluctuations in current and voltage, i.e. timing fluctuation characteristics, and outputting a timing fluctuation feature vector.

[0095] (3) Through the cross-attention mechanism of the multimodal fusion module, visual features and temporal fluctuation features are fused across modes to obtain multimodal fusion features.

[0096] like Figure 3As shown, the multimodal fusion module employs a cross-attention mechanism to interact visual and temporal features across modalities, generating a multimodal fusion feature vector that integrates image and electrical information. This achieves deep association between image features, electrical features, and fault types. The specific process is as follows: 1) Input visual features and temporal fluctuation features into the multimodal fusion module. Based on visual features and temporal fluctuation features, calculate the proportion of the feature difference between the suspected faulty photovoltaic panel group and other normal photovoltaic panel groups in the total daily change, and use it as the fluctuation proportion of the corresponding feature.

[0097] Visual features and temporal fluctuation features are input into the multimodal fusion module. Through fluctuation judgment, the query (Q), key (K) and value (V) are adaptively assigned according to the fluctuation ratio. The similarity between QK and QV is calculated, and the feature with the highest similarity is selected for attention weight calculation. The fluctuation judgment is the proportion of the feature difference between the suspected faulty photovoltaic panel group and other normal photovoltaic panel groups in the total daily change.

[0098] The formula for calculating the fluctuation ratio is: ; Where Flu is the fluctuation ratio; F e These are characteristics of suspected faulty photovoltaic panels, specifically surface physical defects, abnormal temperatures, or temporal fluctuations. (F) all ΔF represents the characteristics of all photovoltaic panels, including physical defects, temperature anomalies, or temporal fluctuations on the surface of all photovoltaic panels; ΔF represents the change in photovoltaic panel characteristics over the past day, which is the rate of change of physical defects, temperature anomalies, or temporal fluctuations on the surface of the photovoltaic panels over the past day.

[0099] Based on the fluctuation weight calculation formula, the fluctuation weights of time-series fluctuation characteristics, temperature anomaly characteristics, and physical surface characteristics are calculated respectively, and used to allocate QKV.

[0100] 2) Based on the fluctuation ratio, the query, key and value of the attention mechanism are adaptively allocated, and the feature with the highest difference is taken as the query.

[0101] The calculation formula is: , ; Among them, IF O This represents optical image features, i.e., surface physical defect features; IF T The text indicates thermal imaging features, specifically temperature anomalies; EPC represents electrical parameter features, i.e., temporal fluctuations; the subscript E indicates anomaly labeling; the overline represents the average data feature of the entire photovoltaic power plant; and max indicates taking the maximum value.

[0102] 3) Calculate the similarity between the query and the key, and between the query and the value, and score the feature with the highest similarity to obtain the similarity score.

[0103] The calculation formula is: ; Here, Score is the similarity score, the superscript T represents the matrix transpose, and max.maen represents the dot product result of the matrix with the largest mean. The matrix with the larger mean is selected as the similarity output when comparing the mean of the matrices.

[0104] Based on the similarity score calculation formula, the similarity scores of query Q and key K, query Q and value V, and the similarity score of the feature with the highest similarity are obtained respectively.

[0105] 4) Scale and normalize the similarity scores to obtain attention weights; use the features with low similarity as value vectors, and multiply the attention weights with the value vectors to obtain multimodal fusion features.

[0106] When the similarity between QK and QV is greater than the similarity between QV, the multimodal fusion feature calculation formula is: ; Where Attention(Q,K,V) represents the multimodal fusion feature; d K It is the dimension of the K matrix; This is a scaling process for similarity scores; Softmax() is an activation function that normalizes the scaled similarity scores.

[0107] (4) Input the multimodal fusion features into the prediction head, and output the fault category and fault area coordinates through the fault classification head and the fault location head respectively.

[0108] The prediction head includes a fault classification head and a fault location head. The fault classification head is based on the fused feature vector and outputs the fault type (fault category label) of the photovoltaic fault through a fully connected layer and a Softmax activation function. The fault location head adopts a fully convolutional network structure and outputs the pixel-level coordinate box of the fault area to achieve accurate fault location.

[0109] The fault types identified by the multimodal large model include at least hot spot effect, microcrack, cell breakage, surface dirt obstruction, diode short circuit, junction box failure, grid breakage, and PID effect. The multimodal large model distinguishes between hot spots caused by internal circuit faults and hot spots caused by external obstruction through joint analysis of electrical timing characteristics and thermal imaging characteristics.

[0110] For hot spot faults, if thermal imaging shows local high temperature, and electrical parameters show current mismatch and increased reverse current, it is determined to be a hot spot caused by internal circuit fault (such as diode short circuit or open gate); if thermal imaging shows local high temperature, electrical parameters are not obviously abnormal, and optical images show that there is dust or foreign object obstruction at the corresponding location, it is determined to be a hot spot caused by external obstruction, providing accurate guidance for subsequent operation and maintenance.

[0111] like Figure 1 As shown, in step S4, an operation and maintenance report and graded maintenance recommendations are generated based on the fault diagnosis results output by the multimodal large model.

[0112] Based on the fault diagnosis results output by the multimodal large model, a standardized operation and maintenance report is generated, and graded maintenance suggestions and priority operation and maintenance plans are generated according to the severity of the fault, and pushed to the operation and maintenance management terminal.

[0113] In this embodiment, based on the fault diagnosis results output in step S3, the system automatically generates a standardized operation and maintenance report. The report includes: the number of the faulty photovoltaic panel group, the array location, the coordinates of the faulty area, the fault type, the fault severity, the fault cause analysis, and historical power generation attenuation data.

[0114] The severity of the fault is divided into three levels: Level 1 minor fault, Level 2 moderate fault, and Level 3 severe fault.

[0115] Based on the severity of the fault, the system generates graded maintenance recommendations and priority operation and maintenance plans.

[0116] Level 1 Minor Fault: Such as slight dust obstruction or minor microcracks, with no immediate safety risk and a power generation efficiency reduction of <5%; Level 2 medium-level faults: such as large-area dirt blockage or localized microcracks, power generation efficiency decreases by 5% to 20%, requiring regular maintenance; Level 3 severe faults: such as diode short circuits, large-area hot spots, junction box failures, which pose safety hazards and reduce power generation efficiency by more than 20%, require emergency handling.

[0117] This implementation method can flexibly adjust parameter configuration and model structure according to the scale, deployment environment, and operation and maintenance needs of the photovoltaic power station, and has strong scenario adaptability and engineering implementation value. At the same time, the system architecture supports modular expansion and can access more data sources such as drone inspection images and EL detection images, continuously improving the comprehensiveness and accuracy of fault detection.

[0118] In another embodiment, for distributed residential photovoltaic scenarios, the preset time period is set to a week and the adjustable coefficient k is set to 1.5 to improve the sensitivity to faults in small photovoltaic systems. At the same time, after the multimodal large model is lightweighted, it is fully deployed on local edge computing devices, which can complete the entire process of fault detection without cloud servers, adapting to the operation and maintenance needs of photovoltaic power stations in remote areas without public network access.

[0119] In another embodiment, an acoustic feature branch is added to the multimodal large model to collect the operating acoustic data of the photovoltaic inverter and photovoltaic panel. The acoustic features are integrated to further improve the ability to identify hidden electrical faults, while expanding the types of identifiable faults, including inverter faults, combiner box faults, etc., to realize fault detection and diagnosis of the entire photovoltaic system.

[0120] In another embodiment, in the preliminary fault screening in step S2, the box plot method is introduced to replace the 3σ criterion to set the dynamic anomaly threshold. This improves the robustness of anomaly screening and further reduces the false detection rate for scenarios where the power generation data distribution is non-normal.

[0121] Example 2 The purpose of this embodiment is to provide a photovoltaic panel fault detection system based on power generation comparison and a large model, including: The data acquisition module is used to collect multi-source data from the photovoltaic panel group, including power generation data, electrical parameters, meteorological environmental data, optical image data, and infrared image data. The anomaly screening module is used to screen out suspected faulty photovoltaic panel groups based on multi-source data and the comparison results of the power generation of each photovoltaic panel group under the same operating conditions. The intelligent diagnostic module is used to input the optical image data, infrared image data and electrical parameters of the suspected faulty photovoltaic panel group into the pre-trained multimodal large model for fault diagnosis and to obtain the fault diagnosis results. In the multimodal large model, the visual encoder and the temporal encoder are connected in parallel and then connected to the multimodal fusion module and the prediction head in sequence. The visual encoder extracts visual features, and the temporal encoder extracts temporal fluctuation features. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used for fault classification and fault location.

[0122] It also includes a data preprocessing module for preprocessing multi-source data from photovoltaic panels, specifically including: cleaning, outlier removal, time alignment and normalization of operating data; and denoising, cropping and scaling of optical and infrared images.

[0123] It also includes a report generation module, which generates operation and maintenance reports and graded maintenance recommendations based on the fault diagnosis results output by the multimodal large model.

[0124] like Figure 4 As shown, the photovoltaic panel fault detection system in this embodiment adopts a cloud-edge collaborative architecture, which is divided into a data acquisition layer, an edge processing layer, a cloud analysis layer, and an operation and maintenance application layer.

[0125] The data acquisition layer (data acquisition module) consists of various sensors (voltage, current, power sensors), visible light cameras, and infrared thermal imagers. It is deployed on-site at the photovoltaic array to realize real-time acquisition of multi-source data and transmits the data to the edge processing layer through a wireless communication module.

[0126] The edge processing layer deploys a lightweight data preprocessing module and a power generation anomaly screening module, which are responsible for processing local sensor data in real time and filtering normal samples, and only uploading suspected fault data to the cloud analysis layer.

[0127] The cloud analytics layer deploys a complete multimodal large-scale model on cloud servers, including a multimodal large-scale model accurate diagnosis module and a model training and update module. This module is responsible for accurately diagnosing received suspected fault data and transmitting the diagnostic results back to edge nodes and operation and maintenance management terminals. This architecture effectively reduces network transmission bandwidth requirements and improves the overall system response speed.

[0128] The operation and maintenance (O&M) application layer includes a large screen in the central control room, computers for O&M personnel, and mobile phones / tablets for O&M personnel. Through the large screen in the central control room, computers for O&M personnel, and mobile terminals (phones / tablets), the O&M application layer visually displays fault diagnosis results, including the number and location of the faulty photovoltaic panel, fault type, coordinates of the fault area, fault severity level, and historical power generation attenuation data. Based on the diagnosis results, it automatically generates standardized O&M reports and tiered maintenance recommendations, and pushes priority O&M plans to different terminals, achieving precise assignment and closed-loop management of O&M tasks. This supports a collaborative working mode where power plant management personnel centrally monitor and on-site O&M personnel respond quickly.

[0129] This example adopts a cloud-edge collaborative system deployment architecture. The edge device realizes real-time data processing and preliminary fault screening, while the cloud device completes high-precision fault diagnosis and model iteration updates. It takes into account both the real-time performance and diagnostic depth of fault detection, significantly reduces data transmission bandwidth requirements and processing latency, and can flexibly adapt to the operation and maintenance needs of distributed and large-scale photovoltaic power plants. At the same time, it supports continuous iterative optimization of the model.

[0130] Based on the provided photovoltaic panel fault detection system based on power generation comparison and large model, the method steps in Example 1 are implemented.

[0131] Example 3 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the method in Embodiment 1.

[0132] Example 4 This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method in Embodiment 1.

[0133] The steps and methods involved in the apparatus of the above embodiments correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0134] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0135] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A photovoltaic panel fault detection method based on power generation versus large model, characterized in that, include: Collect multi-source data from photovoltaic panels, including power generation data, electrical parameters, meteorological environmental data, optical image data, and infrared image data. Based on multi-source data, and by comparing the power generation of each photovoltaic panel group under the same operating conditions, suspected faulty photovoltaic panel groups were screened out. The optical image data, infrared image data and electrical parameters of the suspected faulty photovoltaic panel group are input into the pre-trained multimodal large model for fault diagnosis, and the fault diagnosis results are obtained. In the multimodal large model, the visual encoder and the temporal encoder are connected in parallel and then connected to the multimodal fusion module and the prediction head in sequence. The visual encoder extracts visual features, and the temporal encoder extracts temporal fluctuation features. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used for fault classification and fault location.

2. The photovoltaic panel fault detection method based on power generation amount contrast with large model of claim 1, wherein, Electrical parameters should include at least the real-time current, voltage, and power data of the photovoltaic panel group; meteorological environmental data should include at least irradiance, ambient temperature, humidity, wind speed, and precipitation status.

3. The photovoltaic panel fault detection method based on power generation comparison and large model as described in claim 1, characterized in that, The criteria for determining the same operating conditions are: within the same time period, the difference in irradiance in the area where the photovoltaic panel is located is less than or equal to the preset irradiance threshold, the difference in ambient temperature is less than or equal to the preset temperature threshold, the wind speed is less than or equal to the preset wind speed threshold, and there is no precipitation.

4. The photovoltaic panel fault detection method based on power generation amount contrast with large model of claim 1, wherein, Based on multi-source data, and by comparing the power generation of various photovoltaic panel groups under the same operating conditions, suspected faulty photovoltaic panel groups were screened out. The specific process is as follows: Based on meteorological and environmental data and according to the criteria for determining the same working conditions, multiple photovoltaic panel groups under the same weather conditions were selected and formed into a comparative analysis group. Within a preset time period, calculate and compare the cumulative power generation of each photovoltaic panel group within the group; Calculate and compare the statistical characteristic values ​​of cumulative power generation within the group; Based on statistical feature values, set dynamic anomaly thresholds; Photovoltaic panel groups whose statistical characteristic values ​​are lower than the dynamic anomaly threshold are marked as suspected faulty photovoltaic panel groups.

5. The photovoltaic panel fault detection method based on power generation amount contrast with large model of claim 1, wherein, The optical image data, infrared image data, and electrical parameters corresponding to the suspected faulty photovoltaic panel group are input into a pre-trained multimodal large model for fault diagnosis. The specific process is as follows: The optical and infrared image data corresponding to the suspected faulty photovoltaic panel group are input into the visual encoder for feature extraction to obtain visual features, namely surface physical defect features and temperature anomaly features. The electrical parameters corresponding to the suspected faulty photovoltaic panel group are input into the time encoder for feature extraction to obtain the time fluctuation characteristics. By using the cross-attention mechanism of the multimodal fusion module, visual features and temporal fluctuation features are fused across modally to obtain multimodal fused features; The multimodal fusion features are input into the prediction head, and the fault category and fault area coordinates are output by the fault classification head and the fault location head, respectively.

6. The photovoltaic panel fault detection method based on power generation versus large model of claim 5, wherein, The optical and infrared image data corresponding to the suspected faulty photovoltaic panel group are input into the visual encoder for feature extraction. The visual encoder adopts a dual-branch lightweight ViT structure. The specific process is as follows: The surface physical defect features of the optical image data are extracted through the first branch, including the morphology and location features of the surface physical defects; The second branch extracts temperature anomaly features from infrared image data, including at least hot spots, local overheating, and abnormal temperature gradients. Surface physical defects and temperature anomalies constitute visual features.

7. The photovoltaic panel fault detection method based on power generation amount contrast with large model of claim 5, wherein, The cross-attention mechanism of the multimodal fusion module is used to perform cross-modal fusion of visual features and temporal fluctuation features. The specific process is as follows: Visual features and temporal fluctuation features are input into the multimodal fusion module; Based on visual features and temporal fluctuation features, the proportion of the feature difference between suspected faulty photovoltaic panels and other normal photovoltaic panels in the total daily variation is calculated and used as the fluctuation proportion of the corresponding feature. Based on the fluctuation ratio, the query, key and value of the attention mechanism are adaptively allocated, and the feature with the highest difference is taken as the query; Calculate the similarity between the query and the key, and between the query and the value, and then score the feature with the highest similarity to obtain the similarity score. The similarity scores are scaled and normalized to obtain the attention weights; Features with low similarity are used as value vectors, and attention weights are multiplied and fused with the value vectors to obtain multimodal fused features.

8. The photovoltaic panel fault detection method based on power generation comparison and large model as described in claim 1, characterized in that, The training of the multimodal large model adopts a two-stage training strategy, which includes: the first stage, training using unlabeled photovoltaic data to enable the model to learn the general features of photovoltaic multi-source data; and the second stage, fine-tuning using multimodal data with fault category labels and fault region mask labels to optimize the multimodal large model.

9. The photovoltaic panel fault detection method based on power generation amount contrast with large model of claim 1, wherein, It also includes generating operation and maintenance reports and graded maintenance recommendations based on the fault diagnosis results output by the multimodal large model.

10. A photovoltaic panel fault detection system based on power generation versus large model, characterized in that, include: The data acquisition module is used to collect multi-source data from the photovoltaic panel group, including power generation data, electrical parameters, meteorological environmental data, optical image data, and infrared image data. The anomaly screening module is used to screen out suspected faulty photovoltaic panel groups based on multi-source data and the comparison results of the power generation of each photovoltaic panel group under the same operating conditions. The intelligent diagnostic module is used to input the optical image data, infrared image data and electrical parameters of the suspected faulty photovoltaic panel group into the pre-trained multimodal large model for fault diagnosis and to obtain the fault diagnosis results. In the multimodal large model, the visual encoder and the temporal encoder are connected in parallel and then connected to the multimodal fusion module and the prediction head in sequence. The visual encoder extracts visual features, and the temporal encoder extracts temporal fluctuation features. After the visual features and temporal fluctuation features are fused by the multimodal fusion module, the prediction head is used for fault classification and fault location.

Images