Photovoltaic module fault diagnosis system and method based on deep learning

Abstract

The invention discloses a photovoltaic module fault diagnosis system and method based on deep learning, which significantly improves the accuracy and comprehensiveness of fault diagnosis through multi-modal data fusion and advanced deep learning technology, and improves the system operation efficiency through the application of an adaptive sampling strategy and edge calculation. The system can simultaneously execute fault classification, severity evaluation and life prediction through a multi-task learning framework, the diagnosis depth and breadth are enhanced, a predictive maintenance decision-making mechanism based on reinforcement learning optimizes a maintenance strategy, the service life of equipment is prolonged, the operation and maintenance cost is reduced, and the maintenance efficiency is improved. The distributed learning framework ensures data privacy and continuous optimization of the model, improves the overall power generation efficiency and economic benefits of the photovoltaic power station, and provides powerful support for large-scale application of renewable energy sources.

Description

Technical Field

[0001] The present invention belongs to the technical field of photovoltaic power generation, and specifically relates to a photovoltaic component fault diagnosis system and method based on deep learning. Background Art

[0002] As an important component of clean energy, the reliability and efficiency of photovoltaic power generation systems directly affect power generation and economic benefits. Photovoltaic module fault diagnosis technology is the key to ensuring the stable operation of photovoltaic systems. It analyzes various parameters and operating data of components to timely discover and locate possible faults, thereby guiding the maintenance work. Effective fault diagnosis can not only reduce downtime, but also reduce maintenance costs and extend the service life of the system.

[0003] At present, PV module fault diagnosis mainly relies on manual inspection and simple threshold monitoring methods. Although manual inspection is reliable, it is inefficient and costly, especially for large PV power plants. Although threshold monitoring methods can be automated, they can only detect obvious faults and are difficult to effectively identify potential or complex faults. With the development of photovoltaic technology, the types of component failures are becoming increasingly complex, and traditional methods can no longer meet the needs of modern photovoltaic systems for efficient and accurate fault diagnosis. In addition, existing diagnostic systems usually only focus on a single data source, such as electrical parameters or thermal images, and lack the comprehensive use of multi-source data, which limits the comprehensiveness and accuracy of diagnosis.

[0004] Given the limitations of existing technologies, there is an urgent need for a photovoltaic module fault diagnosis system that can comprehensively analyze multi-source data, accurately identify complex faults, and has adaptive capabilities. The system should be able to overcome the shortcomings of traditional methods in terms of diagnostic accuracy, efficiency, and adaptability, and achieve timely and accurate identification of various types of faults. At the same time, considering the distributed characteristics of photovoltaic power stations and data security requirements, the system also needs to solve the problems of data privacy protection and distributed learning. In addition, how to provide explainable diagnostic results while ensuring diagnostic accuracy, and how to achieve predictive maintenance to reduce operation and maintenance costs are all technical challenges currently faced. Summary of the invention

[0005] The purpose of the present invention is to provide a photovoltaic module fault diagnosis system and method based on deep learning, so as to overcome the problems of low comprehensiveness, accuracy and efficiency of photovoltaic module fault diagnosis in the prior art.

[0006] In order to solve the above problems, the present invention adopts the following technical solutions: A photovoltaic module fault diagnosis system based on deep learning, comprising: Multimodal data acquisition module, used to collect electrical parameters, environmental parameters, image data and acoustic data of photovoltaic modules; A data preprocessing module, used for cleaning, standardizing and feature extracting the multimodal data; Deep learning core modules, including multimodal deep learning architecture, cross-modal attention fusion mechanism, multi-task learning framework and dynamic graph neural network, are used to process pre-processed multimodal data; A fault diagnosis engine, used to identify fault types, assess fault severity, and infer fault causes based on the output of the deep learning core module; A predictive maintenance decision module for generating maintenance strategies and predicting the remaining useful life of components; Distributed learning and deployment framework, used to achieve collaborative learning and model edge deployment among multiple PV plants; Among them, the deep learning core module realizes comprehensive status assessment and fault diagnosis of photovoltaic components by fusing multimodal data, using dynamic graph neural network structure to model the relationship between components, and simultaneously executing multiple related tasks.

[0007] Furthermore, the multimodal data acquisition module includes Electrical parameter sensors, used to collect current and voltage data; Environmental parameter sensors, used to collect temperature and irradiance data; Image acquisition equipment, including infrared thermal imaging cameras and high-resolution visible light cameras; Acoustic sensor, used to collect ultrasonic signals; The multimodal data acquisition module also includes an adaptive sampling strategy to dynamically adjust the sampling frequency according to the component status. Furthermore, the adaptive sampling strategy includes: Use lightweight anomaly detection algorithms to calculate the anomaly score of the current state in real time; Dynamically adjust the sampling frequency based on the anomaly score, and increase the sampling frequency when an anomaly is detected; Adopt smooth transition mechanism to avoid sudden changes in sampling frequency; Set the duration and cooldown period of high-frequency sampling; Maintain a circular buffer to save the high-frequency sampling data of the recent period. Furthermore, the data preprocessing module includes: Data cleaning unit, which uses an improved local anomaly factor algorithm to detect and process outliers; Data normalization unit, which performs Z-score normalization on numerical data, pixel value normalization on image data, amplitude normalization and logarithmic transformation on acoustic data; The feature extraction unit extracts time domain, frequency domain and time-frequency domain features for time series data, extracts deep features and texture features for image data, and extracts time domain, frequency domain and time-frequency domain features for acoustic data. Furthermore, the multimodal deep learning architecture includes: The time series data processing branch uses multi-layer causal convolution and temporal convolutional networks; Image data processing branch, using the EfficientNet-V2 backbone network and dual attention mechanism, including channel attention and spatial attention; Acoustic data processing branch, using one-dimensional convolutional neural networks and fully connected layers; The cross-modal attention fusion mechanism uses the Transformer encoder and gating mechanism to dynamically adjust the weights of different modalities. Furthermore, the multi-task learning framework includes: Shared encoder and task-specific decoder; Perform four tasks: fault classification, fault severity assessment, remaining service life prediction, and fault location; Adopt uncertainty-weighted multi-task loss function; Use the Adam optimizer and the Cyclical Learning Rates learning rate scheduling strategy. Furthermore, the dynamic graph neural network is constructed based on a space-time graph convolutional network, including: The PV modules are represented as nodes in the graph, and the physical connections and dependencies between the modules are represented as edges; Use graph attention network to dynamically update edge weights; Introducing time embedding to capture the time evolution of component states; A graph pooling operation is used to obtain a representation of the entire system. Furthermore, the fault diagnosis engine includes: A hierarchical classifier based on the output of the deep learning model to identify multiple fault types; Regression models to assess fault severity; A reasoning unit based on a structured causal model to analyze the causes of failures; An interpretability analysis module integrating Grad-CAM and LIME is used to provide explanations for diagnostic results. Further, the predictive maintenance decision module generates maintenance strategies based on a reinforcement learning system of a soft actor-critic algorithm and uses a life prediction model of a temporal convolutional network to output a probability distribution of the remaining useful life; The distributed learning and deployment framework includes: A federated learning framework based on the federated averaging algorithm; Use knowledge distillation and quantization techniques to achieve model compression; Adopting gradient accumulation strategy to deal with the imbalanced gradient update problem of different tasks; Enable efficient deployment on edge computing devices. In a second aspect, a diagnostic method of a photovoltaic module fault diagnosis system based on deep learning comprises the following steps: The electrical parameters, environmental parameters, image data and acoustic data of the photovoltaic module are collected by the multimodal data collection module, and the collected data of the multimodal data collection module are preprocessed by the data preprocessing module; The preprocessed multimodal data is input into the deep learning core module. The fault diagnosis engine identifies the fault type, evaluates the fault severity, and infers the fault cause based on the output of the deep learning core module. The fault type identification, fault severity assessment and fault cause reasoning output by the fault diagnosis engine are used as inputs to the predictive maintenance decision module to generate maintenance strategies and predict the remaining service life of components; The distributed learning and deployment framework accepts the output of the fault diagnosis engine and the predictive maintenance decision module and updates the deep learning core module.

[0008] Compared with the prior art, the present invention has the following beneficial technical effects: The present invention provides a photovoltaic module fault diagnosis system based on deep learning. Through multimodal data fusion and advanced deep learning technology, the accuracy and comprehensiveness of fault diagnosis are significantly improved. The application of adaptive sampling strategy and edge computing improves the system operation efficiency and reduces the pressure of data transmission and storage. The multi-task learning framework enables the system to perform fault classification, severity assessment and life prediction at the same time, enhancing the depth and breadth of diagnosis. The predictive maintenance decision mechanism based on reinforcement learning optimizes the maintenance strategy, prolongs the equipment life, and reduces the operation and maintenance cost. The interpretable design of the system provides transparent and reliable diagnosis results, which is convenient for operators to understand and make decisions. The distributed learning framework ensures data privacy and continuous optimization of the model, enables the system to adapt to different environments and new fault modes, can effectively identify complex and potential faults, greatly reduces the false alarm and missed alarm rates, improves the overall power generation efficiency and economic benefits of photovoltaic power stations, and provides strong support for the large-scale application of renewable energy.

[0009] The deep learning core module utilizes a combination of multimodal deep learning architecture, cross-modal attention fusion mechanism, multi-task learning framework and dynamic graph neural network, which can utilize the complementary information of different types of data to improve the accuracy and reliability of fault diagnosis.

[0010] Preferably, the multimodal data acquisition module includes electrical parameter sensors, environmental parameter sensors, image acquisition devices and acoustic sensors. The selection and configuration of these sensors are based on comprehensive consideration of various types of failures that may occur in photovoltaic modules, comprehensively capturing the module status, and optimizing resource utilization while ensuring data quality.

[0011] Preferably, the multimodal data acquisition module adopts an adaptive sampling strategy to ensure the comprehensiveness and real-time nature of the data, which can significantly reduce the pressure of data storage and transmission while ensuring data quality. Preferably, the multi-task learning framework simultaneously performs multiple tasks such as fault classification, severity assessment, and life prediction, which improves the generalization ability of the model and enhances the overall performance of the system.

[0012] Preferably, the fault diagnosis engine integrates multiple classifiers, causal reasoning, and explainability analysis to provide accurate and reliable diagnosis results and support in-depth fault cause analysis.

[0013] Preferably, the predictive maintenance decision module generates maintenance strategies based on reinforcement learning and life prediction based on deep survival analysis to achieve long-term optimal maintenance decisions, extend equipment life, and reduce maintenance costs.

[0014] Preferably, the distributed learning and edge deployment framework utilizes an improved federated learning algorithm and efficient model compression technology to achieve multi-station collaborative learning while protecting data privacy and supports real-time diagnosis of edge devices. The present invention provides a method for electrolytic polishing of the inner surface of a large-sized shielding tube. The accuracy of fault detection and classification is significantly improved through multimodal data fusion and advanced deep learning models. Through deep survival analysis, the system can more accurately predict the remaining life of the component and support active maintenance. The maintenance decision mechanism based on reinforcement learning can automatically generate the optimal maintenance strategy based on long-term benefits. The distributed learning framework enables the system to continuously learn from new data and adapt to different environments and new failure modes. Through predictive maintenance and precise diagnosis, the system helps to reduce maintenance costs and improve the overall power generation efficiency of photovoltaic power stations. The present invention provides a comprehensive, intelligent and reliable solution for fault diagnosis and predictive maintenance of photovoltaic components. It not only improves the accuracy and timeliness of diagnosis, but also realizes continuous optimization of the system through intelligent decision support and distributed learning. This innovation will make an important contribution to the intelligent and efficient development of the photovoltaic industry and promote the wider application of renewable energy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is an overall architecture diagram of a photovoltaic module fault diagnosis system based on deep learning according to an embodiment of the present invention; Figure 2A schematic diagram of a multi-modal deep learning architecture of a photovoltaic module fault diagnosis system based on deep learning according to an embodiment of the present invention; Figure 3 This is a flow chart of a photovoltaic module fault diagnosis method based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] Example 1. Overall system architecture The present invention provides a photovoltaic module fault diagnosis system based on deep learning, and its overall architecture is as follows: Figure 1 The system is designed to achieve high-precision and high-reliability fault diagnosis and includes the following six main modules: Multimodal data acquisition module; Data preprocessing module; Deep learning core modules; Fault diagnosis engine; Predictive maintenance decision module; Distributed learning and deployment framework.

[0019] These modules work closely together to form a complete fault diagnosis and predictive maintenance system. Data starts from the multimodal data acquisition module and enters the deep learning core module after preprocessing. The output of the deep learning module is used by the fault diagnosis engine to produce diagnostic results, which are then used for predictive maintenance decisions. The entire system achieves multi-site collaboration and model updates through a distributed learning and deployment framework.

[0020] 2. Multimodal data acquisition module The multimodal data acquisition module is the foundation of the entire system and is responsible for collecting comprehensive status information of photovoltaic modules. This module uses a variety of sensors and acquisition devices, combined with innovative adaptive sampling strategies to ensure the comprehensiveness and real-time nature of the data.

[0021] In the multi-modal data acquisition module, the early detection capability of microcracks is enhanced. The specific implementation is as follows: Optimized configuration of ultrasonic sensors: In order to achieve early and accurate detection of microcracks, we have optimized the ultrasonic sensor as follows: Ultrasonic sensors with high-frequency sampling rates (1MHz) can capture more subtle changes in acoustic signals.

[0022] Use a bandpass filter (30kHz-80kHz) to effectively remove environmental noise and improve signal quality.

[0023] Apply Hilbert transform to extract signal envelope and enhance the analysis ability of transient signals.

[0024] Implement adaptive threshold technology to accurately segment effective signal segments and improve detection sensitivity.

[0025] Specific implementation of adaptive sampling strategy: Under complex meteorological conditions, we further optimized the adaptive sampling strategy: Integrate additional meteorological sensors, including PM2.5 concentration sensors, to fully capture environmental factors.

[0026] The mapping relationship between meteorological conditions and component performance is established using a Gaussian process regression model.

[0027] Dynamically adjust sampling frequency and sensor configuration based on real-time meteorological data to optimize data quality and resource usage 2.1, Sensor Network: Sensor networks include the following types of devices: a) Electrical parameter sensors: Current transformer: accuracy is ±0.5%, sampling rate is 10kHz. Used to accurately measure the output current of photovoltaic modules.

[0028] Voltage sensor: accuracy of ±0.1%, sampling rate of 10kHz. Used to accurately measure the output voltage of photovoltaic modules.

[0029] b) Environmental parameter sensors: Temperature sensor: accuracy of ±0.5°C, sampling rate of 1Hz. Used to monitor component surface temperature and ambient temperature.

[0030] Irradiance sensor: accuracy is ±2%, sampling rate is 1Hz. It is used to measure the intensity of solar radiation incident on the surface of photovoltaic modules.

[0031] c) Image acquisition equipment: Infrared thermal imaging camera: resolution 640×480, frame rate 30fps. Used to detect abnormal heat distribution on the surface of components.

[0032] High-resolution visible light camera: 4K resolution, 60fps. Used to capture visible defects on component surfaces.

[0033] d) Acoustic Sensor: Ultrasonic sensor: frequency range is 20kHz-100kHz, sampling rate is 1MHz. Used to detect tiny cracks and structural changes inside components.

[0034] The selection and configuration of these sensors are based on comprehensive consideration of the various types of failures that may occur in photovoltaic modules. High-precision electrical parameter sensors can capture subtle performance changes, environmental parameter sensors provide important background information, imaging devices can intuitively display surface anomalies, and acoustic sensors can detect internal defects that are difficult to detect with the naked eye.

[0035] 2.2 Adaptive Sampling Strategy In order to balance data quality and system resource consumption, this system adopts an innovative adaptive sampling strategy. This strategy dynamically adjusts the sampling frequency according to the real-time status of the component, ensuring that more detailed data can be obtained in abnormal situations, while reducing unnecessary data collection during normal operation.

[0036] The core of the adaptive sampling strategy is a sampling frequency adjustment mechanism based on anomaly detection. Its working principle is as follows: 2.2.1, Anomaly score calculation: The system uses a lightweight anomaly detection algorithm (such as Isolation Forest) to calculate the anomaly score S of the current state in real time. The calculation of the anomaly score is based on multimodal data collected in the recent period, including electrical parameters, environmental parameters, and simplified image features.

[0037] 2.2.2, Sampling frequency adjustment: Based on the calculated anomaly score S, the system dynamically adjusts the sampling frequency. The adjustment formula is as follows:

[0038] in: is the actual sampling frequency; The sampling frequency is normal (for example, 1Hz for electrical parameters and 0.017Hz for image data, i.e. one frame per minute). is a high-frequency sampling frequency (e.g., 100 Hz for electrical parameters, 1 Hz for image data); is the calculated anomaly score; It is a preset abnormal threshold, which is determined through historical data analysis and expert experience.

[0039] 2.2.3, Smooth transition mechanism: In order to avoid sudden changes in the sampling frequency, a smooth transition mechanism is introduced. When an abnormality is detected, the sampling frequency will not jump to $f_h$ immediately, but will increase smoothly within T seconds:

[0040] Where t is the time after the abnormality is detected, and T is the preset transition time (e.g., 5 seconds).

[0041] 2.2.4, Duration and Cooldown Period: After high-frequency sampling continues for a preset period of time (e.g. 5 minutes), if the abnormal situation persists, the system will issue an alarm; if it returns to normal, it will enter a cooling period and the sampling frequency will gradually decrease back to normal. .

[0042] 2.2.5, Data caching mechanism: The system maintains a circular buffer that temporarily saves the high-frequency sampling data of the recent period (such as 30 seconds) even at the normal sampling frequency. When an abnormality is detected, this data will be saved for subsequent analysis of the precursors to the abnormality.

[0043] This adaptive sampling strategy can significantly reduce the pressure of data storage and transmission while ensuring data quality. It works closely with the subsequent data preprocessing module and deep learning core module to provide a solid data foundation for early detection and diagnosis of faults.

[0044] 3. Data preprocessing module The data preprocessing module plays a key role as a bridge between raw data and deep learning analysis. It is responsible for cleaning and transforming the raw data from the multimodal data acquisition module to provide high-quality input for subsequent deep learning analysis. This module includes three main steps: data cleaning, data standardization, and feature extraction.

[0045] 3.1 Data Cleaning Data cleaning is the first line of defense to ensure data quality. This system uses an improved local outlier factor (LOF) algorithm to detect and process outliers. The advantage of the LOF algorithm is that it can handle multidimensional data and can work effectively for data sets of different densities.

[0046] The improved LOF algorithm workflow is as follows: 3.1.1. For each data point p, calculate its k-distance and k-distance neighborhood $N_k(p)$; 3.1.2. Calculate the local reachability density (lrd) of each point:

[0047] in is the distance between points p and o; Calculate LOF score:

[0048] 3.1.4. Introduce the time decay factor to make the algorithm focus more on the most recent data;

[0049] in is the attenuation coefficient, is the current time, is the timestamp of data point p; 3.1.5. Setting the threshold ,if , then p is marked as an outlier; For data points marked as abnormal, the system will adopt different processing strategies based on the data type: For time series data (such as electrical parameters), use interpolation methods (such as spline interpolation) to replace outliers; For discrete events (such as equipment status changes), the abnormal markers are retained but not replaced, and special attention is paid to them in subsequent analysis; For image data, if a single frame is marked as an anomaly, we try to replace it with the average of the previous and next frames.

[0050] 3.2 Data Standardization Data standardization is a key step to ensure that data of different scales and units can be processed fairly by deep learning models. This system uses different standardization strategies for different types of data: 3.2.1. Numerical data (such as electrical parameters, environmental parameters): Z-score standardization is used, and the formula is as follows:

[0051] in is the original value, is the mean value of this feature, is the standard deviation. and Dynamic calculation is performed through a sliding window (such as 24 hours) to adapt to the long-term change trend of the data.

[0052] 3.2.2. Image data: For thermal and visible light images, pixel value normalization is used:

[0053] in is the original image, and are the minimum and maximum pixel values ​​of the image, respectively.

[0054] 3.2.3. Acoustic data: For ultrasonic signals, amplitude normalization is first performed:

[0055] The spectrum is then log-transformed:

[0056] This helps to highlight small changes in the signal.

[0057] 3.3 Feature Extraction Feature extraction aims to extract meaningful features from standardized data, reduce data dimensions, and highlight key information. This system uses different feature extraction methods for different types of data: 3.3.1, Time Series Data Feature Extraction: a) Time domain characteristics: Statistics: mean, variance, skewness, kurtosis Trend characteristics: first-order difference, second-order difference Extreme value characteristics: maximum value, minimum value, peak-to-peak value b) Frequency domain characteristics: The power spectral density (PSD) is calculated using fast Fourier transform (FFT), and the energy of key frequency bands is used as features.

[0058] c) Time-frequency domain characteristics: Wavelet packet transform (WPT) is used for multi-scale analysis. The parameters of WPT are set as follows: Wavelet basis: db4 (Daubechies 4th order wavelet); Decomposition layers: 5 layers; The energy entropy and time entropy of each subband are extracted as features.

[0059] 3.3.2, Image data feature extraction: a) Deep Features: Use the pre-trained ResNet-50 model to extract deep features. The specific steps are: Resize the input image to 224×224; Pass the first few layers of ResNet-50 (until the last convolutional layer); The feature map is extracted and a 2048-dimensional feature vector is obtained through global average pooling.

[0060] b) Texture features: Calculate the gray-level co-occurrence matrix (GLCM) and extract the following statistics: Contrast, correlation, energy, and homogeneity.

[0061] 3.3.3, Acoustic Data Feature Extraction: a) Time domain characteristics: Similar to time series data, extract statistics and extreme value features.

[0062] b) Frequency domain characteristics: Spectrograms were calculated using short-time Fourier transform (STFT); Extract Mel-frequency cepstral coefficients (MFCC) from the spectrogram; Calculate the frequency band energy ratio.

[0063] c) Time-frequency domain characteristics: Continuous wavelet transform (CWT) is used to perform time-frequency analysis and extract the statistical characteristics of wavelet coefficients.

[0064] All extracted features are organized into a unified feature vector as the input of the deep learning core module. This multi-modal and multi-scale feature extraction method ensures a comprehensive characterization of the PV module status and lays a solid foundation for subsequent fault diagnosis.

[0065] 4. Deep learning core modules: The deep learning core module is the core of this system, responsible for processing the multimodal data from the data preprocessing module and outputting high-level feature representations; the output of the deep learning core module is a multi-dimensional feature vector and a preliminary analysis result set, where the feature vector specifically includes time series data features, image data features, and acoustic data features.

[0066] The initial analysis result set includes: fault type probability distribution, fault severity score, remaining useful life prediction, fault location heat map, system state diagram embedding, and uncertainty estimation: The innovation of this module lies in its combination of multimodal deep learning architecture, cross-modal attention fusion mechanism, multi-task learning framework and dynamic graph neural network. This complex architecture design aims to make full use of the complementary information of different types of data to improve the accuracy and reliability of fault diagnosis.

[0067] 4.1, Multimodal Deep Learning Architecture: See also Figure 2 ,The multimodal deep learning architecture includes three parallel data processing branches, which process time series data, image data, and acoustic data, respectively.,The design of each branch is optimized for its specific data type.

[0068] 4.1.1, Time Series Data Processing Branch The timing data processing branch is mainly used to process timing data such as electrical parameters and environmental parameters. Its structure is as follows: a) Causal Convolutional Layer: Number of layers: 5; Parameters of each layer: Convolution kernel size: 3; Expansion rate: [1, 2, 4, 8, 16] (increasing layer by layer); Number of filters: [64, 64, 128, 128, 256]; Activation function: LeakyReLU (α = 0.1).

[0069] The use of causal convolutions ensures that the model does not exploit future information, which is crucial for real-time fault detection. The increase in dilation rate enables the model to capture long-term dependencies.

[0070] b) Temporal Convolutional Network (TCN) layer: Number of layers: 2; Parameters of each layer; Number of hidden units: 128; Convolution kernel size: 3; dilation rate: [1, 2]; Residual connection: Use 1x1 convolution for dimension matching.

[0071] The introduction of TCN further enhances the model's ability to capture long-term temporal dependencies while keeping the number of parameters low.

[0072] 4.1.2, Image data processing branch: The image data processing branch is used to process thermal imaging and visible light image data. Its structure is as follows: a) Backbone network: EfficientNet-V2-S: Pre-training: Pre-trained on the ImageNet dataset; Fine-tune: Freeze the first few layers and only fine-tune the later layers; b) Spatial attention module; Channel attention: using Squeeze-and-Excitation (SE) blocks;

[0073] in, is the global average pooling, is the ReLU activation function, is the Sigmoid activation function.

[0074] Spatial attention: Use convolutional layers to generate 2D attention maps;

[0075] in, is the convolution operation, and They are average pooling and maximum pooling respectively.

[0076] This dual attention mechanism can highlight key areas and features in the image, which helps detect local faults such as hot spots and cracks.

[0077] 4.1.3, Acoustic Data Processing Branch: The acoustic data processing branch is specifically used to process acoustic signal data such as ultrasonic waves. Its structure is as follows: a) One-dimensional Convolutional Neural Network (1D-CNN): 3 layers of convolution, parameters for each layer; Convolution kernel size: [64, 32, 16] (decreases layer by layer; Step size: [2, 2, 2]; Number of channels: [32, 64, 128] (increases layer by layer); Each layer is followed by BatchNormalization and ReLU activation functions.

[0078] b) Fully connected layer: 2 layers, number of hidden units: [256, 128]; Dropout (rate 0.5) was used to prevent overfitting.

[0079] This structural design can effectively extract frequency and time features in acoustic signals, which helps detect internal faults such as microcracks.

[0080] 4.2, Cross-modal attention fusion mechanism: The purpose of the cross-modal attention fusion mechanism is to dynamically adjust the importance of data in different modalities to achieve effective feature fusion. This mechanism is based on the encoder structure of Transformer and is implemented as follows: 4.2.1, Feature Alignment: First, the outputs of the three branches are projected to the same dimensional space (e.g., 256 dimensions) through a fully connected layer.

[0081] 4.2.2, Transformer encoder: Number of floors: 6; Multi-head attention: 8 heads; Feedforward network dimension: 1024; LayerNorm and residual connections.

[0082] 4.2.3, cross-modal attention calculation: For each modality $m$, calculate its attention with other modalities:

[0083] in, is the query matrix of mode $m$, and is the key and value matrix of all modal features.

[0084] 4.2.4, Gating Mechanism: The weights of each modality are dynamically adjusted using a Gated Linear Unit (GLU):

[0085] in, is the original feature, is the attention-weighted feature, Represents element-wise multiplication.

[0086] 4.2.5, Feature Fusion: The final fusion features are:

[0087] This fusion mechanism can adaptively adjust the importance of each mode according to different working conditions and fault types, improving the robustness and generalization ability of the model.

[0088] 4.3, Multi-task learning framework: The multi-task learning framework aims to complete multiple related tasks at the same time and make full use of the correlation information between tasks. In this system, the multi-task learning framework includes the following tasks: 4.3.1, Fault classification: Identify the fault type (such as hot spots, PID, micro cracks, etc.); 4.3.2, Fault severity assessment: assess the severity of the fault (0-100 points); 4.3.3, Remaining service life prediction: predict the remaining service life of the component; 4.3.4, Fault location positioning: locate the fault area on the image.

[0089] The specific implementation is as follows: a) Shared encoder: Use cross-modal fused features as shared representations.

[0090] b) Task-specific decoders: Fault classification: fully connected layer + Softmax; Severity assessment: fully connected layer + ReLU; Life span prediction: LSTM + fully connected layer; Position localization: transposed convolutional networks.

[0091] c) Loss function: Multi-task loss function using uncertainty weighting:

[0092] in, For each task-specific loss (e.g. cross entropy, mean squared error, etc.), It is the task uncertainty learned by the network.

[0093] d) Optimization strategy: Optimizer: Adam; Learning rate scheduling: Use Cyclical Learning Rates to better handle multi-task scenarios; Gradient accumulation: handles the problem of unbalanced gradient updates for different tasks.

[0094] 4.4, Dynamic Graph Neural Network: Dynamic graph neural network is used to model the dynamic relationship between photovoltaic modules and capture system-level failures. Its implementation is based on space-time graph convolutional network (ST-GCN), as follows: 4.4.1, Graph Construction: Node: Each photovoltaic module is a node; Edge: Defined based on physical connections and correlation coefficients.

[0095] 4.4.2, Feature Representation: Node characteristics include component electrical parameters, environmental parameters, and high-level characteristics from other branches.

[0096] 4.4.3. ST-GCN layer: Number of layers: 3; Parameters of each layer: Spatial convolution kernel size: 3; Temporal convolution kernel size: 3; Attention Mechanism: Use Graph Attention Network (GAT) to dynamically update edge weights.

[0097] 4.4.4, Time Dynamic Modeling: Introducing time embedding to capture the time evolution of component states:

[0098] in, Is a node In the The characteristics of the layer, is the time embedding, is the attention coefficient.

[0099] 4.4.5, Output layer: The graph pooling operation is used to obtain the representation of the entire system for subsequent fault diagnosis tasks.

[0100] Through this dynamic graph neural network, the system can effectively model the mutual influence between components, which helps to detect system-level problems such as series and parallel failures.

[0101] This deep learning core module achieves comprehensive status assessment and high-precision fault diagnosis of photovoltaic modules by fusing multimodal data, using attention mechanism and graph structure, and performing multiple related tasks at the same time. The output of this module will be passed to the fault diagnosis engine for further analysis and decision-making.

[0102] 5. Fault diagnosis engine: The fault diagnosis engine integrates the output of the deep learning core module to achieve fault type identification, severity assessment and cause reasoning.

[0103] 5.1, Fault type identification: Adopt a hierarchical classification strategy, combining multiple classifiers: 5.1.1, Classifier Structure: First level: classification by major categories (such as electrical failure, optical failure, mechanical failure, etc.); Second level: sub-categories (such as hot spots, PID, micro cracks, bypass diode failure, etc.).

[0104] 5.1.2, Classification Algorithm: Integrate deep learning multi-task output, support vector machine (SVM) and random forest.

[0105] 5.1.3, Decision Fusion: Dempster-Shafer evidence theory is used for decision fusion to calculate the credibility of each fault type. This method can effectively handle the uncertainty and conflict of different classifier outputs.

[0106] 5.1.4, Dynamic Threshold Adjustment: The classification threshold is adaptively adjusted according to the historical diagnosis accuracy to improve the long-term performance of the system.

[0107] 5.2, Fault severity assessment: Using a regression model, output a continuous score from 0 to 100: 5.2.1, Regression Model: The gradient boosted decision tree (GBDT) is used, and XGBoost is used for its specific implementation, considering its advantages in handling complex nonlinear relationships.

[0108] 5.2.2, Feature Importance Analysis: Feature importance is calculated using SHAP (SHapley Additive exPlanations) values, providing interpretable severity assessment results.

[0109] 5.2.3, Uncertainty Quantification: Combining ensemble learning and Monte Carlo Dropout provides reliable uncertainty estimates and helps make more robust decisions.

[0110] 5.3 Reasoning of the cause of the failure: Based on structured causal models (SCM) and Bayesian networks: 5.3.1, Causal Graph Construction: Combining expert knowledge and data-driven methods (such as PC algorithms) to build causal graphs.

[0111] 5.3.2, Parameter Learning: Learn conditional probability tables using maximum likelihood estimation.

[0112] 5.3.3, Inference Algorithm: A variational inference method is adopted, using mean field approximation to improve the inference efficiency of large-scale networks.

[0113] 5.3.4, Counterfactual Analysis: Use do-calculus to conduct intervention analysis to evaluate the impact of different factors on failures and provide in-depth cause analysis.

[0114] 5.4, ​​Interpretability Analysis: Integrate multiple interpretation technologies to improve the interpretability and credibility of diagnostic results: 5.4.1. Global Explanation: Using SHAP Values ​​to Explain the Overall Behavior of the Model; 5.4.2. Local interpretation: using Grad-CAM for image data and LIME for time series data; 5.4.3. Counterfactual explanations: Generate counterfactual explanations that explain which factors might have changed the diagnosis; 5.4.4. Natural Language Explanation: Convert technical explanations into understandable text descriptions.

[0115] 6. Predictive maintenance decision module: 6.1, Maintenance strategy generation: Adopt a model-based reinforcement learning approach, specifically using the Soft Actor-Critic (SAC) algorithm: 6.1.1, State space: including component status, environmental conditions, historical maintenance records, etc.; 6.1.2, Action Space: includes maintenance operations at different levels (such as monitoring, cleaning, repair, replacement, etc.); 6.1.3, Reward function: Comprehensively consider factors such as power generation, maintenance costs, and equipment life; 6.1.4, Policy Network and Value Network: Constructed using Deep Neural Network; 6.1.5, Training algorithm: Use the SAC algorithm to balance exploration and utilization; 6.1.6, Experience Replay: Use Priority Experience Replay (PER) to improve learning efficiency.

[0116] This reinforcement learning-based approach is able to adapt to complex decision-making environments and continuously optimize maintenance strategies as experience accumulates.

[0117] 6.2, Remaining service life prediction: A deep survival analysis model is used, combining a temporal convolutional network (TCN) and Weibull distribution: 6.2.1, Feature extraction: Use TCN to extract time series features; 6.2.2, Weibull distribution parameter prediction: Predict the shape parameter and scale parameter of Weibull distribution based on TCN output; 6.2.3, Survival function and density function: calculation of Weibull parameters based on prediction; 6.2.4, Loss function: Use negative log-likelihood as the loss function; 6.2.5, Forecast: Calculate the expected value of the remaining useful life.

[0118] This method combines the powerful feature extraction capabilities of deep learning with the statistical basis of traditional survival analysis to provide more accurate and reliable lifespan predictions.

[0119] 7. Distributed learning and deployment framework: 7.1, Federated Learning Framework: The improved FedAvg algorithm is used to introduce knowledge distillation and differential privacy: 7.1.1, Client selection: Use importance sampling, based on data quality and diversity; 7.1.2, Local Update: Each client uses SGD to update the local model; 7.1.3, Model Aggregation: Aggregate the updated model on the server side; 7.1.4, Knowledge Distillation: Use the global model as a teacher to guide the learning of the local model; 7.1.5, Differential Privacy: Adding noise to client updates to preserve privacy; This improved federated learning framework can improve the generalization ability and learning efficiency of the model while protecting data privacy.

[0120] 7.2, Edge Deployment; To achieve efficient deployment of models on edge devices, the following technologies are used: 7.2.1, Knowledge Distillation: Transferring knowledge from large models to small models; 7.2.2, Pruning: Use structured pruning to remove unimportant convolution kernels; 7.2.3, Quantization: Use the dynamic range quantization (DRQ) method to perform 8-bit fixed-point quantization; 7.2.4, Model Optimization: Use NVIDIA TensorRT for model optimization, including operator fusion, memory optimization, and precision calibration; 7.2.5, Deployment of hardware: Deploy the optimized model on NVIDIA Jetson Xavier NX; 7.2.6, Reasoning Optimization: Use the CUDA deep neural network library to accelerate convolution operations and adopt batch processing strategies to improve GPU utilization; The combination of these technologies enables complex deep learning models to run efficiently on resource-constrained edge devices, enabling real-time fault diagnosis and prediction.

[0121] The present invention realizes high-precision and high-reliability fault diagnosis and predictive maintenance through innovative multimodal data fusion, deep learning architecture and intelligent decision-making mechanism. The main innovations and beneficial effects of the system are as follows: 1. Multimodal data collection and adaptive sampling strategy: Innovation: Integrates multiple sensors and uses dynamically adjusted sampling frequency.

[0122] Beneficial effects: Comprehensively capture component status and optimize resource usage while ensuring data quality.

[0123] 2. Deep learning core modules: Innovation: Combination of multimodal deep learning architecture, cross-modal attention fusion mechanism and dynamic graph neural network.

[0124] Beneficial effects: Make full use of the complementary information of different types of data to improve the accuracy and comprehensiveness of fault diagnosis.

[0125] 3. Multi-task learning framework: Innovation: Simultaneously performs multiple tasks such as fault classification, severity assessment, and life prediction.

[0126] Beneficial effects: Improve the generalization ability of the model and enhance the overall performance of the system.

[0127] 4. Fault diagnosis engine: Innovation: Integration of multiple classifiers, causal reasoning, and explainability analysis.

[0128] Beneficial Effects: Provides accurate and reliable diagnostic results and supports in-depth fault cause analysis.

[0129] 5. Predictive maintenance decision module: Innovation: Maintenance strategy generation based on reinforcement learning and lifespan prediction based on deep survival analysis.

[0130] Beneficial effects: Achieve long-term optimal maintenance decisions, extend equipment life, and reduce maintenance costs.

[0131] 6. Distributed learning and edge deployment framework: Innovation: Improved federated learning algorithm and efficient model compression technology.

[0132] Beneficial effects: Enable multi-station collaborative learning while protecting data privacy, and support real-time diagnosis of edge devices.

[0133] Through these innovations, this system has the following significant advantages over traditional methods: 1. Improved diagnostic accuracy: Multimodal data fusion and advanced deep learning models significantly improve the accuracy of fault detection and classification.

[0134] 2. Enhanced predictive capabilities: Through deep survival analysis, the system can more accurately predict the remaining life of components and support proactive maintenance.

[0135] 3. Intelligent decision-making: The maintenance decision-making mechanism based on reinforcement learning can automatically generate the optimal maintenance strategy based on long-term benefits.

[0136] 4. Enhanced explainability: Integrate multiple explanation technologies to make diagnostic results more transparent and credible, facilitating human-computer collaboration.

[0137] 5. Adaptability and scalability: The distributed learning framework enables the system to continuously learn from new data and adapt to different environments and new failure modes.

[0138] 6. Resource optimization: Adaptive sampling and edge computing technologies optimize the use of data collection and processing resources.

[0139] 7. Economic benefits: Through predictive maintenance and precise diagnosis, the system helps reduce maintenance costs and improve the overall power generation efficiency of photovoltaic power plants.

[0140] In summary, this invention provides a comprehensive, intelligent, and reliable solution for fault diagnosis and predictive maintenance of photovoltaic modules. It not only improves the accuracy and timeliness of diagnosis, but also achieves continuous optimization of the system through intelligent decision support and distributed learning. This innovation will make an important contribution to the intelligent and efficient development of the photovoltaic industry and promote the wider application of renewable energy.

[0141] In addition, the present invention also provides a diagnostic method for a photovoltaic module fault diagnosis system based on deep learning, such as Figure 3 Said, comprising the following steps: The electrical parameters, environmental parameters, image data and acoustic data of the photovoltaic module are collected by the multimodal data collection module, and the collected data of the multimodal data collection module are preprocessed by the data preprocessing module; The preprocessed multimodal data is input into the deep learning core module. The fault diagnosis engine identifies the fault type, evaluates the fault severity, and infers the fault cause based on the output of the deep learning core module. The fault type identification, fault severity assessment and fault cause reasoning output by the fault diagnosis engine are used as inputs to the predictive maintenance decision module to generate maintenance strategies and predict the remaining service life of components; The distributed learning and deployment framework accepts the output of the fault diagnosis engine and the predictive maintenance decision module and updates the deep learning core module.

[0142] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited thereto. Within the technical concept of the present invention, the technical solution of the present invention can be subjected to a variety of simple modifications, including the combination of various technical features in any other suitable manner, and these simple modifications and combinations should also be regarded as the contents disclosed by the present invention and belong to the protection scope of the present invention.

Claims

1. A photovoltaic module fault diagnosis system based on deep learning, characterized in that: include: Multimodal data acquisition module, used to collect electrical parameters, environmental parameters, image data and acoustic data of photovoltaic modules; A data preprocessing module, used for cleaning, standardizing and feature extracting the multimodal data; Deep learning core modules, including multimodal deep learning architecture, cross-modal attention fusion mechanism, multi-task learning framework and dynamic graph neural network, are used to process pre-processed multimodal data; A fault diagnosis engine, used to identify fault types, assess fault severity, and infer fault causes based on the output of the deep learning core module; A predictive maintenance decision module for generating maintenance strategies and predicting the remaining useful life of components; Distributed learning and deployment framework, used to achieve collaborative learning and model edge deployment among multiple PV plants; Among them, the deep learning core module realizes comprehensive status assessment and fault diagnosis of photovoltaic components by fusing multimodal data, using dynamic graph neural network structure to model the relationship between components, and simultaneously executing multiple related tasks.

2. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The multimodal data acquisition module includes Electrical parameter sensors, used to collect current and voltage data; Environmental parameter sensors, used to collect temperature and irradiance data; Image acquisition equipment, including infrared thermal imaging cameras and high-resolution visible light cameras; Acoustic sensor, used to collect ultrasonic signals; The multimodal data acquisition module also includes an adaptive sampling strategy to dynamically adjust the sampling frequency according to the component status.

3. A photovoltaic module fault diagnosis system based on deep learning according to claim 2, characterized in that: The adaptive sampling strategy includes: Use lightweight anomaly detection algorithms to calculate the anomaly score of the current state in real time; Dynamically adjust the sampling frequency based on the anomaly score, and increase the sampling frequency when an anomaly is detected; Adopt smooth transition mechanism to avoid sudden changes in sampling frequency; Set the duration and cooldown period of high-frequency sampling; Maintain a circular buffer to save the high-frequency sampling data of the recent period.

4. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The data preprocessing module comprises: Data cleaning unit, which uses an improved local anomaly factor algorithm to detect and process outliers; Data normalization unit, which performs Z-score normalization on numerical data, pixel value normalization on image data, amplitude normalization and logarithmic transformation on acoustic data; The feature extraction unit extracts time domain, frequency domain and time-frequency domain features for time series data, extracts deep features and texture features for image data, and extracts time domain, frequency domain and time-frequency domain features for acoustic data.

5. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The multimodal deep learning architecture includes: The time series data processing branch uses multi-layer causal convolution and temporal convolutional networks; Image data processing branch, using the EfficientNet-V2 backbone network and dual attention mechanism, including channel attention and spatial attention; Acoustic data processing branch, using one-dimensional convolutional neural networks and fully connected layers; The cross-modal attention fusion mechanism uses the Transformer encoder and gating mechanism to dynamically adjust the weights of different modalities.

6. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The multi-task learning framework includes: Shared encoder and task-specific decoder; Perform four tasks: fault classification, fault severity assessment, remaining service life prediction, and fault location; Adopt uncertainty-weighted multi-task loss function; Use the Adam optimizer and the Cyclical Learning Rates learning rate scheduling strategy.

7. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The dynamic graph neural network is constructed based on a space-time graph convolutional network, including: The PV modules are represented as nodes in the graph, and the physical connections and dependencies between the modules are represented as edges; Use graph attention network to dynamically update edge weights; Introducing time embedding to capture the time evolution of component states; A graph pooling operation is used to obtain a representation of the entire system.

8. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The fault diagnosis engine comprises: A hierarchical classifier based on the output of the deep learning model to identify multiple fault types; Regression models to assess fault severity; A reasoning unit based on a structured causal model to analyze the causes of failures; An interpretability analysis module integrating Grad-CAM and LIME is used to provide explanations for diagnostic results.

9. A photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The predictive maintenance decision module generates maintenance strategies based on a reinforcement learning system of a soft actor-critic algorithm and uses a life prediction model of a temporal convolutional network to output a probability distribution of the remaining service life; The distributed learning and deployment framework includes: A federated learning framework based on the federated averaging algorithm; Use knowledge distillation and quantization techniques to achieve model compression; Adopting gradient accumulation strategy to deal with the imbalanced gradient update problem of different tasks; Enable efficient deployment on edge computing devices.

10. A diagnostic method for a photovoltaic module fault diagnosis system based on deep learning according to claim 1, characterized in that: The following steps are involved: The electrical parameters, environmental parameters, image data and acoustic data of the photovoltaic module are collected by the multimodal data collection module, and the collected data of the multimodal data collection module are preprocessed by the data preprocessing module; The preprocessed multimodal data is input into the deep learning core module. The fault diagnosis engine identifies the fault type, evaluates the fault severity, and infers the fault cause based on the output of the deep learning core module. The fault type identification, fault severity assessment and fault cause reasoning output by the fault diagnosis engine are used as inputs to the predictive maintenance decision module to generate maintenance strategies and predict the remaining service life of components; The distributed learning and deployment framework accepts the output of the fault diagnosis engine and the predictive maintenance decision module and updates the deep learning core module.

Images