Photovoltaic module hot spot detection method and system based on multi-source data and deep learning

Abstract

The invention belongs to the technical field of photovoltaic module detection, and provides a photovoltaic module hot spot detection method and system based on multi-source data and deep learning, a spectral image and a heat ray image are fused, the spectral image can capture subtle changes of the surface of a photovoltaic module, and the heat ray image directly reflects temperature distribution. The combination of the two provides a more comprehensive information source; by means of data fusion, more accurate detection can be carried out in a complex environment, especially in a hot spot area with the temperature close to a normal value, and therefore the accuracy and reliability of hot spot prediction are improved. Hot spot region prediction is carried out by using a depth transformation auto-encoder model, high-dimensional and multi-modal data can be effectively processed, and useful features can be extracted from the data. Through a real-time prediction and evaluation mechanism, operation and maintenance personnel can identify the hot spot problem in the photovoltaic module in time, the risk of system fault occurrence is reduced, and finally the operation and maintenance efficiency and reliability of the photovoltaic power station are improved.

Description

Technical Field

[0001] This invention belongs to the field of photovoltaic module testing technology, specifically relating to a method and system for detecting hot spots in photovoltaic modules based on multi-source data and deep learning. Background Technology

[0002] With the continuous growth of global demand for renewable energy, photovoltaic (PV) power generation has become one of the important forms of energy. PV power plants convert solar energy into electricity, providing a sustainable solution for the transformation of the global energy structure. However, the operating efficiency of PV modules can be affected by environmental factors, equipment aging, and other factors, leading to hot spot problems in certain areas. Hot spots refer to abnormally high localized temperatures on the surface of PV modules caused by uneven illumination or internal damage to the modules.

[0003] Hot spot detection methods for photovoltaic (PV) modules are mainly divided into traditional infrared thermography and automated detection methods based on computer vision technology. Infrared thermography captures the temperature distribution on the PV module surface in real time using an infrared thermal imager, identifies and marks abnormal temperature areas, and is intuitive and easy to use, capable of identifying hot spot locations to a certain extent. However, this method has some problems. First, it may not be sensitive enough for small hot spot areas. Second, under complex lighting conditions, infrared images may be affected by environmental factors, leading to a decrease in detection accuracy. Furthermore, infrared thermal images often lack sufficient detail, failing to provide accurate hot spot location and severity scores. To improve detection accuracy, some studies have attempted to combine spectral images and thermal X-ray images, utilizing spectral information for a deeper analysis of the PV module surface, thereby improving the accuracy of hot spot detection. However, using spectral images alone for hot spot detection still has limitations, especially when the temperature is close to normal, where spectral information contributes little to the hot spot detection, resulting in poor detection performance. Existing methods lack effective solutions for multimodal data fusion, failing to fully utilize the advantages of spectral and thermal X-ray images to improve the accuracy and robustness of hot spot detection. Therefore, multimodal data fusion methods combining thermal X-ray data and spectral data have become an effective approach to improve detection accuracy. However, existing methods lack precise hotspot region prediction and assessment mechanisms, failing to provide detailed information on hotspot location, size, and severity scores, thus impacting decision support for maintenance personnel. Summary of the Invention

[0004] The purpose of this invention is to solve the problems of low accuracy, difficulty in data fusion, and poor real-time performance in existing photovoltaic module hot spot detection technologies, and to provide a photovoltaic module hot spot detection method and system based on multi-source data and deep learning.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0006] A method for detecting hot spots in photovoltaic modules based on multi-source data and deep learning includes:

[0007] (1) Data acquisition and preprocessing: Spectral images and thermal images of photovoltaic modules are acquired by a spectral imager and an infrared thermal imager, respectively. The acquired images are preprocessed and then stitched together with the thermal images to obtain spectral-thermal fusion data containing spectral and temperature information.

[0008] (2) Construction of deep transform autoencoder model: The spectral-thermal ray fusion data is input into the deep transform autoencoder model containing encoder and decoder. The deep transform autoencoder model learns the joint features of spectral and thermal ray data and captures key information of hot spot region in latent space. The encoder compresses the input data into latent space through multi-layer convolutional neural network, and the decoder reconstructs the latent space features into hot spot prediction image through deconvolutional network.

[0009] (3) Hot spot region prediction and evaluation: The input spectral-thermal ray fusion data is inferred based on the deep transform autoencoder model, and the hot spot prediction results are output; the hot spot region is extracted by binarization and connected component analysis, and multiple evaluation indicators are used to quantitatively evaluate the hot spot prediction results;

[0010] (4) Model training and optimization: Based on the spectral-thermal ray fusion data obtained in step (1), train the deep transform autoencoder model and monitor the loss and evaluation metrics on the validation set until convergence;

[0011] (5) Deployment and application: The trained depth transform autoencoder model is deployed to the edge computing device. The real-time collected spectral and thermal ray data are input into the edge computing device for real-time processing, and the hot spot location, size and severity score are output and displayed through the user interface.

[0012] Furthermore, in step (1), the preprocessing of the spectral image includes grayscale conversion, normalization and cropping of each band, and the preprocessing of the thermal ray image includes Gaussian filtering for noise reduction and temperature normalization.

[0013] Furthermore, in step (2), the encoder adopts a multi-layer convolutional neural network structure, with a BatchNorm layer and a ReLU activation function added after each convolutional layer;

[0014] The latent space is sampled using a reparameterization technique, as shown in the formula:

[0015] z=μ(x)+σ(x·∈

[0016] Where ∈ represents the standard normal distribution noise, and μ(x) and σ(x) are the mean and standard deviation of the encoder output;

[0017] The decoder uses deconvolutional layers to progressively upsample, and adds a BatchNorm layer and a ReLU activation function after each deconvolution layer.

[0018] Furthermore, in step (2), the constructed depth transform autoencoder model is optimized using reconstruction loss and KL divergence loss. The reconstruction loss is used to measure the difference between the input image and the reconstructed image, and the mean squared error is used as the reconstruction loss, expressed as:

[0019]

[0020] Where, x i For the input image, To reconstruct the image, N is the total number of pixels in the image;

[0021] KL divergence loss is used to measure the difference between the latent spatial distribution and the standard normal distribution, and is expressed as:

[0022]

[0023] Where μ and σ 2 These are the mean and variance of the encoder output, respectively.

[0024] The weighted sum of reconstruction loss and KL divergence loss is defined as the final loss function, expressed as:

[0025] L total =L recon +λ*L KL

[0026] Wherein, λ is a hyperparameter that adjusts the weight between the reconstruction loss and the KL divergence loss, and is set to 0.5.

[0027] Furthermore, in step (3), the evaluation metrics include mean squared error, crossover ratio, precision, and recall.

[0028] Furthermore, in step (4), the specific steps for model training and optimization include:

[0029] Dataset construction and model preparation: The spectral-thermal ray fusion data is divided into training, validation, and test sets;

[0030] Hyperparameter settings: The Adam optimizer is used for training. The initial learning rate, batch size and training period are set. The learning rate adopts a gradual decay strategy. The training stops when the validation set loss fails to decrease effectively over multiple consecutive training periods.

[0031] Data augmentation: During training, data augmentation is performed using rotation, translation, and brightness adjustment methods;

[0032] Validation and tuning: After each training cycle, the loss is calculated on the validation set and the hyperparameters are tuned to further optimize the model's performance.

[0033] Another objective of this invention is to provide a photovoltaic module hot spot detection system based on multi-source data fusion and deep learning, comprising:

[0034] The data acquisition and preprocessing module is used to acquire spectral images and thermal ray images of photovoltaic modules through a spectral imager and an infrared thermal imager, respectively, and to preprocess the acquired images. The preprocessed spectral images are then stitched together with the thermal ray images to obtain spectral-thermal ray fusion data containing spectral and temperature information.

[0035] The deep transform autoencoder model building module is used to input spectral-thermal ray fusion data into a deep transform autoencoder model containing an encoder and a decoder. The deep transform autoencoder model learns the joint features of spectral and thermal ray data and captures key information of hot spot regions in the latent space.

[0036] The hot spot region prediction and evaluation module is used to perform inference on the input spectral-thermal ray fusion data based on the depth transform autoencoder model and output hot spot prediction results; hot spot regions are extracted through binarization processing and connected component analysis, and multiple evaluation indicators are used to quantitatively evaluate the hot spot prediction results;

[0037] The model training and optimization module is used to train a deep transform autoencoder model using spectral-thermal ray fusion data, and monitor the loss and evaluation metrics on the validation set until convergence.

[0038] The model deployment and application module is used to deploy the trained deep transform autoencoder model to the edge computing device. The real-time collected spectral and thermal radiation data are input into the edge computing device for real-time processing, and the hot spot location, size and severity score are output and displayed through the user interface.

[0039] In combination with the above technical solutions, the beneficial effects of the present invention compared with the prior art are as follows:

[0040] Innovative Application of Multimodal Data Fusion: Traditional photovoltaic hot spot detection methods typically use spectral data or thermal X-ray images as input data alone, which limits the complementarity and comprehensiveness of information. This invention, however, effectively combines the high spatial resolution of spectral data with the temperature distribution advantages of thermal X-ray images by fusing spectral data and thermal X-ray images, overcoming the limitations of a single data source. By deeply fusing these two types of data, the system can more accurately identify and locate hot spot regions, especially in small hot spot areas with temperatures close to normal values, improving detection sensitivity and accuracy.

[0041] Innovative Application of Deep Transformation Variational Autoencoder (DT-VAE) Structure: While autoencoders are widely used in feature learning and dimensionality reduction in existing technologies, this invention uniquely employs a deep transformation autoencoder, enabling it to fully learn the nonlinear relationships and high-dimensional features of multimodal data. Unlike traditional autoencoder methods, this invention uses a transformation strategy during the latent space transformation process, effectively enhancing feature extraction capabilities through a deep network structure and reducing noise interference with prediction results. This deep transformation autoencoder not only better compresses input data but also performs efficient data reconstruction, further improving the accuracy and stability of hotspot prediction.

[0042] As can be seen, this invention overcomes the limitations of traditional methods that rely on only a single data source by combining multimodal data from spectral and thermal X-ray images and utilizing a deep transform autoencoder model for hot spot region prediction. Spectral images can capture subtle changes on the surface of photovoltaic modules, while thermal X-ray images directly reflect temperature distribution; their combination provides a more comprehensive information source. This data fusion enables the invention to perform more accurate detection in complex environments, especially in hot spot regions with temperatures close to normal values, thereby improving the accuracy and reliability of hot spot prediction. Furthermore, the deep transform autoencoder model effectively processes high-dimensional, multimodal data and extracts useful features. The model compresses the input data into a latent space through an encoder, and the decoder reconstructs the image to form an accurate prediction of the hot spot region. Compared to traditional VAE models, the DT-VAE architecture of this invention fully utilizes spectral and temperature information through the combination of convolutional neural networks and deconvolutional networks, significantly improving the effectiveness of hot spot detection. Through a real-time prediction and evaluation mechanism, maintenance personnel can promptly identify hot spot problems in photovoltaic modules, reduce the risk of system failures, and ultimately improve the operation and maintenance efficiency and reliability of photovoltaic power plants.

[0043] This invention, by combining spectral and thermal imaging and employing a depth transform autoencoder model, effectively solves the challenges of multimodal data fusion and high-dimensional data processing. This not only improves the accuracy of hotspot prediction but also enhances detection robustness, particularly in complex backgrounds and with micro-cracks, enabling more precise identification of hotspot regions. Simultaneously, it can predict hotspot regions in real time and provide detailed evaluation results, offering precise decision support for photovoltaic power plant operation and maintenance personnel, thereby improving the efficiency and reliability of photovoltaic power plant operation and maintenance.

[0044] By combining spectral and thermal radiation data, a deep learning model is used to perform efficient and accurate hotspot region prediction. This not only maintains high detection accuracy under different lighting conditions but also fully leverages the complementarity of spectral and thermal radiation data through multimodal data fusion, significantly improving the accuracy of hotspot detection. This effectively overcomes the shortcomings of existing technologies, providing more reliable hotspot monitoring and maintenance decision support for photovoltaic power plants. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0046] Figure 1 This is a flowchart of a photovoltaic module hot spot detection method based on multi-source data and deep learning provided in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the structure of a photovoltaic module hot spot detection system based on multi-source data and deep learning, provided in an embodiment of the present invention. Detailed Implementation

[0048] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0049] Example 1:

[0050] like Figure 1 The image shows an embodiment of the photovoltaic module hot spot detection method based on multi-source data and deep learning provided by the present invention, which specifically includes the following steps:

[0051] S1: Data Acquisition and Preprocessing: Spectral images and thermal images of photovoltaic modules are acquired by a spectral imager and an infrared thermal imager, respectively. The acquired images are preprocessed, and the preprocessed spectral images are stitched together with the thermal images to obtain spectral-thermal fusion data containing spectral and temperature information.

[0052] S2: Construction of the deep transform autoencoder model: The spectral-thermal ray fusion data is input into the deep transform autoencoder model containing the encoder and decoder. The deep transform autoencoder model learns the joint features of the spectral and thermal ray data and captures key information of the hot spot region in the latent space. The encoder compresses the input data into the latent space through a multi-layer convolutional neural network, and the decoder reconstructs the latent space features into a hot spot prediction image through a deconvolutional network.

[0053] S3: Hot Spot Region Prediction and Evaluation: Based on the deep transform autoencoder model, the input spectral-thermal ray fusion data is inferred to output hot spot prediction results; hot spot regions are extracted through binarization processing and connected component analysis, and multiple evaluation indicators are used to quantitatively evaluate the hot spot prediction results;

[0054] S4: Model Training and Optimization: Train the deep transform autoencoder model based on the spectral-thermal ray fusion data obtained in step S1, and monitor the loss and evaluation metrics on the validation set until convergence;

[0055] S5: Deployment and Application: Deploy the trained deep transform autoencoder model to the edge computing device. Real-time collected spectral and thermal ray data are input into the edge computing device for real-time processing, outputting hot spot location, size, and severity scores, and displaying them through the user interface.

[0056] Specifically, this invention proposes a photovoltaic module hot spot prediction method based on spectral-thermal X-ray data fusion and a depth transform autoencoder. Combining multimodal data from spectral and thermal X-ray images, and utilizing a depth transform autoencoder model, it can efficiently and accurately predict hot spot regions in photovoltaic modules. This method significantly improves the accuracy of hot spot detection through data fusion and the autoencoder structure, particularly excelling in detecting complex backgrounds and micro-cracks. The model's real-time prediction capability, combined with an operation and maintenance decision support system, not only provides an efficient fault early warning mechanism for photovoltaic power plants but also reduces the risk of system failures, improves the long-term stability of modules, and provides reliable technical support for intelligent operation and maintenance in the photovoltaic industry.

[0057] The steps of this invention will be described in detail below with specific examples:

[0058] Step 1: Image Acquisition and Data Preprocessing

[0059] 1. Image Acquisition:

[0060] This invention uses two types of imaging devices to acquire image data of photovoltaic modules: a spectral imaging instrument and an infrared thermal imager.

[0061] Spectral imaging was performed using NASA's AVIRIS imager to acquire spectral images of the photovoltaic modules. This instrument can acquire spectral information across multiple bands, covering the visible, near-infrared, and short-wave infrared bands. The spectral images have a resolution of 512×512, enabling detailed capture of the spectral characteristics of the module surface. During acquisition, the instrument acquired reflectivity data for each band while in flight and recorded the reflectivity characteristics of the photovoltaic module surface for subsequent analysis.

[0062] Thermal imaging utilizes the FLIR T640 infrared thermal imager, which boasts a resolution of 640×480 and can accurately capture temperature changes in hotspot areas on the surface of photovoltaic modules. The infrared thermal imager allows for real-time monitoring of the module's heat distribution, particularly areas of abnormal temperature, to identify hotspot problems.

[0063] 2. Data preprocessing:

[0064] After acquiring spectral and thermal images, the data needs to be preprocessed to ensure it is suitable for use in the model built in subsequent steps.

[0065] 1. Preprocessing of spectral images:

[0066] (1) Grayscale Conversion and Normalization: Since spectral images typically contain data from multiple bands, each band's image is first converted to grayscale before being input into the model. The pixel values ​​of each band need to be normalized, mapping them to the [0,1] interval. The calculation method is as follows:

[0067]

[0068] (2) Cropping: Cropping the image removes irrelevant areas and retains only the photovoltaic module area to ensure that background noise is not included in subsequent processing.

[0069] 2. Preprocessing of thermal X-ray images:

[0070] (1) Denoising and smoothing: A 5×5 Gaussian filter was used to denoise the thermal ray image to reduce the impact of background noise. The standard deviation was set to 2 to ensure that the image was smoothed and important hot spot information was preserved.

[0071] (2) Temperature normalization: The temperature values ​​in the thermal X-ray images are normalized to ensure that images of different photovoltaic modules and under different acquisition conditions can be effectively compared.

[0072] 3. Data Fusion: The preprocessed spectral image is fused with the thermal ray image. The spectral data at each corresponding location is stitched together with the temperature data of the thermal rays to obtain a multi-channel image containing both spectral and temperature information for subsequent network processing.

[0073] Step 2: Design and Construction of the Depth Transform Autoencoder (DT-VAE) Model:

[0074] 1. Depth Transform Autoencoder Structure:

[0075] The innovation of this invention lies in employing a deep transform autoencoder to fuse spectral and thermal image data. While multimodal data fusion and autoencoder techniques are already applied in existing technologies, this invention innovates in both aspects to improve the model's performance in high-dimensional data fusion. Traditional multimodal data fusion methods often lack sufficient feature complementarity when processing spectral data and thermal images, and while autoencoders are widely used for data dimensionality reduction, they have not been fully optimized in deep learning architectures for multimodal data fusion. Compared to existing methods, this invention enhances the model's expressive power for high-dimensional data, particularly in the fusion processing of spectral and thermal images, by introducing a deep transform autoencoder. DT-VAE comprises two main parts: an encoder and a decoder, and employs a transform autoencoder structure to enhance the model's expressive power in high-dimensional data.

[0076] Encoder: The encoder compresses the input spectral-thermal fusion image into a latent space representation. The encoder uses a multi-layer convolutional neural network (CNN) structure for feature extraction. Each convolutional layer is followed by a BatchNorm layer and a ReLU activation function. The number of convolutional layers increases progressively with the complexity of the input data until the latent features of the image are extracted. Finally, fully connected layers compress the image features into a latent space, yielding latent variables μ and σ, representing the probability distribution of the image.

[0077]

[0078] Where μ(x) and σ 2 (x) represents the mean and variance of the latent space, respectively, and z is a latent variable sampled from the latent space.

[0079] Latent Space and Reparameterization: To ensure good controllability and generativeability of the latent space, reparameterization techniques are employed, as detailed in the following calculations:

[0080] z=μ(x)+σ(x·∈

[0081] Where ∈ is from the standard normal distribution The noise sampled in the middle, μ(x) and σ(x) are the mean and standard deviation of the encoder output.

[0082] Decoder: The decoder maps the variable z in the latent space back to the image space. The decoder structure uses deconvolutional layers (transposed convolutions) to progressively upsample the low-dimensional representation of the latent space and reconstruct an image of the original size. Each deconvolutional layer is followed by a BatchNorm and ReLU activation function. Finally, the decoder outputs a reconstructed image. That is, the predicted hot spot area.

[0083] This invention enhances the model's ability to express high-dimensional data by employing a transform autoencoder (VAE) structure and combining convolutional neural networks and deconvolutional networks, making it particularly suitable for the fusion processing of spectral and thermal X-ray images.

[0084] Specifically, the learning and information capture principle of the deep transform autoencoder lies in the following: The encoder automatically extracts and learns joint features from the input spectral-thermal ray fusion data through a multi-layer convolutional neural network. These features integrate complementary information from the two modalities and compress them into a low-dimensional latent space. This latent space is designed to capture and represent key information about the hotspot region, such as its location, size, and severity. Through reparameterization techniques, the model can sample from this latent space, making the training process differentiable. Subsequently, the decoder uses a deconvolutional network to progressively upsample the abstract features representing the core information of the hotspot in the latent space and reconstruct a predicted hotspot image of the same size as the original image. The entire learning process is driven by reconstruction loss (ensuring the reconstructed image is similar to the original input) and KL divergence loss (normalizing the distribution of the latent space), thereby forcing the model to efficiently and meaningfully capture the "key information" of the hotspot region in the latent space.

[0085] 2. Definition of loss function:

[0086] The model is optimized using reconstruction loss and KL divergence loss; where reconstruction loss measures the difference between the input image and the reconstructed image, and mean squared error (MSE) is used as the reconstruction loss, defined as follows:

[0087]

[0088] Where, x i For the input image, To reconstruct the image, N is the total number of pixels in the image. KL divergence loss: used to measure the difference between the latent spatial distribution and the standard normal distribution, defined as follows:

[0089]

[0090] Where μ and σ 2 Let these be the mean and variance of the encoder output, respectively. The final loss function is a weighted sum of these two values, defined as follows:

[0091] L total =L recon +λ*L KL

[0092] Wherein, λ is a hyperparameter that adjusts the weight between the reconstruction loss and the KL divergence loss, and is set to 0.5.

[0093] Step 3: Hot Spot Area Prediction and Assessment

[0094] Using the deep transform autoencoder model trained in step two, the hot spot region of the photovoltaic module is predicted and evaluated by inputting the fused spectral-thermal X-ray data. Although multimodal data fusion and autoencoder technology have been applied in existing technologies, this invention overcomes the limitations of existing technologies in multimodal data fusion and hot spot prediction through the innovative application of deep transform autoencoders, especially in terms of accuracy and robustness. Existing methods often fail to effectively capture fine-grained information of different modal data during the fusion process, while this invention, through the latent space learning of the DT-VAE model, fully utilizes the complementarity of spectral and thermal X-ray images, significantly improving the prediction accuracy of hot spot regions.

[0095] 1. Hot spot region prediction:

[0096] After step two, the image encoding and decoding process has been completed using a depth transform autoencoder. Using this model, the input is a fused spectral-thermal image (i.e., a multi-channel image after preprocessing and data fusion), and the model outputs an image containing the predicted hotspot region. Specifically:

[0097] Model Input: The input image is a fusion of a spectral image and a thermal ray image, with a size of 256×256. Each pixel contains spectral information from different wavelengths and the corresponding temperature value. The data shape of the input image is $(256,256,9)$, where the 9 channels contain 8 spectral bands (including specific bands of visible light, near-infrared, and short-wave infrared: 450nm, 550nm, 650nm, 750nm, 850nm, 950nm, 1050nm, 1150nm) and 1 thermal ray temperature channel.

[0098] Model Output: The DT-VAE model transforms the input image into an output image of the same size as the input image. Each pixel in the output image contains a prediction of whether the region is a hotspot (based on temperature anomalies). The model's predictions are passed through the decoder's deconvolution process to generate predicted hotspot images. Each pixel value in this output image represents the degree of temperature anomaly and the probability of a hotspot at the corresponding location.

[0099] 2. Post-processing and hot spot region extraction:

[0100] To accurately extract hotspot regions from the output image of DT-VAE, post-processing of the prediction results is required. The specific processing steps are as follows:

[0101] 1. Hotspot Region Binarization: Each pixel value in the hotspot image output by the DT-VAE model is compared with a set temperature threshold. If the pixel value is greater than the set threshold (temperature threshold is 50℃), it is determined to be a hotspot region; otherwise, it is a non-hotspot region. The determination formula is as follows:

[0102]

[0103] Where T(i,j) is the predicted temperature at position (i,j), 50℃ is the set temperature threshold, and H pred (i,j) is the binary hot spot region label (1 represents hot spot, 0 represents non-hot spot).

[0104] 2. Region Extraction and Labeling: The locations of hotspot regions in the image are extracted using a connected component analysis algorithm (using the `skimage.measure.label` function). This step labels connected hotspot regions as individual regions, providing their coordinates, size, and bounding box information. The bounding box is assumed to have a width of W and a height of H. The extracted hotspot regions are used for subsequent decision support and intervention by maintenance personnel.

[0105] 3. Hot spot area assessment:

[0106] The accuracy of the hot spot regions predicted by the model is mainly evaluated using the following criteria:

[0107] (1) Mean Squared Error (MSE): Measures the pixel-level difference between the predicted hotspot area and the actual area. For each pixel, the error between the predicted value and the true value is calculated, and the error is averaged over all pixels. The formula is as follows:

[0108]

[0109] Among them, y i These are the actual values ​​(i.e., the pixel values ​​of the actual hotspot area). Here is the predicted value, and $N$ is the total number of pixels.

[0110] (2) Crossover-Union Ratio (IoU): IoU measures the degree of overlap between the predicted hotspot region and the actual hotspot region. The calculation formula is:

[0111]

[0112] Where A represents the predicted hotspot region, B represents the actual hotspot region, |A∩B| is the intersection area of ​​the predicted region and the actual region, and |A∪B| is the union area of ​​the predicted region and the actual region. A larger IoU value indicates a more accurate prediction. For this invention, the IoU threshold for the hotspot region is set to 0.5; when the IoU is greater than this value, the hotspot prediction is considered correct.

[0113] (3) Accuracy: Accuracy measures the overall performance of the model in predicting hot spot regions. The formula is as follows:

[0114]

[0115] Wherein, TP represents true positives (the number of regions correctly predicted as hot spots), TN represents true negatives (the number of regions correctly predicted as non-hot spots), FP represents false positives (the number of regions incorrectly predicted as hot spots), and FN represents false negatives (the number of regions incorrectly predicted as non-hot spots). The higher the value, the more accurate the model prediction.

[0116] (4) Recall: Recall measures the proportion of hotspots successfully identified by the model in actual hotspot regions. The calculation formula is:

[0117]

[0118] A higher recall rate indicates that the model can identify more real hot spot areas.

[0119] Step 4: Model Training and Optimization

[0120] 1. Dataset Construction and Model Preparation:

[0121] Before training, the data needs to be preprocessed and then input into the DT-VAE model. The data preparation process has been described in detail in steps one and two, including preprocessing of spectral and thermal images, data normalization, image cropping, and fusion of spectral and thermal images.

[0122] Divide all datasets into training, validation, and test sets. Generally, the dataset split should be 70% training, 15% validation, and 15% test. Ensure the training set has diverse images to avoid overfitting.

[0123] 2. Hyperparameter settings:

[0124] Training was performed using the Adam optimizer with an initial learning rate of 0.001, a batch size of 32, and a training period of 10 epochs. A gradual decay strategy was employed, reducing the learning rate to 0.5 times its original value every 5 epochs. Training stopped when the validation set loss failed to decrease effectively within 5 consecutive epochs.

[0125] 3. Data Augmentation:

[0126] During training, the following data augmentation methods were used:

[0127] (1) Rotation: Randomly rotate the image by a maximum of 45 degrees;

[0128] (2) Translation: Randomly translate the image by a maximum of 10%;

[0129] (3) Brightness adjustment: Randomly change the brightness of the image.

[0130] 4. Verification and Optimization:

[0131] After each epoch, the loss is calculated on the validation set, and the hyperparameters are tuned to further optimize the model's performance.

[0132] It should be noted that step four and step two are complementary optimization operations. Step two is mainly responsible for defining the structure of the model (encoder, decoder, loss function and its components). It determines what the model will optimize and how to calculate the optimization objective (loss function). For example, in step two, the weighted sum of reconstruction loss and KL divergence loss is defined as the total loss function, and λ is set to 0.5.

[0133] Step four focuses on how to efficiently and effectively achieve the optimization objectives defined in step two, and further improve the model's performance. This is specifically reflected in the following optimization operations:

[0134] Hyperparameter tuning (Validation and Tuning): In addition to preset parameters such as λ=0.5, different learning rates, batch sizes, and even certain configurations in the network structure will be dynamically adjusted or tried based on the performance of the validation set in order to find the best combination.

[0135] Learning rate decay strategy: The "gradual decay strategy" is a classic optimization technique that aims to achieve rapid convergence in the early stages of training and then fine-tune the model in the later stages to avoid oscillations and help the model reach a better local optimum.

[0136] Early Stopping: The training stops when the validation set loss fails to decrease effectively over multiple consecutive training cycles. This is an important optimization strategy to prevent model overfitting and ensure that the model's generalization ability on the validation set reaches its optimal point rather than just performing well on the training set.

[0137] Data augmentation: Although data augmentation works directly on the input data rather than the model itself, it indirectly improves the model's generalization ability and robustness by expanding the diversity of the training set. It is also an important optimization technique that helps the model learn more robust features.

[0138] Therefore, step four involves optimizing and tuning the actual training process within the optimization problem and framework defined in step two. These two steps are closely linked and indispensable.

[0139] Step 5: Deployment and Application

[0140] In step four, the trained and optimized Deep Transform Autoencoder (DT-VAE) model is already able to accurately predict hot spot areas on photovoltaic modules. Step five describes how to deploy this model to actual equipment and how to apply it in real time at a photovoltaic power plant. The goal of deployment and application is to ensure that the model can operate efficiently under real-time conditions and provide accurate hot spot prediction results for operation and maintenance personnel.

[0141] 1. Model Deployment:

[0142] The trained DT-VAE model needs to be converted into an inference format suitable for embedded devices so that it can perform prediction tasks on field devices. The model is converted into a format suitable for edge computing devices using TensorFlow Lite format; then, using an NVIDIA Jetson Xavier NX as an edge computing device, the converted TensorFlow Lite model is uploaded to the device, and the model is run on the device through the TensorFlow Lite inference engine.

[0143] 2. The model application consists of the following five steps:

[0144] (1) Real-time data acquisition: Drones and fixed cameras installed in the photovoltaic power station are responsible for acquiring image data of the photovoltaic modules in real time. Spectral imagers are responsible for acquiring spectral data of the photovoltaic module surface, and infrared thermal imagers are responsible for acquiring thermal ray images of the module surface. Both data are transmitted wirelessly to the inference system deployed on edge devices.

[0145] (2) Image Preprocessing and Data Fusion: On the inference device, the acquired spectral and thermal images are first preprocessed. Specific operations include grayscale conversion, denoising, normalization, and resizing. Then, the spectral and thermal images are fused at pixel positions to form a multi-channel image containing multiple band spectral and temperature information. This fused image will serve as the input data for the model.

[0146] (3) Hot Spot Prediction: The preprocessed and fused images are input into the trained DT-VAE model for inference. The model outputs the predicted hot spot region of the photovoltaic module through a depth transform autoencoder structure. The output result is the temperature anomaly value of each pixel, indicating whether the region is a hot spot region.

[0147] (4) Post-processing and hotspot region extraction: The hotspot region image output by the model undergoes a post-processing step, which binarizes it. Specifically, the pixel values ​​in the predicted image are compared with a temperature threshold. If the predicted temperature is greater than the set threshold (50℃), the region is marked as a hotspot; otherwise, it is a non-hotspot. The coordinates, size, and bounding box information of the hotspot region are extracted through connected component analysis.

[0148] (5) Result Feedback and Display: The prediction results are displayed to the operation and maintenance personnel of the photovoltaic power station through a graphical interface. The interface will display the specific location, size, severity score, and temperature value of the hot spot in real time. The hot spot area is marked with a red box, and the non-hot spot area is marked with a green box. The interface will also provide the temperature value of the hot spot area and the corresponding risk assessment to help the operation and maintenance personnel determine whether maintenance measures are needed.

[0149] It should be noted that, in addition to using the Deep Transform Autoencoder (DT-VAE) model, traditional machine learning algorithms (such as Support Vector Machines (SVM), Random Forests, or K-Nearest Neighbors) can also be used to combine spectral and thermal X-ray data for hotspot prediction. When using traditional machine learning algorithms, features can first be extracted from the spectral and thermal X-ray images (e.g., Principal Component Analysis (PCA) or feature selection), and then these features are input into a traditional classifier for hotspot region identification. Although this method does not have the powerful automatic feature learning capabilities of deep learning methods, it can still effectively predict hotspots in some simple scenarios.

[0150] Example 2:

[0151] like Figure 2 As shown in the figure, this embodiment of the invention provides a photovoltaic module hot spot detection system based on multi-source data and deep learning, comprising:

[0152] The data acquisition and preprocessing module is used to acquire spectral images and thermal ray images of photovoltaic modules through a spectral imager and an infrared thermal imager, respectively, and to preprocess the acquired images. The preprocessed spectral images are then stitched together with the thermal ray images to obtain spectral-thermal ray fusion data containing spectral and temperature information.

[0153] The deep transform autoencoder model building module is used to input spectral-thermal ray fusion data into a deep transform autoencoder model containing an encoder and a decoder. The deep transform autoencoder model learns the joint features of spectral and thermal ray data and captures key information of hot spot regions in the latent space.

[0154] The hot spot region prediction and evaluation module is used to perform inference on the input spectral-thermal ray fusion data based on the depth transform autoencoder model and output hot spot prediction results; hot spot regions are extracted through binarization processing and connected component analysis, and multiple evaluation indicators are used to quantitatively evaluate the hot spot prediction results;

[0155] The model training and optimization module is used to train a deep transform autoencoder model using spectral-thermal ray fusion data, and monitor the loss and evaluation metrics on the validation set until convergence.

[0156] The model deployment and application module is used to deploy the trained deep transform autoencoder model to the edge computing device. The real-time collected spectral and thermal radiation data are input into the edge computing device for real-time processing, and the hot spot location, size and severity score are output and displayed through the user interface.

[0157] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in the present invention, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0158] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0159] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting hot spots in photovoltaic modules based on multi-source data and deep learning, characterized in that, The method includes: (1) Data acquisition and preprocessing: Spectral images and thermal images of photovoltaic modules are acquired by a spectral imager and an infrared thermal imager, respectively. The acquired images are preprocessed and then stitched together with the thermal images to obtain spectral-thermal fusion data containing spectral and temperature information. (2) Construction of deep transform autoencoder model: The spectral-thermal ray fusion data is input into the deep transform autoencoder model containing encoder and decoder. The deep transform autoencoder model learns the joint features of spectral and thermal ray data and captures key information of hot spot region in latent space. The encoder compresses the input data into latent space through multi-layer convolutional neural network, and the decoder reconstructs the latent space features into hot spot prediction image through deconvolutional network. (3) Hot spot region prediction and evaluation: The input spectral-thermal ray fusion data is inferred based on the deep transform autoencoder model, and the hot spot prediction results are output; the hot spot region is extracted by binarization and connected component analysis, and multiple evaluation indicators are used to quantitatively evaluate the hot spot prediction results; (4) Model training and optimization: Based on the spectral-thermal ray fusion data obtained in step (1), train the deep transform autoencoder model and monitor the loss and evaluation metrics on the validation set until convergence; (5) Deployment and application: The trained depth transform autoencoder model is deployed to the edge computing device. The real-time collected spectral and thermal ray data are input into the edge computing device for real-time processing, and the hot spot location, size and severity score are output and displayed through the user interface.

2. The photovoltaic module hot spot detection method based on multi-source data and deep learning according to claim 1, characterized in that, In step (1), the preprocessing of the spectral image includes grayscale conversion, normalization and cropping of each band, and the preprocessing of the thermal ray image includes Gaussian filtering for noise reduction and temperature normalization.

3. The photovoltaic module hot spot detection method based on multi-source data and deep learning according to claim 1, characterized in that, In step (2), the encoder adopts a multi-layer convolutional neural network structure, with a BatchNorm layer and a ReLU activation function added after each convolutional layer; The latent space is sampled using a reparameterization technique, as shown in the formula: z=μ(x)+σ(x·∈ Where ∈ represents the standard normal distribution noise, and μ(x) and σ(x) are the mean and standard deviation of the encoder output; The decoder uses deconvolutional layers to progressively upsample, and adds a BatchNorm layer and a ReLU activation function after each deconvolution layer.

4. The photovoltaic module hot spot detection method based on multi-source data and deep learning according to claim 1, characterized in that, In step (2), the constructed depth transform autoencoder model is optimized using reconstruction loss and KL divergence loss. The reconstruction loss is used to measure the difference between the input image and the reconstructed image, and the mean squared error is used as the reconstruction loss, expressed as: Where, x i For the input image, To reconstruct the image, N is the total number of pixels in the image; KL divergence loss is used to measure the difference between the latent spatial distribution and the standard normal distribution, and is expressed as: Where μ and σ 2 These are the mean and variance of the encoder output, respectively. The weighted sum of reconstruction loss and KL divergence loss is defined as the final loss function, expressed as: L total L recon +λ*L KL Wherein, λ is a hyperparameter that adjusts the weight between the reconstruction loss and the KL divergence loss, and is set to 0.

5.

5. The photovoltaic module hot spot detection method based on multi-source data and deep learning according to claim 1, characterized in that, In step (3), the evaluation metrics include mean squared error, crossover ratio, precision and recall.

6. The photovoltaic module hot spot detection method based on multi-source data and deep learning according to claim 1, characterized in that, In step (4), the specific steps for model training and optimization include: Dataset construction and model preparation: The spectral-thermal ray fusion data is divided into training, validation, and test sets; Hyperparameter settings: The Adam optimizer is used for training. The initial learning rate, batch size and training period are set. The learning rate adopts a gradual decay strategy. The training stops when the validation set loss fails to decrease effectively over multiple consecutive training periods. Data augmentation: During training, data augmentation is performed using rotation, translation, and brightness adjustment methods; Validation and tuning: After each training cycle, the loss is calculated on the validation set and the hyperparameters are tuned to further optimize the model's performance.

7. A photovoltaic module hot spot detection system based on multi-source data fusion and deep learning, characterized in that, The system includes: The data acquisition and preprocessing module is used to acquire spectral images and thermal ray images of photovoltaic modules through a spectral imager and an infrared thermal imager, respectively, and to preprocess the acquired images. The preprocessed spectral images are then stitched together with the thermal ray images to obtain spectral-thermal ray fusion data containing spectral and temperature information. The deep transform autoencoder model building module is used to input spectral-thermal ray fusion data into a deep transform autoencoder model containing an encoder and a decoder. The deep transform autoencoder model learns the joint features of spectral and thermal ray data and captures key information of hot spot regions in the latent space. The hot spot region prediction and evaluation module is used to perform inference on the input spectral-thermal ray fusion data based on the depth transform autoencoder model and output hot spot prediction results; hot spot regions are extracted through binarization processing and connected component analysis, and multiple evaluation indicators are used to quantitatively evaluate the hot spot prediction results; The model training and optimization module is used to train a deep transform autoencoder model using spectral-thermal ray fusion data, and monitor the loss and evaluation metrics on the validation set until convergence. The model deployment and application module is used to deploy the trained deep transform autoencoder model to the edge computing device. The real-time collected spectral and thermal radiation data are input into the edge computing device for real-time processing, and the hot spot location, size and severity score are output and displayed through the user interface.

Images