Photovoltaic module fault detection method and device, computer equipment and medium

Abstract

The invention provides a photovoltaic module fault detection method and device, computer equipment and a medium, and belongs to the field of image recognition, and the method comprises the steps: taking MobileNetV3-Small as a backbone network, dividing the backbone network into three stages according to the hierarchy of feature extraction, and respectively inserting a CNN-ViT mixing block between the first stage and the second stage and between the second stage and the third stage; inserting a space attention mechanism module after the third stage to obtain a hybrid classification network; taking the marked historical infrared image of the photovoltaic module as a training set, and training the hybrid classification network by adopting a composite loss function combining cross entropy loss and triple loss to obtain a trained fault detection model; and obtaining an infrared image of the target photovoltaic module, inputting the infrared image into the fault detection model, and obtaining output fault information. Therefore, the scheme realizes light and accurate fault detection through collaborative design of a light trunk, a global-local feature fusion module and an attention mechanism.

Description

Technical Field

[0001] This invention belongs to the field of image recognition, and specifically relates to a method, device, computer equipment, and medium for detecting faults in photovoltaic modules. Background Technology

[0002] As the global energy structure gradually shifts towards cleaner and lower-carbon energy, photovoltaic power generation, as one of the most mature technologies for utilizing solar energy, has seen rapid growth in installed capacity. However, photovoltaic modules are exposed to a complex and ever-changing natural environment for extended periods, making them highly susceptible to various faults such as hot spots and diode defects. These faults not only significantly reduce the power generation efficiency of the modules but may also trigger serious safety accidents such as fires, posing a significant threat to the operational efficiency and safety of power plants.

[0003] Currently, photovoltaic (PV) fault detection methods based on infrared images have gradually become a research hotspot. This method identifies local abnormal regions by analyzing the temperature distribution of the module, enabling long-distance, real-time, and non-contact detection. In recent years, some studies have used lightweight convolutional neural networks (CNNs) such as MobileNet and ShuffleNet to extract image features, which can identify PV faults relatively well. However, fault identification depends on the difference between the fault area and the whole module. CNNs alone cannot fully capture the rich semantic information in the image, resulting in limited recognition performance in complex backgrounds. To improve the accuracy of fault detection, the VisionTransformer model has been widely used in fault detection because it can capture long-distance dependencies. Although it has achieved higher accuracy, it ignores local image features and is difficult to accurately capture faults with strong local detail dependencies in PV modules. Moreover, its self-attention mechanism is complex, resulting in long model training and inference times, making it difficult to deploy on edge devices or drone terminals.

[0004] Therefore, existing technologies still face many challenges. On the one hand, while lightweight CNN models have low computational cost, they struggle to effectively extract deep semantic features, leading to inaccurate photovoltaic module fault detection. On the other hand, while high-precision models such as Vision Transformer can extract global image information, their complex structures make them difficult to deploy on resource-constrained devices. Therefore, there is an urgent need for a lightweight infrared image fault detection method for photovoltaic modules that integrates local and global information to meet the deployment requirements of actual power plant inspections. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a photovoltaic module fault detection method, apparatus, computer equipment, and medium.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A method for detecting faults in photovoltaic modules, the method comprising: Using MobileNetV3-Small as the backbone network, the Bottleneck module sequence in the backbone network is divided into three stages according to the depth of feature extraction. A CNN-ViT hybrid block is inserted between the first and second stages and between the second and third stages. A spatial attention mechanism module is inserted between the third stage and the average pooling layer to obtain a hybrid classification network. The historical infrared images of the photovoltaic modules labeled with annotations are used as the training set. The hybrid classification network is trained using a composite loss function that combines cross-entropy loss and triplet loss to obtain a trained fault detection model. The infrared image of the target photovoltaic module is obtained and input into the fault detection model to obtain the output fault information.

[0007] Optionally, the Bottleneck module sequence includes multiple Bottleneck modules and the output of a convolutional layer; dividing the Bottleneck module sequence in the backbone network into three stages according to the depth of feature extraction includes: The sequence portion for extracting shallow detail features is taken as the first stage. The first stage includes a 16-channel Bottleneck module with a kernel size of 3×3 and two 24-channel Bottleneck modules with a kernel size of 3×3. The sequence portion from which the mid-level semantic features are extracted is taken as the second stage. The second stage includes three 40-channel Bottleneck modules with a kernel size of 5×5 and two 48-channel Bottleneck modules with a kernel size of 5×5. The sequence portion from which deep abstract features are extracted is taken as the third stage, which includes three 96-channel Bottleneck modules with a kernel size of 5×5 and a 576-channel convolutional layer output with a kernel size of 1×1.

[0008] Optionally, the CNN-ViT hybrid block is used to sequentially perform depthwise convolution and pointwise convolution on the input feature map, then extract global features through the Vision Transformer module, perform pointwise convolution for dimensionality reduction, and perform residual connection with the input feature map to output a feature map that fuses local and global information.

[0009] Optionally, the multi-head self-attention layer in the Vision Transformer module is replaced with a linear attention layer, calculated as follows: ; in, Let V be the feature mapping function that maps Q and K to a nonnegative space, where Q, K, and V are the query vector, key vector, and value vector, respectively.

[0010] Optionally, the spatial attention mechanism module is used to perform global max pooling and global average pooling on the input feature map respectively, concatenate the two results and extract spatial weights through a convolutional layer, activate them with Sigmoid and multiply them with the original feature map to obtain the weighted spatial features.

[0011] Optionally, the expressions for the cross-entropy loss function and the triplet loss function are respectively: ; ; The expression for the composite loss function is: ; in, The loss function is called cross-entropy loss, where N is the number of classes and yi is the true label. Output the results for the model. The loss is the triplet loss, and d(·) is the distance between the two classes of samples. a For example, an anchor p To and a Samples of the same category, n To and a Different categories of samples, and These are the weighting coefficients.

[0012] Optionally, after obtaining the output fault information, the method further includes: Heat maps are generated using Grad-CAM technology to provide a visual explanation of the decision-making process for fault detection. The Grad-CAM technique calculates the gradient of the target class score with respect to the feature map of the last convolutional layer, performs global average pooling on the gradient to obtain weights, performs weighted summation on the feature map, and then activates it through ReLU to generate a heatmap representing the key region.

[0013] A photovoltaic module fault detection device, the device comprising: The construction module is used to divide the Bottleneck module sequence in the backbone network into three stages according to the depth of feature extraction, with MobileNetV3-Small as the backbone network. A CNN-ViT hybrid block is inserted between the first stage and the second stage and between the second stage and the third stage. A spatial attention mechanism module is inserted between the third stage and the average pooling layer to obtain a hybrid classification network. The training module is used to use the labeled historical infrared images of the photovoltaic modules as the training set, and to train the hybrid classification network using a composite loss function that combines cross-entropy loss and triplet loss to obtain a trained fault detection model. The identification module is used to acquire infrared images of the target photovoltaic module, input them into the fault detection model, and obtain output fault information.

[0014] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned photovoltaic module fault detection method.

[0015] A computer device is characterized by comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned photovoltaic module fault detection method.

[0016] The photovoltaic module fault detection method provided by this invention has the following beneficial effects: This invention uses the lightweight MobileNetV3-Small as the backbone network, which employs efficient structures such as depthwise separable convolutions, significantly reducing the number of model parameters and computational cost, laying a lightweight foundation for deployment on edge devices with limited computing resources. Secondly, by dividing the backbone network into three feature extraction stages and inserting CNN-ViT hybrid blocks between each stage, this design incorporates the global modeling capabilities of the Vision Transformer in the early and mid-stages of the model. This compensates for the limitations of purely lightweight convolutional networks, such as limited receptive fields and difficulty in capturing long-distance dependencies and global context, enabling the model to integrate the overall semantic information of the image while extracting local details, thereby improving its ability to represent deep fault features. Finally, a spatial attention mechanism is introduced after the deep feature stage, allowing the model to adaptively focus on key spatial features of fault regions in infrared images, enhancing the sensitivity to identify local and diverse faults such as hot spots and cracks. The training strategy combining cross-entropy loss and triplet loss further optimizes the intra-class aggregation and inter-class separability of the feature space. Therefore, this scheme significantly enhances the ability to extract and identify deep features of photovoltaic module faults while maintaining low model complexity through the collaborative design of a lightweight backbone, a global-local feature fusion module, and an attention mechanism. Ultimately, it achieves both lightweight and accurate fault detection in actual power plant inspections. Attached Figure Description

[0017] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a photovoltaic module fault detection method provided by the present invention according to an exemplary embodiment.

[0019] Figure 2 This is a schematic diagram of a separable convolutional layer structure provided by the present invention according to an exemplary embodiment.

[0020] Figure 3 This is a flowchart illustrating a CNN-ViT hybrid block provided by the present invention according to an exemplary embodiment.

[0021] Figure 4 This is a schematic diagram of the overall framework of a spatial attention mechanism module provided by the present invention according to an exemplary embodiment.

[0022] Figure 5 This is a schematic diagram of the overall structure of a hybrid classification network according to an exemplary embodiment of the present invention.

[0023] Figure 6 This invention provides a visual heatmap according to an exemplary embodiment.

[0024] Figure 7 This is a block diagram of a photovoltaic module fault detection device provided by the present invention according to an exemplary embodiment. Detailed Implementation

[0025] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0026] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0027] First, this invention provides a method for detecting faults in photovoltaic modules, specifically as follows: Figure 1 As shown, it includes the following steps: S101. Using MobileNetV3-Small as the backbone network, the Bottleneck module sequence in the backbone network is divided into three stages according to the depth of feature extraction. A CNN-ViT hybrid block is inserted between the first and second stages and between the second and third stages. A spatial attention mechanism module is inserted between the third stage and the average pooling layer to obtain a hybrid classification network.

[0028] The Bottleneck module sequence comprises multiple Bottleneck modules and the output of a convolutional layer. The first stage extracts shallow detail features and includes one 16-channel 3×3 Bottleneck module and two 24-channel 3×3 Bottleneck modules. The second stage extracts mid-level semantic features and includes three 40-channel 5×5 Bottleneck modules and two 48-channel 5×5 Bottleneck modules. The third stage extracts deep abstract features and includes three 96-channel 5×5 Bottleneck modules and the output of a 576-channel 1×1 convolutional layer. This CNN-ViT hybrid block sequentially performs depthwise convolution and pointwise convolution on the input feature map. Global features are then extracted using the Vision Transformer module, reduced in dimensionality through pointwise convolution, and residually connected to the input feature map to output a feature map that fuses local and global information. The spatial attention mechanism module is used to perform global max pooling and global average pooling on the input feature map respectively. The two results are concatenated and then the spatial weights are extracted through a convolutional layer. After activation by Sigmoid, the weights are multiplied with the original feature map to obtain the weighted spatial features.

[0029] In one embodiment, Vision Transformer (ViT) is the embodiment of the Transformer model in computer vision tasks. Its core is to split the input image x∈RH×W×C into image patches, i.e., xp∈RN×(P2·C), where (H, W) is the resolution of the image, C is the number of channels of the image, (P, P) is the resolution of each image patch, and N=HW / P2 is the number of image patches. ViT first maps the image patches through the embedding layer E∈R(P2·C)×D, uses the classification token xClass to extract global feature representations, and adds positional encoding information EPOS∈R(N+1)×D to obtain the input sequence z0, as shown in formula (1):

[0030] The input sequence is transformed through M Transformer layers. Each Transformer layer mainly consists of a multi-head self-attention layer (MSA) and a multilayer perceptron (MLP) to obtain the output sequence zl, which can be represented as: Where Norm(·) denotes layer normalization, MSA is a multi-head self-attention layer, and MLP is a multilayer perceptron. Finally, the feature z0 L is processed by the classification head of the MLP to obtain the final classification result y:

[0031] Among them, MLPHead is a classification head based on MLP.

[0032] In ViT, the multi-head self-attention layer linearly projects the input into a query (Q), key (K), and value (V), as follows: Here, dk represents the dimension of the query (key). The complexity of this attention layer is O(n²) (where n is the sequence length of image pixels), leading to a quadratic increase in computational cost and significantly reducing computational efficiency. Therefore, this invention replaces the multi-head attention layer with linear attention. Linear attention decomposes multi-head self-attention into linear operations, reducing the complexity of standard self-attention to O(n), as shown in the formula:

[0033] in, Let V be the feature mapping function, which maps Q and K to a non-negative space, where Q, K, and V are the query vector, key vector, and value vector, respectively.

[0034] In addition, ViT flattens the feature map into a sequence, which loses edge texture details and destroys the original spatial structure. Fault features in photovoltaic module infrared images, such as hot spots and cracks, are highly sensitive to local spatial structure. Therefore, this invention adds a depth-wise (DW) separable convolutional layer between the input feature map and ViT to construct a CNN-ViT hybrid block. The DW separable convolutional layer is as follows: Figure 2 As shown, it decomposes standard convolution into DW convolution and point-wise (PW) convolution. In the fusion block, the input feature map is first smoothed locally using DW convolution to preserve the local positional relationships of features. Then, PW convolution is used to achieve channel fusion and project it into a high-dimensional space. Next, the ViT module obtains global information, and PW convolution is used for dimensionality reduction. The output is then residually connected to the input feature map to finally obtain the fused local and global features, such as... Figure 3 As shown.

[0035] In infrared image fault detection of photovoltaic modules, the fault area typically occupies only a local space in the image and exhibits characteristics of varying scales, diverse shapes, and random spatial distribution. This invention introduces a Spatial Attention (SAM) mechanism, enabling the model to adaptively focus on spatial features related to the fault, thereby improving the model's detection performance. The overall framework of the SAM module is as follows: Figure 4 As shown, the input feature map is first subjected to global max pooling and global average pooling operations to extract global features. Then, the two feature maps are concatenated and input into a convolutional layer to capture spatial features. A sigmoid layer is used to convert its output into weights for each spatial location, and then multiplied with the original feature map to obtain the final spatial features.

[0036] MobileNetV3 is a lightweight convolutional neural network that inherits the DW separable convolution from MobileNetV1 and the linear bottleneck and inverse residual structure from MobileNetV2. In addition, it innovatively employs the h-swish activation function and SE attention mechanism, and uses Neural Architecture Search (NAS) to find the optimal network architecture, significantly reducing the number of parameters and computational complexity while maintaining high accuracy. MobileNetV3 has two versions: Small and Large. MobileNetV3-Small has fewer parameters and lower computational cost, enabling faster inference speed and lower energy consumption, making it suitable for scenarios with limited computing resources. Therefore, this invention implements fault detection based on the MobileNetV3-Small model.

[0037] MobileNetV3-Small, primarily based on DW separable convolutions, effectively extracts local image features but struggles to capture long-range dependencies and subtle feature differences between different fault types. To address this, this invention divides the MobileNetV3-Small backbone into three feature extraction stages: shallow details, mid-level semantics, and deep abstract features. A CNN-ViT hybrid block and a SAM mechanism are inserted into each stage to comprehensively integrate local and global image features. While the first and second stages primarily extract low- and mid-level features, containing significant detail but lacking global context, the third stage extracts deep abstract features, where the differences between faulty and normal regions are more pronounced. Furthermore, the CNN-ViT hybrid block includes a ViT module capable of modeling global information; smaller feature maps may lose global information, and the low contrast between faulty and normal regions in low-level features may prevent the SAM mechanism from effectively distinguishing faulty regions. Therefore, this invention inserts a CNN-ViT hybrid block after the first and second stages to fuse global information while preserving local details, compensating for the locality defects of convolution operations. Furthermore, it introduces a SAM mechanism after the third stage to further refine high-level features and enhance critical fault regions. The overall structure of the hybrid network in this invention is as follows: Figure 5 As shown.

[0038] S102. Using the labeled historical infrared images of the photovoltaic module as the training set, the hybrid classification network is trained using a composite loss function that combines cross-entropy loss and triplet loss to obtain the trained fault detection model.

[0039] The expressions for the cross-entropy loss function and the triplet loss function are as follows: ; ; The expression for the composite loss function is: ; in, The loss function is called cross-entropy loss, where N is the number of classes and yi is the true label. Output the results for the model. Let be the triplet loss, d(·) be the distance between the two classes of samples, a be the anchor example, p be the sample of the same class as a, and n be the sample of a different class. and These are the weighting coefficients.

[0040] In addition, during model training, a fixed random seed was used to ensure the reproducibility of the experiment. The dataset was divided into training, validation, and test sets in a 7:3:2 ratio. The AdamW optimizer was used, with an initial learning rate of 5e-4 and a weight decay coefficient of 1e-4. The ReduceLROnPlateau learning rate scheduler was used to monitor the loss. If the loss did not improve within 3 epochs, the learning rate was reduced by 0.5 times, with a minimum learning rate of 1e-7. In the experiment, the batch size was set to 8, and the model was trained on the training set for 100 epochs. An early stopping mechanism was set; if the performance metrics did not improve within 15 epochs, training was stopped, which could alleviate the overfitting problem to some extent. For the construction of the dataset, infrared images of photovoltaic modules were collected by a drone equipped with an infrared imager. Images with distorted images or without photovoltaic modules were removed to create the dataset. The original images were randomly rotated, flipped, had their brightness changed, and Gaussian noise added to generate multiple images. Next, the input images were segmented to form images containing only photovoltaic modules, and the images were preprocessed, including normalization. Finally, the preprocessed images are used as input images and divided into training, validation, and test sets.

[0041] To quantitatively evaluate the model's performance, the experiment used accuracy, recall, precision, and F1 score as evaluation metrics. Accuracy represents the model's prediction accuracy across all test samples, i.e., the proportion of correctly predicted samples out of the total test samples. Precision is the proportion of samples predicted as positive by the model that were actually positive. Recall represents the proportion of samples correctly predicted as positive out of all actual positive samples. The F1 score is the harmonic mean of precision and recall, used to comprehensively evaluate the model's performance. The formulas for calculating the four evaluation metrics are as follows:

[0042] Where TP represents the number of positive examples predicted as positive by the model, FP represents the number of negative examples predicted as positive by the model, TN represents the number of negative examples predicted as negative by the model, and FN represents the number of positive examples predicted as negative by the model.

[0043] S103. Obtain the infrared image of the target photovoltaic module and input it into the fault detection model to obtain the output fault information.

[0044] In this step, after obtaining the output fault information, a heatmap is generated using Grad-CAM technology to visualize and explain the fault detection decision-making process. Grad-CAM technology calculates the gradient of the target class score with respect to the feature map of the last convolutional layer, performs global average pooling on the gradient to obtain weights, performs weighted summation on the feature map, and then activates it using ReLU to generate a heatmap representing the key region. This heatmap may include the predicted class and confidence level.

[0045] Grad-CAM technology utilizes gradient information flowing towards the target class to weight the feature maps of the last convolutional layer in a convolutional neural network, generating a heatmap that visually displays key regions for model decision-making. Its implementation can be divided into three steps: gradient calculation, global gradient averaging and weight calculation, and weighted summation and ReLU activation. Gradient calculation involves feeding the input image into the network and performing forward propagation to obtain the raw output score for a specific target class. Then, backpropagation is performed to calculate the gradient of the target class score relative to each feature map output by the last convolutional layer. The last convolutional layer is chosen because it preserves spatial information while containing high-level semantic features. Global gradient averaging and weight calculation involves globally averaging the gradients of each feature map obtained in the first step to obtain a global average. This average represents the importance of each feature map in predicting the target class. A larger weight indicates that the feature map contains more information related to the target class. Weighted summation and ReLU activation use the calculated weights to perform a weighted summation of all feature maps in the last convolutional layer, resulting in a coarse class activation map. Finally, a ReLU activation function is used for filtering, with the aim of retaining only features that positively contribute to the predicted target category while suppressing negatively correlated regions, thereby highlighting key regions in the image that support the model's decision.

[0046] In addition, based on the above method steps, the present invention also provides experimental verification.

[0047] This invention verifies the performance of hybrid networks in infrared image fault detection of photovoltaic modules through ablation experiments. Each experiment used the same environment and training strategy, and the results are shown in Table 1. The MobileNetV3-Small model has high precision but low recall and F1 score. Introducing the SAM mechanism improves all metrics, with the most significant improvement in recall (10%). This indicates that the SAM mechanism enables the model to focus on key fault regions, significantly enhancing fault identification capabilities. The combination of MobileNetV3-Small and CNN-ViT hybrid blocks achieves a recall of 98.57%, which is 15.17% and 5.71% higher than the MobileNetV3-Small and MobileNetV3-Small+SAM models, respectively. However, compared to the MobileNetV3-Small+SA model, the precision decreases by 1.3%. This suggests that while the CNN-ViT hybrid block enhances global feature capture capabilities, its ability to focus on key fault regions is slightly insufficient. Finally, the model combining the three achieved optimal results on all four evaluation metrics, with accuracy, precision, recall, and F1 score of 98.76%, 98.57%, 98.57%, and 98.57%, respectively. This demonstrates that the hybrid network proposed in this invention can achieve a balance in fault detection performance while taking into account both global features and local fault regions.

[0048] Table 1 Ablation Experiment Results To verify the effectiveness of the method of this invention, the model of this invention was compared with other advanced models in terms of parameter count, computational cost, inference time, and evaluation metrics. To ensure a fair comparison of model performance, all models were tested under the same environment. The comparison results are shown in Table 2. Regarding parameter count and computational cost, the model of this invention has the fewest parameters, while its computational cost is slightly higher than the MobileFormer-96M model. This indicates that the model of this invention is more lightweight and saves computational resources compared to the other models. In terms of inference time, the inference time of the model of this invention is 16.48 ms, which is 3.51 ms higher than the MobileNetV3-Large model, but significantly lower than the other models, thus improving the inference speed to some extent. In terms of evaluation metrics, the model of this invention achieved the best results in all four evaluation metrics. Compared with the MobileNetV3-Large, ResNet50 and MobileViTv3-S models, the model of this invention improved the evaluation metrics while reducing the number of parameters and computational cost. Although the computational cost was only 0.1G higher than MobileFormer-96M, the accuracy, recall and F1 score were improved by about 1.99%, 4.05% and 2.07% respectively. It can be seen that the model of this invention significantly reduced the model complexity while maintaining high accuracy, and has the advantage of being applied to edge devices with limited computing resources.

[0049] Table 2 Model Comparison Results Finally, this invention employs Grad-CAM technology to visualize the detection results, enhancing the model's interpretability. The visualized heatmap is shown below. Figure 6 As shown.

[0050] Secondly, the present invention also provides a photovoltaic module fault detection device, such as... Figure 7 As shown, it includes: Module 201 is used to construct a backbone network based on MobileNetV3-Small. The Bottleneck module sequence in the backbone network is divided into three stages according to the depth of feature extraction. A CNN-ViT hybrid block is inserted between the first stage and the second stage, and between the second stage and the third stage. A spatial attention mechanism module is inserted between the third stage and the average pooling layer to obtain a hybrid classification network.

[0051] Training module 202 is used to train the hybrid classification network using the labeled historical infrared images of the photovoltaic module as the training set, and to obtain the trained fault detection model by using a composite loss function that combines cross-entropy loss and triplet loss.

[0052] The identification module 203 is used to acquire the infrared image of the target photovoltaic module and input it into the fault detection model to obtain the output fault information.

[0053] Using the aforementioned device, with the lightweight MobileNetV3-Small as the backbone network, which employs efficient structures such as depthwise separable convolutions, the model's parameter count and computational cost are significantly reduced, laying a lightweight foundation for deployment on edge devices with limited computing resources. Secondly, by dividing the backbone network into three feature extraction stages and inserting CNN-ViT hybrid blocks between each stage, this design incorporates the global modeling capabilities of the Vision Transformer in the early and mid-stages of the model. This compensates for the limitations of purely lightweight convolutional networks, such as limited receptive fields and difficulty in capturing long-distance dependencies and global context, enabling the model to integrate the overall semantic information of the image while extracting local details, thereby improving its ability to represent deep fault features. Finally, a spatial attention mechanism is introduced after the deep feature stage, allowing the model to adaptively focus on key spatial features of fault regions in infrared images, enhancing the sensitivity to identify local and diverse faults such as hot spots and cracks. The training strategy combining cross-entropy loss and triplet loss further optimizes the intra-class aggregation and inter-class separability of the feature space. Therefore, this scheme significantly enhances the ability to extract and identify deep features of photovoltaic module faults while maintaining low model complexity through the collaborative design of a lightweight backbone, a global-local feature fusion module, and an attention mechanism. Ultimately, it achieves both lightweight and accurate fault detection in actual power plant inspections.

[0054] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The steps of the provided photovoltaic module fault detection method.

[0055] This invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for various operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above-mentioned functions. Figure 1 The steps of the provided photovoltaic module fault detection method.

[0056] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0057] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0058] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0060] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the patent of the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A photovoltaic module fault detection method, characterized by, The method comprises: Taking MobileNetV3-Small as a backbone network, a Bottleneck module sequence in the backbone network is divided into three stages according to a deep level of feature extraction, a CNN-ViT hybrid block is inserted between the first stage and the second stage and between the second stage and the third stage, and a spatial attention mechanism module is inserted between the third stage and an average pooling layer to obtain a hybrid classification network; A historical infrared image of the photovoltaic module is labeled as a training set, a composite loss function combining a cross-entropy loss and a triplet loss is used to train the hybrid classification network, and a trained fault detection model is obtained; An infrared image of a target photovoltaic module is input into the fault detection model to obtain output fault information.

2. The method of claim 1, wherein, The Bottleneck module sequence comprises a plurality of Bottleneck modules and an output of a convolutional layer; the Bottleneck module sequence in the backbone network is divided into three stages according to a deep level of feature extraction, which comprises: A sequence part for extracting shallow detail features is taken as the first stage, and the first stage comprises one 16-channel Bottleneck module with a convolution kernel size of 3x3 and two 24-channel Bottleneck modules with a convolution kernel size of 3x3; A sequence part for extracting middle layer semantic features is taken as the second stage, and the second stage comprises three 40-channel Bottleneck modules with a convolution kernel size of 5x5 and two 48-channel Bottleneck modules with a convolution kernel size of 5x5; A sequence part for extracting deep abstract features is taken as the third stage, and the third stage comprises three 96-channel Bottleneck modules with a convolution kernel size of 5x5 and one 576-channel convolutional layer output with a convolution kernel size of 1x1.

3. The method of claim 1, wherein, The CNN-ViT hybrid block is used for sequentially performing deep convolution and point-by-point convolution on an input feature map, extracting global features through a Vision Transformer module, reducing the dimension through point-by-point convolution, and performing residual connection with the input feature map to output a feature map that fuses local and global information.

4. The method of claim 3, wherein, The multi-head self-attention layer in the Vision Transformer module is replaced by a linear attention layer, and the calculation formula is: ； wherein, is a feature mapping function that maps Q and K to non-negative space, Q, K, V are query vector, key vector and value vector respectively.

5. The method of claim 1, wherein, The spatial attention mechanism module is used for performing global maximum pooling and global average pooling on an input feature map respectively, splicing the two results, extracting spatial weights through a convolutional layer, multiplying the spatial weights with the original feature map after Sigmoid activation to obtain weighted spatial features.

6. The method of claim 1, wherein, Expressions of the cross-entropy loss function and the triplet loss function are respectively: ； ； An expression of the composite loss function is: ； wherein, is the cross-entropy loss, N is the number of classes, yi is the real label, is the model output result, is the triplet loss, d(·) is the distance between two classes of samples, a is the anchor example, p is the sample of the same class as a , n is the sample of a different class from a , and is the weight coefficient.

7. The method of claim 1, wherein, After obtaining the output fault information, the method further comprises: Generating a heat map through Grad-CAM technology to visually explain the decision-making process of fault detection; The Grad-CAM technology is used to calculate the gradient of the last convolutional layer feature map with respect to the target class score, to obtain a weight by globally average pooling the gradient, to perform weighted summation on the feature map, and to generate a heat map representing a key region by ReLU activation.

8. A photovoltaic module fault detection apparatus, characterized by, The device comprises: The construction module is configured to take MobileNetV3-Small as a backbone network, divide a Bottleneck module sequence in the backbone network into three stages according to a depth level of feature extraction, insert a CNN-ViT hybrid block between the first stage and the second stage and between the second stage and the third stage, respectively, and insert a spatial attention mechanism module between the third stage and an average pooling layer to obtain a hybrid classification network. The training module is configured to take the historical infrared images of the photovoltaic module labeled as a training set, train the hybrid classification network by using a composite loss function combining a cross-entropy loss and a triplet loss, and obtain a trained fault detection model. The recognition module is configured to input the infrared image of the target photovoltaic module into the fault detection model to obtain output fault information.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-7.

10. A computer device, comprising: The device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the processor implements the method in any one of claims 1-7 when executing the program.

Images