A method and device for photovoltaic module fault detection based on YOLOv7

Abstract

The application provides a photovoltaic module fault detection method based on YOLOv7 and a device thereof, and relates to the technical field of target detection. The method introduces a thermal infrared and temperature information fusion module in the network front end, aligns, encodes and fuses the thermal infrared image and its pixel-level temperature matrix through an attention mechanism to enhance the representation ability of multi-modal features. A shallow-deep information aggregation module is introduced in the neck network to improve the context perception and feature discrimination of the model for multi-scale small targets. A global context perception module is introduced in front of the detection head, and the direction channel context branch and the query-key value branch thereof capture the directional global dependence and spatial context relationship respectively to realize global modulation of the features. The application significantly improves the detection accuracy and robustness of multiple faults such as photovoltaic module junction box faults, fragmentation, hot spots and shielding, optimizes the positioning ability of weak and small targets, is suitable for deployment on mobile or embedded devices, and realizes efficient and intelligent inspection of photovoltaic power stations.

Description

A method and apparatus for fault detection of photovoltaic modules based on YOLOv7 Technical Field

[0001] This application relates to the field of target detection technology, and more specifically, to a method and apparatus for photovoltaic module fault detection based on YOLOv7. Background Technology

[0002] In recent years, the development and utilization of solar energy have been vigorously promoted, leading to the rapid development of the photovoltaic power generation industry. However, under long-term operation and frequent exposure to severe weather conditions, photovoltaic modules are prone to problems such as material fatigue, thermal stress concentration, and mechanical damage, resulting in the formation of microcracks inside. This not only causes a decrease in photoelectric conversion efficiency and power output attenuation but may also induce secondary failure modes such as hot spot effect and potential-induced decay. Therefore, how to achieve efficient and accurate fault detection in the operation and maintenance of large-scale photovoltaic power plants has become an urgent problem to be solved.

[0003] Photovoltaic module fault detection is a crucial step in ensuring the safe and stable operation of photovoltaic power generation systems. Traditional photovoltaic module inspection methods typically rely on manual visual inspection. However, these methods generally suffer from low detection efficiency, high subjectivity, high equipment costs, and long processing times per point, making it difficult to meet the dual requirements of high detection accuracy (≥95%) and high processing efficiency (hundreds of modules / hour) in industrial scenarios. With the continuous development of deep learning and convolutional neural networks, intelligent detection methods based on automatic learning of image features are gradually becoming mainstream. Existing deep learning-based detection models have significantly improved detection accuracy, but still suffer from limited feature extraction capabilities and low detection efficiency, making them unsuitable for real-time monitoring needs in complex environments. Traditional methods have advantages in interpretability and simplicity, but they have significant limitations in complex operating conditions, large-scale, and high real-time scenarios. For example, interference factors such as changes in ambient temperature and illumination can easily cause feature ambiguity; existing models have a high false detection rate in scenarios with multiple faults overlapping; and it is difficult to simultaneously ensure detection accuracy and processing efficiency when inspecting large-area photovoltaic modules.

[0004] Therefore, there is an urgent need for a method and apparatus for photovoltaic module fault detection based on YOLOv7 to solve one of the aforementioned technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for fault detection of photovoltaic modules based on YOLOv7, which can solve at least one of the aforementioned technical problems. The specific solution is as follows:

[0006] According to a specific embodiment of this application, this application provides a method for photovoltaic module fault detection based on YOLOv7, including:

[0007] A dataset containing thermal infrared images of photovoltaic modules and their corresponding temperature matrices was constructed and preprocessed.

[0008] Construct a network based on YOLOv7-tiny;

[0009] Specifically, a thermal infrared and temperature information fusion module is introduced into the backbone of the network, a shallow-deep information aggregation module is introduced into the neck of the network, and a global context awareness module is introduced into the detection head of the network.

[0010] The network is trained using the dataset to obtain a photovoltaic fault detection model;

[0011] The thermal infrared image of the photovoltaic module to be detected and its corresponding temperature matrix are input into the photovoltaic fault detection model, and the fault detection results are output.

[0012] Furthermore, the thermal infrared and temperature information fusion module includes:

[0013] Spatial alignment and normalization are performed on the input thermal infrared image and temperature matrix;

[0014] Spatial texture features and temperature distribution features are extracted using parallel image coding branches and temperature coding branches, respectively.

[0015] The extracted spatial texture features and temperature distribution features are then combined by channel stitching and convolution fusion.

[0016] An improved convolutional block attention module is used to adaptively enhance the channel and spatial dimensions of the fused features.

[0017] Furthermore, the improved convolutional block attention module includes:

[0018] The features obtained after global average pooling and global max pooling are processed and summed using a multilayer perceptron to generate a channel attention weight map.

[0019] The channel attention weight map is multiplied by the fused feature to obtain the channel enhancement feature;

[0020] The channel enhancement features are processed by global average pooling and global max pooling, then concatenated along the channel dimension, and a spatial attention weight map is generated by a convolutional layer.

[0021] The spatial attention weight map is multiplied by the channel enhancement feature to output the enhanced multimodal feature map.

[0022] Furthermore, the shallow-deep information aggregation module includes:

[0023] Multi-scale convolutional kernels are used to extract contextual information of different receptive fields of the input features in parallel, and then channel concatenation is performed;

[0024] A channel attention mechanism is introduced to perform saliency modeling and adaptive weighting on the spliced ​​multi-scale features;

[0025] The expressive power and diversity of features are enhanced by using a lightweight channel transformation module and a nonlinear activation function.

[0026] Group normalization is used to stabilize the transformed features, and the aggregated features are output.

[0027] Furthermore, the channel attention mechanism includes:

[0028] Global average pooling is performed on the input features to generate channel description vectors;

[0029] The channel description vector is sequentially passed through a fully connected layer and an asymmetric activation function for nonlinear mapping;

[0030] The mapped vector is activated by the Sigmoid function to generate the weight coefficients for each channel;

[0031] The generated weight coefficients are multiplied by the original input features channel by channel to complete channel weighting.

[0032] Furthermore, the global context-aware module includes:

[0033] Directional channel context branch and query-key value branch; where,

[0034] The directional channel context branch performs one-dimensional global pooling on the input feature map in the horizontal and vertical directions to capture directional global dependencies.

[0035] The query-key-value branch uses convolution operations to simulate the interaction between query, key and value in the attention mechanism and calculates spatial context weights.

[0036] The output of the directional channel context branch and the output of the query-key value branch are used to perform operations to achieve global context modulation of the input features.

[0037] Furthermore, the directional channel context branch includes:

[0038] One-dimensional average pooling is performed along the height and width directions of the input feature map to generate feature vectors in two directions.

[0039] The feature vectors from the two directions are concatenated and then dimensionality reduced using convolution.

[0040] The dimensionality-reduced features are split into horizontal features and vertical features;

[0041] The dimensions of the horizontal and vertical features are recovered through convolution operations, and the directional channel attention weights are calculated for each spatial location.

[0042] Furthermore, the construction and preprocessing of the dataset includes:

[0043] Collect infrared images and corresponding temperature matrix data of photovoltaic power plants;

[0044] The collected data should be labeled, and the fault types should include at least junction box failure, component breakage, hot spots and bird droppings obstruction;

[0045] Offline augmentation of labeled data can be performed by flipping, rotating, or cropping and splicing to expand the dataset size.

[0046] Furthermore, training the constructed network includes:

[0047] The K-means clustering algorithm is used to re-cluster the sizes of the labeled boxes in the dataset to generate prior anchor boxes that match photovoltaic fault targets;

[0048] The preprocessed training data is input into the network, and the prediction results are calculated through forward propagation.

[0049] We combine localization loss, classification loss, and confidence loss for multi-task joint optimization, and update the network weights through backpropagation.

[0050] According to a specific embodiment of this application, this application also provides a photovoltaic module fault detection device based on YOLOv7, comprising:

[0051] The data preparation module is used to construct and preprocess a dataset containing thermal infrared images and temperature matrices.

[0052] The network construction module is used to construct a YOLOv7-tiny network that integrates the thermal infrared and temperature information fusion module, the shallow-deep information aggregation module, and the global context awareness module.

[0053] The model training module is used to train the network constructed by the network construction module using the data provided by the data preparation module to obtain a photovoltaic fault detection model.

[0054] The fault detection module is used to send the input data to be detected into the photovoltaic fault detection model and output the final fault location and classification results.

[0055] Compared with the prior art, the above-described solutions of this application have at least the following beneficial effects:

[0056] 1. This application discloses a method and apparatus for photovoltaic module fault detection based on YOLOv7. A thermal infrared and temperature information fusion module (ITFM) is introduced at the front end of the backbone network. It performs dual-branch encoding using aligned infrared images and temperature matrices, and utilizes an improved attention mechanism for adaptive fusion and enhancement, generating spatial-channel co-optimized multimodal features. The deeply complementary fusion mechanism of this application's technical solution enables the model to simultaneously utilize texture and temperature distribution information, effectively mining the salient features of the fault area. This results in higher detection accuracy and stronger anti-interference capability when facing complex backgrounds and variable environments.

[0057] 2. This application discloses a method and apparatus for photovoltaic module fault detection based on YOLOv7. A shallow-deep information aggregation (SDA) module and a global context-aware (GCAM) module are introduced into the neck network and detection head, respectively. The SDA module aggregates contextual information from different receptive fields through multi-scale convolution and channel attention mechanisms, enhancing the feature representation of targets with varying scales. The GCAM module captures long-range directional dependencies through directional channel context branches and models spatial relationships using query-key-value branches, achieving global context modulation of the feature map. The synergistic effect of these two modules significantly improves the model's sensitivity and accuracy in perceiving and locating small and ambiguous targets.

[0058] 3. This application discloses a method and apparatus for photovoltaic module fault detection based on YOLOv7. Using the lightweight YOLOv7-tiny as the baseline network, by introducing carefully designed lightweight modules at key locations and focusing on computational efficiency in feature fusion and attention mechanisms, the detection accuracy is significantly improved with only a slight increase in the number of parameters and computational load. This makes the improved model still suitable for deployment on mobile or embedded platforms with limited computing resources, meeting the urgent need for real-time and efficient on-site inspection of photovoltaic power plants, and achieving a balance between lightweight and high precision. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0060] Figure 1 is a flowchart illustrating a method for photovoltaic module fault detection based on YOLOv7, as shown in an embodiment of this application.

[0061] Figure 2 is a schematic diagram of the structure of a YOLOv7 network shown in an embodiment of this application.

[0062] Figure 3 is a schematic diagram of the thermal infrared and temperature information fusion module shown in an embodiment of this application.

[0063] Figure 4 is a schematic diagram of the shallow-deep information aggregation module shown in an embodiment of this application.

[0064] Figure 5 is a schematic diagram of the structure of the global context-aware module shown in an embodiment of this application.

[0065] Figure 6 is a schematic diagram of the clustering results when the number of cluster centroids is 9, as shown in the embodiment of this application.

[0066] Figure 7 is a schematic diagram of the clustering results when the number of cluster centroids is 12, as shown in the embodiment of this application. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0068] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0069] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0070] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0071] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0072] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0073] As shown in Figure 1, a method for photovoltaic module fault detection based on YOLOv7 includes:

[0074] S1. Construct a dataset containing thermal infrared images of photovoltaic modules and their corresponding temperature matrices, and perform preprocessing.

[0075] A photovoltaic module fault dataset was constructed for training and testing. A drone equipped with an infrared thermal imager was used to autonomously fly over a large photovoltaic power plant along a preset route to collect thermal infrared images of the photovoltaic modules and their corresponding pixel-level temperature matrices. Each thermal infrared image covers one photovoltaic string. In this embodiment, each string contains 16 or 22 photovoltaic modules. Based on the PASCAL VOC dataset format, the LabelImg tool was used to finely annotate the 503 original images, generating corresponding label files for subsequent model training. Fault categories were defined as four types: junction box failure, module breakage, hot spots, and bird droppings obstruction.

[0076] In this embodiment, the various fault types and their corresponding numbers were statistically analyzed, as follows: 37 images of junction box faults; 79 images of component breakage faults; 299 images of component hot spot faults; and 88 images of bird droppings obstruction faults, totaling 503 images. 50 original images were reserved as a test set. The remaining 453 images underwent offline data augmentation through methods such as horizontal flipping, scaling, rotation, and cropping / stitching to expand the dataset size and improve the model's generalization ability. After augmentation, a total of 3859 images were obtained, which were divided into training and validation sets in a 4:1 ratio.

[0077] S2. Construct a network based on YOLOv7-tiny. As shown in Figure 2, a thermal infrared and temperature information fusion module (ITFM) is introduced into the backbone of the network, a shallow-deep information aggregation module (SDA) is introduced into the neck of the network, and a global context awareness module (GCAM) is introduced into the detection head of the network.

[0078] This application provides a preferred technical solution, a thermal infrared and temperature information fusion module (ITFM), as shown in Figure 3, including:

[0079] The input thermal infrared image and temperature matrix are spatially aligned and normalized. The original thermal infrared image... and temperature matrix Spatial alignment is performed using bilinear interpolation, unifying the dimensions to the input dimensions (H×W) of the backbone network. And T, to ensure spatial consistency of subsequent feature encoding operations.

[0080]

[0081] in, This indicates the thermal infrared image before alignment; This represents the temperature matrix before alignment. express OK A matrix of real numbers in columns; This represents the height of the original input image, i.e., the number of rows, which is the number of pixels in the vertical direction of the image; H represents the width of the original input image, i.e., the number of columns, which is the number of pixels in the horizontal direction of the image; W represents the number of rows of the backbone network input; W represents the number of columns of the backbone network input. This represents the aligned thermal infrared image; T represents the aligned temperature matrix.

[0082] Will Both T and T are linearly normalized. Let the pixel range of the infrared image be [ The temperature range is []. The normalized expression is:

[0083]

[0084]

[0085] in, This represents the normalized thermal infrared image. This represents the normalized temperature matrix; This represents the thermal infrared image before normalization. This represents the temperature matrix before normalization. These represent the minimum and maximum values ​​of the pixel range, respectively. These represent the minimum and maximum temperature ranges, respectively.

[0086] Parallel image coding and temperature coding branches, namely infrared and temperature branches, are used to extract spatial texture features and temperature distribution features, respectively. The normalized... and Two parallel lightweight convolutional encoding branches, CNN-I and CNN-T, are input respectively to extract spatial texture features and temperature distribution features. The CNN-I branch extracts spatial texture features through 3×3 convolution, batch normalization (BN), ReLU activation, and max pooling operations. The CNN-T branch extracts temperature distribution features through 1×1 convolution, batch normalization (BN), ReLU, and max pooling. . and The expression is:

[0087]

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] in, This represents the feature map output from the first stage of the infrared branch. This represents a 3×3 convolution; This represents the normalized input thermal infrared image; This represents the max pooling operation; This represents the feature map output of the second stage of the infrared branch; This represents the feature map output of the third stage of the infrared branch; This indicates the number of channels in the feature map output from the third stage of the infrared branch. The feature map representing the output of the first stage of the temperature branch; Represents a 1×1 convolution; This represents the normalized input temperature matrix; The feature map representing the output of the third stage of the temperature branch; The feature map representing the output of the second stage of the temperature branch; This represents the number of channels in the feature map output from the third stage of the temperature branch. This indicates the height of the output feature map, i.e., the number of rows; This indicates the width of the output feature map, i.e., the number of columns.

[0094] The extracted spatial texture features and temperature distribution features are then fused through channel concatenation and convolution. and Channel-dimensional concatenation is performed, and then fused using 1×1 convolutions and non-linear activations are applied to obtain... This process further compresses the channel dimension from 2C to C, effectively halving the number of channels and achieving modal information fusion while reducing redundancy. Subsequently, a 1×1 convolution is used for channel compression and initial fusion to obtain the feature map. The specific expression is as follows:

[0095]

[0096]

[0097]

[0098]

[0099] in, C represents the normalized input temperature matrix; C represents the number of channels in the feature map after concatenating spatial and temperature features. This represents the spliced ​​feature map; This represents the feature map after compression and fusion; This represents the activation function.

[0100] An improved convolutional block attention module is used to adaptively enhance the channel and spatial dimensions of the fused features.

[0101] This application provides a preferred technical solution, which introduces an improved convolutional block attention module (CBAM) to enhance the model's ability to perceive key regions and significant temperature differences. The CBAM sequentially performs channel attention and spatial attention calculations.

[0102] The features obtained after global average pooling and global max pooling are processed separately using a multilayer perceptron and then summed to generate a channel attention weight map. The channel attention weight map is multiplied with the fused features to obtain the channel-enhanced features. These channel-enhanced features are then concatenated along the channel dimension after undergoing global average pooling and global max pooling, and a spatial attention weight map is generated through a convolutional layer. Finally, the spatial attention weight map is multiplied with the channel-enhanced features to output the enhanced multimodal feature map.

[0103] In this embodiment, a shared multilayer perceptron (MLP) is used. Channel attention is processed by the MLP, which combines the features obtained from global average pooling and global max pooling, and then summed. The sum is then processed by a sigmoid function to generate a channel attention weight map. Spatial attention then enhances the features of the channel. After global average pooling and global max pooling, the data is concatenated along the channel dimension, and then a spatial attention weight map is generated by a 7×7 convolution and a sigmoid function. The final output is the feature modulated by dual attention. This provides a more discriminative multimodal input for subsequent networks. The specific expression is as follows:

[0104]

[0105]

[0106]

[0107]

[0108] in, Represents the channel attention weight map; This represents the intermediate enhanced feature map after channel weighting; Represents the spatial attention weight map; This represents the enhanced multimodal feature map; This indicates an average pooling operation; This represents the maximum pooling operation; This represents a 7×7 convolution.

[0109] This application provides a preferred technical solution, as shown in Figure 4, wherein the shallow-deep information aggregation module (SDA) introduced at the neck of the network includes:

[0110] Multi-scale convolutional kernels are used to extract contextual information from different receptive fields of the input features in parallel, followed by channel concatenation. First, convolutional layers of three different scales (3×3, 5×5, and 7×7) are used to extract features from the input features. The input feature map is then fed into these three parallel convolutional layers, which use the same number of input / output channels, thereby extracting features from different receptive fields. , , By concatenating the three features mentioned above along the channel dimension, a feature map with three times the number of channels is obtained. Subsequently, a 1×1 convolutional layer is immediately used to compress the channels of the concatenated features, reducing the number of channels to a preset dimension. Group normalization is then introduced to improve the model's stability and generalization ability under mini-batch training, achieving efficient fusion of multi-scale semantics and preparing for subsequent processing. The specific formula is shown below:

[0111]

[0112] in, Represents the feature map; This represents the features under the receptive field of a 3×3 convolution; This indicates the characteristics of the 5×5 sensory field. This indicates the characteristics of the 7×7 sensory field.

[0113] A channel attention mechanism is introduced to perform saliency modeling and adaptive weighting on the concatenated multi-scale features. For the features... Global average pooling (GAP) is performed to obtain the channel description vector. This vector is then passed sequentially through a first fully connected layer, an asymmetric activation function, and a second fully connected layer to obtain the corresponding weights. In this embodiment, the first fully connected layer serves to reduce dimensionality; the asymmetric activation function is ReLU; and the second fully connected layer is used to restore the channel dimension. The output of the second fully connected layer is activated by a sigmoid function, generating a channel attention weight vector ranging from 0 to 1.

[0114] The generated channel attention weight vector is combined with the multi-scale fused features. Perform channel-wise multiplication. Then, add the weighted features element-wise to the original input features of this module, i.e., perform residual connection, to obtain the channel-enhanced features. The technical solution presented in this application effectively preserves the original gradient flow and stabilizes the training process. The specific expression is as follows:

[0115]

[0116] in, This represents the channel enhancement feature, which is the output feature after combining channel attention and residual connection operations, i.e., the feature after channel enhancement. This represents the weight matrix of the first fully connected layer; This represents the weight matrix of the second fully connected layer; Represents the original input features; represents the Sigmoid function; GAP represents global average pooling.

[0117] The technical solution of this application embodiment captures the nonlinear relationship between channels through the first fully connected layer and performs dimensionality reduction to reduce computational load; the number of neurons in the second fully connected layer is restored to the original number of channels, and its output restores the channel dimension. The generated channel attention weight vector is multiplied channel-by-channel with the original multi-scale fusion features, so that the feature map of each channel is multiplied by the corresponding weight coefficient, realizing adaptive weighting, which can improve the ability to obtain information useful for fault identification, while suppressing redundant or noisy channels.

[0118] By using a lightweight channel transformation module and a non-linear activation function, the expressive power and diversity of features are enhanced. First, regularization is applied using a Dropout layer to prevent the network from relying on a few specific neurons, thereby enhancing the model's generalization ability. The features regularized by Dropout are then input into a lightweight shared multilayer perceptron (MLP) to perform cross-channel nonlinear mapping and feature transformation, resulting in further optimized features. The specific expression is:

[0119]

[0120] in, This indicates a feature that is optimized in one step.

[0121] The features are processed in one step using a 1×1 convolutional layer, adjusting their channel dimensions to the target size. The output of the 1×1 convolutional layer is then fed into a group normalization layer. GN (Group Normalization) effectively avoids the performance instability of batch normalization (BN) during mini-batch training by normalizing the channels by grouping them, thus significantly improving the model's generalization ability and training stability. The output of the group normalization layer is the final feature of this module. The data is then fed into the subsequent parts of the network for final target detection. The specific expression is:

[0122]

[0123] in, This represents the aggregated features.

[0124] This application provides a preferred technical solution, as shown in Figure 5. The Global Context Awareness (GCAM) module includes two parallel branches. Branch 1 is the directional channel context branch, which is the Directional ChannelContext branch in this embodiment. Branch 2 is the query-key branch, which is the QK branch in this embodiment. Ultimately, feature modulation is achieved through computation. Branch 1 is used to model the one-dimensional structural information of the feature map in the horizontal and vertical directions. Branch 2 is independent of specific directions and processes the global spatial context relationship.

[0125] By employing a directional channel context branch, one-dimensional global pooling is performed on the input feature map in both the horizontal and vertical directions to capture directional global dependencies. This module enhances the model's ability to discriminate fault regions, especially weak targets, by jointly modeling directional global dependencies and spatial context relationships.

[0126] This application provides a preferred technical solution, employing directional channel context branching and integrating global information using AvgPool and MaxPool. One-dimensional average pooling is performed along the height and width directions of the input feature map, generating feature vectors in two directions. The input feature map is denoted as M. Along the horizontal direction of the feature map, one-dimensional global average pooling and one-dimensional global max pooling are performed. For the c-th channel and h-th row, average pooling is calculated, and max pooling takes the maximum value in that row, generating two feature vectors with dimensions C×H×1. Along the vertical direction of the feature map, one-dimensional global average pooling and one-dimensional global max pooling are performed. For the c-th channel and w-th column, average pooling is calculated, and max pooling takes the maximum value in that column, generating two feature vectors with dimensions C×1×W.

[0127] The technical solution of this application embodiment adds and fuses the results of average pooling and max pooling in the same direction to combine overall statistical information and the most salient features, thereby obtaining enhanced horizontal and vertical feature vectors respectively. The directional pooling strategy in this embodiment can effectively aggregate long-range dependencies, help separate redundant backgrounds and enhance fault regions, and prioritize the location information of weak targets.

[0128] The two directional feature vectors are concatenated and then dimensionality is reduced by convolution.

[0129] The specific expression is:

[0130]

[0131] in, This represents the value of the c-th channel of the input feature map at position (h,j), and the first summation term corresponds to horizontal pooling; The value of the c-th channel of the input feature map at position (i, w) is represented by the second summation term, which corresponds to vertical pooling. f represents the output feature after dimensionality reduction. C, H, and W represent the number of channels, height, and width of the input feature map, respectively.

[0132] The dimensionality-reduced features are split into horizontal and vertical features, and dimensionality is restored through 1×1 convolutions for each. Finally, directional channel attention weights are calculated at each position (i,j). The horizontal features after dimensionality restoration are then selected. The vector in the i-th row , and vertical features vector in the j-th column After final feature adjustment through independent 1×1 convolutions, a two-dimensional directional channel attention weight map is generated. The weight value of each position combines the global dependency information of the row and column in which that position is located.

[0133] The specific expression is:

[0134]

[0135] in, This indicates that a 1×1 convolution operation is performed, i.e., feature adjustment is performed.

[0136] This application provides a preferred technical solution. Through branch 2, the query-key-value branch, convolutional operations are used to simulate the interaction between query, key, and value, calculating spatial context weights. The query-key-value branch is a parallel structure. One branch directly uses a 1×1 convolution to obtain a linear transformation result, generating value features and obtaining a value feature matrix. The other branch uses a 1×1 convolution and a sigmoid function to simplify the multiples of query and key, calculating an attention distribution to generate simplified query-key interaction weights, i.e., generating a QK attention weight matrix. The value feature matrix and the QK attention weight matrix are multiplied to aggregate the spatial context. The aggregated context information is then passed through a 1×1 convolutional layer, a batch normalization layer, and a ReLU activation function for nonlinear enhancement and transformation. Finally, the enhanced features are residually concatenated with the original input features to complete the processing within the branch. The resulting output is the spatial context feature map calculated by this branch, i.e., the feature map output by the query-key-value branch. The expression for the spatial context feature map is:

[0137]

[0138] in, This represents the enhanced feature vector output from the spatial context feature map at the i-th location; This represents the original input feature vector at the i-th spatial location, i.e., the features before processing by this attention module; This represents the weight matrix of the second linear transformation layer; This represents the weight matrix of the first linear transformation layer; This represents the total number of spatial locations in the input feature map; The feature x representing position j j After linear transformation W kThe unnormalized attention score was calculated subsequently; The feature x representing position m m After linear transformation W k The unnormalized attention score is calculated subsequently; m represents the index variable for traversing all spatial locations in the feature map; This represents the linear transformation weight matrix used to calculate the attention score.

[0139] The output of the directional channel context branch is processed in conjunction with the output of the query-key branch to achieve global context modulation of the input features. (Directional channel attention weight graph) and spatial context feature map After performing the Hadamard product, i.e., the element-wise multiplication operation of the broadcast, an enhanced feature map is obtained. Finally, the global context awareness module outputs an enhanced feature map that has been jointly modulated by the global and local contexts, which significantly improves the model's feature discrimination of fault areas, especially in scenarios with weak target information.

[0140] The network in this application uses YOLOv7-tiny as its basic framework. The backbone incorporates a thermal infrared and temperature information fusion module as the network's information entry point. This module aligns, fuses, and enhances the attention of the spatial texture features of the thermal infrared image with the quantized distribution features of the temperature matrix, providing richer and more complementary information from the source. This significantly improves the model's sensitivity and discriminative power for faults, especially temperature anomalies such as hot spots, fundamentally reducing the false detection rate. It solves the technical problem that in complex environments, such as those with varying lighting and background interference, it is difficult to clearly distinguish faulty areas from normal areas based solely on the texture information of thermal infrared images, easily leading to false detections and missed detections.

[0141] A shallow-deep information aggregation module is introduced in the neck area. Multi-scale convolution is used to extract features from different receptive fields in parallel, and channel attention is used for adaptive weighting. This allows for the simultaneous capture of details and context, focusing on the most effective feature channels. This addresses the technical problem in photovoltaic modules where targets such as microcracks and small hot spots have a small pixel ratio. In existing downsampling techniques, shallow localization information and deep semantic information are easily lost or difficult to effectively fuse, leading to poor small target detection performance.

[0142] The detection head incorporates a global context-aware module. Through a directional channel context branch, it captures long-range dependencies of feature maps in the horizontal and vertical directions, aiding in the accurate perception of target edges and orientation, thus improving localization accuracy. The query-key branch, using a simplified self-attention mechanism, calculates spatial context weights across all locations, ensuring that the judgment of each point incorporates global information. This addresses the technical challenge of fault regions often being intertwined with the background, making it difficult for models lacking a global perspective to accurately determine whether a local area is faulty and where its precise boundaries lie.

[0143] S3. Use the dataset to train the network and obtain a photovoltaic fault detection model.

[0144] This application provides a preferred technical solution, which uses the K-means algorithm to re-cluster the target boxes to generate prior anchor boxes that are more closely matched to the size of photovoltaic fault targets.

[0145] All bounding boxes in the training dataset are selected as samples. Depending on the number of YOLOHeads in the network, such as 3 or 4, the number of cluster centroids k is set to 9 or 12. Three YOLOHeads typically correspond to large, medium, and small scale feature maps, responsible for detecting large, medium, and small targets respectively. This is a very classic and common setup. Four YOLOHeads introduce finer-grained scale division, adding one more head to detect extremely small targets compared to three, further improving the detection capability for small targets. The number of YOLOHeads can be set according to actual needs; this application does not impose a limit on this.

[0146] In this embodiment, the examples illustrate a scenario with 3 YOLOHeads and 9 cluster centroids (k), and 4 YOLOHeads and 12 cluster centroids (k). Each YOLOHead requires a set of predefined prior anchor boxes as initial references for prediction, and the total number of anchor boxes is three times the number of YOLOHeads. The number of centroids (k) in K-means clustering is equal to the total number of anchor boxes in all YOLOHeads, ensuring that each prediction location generates a prior box size that best matches the dataset. Data points are iteratively assigned to the nearest centroid, and the centroid positions are updated until the centroids stabilize and no longer change. Finally, nine candidate box sizes are obtained: (5,5), (5,7), (7,5), (7,8), (8,11), (7,34), (11,65), (28,60), and (35,75). These anchor box sizes better match the actual shape and size distribution of faults such as small hot spots and large targets in component fragmentation in the dataset, providing a better initial reference for the model, helping to accelerate training convergence and improve localization accuracy. When the number of cluster centroids k is set to 12, the candidate box sizes are: (0,10), (5,6), (5,7), (7,7), (6,8), (7,9), (9,13), (7,33), (11,65), (27,59), (33,69), (39,84). The clustering results are shown in Figures 6 and 7. Each dot in the figure represents a sample of a target box in the training set, and the origin corresponds to a unique coordinate in the figure, namely the box width and box length data. The K-means algorithm divides the target box width and length into groups. Dots of the same color represent target boxes with similar widths and lengths. Their dimensions are similar, so they are grouped into the same group, i.e., the same color. In the diagram, X represents this color, which is the average size of the boxes in this group. They are grouped together because their sizes are similar, and the average value represents the average size of the boxes in this group: width and length. If K=9, there are 9 groups; if K=12, there are 12 groups; the number of groups corresponds to the number of X values.

[0147] S4. Input the preprocessed and enhanced training set of 3859 images into the constructed network. The network performs forward propagation, with the input data passing sequentially through the backbone network integrating the ITFM module, the neck network integrating the SDA module, and the detection head integrating the GCAM module. Finally, it outputs the prediction results of the target location, i.e., the bounding box coordinates, the category, and the presence confidence.

[0148] The network's predictions are compared with the ground truth labels to calculate the multi-task joint loss. In this embodiment, the loss functions include: localization loss, classification loss, and confidence loss. CIoU Loss is used to measure the difference in position and shape between the predicted bounding box and the ground truth bounding box, ensuring accurate box regression. Cross-entropy loss is used to measure the difference between the predicted fault category and the ground truth category. This is used to determine whether the predicted bounding box contains the target and the reliability of the prediction.

[0149] The calculated total loss gradient is propagated from the output layer to the input layer using the backpropagation algorithm, and all learnable parameters in the network, such as convolutional kernel weights and fully connected layer parameters in the attention mechanism, are updated using an optimizer such as Adam or SGD. This process is repeated iteratively until the model performance stabilizes on the validation set.

[0150] After training, the final photovoltaic fault detection model is obtained. In practical applications, the thermal infrared image of the photovoltaic module to be detected and its corresponding pixel-level temperature matrix are used as input. The input data first undergoes the same preprocessing procedure as during training, and then is fed into the trained photovoltaic fault detection model, namely the IGS-YOLO model. The model performs forward propagation calculations and finally outputs the fault detection results. Each fault detection result contains the following information: bounding box, representing the specific location of the fault in the image in pixel coordinates; category label, indicating the fault type; and confidence score, representing the model's assessment of the reliability of the detection result.

[0151] This application also provides a device for photovoltaic module fault detection based on YOLOv7, including: a data preparation module, a network construction module, a model training module, and a fault detection module.

[0152] In this embodiment, the data preparation module is responsible for the acquisition and preprocessing of multimodal data from photovoltaic modules. This module controls an infrared thermal imager mounted on a drone to automatically acquire thermal infrared images and temperature matrices along a preset flight path. The module has an embedded data preprocessing unit that can label and convert the raw data, and perform offline data augmentation operations (such as flipping, rotating, cropping, and stitching), ultimately generating structured training, validation, and test sets.

[0153] In this embodiment, the network construction module is responsible for defining and building the improved YOLOv7-tiny detection model. Based on the standard YOLOv7-tiny architecture, this module integrates three core sub-modules: a thermal infrared and temperature information fusion module, a shallow-deep information aggregation module, and a global context awareness module. This module instantiates these components through code to construct a complete, trainable network computation graph.

[0154] The model training module is responsible for driving the model's learning process. This module loads the dataset processed by the data preparation module and calls the model defined by the network building module. It includes a built-in K-means clustering program for calculating optimal anchor boxes. The module encapsulates the complete training loop, including forward propagation, loss calculation (using DSCIOU for localization loss), and combining classification and confidence losses, backpropagation, and parameter updates using the Adam optimizer. It also manages training strategies, such as partial network freezing, and is responsible for saving the optimal model weights.

[0155] Fault Detection Module: Responsible for model deployment and real-time inference. This module loads the final weight file produced by the model training module, deploys the trained model to computing devices such as ground servers or embedded platforms. It provides an application programming interface (API) to receive the thermal infrared image and temperature matrix to be detected, performs necessary preprocessing such as resizing and normalization, and then performs model inference. Finally, this module performs post-processing on the output predicted bounding boxes, such as non-maximum suppression, and provides clear fault detection results—location, category, and confidence level—either as a visual output or transmitted to the photovoltaic power plant intelligent operation and maintenance platform.

[0156] This application's embodiments verify the effectiveness of its technical solution through comparative experiments, comparing it with models such as YOLOv8-GD, YOLOv5-SC, LFS-Yolo, FAA-Yolo, GBM-YoLOv8, EL-LwNet, and SCNet. The comparative experiments used parameters, mAP50, mAP50:95, and FLOPs (computational cost) to measure the superior performance of the models.

[0157] Table 1 Comparison of Experimental Results

[0158]

[0159] The above comparisons show that the technical solution of this application achieves the best performance in all detection accuracy indicators, with mAP50 and mAP50:95 reaching 0.935 and 0.594 respectively, both exceeding the highest values ​​of the current comparative methods. Compared with FAA-Yolo (mAP50 = 0.922, mAP50:95 = 0.589), which has the closest accuracy, the technical solution of this application improves the accuracy by 1.3 and 0.9 percentage points in the two indicators respectively. Compared with other technical solutions, the technical solution of this application maintains a stable advantage at high IoU thresholds, demonstrating a precise fitting ability to the target boundary. Based on the original YOLOv7-tiny, this application improves mAP50 by 17.5% and mAP50:95 by 21.7% while increasing the number of parameters by approximately 42.6% (from 16.2M to 23.1M) and the computational load by approximately 103% (from 56.8G to 115.2G).

[0160] In summary, the technical solution of this application introduces a thermal infrared image and temperature information fusion module (ITFM), combined with a shallow-deep information aggregation module (SDA) and a global context awareness module (GCAM), significantly enhancing the model's ability to detect four types of faults in photovoltaic module junction boxes: faults, cracks, hot spots, and shading. This is particularly effective for detecting small targets like hotspots, maintaining a lightweight design while significantly improving detection accuracy and robustness. The optimized design of feature representation and multi-scale information fusion in this application addresses these issues. Firstly, the introduced shallow-deep information aggregation module enhances the collaborative modeling capability of local structure and global context, effectively alleviating detection difficulties caused by complex backgrounds and varying target sizes. Secondly, the global context awareness module guides the strategy, enabling the network to dynamically focus on key target regions during training, thereby improving detection accuracy under conditions of small targets, shading, and low contrast. This method achieves a significant improvement in accuracy while reducing the number of parameters and computational cost, making it more competitive in high-accuracy applications such as photovoltaic panel fault detection. Furthermore, it demonstrates that the proposed structural optimization strategy can effectively improve detection performance without significantly increasing computational burden.

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0162] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

Claims

1. A method for fault detection of photovoltaic modules based on YOLOv7, characterized in that, include: A dataset containing thermal infrared images of photovoltaic modules and their corresponding temperature matrices is constructed and preprocessed. A YOLOv7-tiny-based network is constructed. A thermal infrared and temperature information fusion module is introduced into the backbone of the network, a shallow-deep information aggregation module is introduced into the neck of the network, and a global context awareness module is introduced into the detection head of the network. The network is trained using the dataset to obtain a photovoltaic fault detection model. The thermal infrared image and corresponding temperature matrix of the photovoltaic module to be detected are input into the photovoltaic fault detection model, and the fault detection result is output. The thermal infrared and temperature information fusion module includes: spatial alignment and normalization of the input thermal infrared image and temperature matrix; extraction of spatial texture features and temperature distribution features using parallel image coding and temperature coding branches, respectively; channel concatenation and convolutional fusion of the extracted spatial texture features and temperature distribution features; and channel concatenation and spatial alignment of the fused features using an improved convolutional block attention module. The improved convolutional block attention module includes: adaptive enhancement of inter-dimensional features; processing and summing the features after global average pooling and global max pooling respectively through a multilayer perceptron to generate a channel attention weight map; multiplying the channel attention weight map with the fused features to obtain channel enhanced features; concatenating the channel enhanced features along the channel dimension after global average pooling and global max pooling, and generating a spatial attention weight map through a convolutional layer; multiplying the spatial attention weight map with the channel enhanced features to output an enhanced multimodal feature map; the shallow-deep information aggregation module includes: extracting different receptive field context information of the input features in parallel using multi-scale convolutional kernels and concatenating the channels; introducing a channel attention mechanism to perform saliency modeling and adaptive weighting on the concatenated multi-scale features; improving the expressive power and diversity of features through a lightweight channel transformation module and a nonlinear activation function; stabilizing the transformed features using group normalization and outputting the aggregated features.

2. The method according to claim 1, characterized in that, The channel attention mechanism includes: performing global average pooling on the input features to generate channel description vectors; sequentially mapping the channel description vectors through fully connected layers and asymmetric activation functions nonlinearly; activating the mapped vectors through the Sigmoid function to generate weight coefficients for each channel; and multiplying the generated weight coefficients with the original input features channel by channel to complete channel weighting.

3. The method according to claim 1, characterized in that, The global context awareness module includes: a directional channel context branch and a query-key-value branch; wherein, the directional channel context branch performs one-dimensional global pooling on the input feature map in the horizontal and vertical directions to capture directional global dependencies; the query-key-value branch uses convolution operations to simulate the interaction of query, key, and value in the attention mechanism to calculate spatial context weights; the output of the directional channel context branch and the output of the query-key-value branch are combined to achieve global context modulation of the input features.

4. The method according to claim 3, characterized in that, The directional channel context branch includes: performing one-dimensional average pooling along the height and width directions of the input feature map to generate feature vectors in two directions; concatenating the feature vectors in the two directions and reducing their dimensionality through convolution; splitting the dimensionality-reduced features into horizontal and vertical features; restoring the dimensions of the horizontal and vertical features through convolution operations, and calculating the directional channel attention weights at each spatial location.

5. The method according to claim 1, characterized in that, The dataset was constructed and preprocessed, including: collecting infrared images of photovoltaic power plants and corresponding temperature matrix data; labeling the collected data, with fault types including at least: junction box failure, component breakage, hot spots and bird droppings obstruction; and performing offline augmentation on the labeled data by flipping, rotating or cropping and splicing to expand the dataset size.

6. The method according to claim 1, characterized in that, The training of the network includes: using the K-means clustering algorithm to re-cluster the sizes of the labeled boxes in the dataset to generate prior anchor boxes that match photovoltaic fault targets; inputting the preprocessed training data into the network and calculating the prediction results through forward propagation; performing multi-task joint optimization by combining localization loss, classification loss and confidence loss, and updating the network weights through backpropagation.

7. An apparatus for implementing the photovoltaic module fault detection based on YOLOv7 as described in any one of claims 1-6, characterized in that, include: The data preparation module is used to construct and preprocess a dataset containing thermal infrared images and temperature matrices. The network construction module is used to construct a YOLOv7-tiny network integrating the thermal infrared and temperature information fusion module, the shallow-deep information aggregation module, and the global context awareness module; the model training module is used to train the network constructed by the network construction module using the data provided by the data preparation module to obtain a photovoltaic fault detection model; the fault detection module is used to feed the input data to be detected into the photovoltaic fault detection model and output the final fault location and classification results.

Images