A photovoltaic panel defect category detection method based on feature pyramid and cascaded group attention

Abstract

The present application relates to the field of computer vision, and discloses a photovoltaic panel defect category detection method based on feature pyramid and cascade group attention, comprising the following steps: S1. Data acquisition, labeling and preprocessing; S2. Model training using improved YOLOv11 algorithm; S3. Predicting the input infrared picture, accurately positioning the defect and labeling the category. The photovoltaic panel defect category detection algorithm based on feature pyramid and cascade group attention according to the internal circuit structure of the photovoltaic panel and the infrared imaging picture accurately positions, labels and classifies the defects on the photovoltaic panel; the latest YOLOv11 target detection algorithm is adopted and improved, part of the C3k2 module is replaced with SCC3k2, so that the information capture on space and channel is more accurate, and the adverse effects caused by redundant information extraction are reduced; at the same time, different input segmentation is provided for each attention head in the attention module, and cross-head cascade output features are used to enhance the feature diversity of the input to the attention head.

Description

Technical Field

[0001] This invention relates to the field of computer vision, specifically to a method for detecting defect categories in photovoltaic panels based on feature pyramids and cascaded group attention. Background Technology

[0002] Photovoltaic power generation converts solar energy into electrical energy through the photovoltaic effect, providing a renewable energy solution to address the energy crisis and environmental pollution. Large-scale deployment of photovoltaic panels can effectively meet energy demand and promote sustainable development; however, issues such as equipment aging and malfunctions urgently need to be addressed. Therefore, timely detection of photovoltaic panel faults is crucial to ensuring their stable operation.

[0003] Defect detection of photovoltaic panels can be achieved by handheld camera data collection. In recent years, with the popularization of drones, more industries have adopted drones for real-time shooting and infrared data collection, which collects more, more complete, more stable, and higher quality information, which is conducive to providing data support for subsequent defect detection.

[0004] Currently, photovoltaic panel defect detection technology is widely used. However, there is little research on how to accurately classify the detected faults. In practice, it is not only necessary to detect which photovoltaic panels have defects, but also to determine the type of defect in order to know the severity of the specific defect and whether it can be repaired, so as to facilitate subsequent repair or replacement of photovoltaic panels.

[0005] In research on defect classification and detection of photovoltaic panels, traditional methods rely on machine vision for defect detection, employing techniques such as Random Forest (RF), Support Vector Machine (SVM), and K-means clustering for classification. However, these methods have limited capabilities in handling complex features and high-dimensional data. They not only consume significant computational resources but also struggle to scale to larger datasets, limiting their feasibility in practical industrial applications. For instance, in high-resolution infrared images of photovoltaic panels, the size, texture, and other features of different defect categories vary considerably. Traditional methods are susceptible to interference from redundant features, leading to misjudgments.

[0006] Deep learning, with its powerful computational and learning capabilities, has demonstrated remarkable performance in object detection in recent years, with the YOLO algorithm being the most representative. However, since most defects in photovoltaic panels are small (such as debris obscuring and small hot spots), their feature information is insufficient, resulting in poor performance of the YOLO algorithm in small object detection. Therefore, this invention, based on the YOLOv11 algorithm, adaptively improves some modules, enabling the model to focus more on refined feature extraction and reduce redundant feature computation, while also utilizing task alignment to achieve better synergistic effects in photovoltaic panel defect localization and classification detection. Summary of the Invention

[0007] Technical problems solved

[0008] In view of the defects of the prior art, the photovoltaic panel defect category detection method based on feature pyramid and cascade group attention is provided to solve the above problems.

[0009] Technical solutions

[0010] To achieve the above object, the application provides the following technical scheme: a photovoltaic panel defect category detection method based on feature pyramid and cascade group attention, comprising the following steps:

[0011] S1. Data acquisition, labeling and preprocessing;

[0012] S2. Model training using an improved YOLOv11 algorithm;

[0013] S3. Predicting the input infrared picture, accurately positioning the defect and labeling the category.

[0014] Preferably, the specific content of S1 is:

[0015] S1.1: Obtain the infrared imaging picture shot by the unmanned aerial vehicle;

[0016] S1.2: Analyze the internal circuit of the photovoltaic panel, and process the one-to-one correspondence relationship between fault types and manifestations;

[0017] S1.3: Use the labeling tool to finely label the defect fault, including position information and classification group;

[0018] S1.4: Use the opencv library to perform rotation, brightness contrast adjustment, scaling data enhancement operations on the photovoltaic panel picture to generate more samples for subsequent model training.

[0019] Preferably, in S2, the YOLOv11 network is adaptively improved, the picture is transmitted into the pre-trained network, and the model is trained, and the specific content is:

[0020] S2.1: The detection module is divided into backbone network, feature pyramid neck, and final detection head;

[0021] Among them, the down-sampling layer in the backbone network uses multi-layer convolution to extract multi-dimensional features from the input infrared shot photovoltaic panel picture, and uses the SPPF module to enhance the features of important information to obtain defect information;

[0022] In the Neck part, a feature pyramid structure is constructed by introducing an up-sampling layer and multiple convolution modules, and is connected with multiple layers of features of the backbone network to realize multi-scale feature fusion.

[0023] The detection head contains several Detect modules, in which anchor boxes are drawn, and through non-maximum suppression, redundant repeated boxes are removed. The maximum probability class is selected on the output probability distribution, and the defect conditions of small and large samples are output on multiple channels.

[0024] S2.2: In the backbone network, multiple Conv convolution layers C3k2 layers and SCC3k2 layers are alternately connected to build a deep network to continuously sample and extract information. The SCC3k2 module uses a spatial and channel reconstruction convolution SCConv module;

[0025] The SCConv module is composed of a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU) in sequence. First, SRU reduces the redundancy in the spatial dimension, and then CRU reduces the redundancy in the channel dimension. The SCConv module can be seamlessly integrated into the existing CNN characteristics to replace the standard convolution operation;

[0026] After several Conv convolution layers C3k2 layers and SCC3k2 layers are alternately connected, SPPF parallel maximum pooling processing is performed to further enhance the effective defect information.

[0027] At the end of the backbone network, the C2WCGA module is used to replace the original C2PSA module. The cascaded group attention based on local window Window parallel processing pays more attention to learning on the refined features with rich information, and improves the diversity of attention.

[0028] Due to the high resolution of the input picture, a local window attention mechanism is introduced before the input to the module, which is used to split the input width and height into multiple sub-regions according to a certain resolution, so that the width and height are reduced, the channel remains unchanged, and the batch number increases. After dividing multiple local windows, the cascaded group attention mechanism is applied independently to each window. This method can reduce the amount of calculation, especially when processing high-resolution inputs, and can improve the model's attention to local features, thereby improving performance;

[0029] The CGA cascaded group attention part under the C2WCGA module contains more attention heads for attention calculation on different channels, and through cascading operation, the output of the previous head is spliced into the input of the next head, not only enhancing the information flow within each window, but also promoting the information fusion across windows, which is conducive to the model gradually generating a global understanding of the entire image.

[0030] S2.3: The core idea of the feature pyramid FPN is to generate feature maps of multiple scales through different convolutional layers, and to enrich the representation by gradually fusing high-level and low-level features; in the neck layer, the UnSample up-sampling layer refines and expands the features, and is connected with the C3k2 and SCC3k2 modules in the backbone network through concat, so as to fuse the information of high-level and low-level features to enrich the features; in addition, connecting different SCC3k2 modules on different feature output dimensions can further strengthen the effect of multi-scale fusion;

[0031] S2.4: In the last Head layer, the Anchor-Free idea is adopted, compared with Anchor-Based, Anchor-Free does not need to generate pre-defined anchor boxes, so that the model is more flexible, and can more effectively identify various defects of photovoltaic panels;

[0032] At the same time, the Anchor-Free module adopts a decoupled manner, which can make multiple Detect modules play a role to the greatest extent, and output small samples and large samples of defects on multiple channels; in addition, the original Detect module is changed to DTADetect module, which refers to the TOOD (Task-aligned One-stage Object Detection) idea, and performs corresponding dynamic task alignment (Dynamic Task Align) structure design on the detection head, so as to strengthen the cooperative performance of the two tasks of photovoltaic panel defect positioning and category prediction. The extractor can extract different refined features from different convolutional layers and interact with each other, thereby improving the performance of defect classification and detection;

[0033] In this part, first, a plurality of anchor boxes are generated, and then the IOU (Intersection over Union) is analyzed, which is a key indicator for measuring the overlap degree of the predicted box. The larger the IOU value, the more serious the overlap of the predicted box. Then, the NMS (Non-Maximum Suppression) technique is used to remove redundant anchor boxes with high overlap degree, to ensure the reliability of the final prediction result. On this basis, according to the generated category probability, the labeled defect anchor boxes after screening are assigned with corresponding defect categories.

[0034] Preferably, the specific content of S3 is:

[0035] S3.1: Specify the file directory, and use the batch photovoltaic panel infrared pictures therein as input;

[0036] S3.2: Specify the configuration file, and load the model trained by the improved YOLOv11 algorithm;

[0037] S3.3: Traverse each picture in the folder, use the model to classify and detect the defects of the photovoltaic panel, and generate anchor boxes in the picture to locate the defect position and size, and each anchor box will be labeled with the class name and confidence size.

[0038] Preferably, the SCConv module in the SCC3k2 module is used for convolution to efficiently capture information in deep space and channels, reduce the extraction of redundant information, and improve training efficiency.

[0039] In the SRU module of the SCConv module, separation operation and reconstruction operation are included to reduce the redundancy of spatial information. Although the traditional C3k2 reduces the computational complexity to a certain extent through the residual connection block, the part of the data processing still includes redundant feature information, which increases the computational complexity and brings unnecessary noise interference.

[0040] In the separation operation of SRU, Group Normalization is used to extract the scaling factor of the feature map. The larger the variance, the more effective information about the photovoltaic panel defects. By comparing the values in the variance, the spatial distribution of effective information can be known, and according to a certain threshold, it can be divided into two parts, the effective information above the threshold and the redundant information below the threshold. In order to further integrate these two parts of information, reconstruction operation is needed to rearrange and integrate the features through cross-reconstruction to enhance the expression ability of the features.

[0041] The formula of the separation operation is as follows:

[0042] W = T (Sigmoid (W γ(GN(X)) ));

[0043] The formula indicates that in the separation operation, after Group Normalization processing of the input X, the scaling factor W γ is combined for feature extraction, and then the Sigmoid activation function is applied to make the weight value smoother, and finally the threshold T is applied to divide the features into rich features and redundant features, and the final feature map weight W is obtained.

[0044] In the SRU module of SCC3k2, the threshold T is set to 0.4 to extract more and richer photovoltaic panel defect features. Because the debris cover and other categories in the photovoltaic panel cause defects, they are usually small targets on the photovoltaic panel and have more quantity. If the threshold is not changed, there will be a greater probability of identifying them as redundant features.

[0045] The related formula of threshold assignment is as follows:

[0046]

[0047] Wherein W1 is the upper layer rich feature map, and W2 is the shallow layer redundant feature map.

[0048] Preferably, in the backbone network construction and feature extraction, the channel reconstruction unit CRU is divided into three steps of segmentation, transformation and fusion.

[0049] In the segmentation step, the CRU first obtains the spatially refined feature map obtained in the previous step, and divides the channels into two parts, each containing aC channels and (1-a)C channels, and then compresses the feature map through 1x1 convolution to improve the calculation efficiency.

[0050] In the transformation step, the upper half of the separated feature map is transformed through group-wise convolution (GWC) and point-wise convolution (PWC) to extract rich high-level features, and the lower half is connected in residual form with the original shallow input through cheap 1x1 convolution to supplement the shallow feature information.

[0051] In the fusion part, the upper and lower parts of information are weighted and fused through global average pooling and beta parameters related to soft attention mechanism to extract the final channel refined feature map. Compared with direct connection, the fusion of extracted information is more efficient.

[0052] In the bottleneck structure contained in the original C3k2, the Conv of the second layer is replaced by the SCConv module. In order to reduce the calculation burden, the present application selects to replace only the convolution layers in the latter half of the model, so as to realize the performance improvement while effectively controlling the calculation cost, and finally forms the improved SCC3k2 module by replacing the Bottleneck with the SCBottleneck through residual connection.

[0053] Preferably, the step of applying a cascade group attention mechanism (CGA) to enhance feature representation, the CGA module is adopted in the last C2WCGA module of the backbone network, different segmentation of complete features is performed on each head in the multi-head attention, so that each sub-task can extract more accurate refined features, and finally the more effective feature output is obtained to provide valuable context information.

[0054] Specifically, first, the input feature map is spatially segmented into multiple local windows to form multiple sub-regions, in this process, the channel number is kept unchanged, the height and width dimensions are reduced, and the batch size is increased, so that the efficiency of parallel calculation can be effectively improved.

[0055] Next, the multi-head attention mechanism is applied on each window in parallel. First, the input feature map is passed through the QKV layer to calculate the query, key and value, and then it is divided into multiple sub-features and each sub-task is processed respectively. Each head models the features through the self-attention mechanism to capture the local context information, and an attention bias is added in the attention module to guide the attention mechanism to focus more finely on different regions and features, improving the performance of the model. The specific formula is as follows:

[0056]

[0057] where d k is the dimension of the key vector, used to scale the dot product to avoid numerical overflow, B is the bias term, used to adjust the similarity of query Q and key K, the softmax operation converts the similarity into weight, and the value vector V is weighted through these weights, and the weighted value vector V is finally output. In the code implementation, a feat feature map array is introduced to record the multi-head output results obtained by parallel calculation. The output results of each head are concatenated to the input of the next head one by one, and multiple feature fusion and information transmission are performed to ensure effective information exchange between different heads. The specific concatenation formula is as follows:

[0058]

[0059] where the actual input feature in the first layer is the same as the original input feature, and the actual input feature f i of the i-th layer is obtained by superimposing the current original input x i and the output of the previous layer.

[0060] Advantages

[0061] Compared with the prior art, the present application provides a photovoltaic panel defect category detection method based on feature pyramid and cascaded group attention, which has the following advantages:

[0062] The photovoltaic panel defect category detection method based on feature pyramid and cascaded group attention accurately locates and classifies the defects on the photovoltaic panel according to the internal circuit structure of the photovoltaic panel and the infrared imaging diagram.

[0063] The latest YOLOv11 target detection algorithm is adopted and improved, and part of the C3k2 module is replaced with SCC3k2, so that the information capture on space and channel is more accurate, and the adverse effects caused by redundant information extraction are reduced.

[0064] Recycle C2WCGA module instead of the original basic C2PSA, strengthen the fusion performance of the refined features, so that the recognition degree of photovoltaic panel defect classification detection is higher, and the effect is better;

[0065] The detection head part improves the Detect module to the DTADetect module, learns the interaction features between multiple tasks, to extract more refined information;

[0066] After the infrared picture is processed by the trained model, the picture with defect label and classification information and the processed table data are output, so that the photovoltaic panel workers can repair or replace the corresponding position photovoltaic panel in the later period. BRIEF DESCRIPTION OF DRAWINGS

[0067] Figure 1 The algorithm processing flowchart of the photovoltaic panel defect classification detection method based on feature pyramid and cascade group attention;

[0068] Figure 2 The improved YOLOv11 structure diagram in the photovoltaic panel defect classification detection method based on feature pyramid and cascade group attention;

[0069] Figure 3 The SCConv module structure diagram in the photovoltaic panel defect classification detection method based on feature pyramid and cascade group attention;

[0070] Figure 4 The photovoltaic panel defect classification detection effect schematic diagram in the photovoltaic panel defect classification detection method based on feature pyramid and cascade group attention. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0072] It should be understood that in various embodiments of the present application, the size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0073] It should be understood that in the present application, "comprising" and "having" and any variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that comprises a list of steps or units need not necessarily be limited to those clearly recited but can include other steps or units that are not expressly listed or inherent to such processes, methods, products or apparatuses.

[0074] Referring to Figures 1-4 A photovoltaic panel defect category detection method based on feature pyramid and cascade group attention, comprising the following steps:

[0075] S1. Data acquisition, labeling and preprocessing;

[0076] S2. Training the model using an improved YOLOv11 algorithm;

[0077] S3. Predicting the input infrared picture, accurately positioning the defect and labeling the category.

[0078] It should be noted that the specific content of S1 is:

[0079] S1.1: Obtain the infrared imaging picture taken by the unmanned aerial vehicle;

[0080] S1.2: Analyze the internal circuit of the photovoltaic panel, and process the one-to-one correspondence relationship between fault types and manifestations;

[0081] S1.3: Use the labeling tool to finely label the defect faults, including position information and classification groups;

[0082] S1.4: Use the opencv library to rotate, adjust brightness and contrast, and zoom the photovoltaic panel picture for data enhancement operations to generate more samples for subsequent model training.

[0083] It should be noted that S2 adaptively improves YOLOv11 network, transmits the picture into the pre-trained network, and trains the model, and the specific content is:

[0084] S2.1: The detection module is divided into backbone network, feature pyramid neck, and final detection head;

[0085] Among them, the down-sampling layer in the backbone network uses multi-layer convolution to extract multi-dimensional features from the input infrared imaging photovoltaic panel picture, and uses the SPPF module to enhance the features of important information to obtain valuable defect information;

[0086] In the neck part, the feature pyramid structure is constructed by introducing the up-sampling layer and multiple convolution modules, and connected with the multi-layer features of the backbone network to realize multi-scale feature fusion;

[0087] The detection head comprises a plurality of Detect modules, in which anchor boxes are drawn, and through non-maximum suppression, redundant repeated boxes are removed, the maximum probability category is selected on the output probability distribution, and the small sample and the large sample are output on the multi-channel respectively;

[0088] The specific structure of the backbone network is as follows:

[0089] S2.2: In the backbone network, a plurality of Conv convolutional layers C3k2 layers and SCC3k2 layers are alternately connected to build a deep network to continuously sample and extract information. The specific connection mode is shown in the following Figure 2 , and the gray part in the figure is the improved module. The SCC3k2 module in the SCC3k2 module uses a space and channel reconstruction convolution SCConv module to replace the convolution part in the traditional C3k2 module to enhance the deep feature extraction effect;

[0090] The SCConv module is composed of a space reconstruction unit (SRU) and a channel reconstruction unit (CRU) in sequence, which first reduces the redundancy in the spatial dimension through SRU, and then reduces the redundancy in the channel dimension through CRU; the SCConv module can be seamlessly integrated into the existing CNN characteristics, and is used to replace the standard convolution operation;

[0091] After a plurality of Conv convolutional layers C3k2 layers and SCC3k2 layers are alternately connected, SPPF parallel maximum pooling processing is performed, and effective defect information is further enhanced;

[0092] At the end of the backbone network, the C2WCGA module is used to replace the original C2PSA module. The cascaded group attention based on local window Window parallel processing pays more attention to learning on the refined features with rich information, and improves the diversity of attention;

[0093] Due to the high resolution of the input picture, a local window attention mechanism is introduced before the input into the module, which is used to split the input width and height into a plurality of sub-regions according to a certain resolution, so that the width and height are reduced, the channel is unchanged, and the batch number is increased. After a plurality of local windows are divided, the cascaded group attention mechanism is independently applied to each window. This method can reduce the amount of calculation, especially when processing high-resolution input, and can improve the attention of the model to local features, thereby improving the performance;

[0094] In the CGA cascade group attention part under this module, more attention heads are included to perform attention calculation on different channels, and the cascade operation is performed, that is, the output of the previous head is spliced into the input of the next head, which not only enhances the information flow within each window, but also promotes the information fusion across windows, which is beneficial to the model to gradually generate the global understanding of the entire image;

[0095] S2.3: The core idea of the feature pyramid FPN is to generate feature maps of multiple scales through different convolution layers, and to enrich the representation by gradually fusing high-level and low-level features; in the neck layer, the UnSample up-sampling layer refines and expands the features, and is connected with the C3k2 and SCC3k2 modules in the backbone network through concat, so as to fuse the information of high-level and low-level to enrich the features; in addition, different SCC3k2 modules are connected on different feature output dimensions, which can further strengthen the effect of multi-scale fusion;

[0096] S2.4: In the last Head layer, the Anchor-Free idea is adopted, which does not need to generate predefined anchor boxes compared with Anchor-Based, so that the model is more flexible and can more effectively identify various defects of photovoltaic panels;

[0097] At the same time, this module adopts a decoupled manner, which can make multiple Detect modules play a role to the greatest extent, and output small sample and large sample defects on multiple channels respectively; in addition, the original Detect module is changed to DTADetect module, which refers to the TOOD (Task Aligned One-Stage Object Detection) idea, and performs corresponding Dynamic Task Align structure design on the detection head, so as to strengthen the cooperative performance of photovoltaic panel defect positioning and class prediction. The extractor in it can extract different refined features from different convolution layers and interact with each other, thereby improving the performance of defect classification and detection;

[0098] In this part, a plurality of anchor boxes are first generated, and then analyzed according to the IOU (Intersection over Union), a key indicator for measuring the overlapping degree of the predicted box. The larger the IOU value is, the more serious the overlap of the predicted box is. Then, the NMS (Non-Maximum Suppression) technology is used to remove redundant anchor boxes with high overlap, to ensure the reliability of the final prediction result. On this basis, according to the generated class probability, the labeled defect anchor boxes after screening are assigned with corresponding defect classes.

[0099] It should be noted that the data acquisition is performed by using a UAV, and high-resolution infrared TIR photovoltaic shooting pictures are obtained through infrared imaging technology;

[0100] Using the opencv library, the picture is rotated, the brightness contrast picture is adjusted, and other operations are performed to enhance the data, and more samples are obtained as model input;

[0101] Since the infrared acquisition picture has most of the information, such as local temperature rise caused by hot spots, local image color brightening, therefore, using the labelimg labeling tool, according to the characteristics corresponding to different defects of the photovoltaic panel, the infrared shooting picture is finely classified and labeled, including broken plate, battery damage, tree shadow and other shadow coverings.

[0102] It should be noted that in the main network, the SCConv module in the SCC3k2 module is used for convolution to efficiently capture information in deep space and channels, reduce redundant information extraction and improve training efficiency;

[0103] In the SRU module of the SCConv module, separation operation and reconstruction operation are included to reduce the redundancy of spatial information. Although the traditional C3k2 reduces the computational complexity to a certain extent through the residual connection block, the part of the data processing still includes redundant feature information, which increases the computational complexity and brings unnecessary noise interference;

[0104] In the separation operation of SRU, Group Normalization is used to extract the scaling factor of the feature map, and the greater the variance represents the more effective information about the photovoltaic panel defects. By comparing the values in the variance, the spatial distribution of effective information can be known, and according to a certain threshold, it can be divided into two parts, the effective information above the threshold and the redundant information below the threshold. In order to further integrate the two parts of information, reconstruction operation is needed to rearrange and integrate the features through cross reconstruction to enhance the expression ability of the features;

[0105] The formula of the separation operation is as follows:

[0106] W = T(Sigmoid(W γ(GN(X)) ));

[0107] The formula indicates that in the separation operation, after Group Normalization processing of the input X, the scaling factor W γ is used for feature extraction, and then the Sigmoid activation function is applied to make the weight value smoother, and finally the threshold gate T is used to divide the features into rich features and redundant features, and the final feature map weight W is obtained.

[0108] In the SRU module of the SCC3k2, the threshold T is changed from the default 0.5 to 0.4 for extracting more and richer photovoltaic panel defect features, because the defects caused by the debris cover and the like in the photovoltaic panel are usually small targets on the photovoltaic panel and have a larger number, and if the threshold is not changed, there will be a greater probability of identifying them as redundant feature processing;

[0109] The related formula of threshold assignment is as follows:

[0110]

[0111] Wherein W1 is the upper layer rich feature map, and W2 is the shallow layer redundant feature map.

[0112] It should be noted that in the backbone network construction and feature extraction, the channel reconstruction unit CRU in the SCConv module is divided into three steps of segmentation, transformation and fusion.

[0113] In the segmentation step, the CRU first obtains the spatially refined feature map obtained in the previous step, and divides the channels into two parts, respectively containing aC channels and (1-a)C channels, and then compresses the feature map through 1*1 convolution to improve the calculation efficiency.

[0114] In the transformation step, the upper half of the separated feature map is transformed through group-wise convolution (GWC) and point-wise convolution (PWC) to extract rich high-level features; and the lower half is connected in residual through a cheap 1*1 convolution and the original shallow layer input to supplement the shallow feature information.

[0115] In the fusion part, the upper and lower parts of information are weighted and fused through global average pooling and a beta parameter related to the soft attention mechanism to extract the final channel refined feature map. Compared with direct connection, the fusion of extracted information is more efficient.

[0116] In the bottleneck structure originally contained in the C3k2, the second layer of Conv is replaced with the SCConv module. In order to reduce the calculation burden, the present application selects to replace only the convolution layers in the latter half of the model, so as to realize performance improvement while effectively controlling the calculation cost, and finally forms an improved SCC3k2 module by replacing the Bottleneck with a residual connection to form an SCBottleneck.

[0117] It should be noted that the step of applying the cascaded group attention mechanism (CGA) to enhance feature representation adopts the CGA module in the last C2WCGA module of the backbone network, which makes each sub-task extract more accurate and refined features by different segmentation of complete features in each head of the multi-head attention, and finally connects to obtain more effective feature output to provide valuable context information.

[0118] Specifically, first, the input feature map is spatially segmented and divided into multiple local windows, thereby forming multiple sub-regions. In this process, the number of channels is kept unchanged, the height and width dimensions are reduced, and the batch size is increased, thereby effectively improving the efficiency of parallel computing.

[0119] Next, the multi-head attention mechanism is applied in parallel on each window. First, the input feature map is subjected to QKV layer to calculate the query, key and value, and then it is divided into multiple sub-features and each sub-task is processed. Each head performs feature modeling through self-attention mechanism to capture local context information. An attention bias is added in the attention module to guide the attention mechanism to focus more finely on different regions and features, thereby improving the performance of the model. The specific formula is as follows:

[0120]

[0121] where d k is the dimension of the key vector, used to scale the dot product to avoid large numbers, B is the bias term, used to adjust the similarity of query Q and key K, the softmax operation converts the similarity into weight, and the value vector V is weighted through these weights, and finally the weighted value vector V is output. In the code implementation, a feat feature map array is introduced to record the multi-head output results obtained by parallel computing. The output results of each head are sequentially concatenated to the input of the next head for multiple feature fusion and information transmission to ensure effective information exchange between different heads. The specific concatenation formula is as follows:

[0122]

[0123] where the actual input feature in the first layer is the same as the original input feature, and the actual input feature f i of the i-th layer is obtained by stacking the current original input x i and the output of the previous layer .

[0124] Finally, the outputs of all heads are reconnected to form the final concatenated group attention result. After shape transformation, this result will be passed as input to the next layer network for subsequent processing.

[0125] The neck layer also replaces the traditional C3k2 structure by introducing the SCC3k2 module, and through more refined feature extraction and multi-scale information fusion, the perception ability of the model for defects of different scales is significantly improved, especially the detection accuracy in complex background. Specifically, the SCC3k2 module uses spatial channel reconstruction convolution to reduce the calculation of redundant information, so that the model can pay more attention to the fusion of rich and refined features, not only enhancing the expression ability of features at different levels, but also effectively solving the problem caused by scale change, so that the network can more accurately capture defects of different sizes and different categories on the surface of the photovoltaic panel, including large-area circuit board failures and tiny bird droppings and debris coverings, etc. In complex scenes with different factors such as light and angle, the model also shows high accuracy, further enhancing the robustness of the model.

[0126] In the final Head layer, multiple DTADetect modules are used for output. This model uses the dynamic task alignment idea to improve the target detection accuracy and efficiency through task decomposition and dynamic convolution mechanism. The specific content is as follows:

[0127] First, multi-scale feature processing is performed. For each input scale x i , the features are first extracted through the shared convolution layer ShareConv, and then the features are spliced to obtain a fused feature map.

[0128] For the spliced feature map, first, the adaptive average pooling is used to reduce the dimension to an average feature to obtain the global context information, and then the task decomposition module is used to perform regression and classification feature extraction on the feature map respectively.

[0129] In the regression task, first, a spatial convolution offset is used to generate an offset and a mask. These offsets will help adjust the regression features to make the target box regression more accurate. Then, the DyDCNV2 dynamic convolution module is used to input the regression features, offset and mask together, so as to further adjust and align the regression features.

[0130] For the classification task, first, a convolution layer is used to obtain the classification features, which are activated by ReLU, and then a class convolution layer is used to generate the class probability of each region. The classification features and the class probability are multiplied to obtain the final class prediction.

[0131] For each scale output, the regression prediction and the class prediction are spliced together to merge the results, and the final prediction output of each anchor box is obtained. The predicted target box position is decoded and converted into actual coordinates to complete the drawing of the final bounding box.

[0132] By sharing convolutional layers at multiple scales, this module not only reduces the amount of calculation, but also effectively deals with the detection needs of targets of different scales. Using adaptive feature processing and task alignment strategies, the model can efficiently process input features during training and inference, output accurate target boxes and class probabilities, and further improve the adaptability of the detection head to multi-scale and multi-class targets.

[0133] During the training process, the model is constantly converging, and since its convergence speed is not completely monotonic, the last model is recorded, as well as all the models with the minimum loss during the process.

[0134] Using the best model file trained, the photovoltaic panel picture is classified and detected for defects, and after obtaining the results, the defect information is integrated and processed to achieve accurate positioning of the defects.

[0135] Different color anchor boxes are labeled on the image, and the image results are output. At the same time, the processed defect fault coordinates, categories and other information are sorted into an excel table for storage.

[0136] Related photovoltaic staff can locate and analyze the defect photovoltaic panel according to the visual defect classification detection image and excel data table, which is convenient for subsequent photovoltaic panel repair and replacement work

[0137] It should be noted that the specific content of S3 is:

[0138] S3.1: Specify the specified file directory, and use the batch photovoltaic panel infrared pictures in it as input;

[0139] S3.2: Specify the configuration file and load the model trained by the improved YOLOv11 algorithm;

[0140] S3.3: Traverse each picture in the folder, use the model to classify and detect the defects of the photovoltaic panel, and generate anchor boxes in the picture to locate the defect position and size. Each anchor box will be labeled with the category name and confidence size.

[0141] In summary, the application provides a photovoltaic panel defect category detection method based on feature pyramid and cascaded group attention, which can accurately locate and label the defects on the photovoltaic panel and classify them according to the internal circuit structure of the photovoltaic panel and the infrared imaging diagram; the latest YOLOv11 target detection algorithm is adopted and improved, and part of the C3k2 module is replaced with SCC3k2, so that the information capture on space and channel is more accurate, and the adverse effects caused by redundant information extraction are reduced; then the C2WCGA module is used instead of the original basic C2PSA to strengthen the fusion performance of the refined features, so that the recognition degree of photovoltaic panel defect classification detection is higher and the effect is better; the Detect module in the detection head part is improved to DTADetect module to learn the interaction features between multiple tasks to extract more refined information; after the infrared picture is processed by the trained model, the picture with defect label and classification information and the processed table data are output, which is convenient for photovoltaic panel workers to repair or replace the corresponding position photovoltaic panel.

[0142] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A photovoltaic panel defect class detection method based on feature pyramid and cascaded group attention, characterized in that, The method comprises the following steps: S1. Data acquisition, labeling and preprocessing; S2. Training of the model using an improved YOLOv11 algorithm; S3. Prediction of the input infrared picture, accurate positioning of defects and labeling of categories; In S2, the YOLOv11 network is adaptively improved, the picture is input into the pre-trained network, and the model is trained, and the specific content is as follows: S2.1: The detection module is divided into a backbone network, a feature pyramid neck and a final detection head; In the backbone network, the down-sampling layer uses multi-layer convolution to extract multi-dimensional features from the input infrared photovoltaic panel picture, and uses an SPPF module to enhance the features of important information to obtain defect information; In the neck part, an up-sampling layer and a plurality of convolution modules are introduced to construct a feature pyramid structure, and the multi-scale feature fusion is realized by connecting the multi-layer features of the backbone network; The detection head contains a plurality of Detect modules, in which the anchor frame is drawn, and the redundant repeated frame is removed through non-maximum suppression, and the maximum probability category is selected on the output probability distribution, and the small sample and large sample defect conditions are output on the multi-channel; S2.2: In the backbone network, a plurality of Conv convolution layers C3k2 layers and SCC3k2 layers are alternately connected to build a deep network to continuously sample and extract information, and the SCC3k2 module utilizes a spatial and channel reconstruction convolution SCConv module; The SCConv module is composed of a spatial reconstruction unit (SRU) and a channel reconstruction unit (CRU) in sequence, which first reduces the redundancy in the spatial dimension through SRU, and then reduces the redundancy in the channel dimension through CRU; After the plurality of Conv convolution layers C3k2 layers and SCC3k2 layers are alternately connected, the effective defect information is further enhanced through SPPF parallel maximum pooling processing; At the end of the backbone network, a C2WCGA module is used to replace the original C2PSA module, which utilizes a local window Window-based parallel processing cascade group attention; Before the input picture is input into the C2WCGA module, a local window attention mechanism is introduced to split the input width and height into multiple sub-regions according to a certain resolution, so that the width and height are reduced, the channel remains unchanged, and the batch number increases, and after the multiple local windows are divided, the cascade group attention mechanism is independently applied to each window; In the CGA cascade group attention part of the C2WCGA module, a plurality of attention heads are included for attention calculation on different channels, and the cascade operation is performed, that is, the output of the previous head is spliced into the input of the next head; S2.3: In the neck layer, the UnSample up-sampling layer refines and expands the features, and is connected with the C3k2 and SCC3k2 modules in the backbone network through concat; S2.4: In the last Head layer, an Anchor-Free module is adopted; At the same time, the Anchor-Free module adopts a decoupled manner to output small sample and large sample defects on multiple channels respectively; in addition, the original Detect module is changed to a DTADetect module, and the extractor in the module can extract different refined features from different convolution layers and interact with each other; First, a plurality of anchor boxes are generated, and then the key indicator of IOU is used to analyze the overlapping degree of the predicted boxes, then the NMS technology is used to remove redundant anchor boxes with high overlapping degree, and according to the generated class probability, the corresponding defect class is assigned to the labeled defect anchor box after screening.

2. The photovoltaic panel defect classification method based on feature pyramid and cascade group attention according to claim 1, characterized in that: The specific content of S1 is: S1.1: Obtain the infrared imaging picture shot by the unmanned aerial vehicle; S1.2: Analyze the internal circuit of the photovoltaic panel, and process the one-to-one correspondence relationship between fault types and manifestations; S1.3: Use a labeling tool to finely label the defect faults, including position information and classification groups; S1.4: Use the opencv library to perform rotation, brightness contrast adjustment, and scaling data enhancement operations on the photovoltaic panel picture to generate more samples for subsequent model training.

3. The photovoltaic panel defect classification method based on feature pyramid and cascade group attention of claim 1, characterized in that: The specific content of S3 is: S3.1: Specify the file directory, and use the batch photovoltaic panel infrared pictures in it as input; S3.2: Specify the configuration file, and load the model trained using the improved YOLOv11 algorithm; S3.3: Traverse each picture in the folder, use the model to perform photovoltaic panel defect classification and detection, and generate anchor boxes in the picture to locate the defect position and size, and each anchor box will be labeled with the class name and confidence size.

4. The photovoltaic panel defect classification method based on feature pyramid and cascade group attention of claim 1, characterized in that: The SCConv module in the SCC3k2 module is used for convolution to efficiently capture deep spatial and channel information; In the SRU module of the SCConv module, there are separation and reconstruction operations; In the separation operation of SRU, Group Normalization is used to extract the scaling factor of the feature map, and compared with the values in the variance, the spatial distribution of the effective information can be known, and according to a certain threshold, the information is divided into two parts, the information above the threshold is effective information, and the information below the threshold is redundant information, and the cross-reconstruction method is used to rearrange and integrate the features; The formula of the separation operation is as follows: W = T(Sigmoid(W γ(GN(X) )); The formula represents that in the separation operation, after GroupNormalization processing is performed on the input X, a scaling factor W is combined γ feature extraction, and then a Sigmoid activation function is applied to make the weight value smoother, and finally a T threshold gate is used to divide the features into rich features and redundant features, and the final feature map weight W is obtained. In the SRU module of SCC3k2, the threshold T is set to 0.4 to extract the photovoltaic panel defect features; The related formula of threshold allocation is as follows: Where W1 is the upper layer rich feature map, and W2 is the shallow layer redundant feature map.

5. The photovoltaic panel defect classification method based on feature pyramid and cascade group attention of claim 1, characterized in that: In the backbone network construction and feature extraction, the channel reconstruction unit CRU in the SCConv module is divided into three steps of segmentation, transformation and fusion. In the segmentation step, CRU first obtains the spatial fine-grained feature map obtained in the previous step, and divides the channel into two parts, containing αC channels and (1-α)C channels respectively, and then performs compression on the feature map through 1×1 convolution; In the transformation step, the upper half of the separated feature map is transformed through group-wise convolution (GWC) and point-wise convolution (PWC) to extract rich high-level features; The lower part is concatenated with the original shallow input by 1x1 convolution to supplement the shallow feature information; In the fusion part, the upper and lower part information is weighted and fused by global average pooling and β parameter related to soft attention mechanism to extract the final channel refined feature map.

6. The photovoltaic panel defect classification method based on feature pyramid and cascade group attention of claim 1, wherein: The step of applying a cascaded group attention mechanism (CGA) to enhance the feature representation, in the last C2WCGA module of the backbone network, adopts a CGA module, which performs different segmentation of complete features for each head in the multi-head attention mechanism; First, the input feature map is spatially segmented into multiple local windows, forming multiple sub-regions; The multi-head attention mechanism is applied in parallel on each window. First, the input feature map is processed by the QKV layer to calculate the query, key, and value. Then, it is divided into multiple sub-features, and each sub-task is processed. Each head models the features through self-attention mechanism, and an attention bias is added in the attention module. The specific formula is as follows: where d k is the dimension of the key vector, used to scale the dot product to avoid numerical overflow, B is the bias term, used to adjust the similarity of the query Q and the key K, the softmax operation converts the similarity into weights, and weights the value vector V through these weights, and finally outputs the weighted value vector V. In the code implementation, a feat feature map array is introduced to record the multi-head output results obtained by parallel calculation. The output results of each head are sequentially concatenated to the input of the next head for multiple feature fusion and information transmission. The specific concatenation formula is as follows: wherein the actual input feature in the first layer is the same as the original input feature, and the actual input feature f i needs to be obtained by superimposing the current original input x i and the output of the previous layer .

Images