Photovoltaic cell defect detection method fusing multi-scale features and re-parameterization strategy

Abstract

The invention relates to the technical field of deep learning, in particular to a photovoltaic cell defect detection method fusing multi-scale features and a re-parameterization strategy, and the method comprises the steps: obtaining a to-be-detected image; the image is input into a defect detection model, in the defect detection model, the backbone network is used for extracting a feature map of the image, and a first C3K2MSDA module is used for carrying out multi-scale cavity sliding window self-attention aggregation on the feature map to output a multi-scale feature map; the neck network is used for performing up-sampling on the multi-scale feature map and splicing the up-sampled feature map with the feature map from the backbone network to generate a fusion feature map, and performing multi-scale feature aggregation again by using a second C3K2MSDA module and an EMA-AFF module and generating a cross-scale fusion feature map; a DEC-Head module in the head network performs detection processing on the cross-scale fusion feature map, and outputs a defect category probability of each candidate region and a corresponding bounding box position as a detection result; and designing a loss function and optimizing the defect detection model by using the loss function. According to the method, the model defect detection performance can be improved.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a method for detecting defects in photovoltaic cells that integrates multi-scale features and reparameterization strategies. Background Technology

[0002] With the rapid development of renewable energy, photovoltaic (PV) power generation has become an important direction for global energy transition due to its advantages such as being clean, renewable, and widely distributed. As the core of a photovoltaic system, the reliability of PV modules in manufacturing and operation directly determines the overall performance of the system. However, during production, transportation, and use, PV cells often exhibit various defects such as cracks, broken grids, and black cores. These defects not only reduce the photoelectric conversion efficiency of the modules but may also lead to performance degradation or safety hazards. Therefore, achieving efficient and accurate detection of PV cell defects has become a crucial aspect of ensuring the quality and reliability of PV modules. Currently, deep learning methods, by constructing advanced network structures, can achieve higher-precision target detection, and thus have received widespread attention in the field of PV cell defect detection. Existing deep learning detection algorithms are mainly divided into two categories: two-stage and single-stage. Two-stage algorithms (such as Faster R-CNN and Cascade R-CNN) generate candidate regions first and then classify and regress them, which has high detection accuracy, but the computational cost is large and it is difficult to meet the real-time requirements of actual detection. Single-stage algorithms (such as SSD and YOLO series) complete classification and regression simultaneously through an end-to-end network structure, which has faster inference speed and stronger parallel processing capabilities, and therefore has higher practical value in photovoltaic cell defect detection scenarios.

[0003] Among them, the YOLO series, as a representative of single-stage object detection algorithms, has consistently maintained a leading advantage in real-time detection scenarios despite multiple version iterations. For example, some studies have enhanced the representation of key information by introducing dynamic snake convolution and channel attention mechanisms; others have used deformable convolution to improve the feature extraction capability of irregular defects to adapt to diverse target shapes; some works have optimized the feature fusion structure through multi-scale pooling pyramids or cross-layer feature connections, thereby improving the model's detection performance for multi-scale defects; in addition, some studies have reduced model complexity and improved detection speed by using lightweight convolution, partial convolution, or quantization modules. Although the above methods have made some progress in feature extraction and structure optimization, existing methods still have limitations in complex detection scenarios: first, insufficient multi-scale feature extraction capability, resulting in inadequate representation of defects at different scales; second, the feature fusion mechanism is difficult to balance shallow localization information and deep semantic information, causing small target features to be easily weakened during the fusion process; third, the detection head design is complex and has a large number of parameters, affecting training efficiency and model convergence stability. In summary, the main technical problem of existing photovoltaic cell defect detection methods is that the defect detection model has insufficient ability to represent small target defects during multi-scale feature extraction and fusion, making it difficult to balance detection accuracy and stability. Summary of the Invention

[0004] This application provides a photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies, which can improve the performance of model defect detection. The technical solution provided in this application is as follows: In a first aspect, this application provides a photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies, the method comprising: Acquire images of the photovoltaic cells to be detected and preprocess the photovoltaic cell images; The preprocessed photovoltaic cell image is input into a pre-defined defect detection model, which includes a backbone network, a neck network, and a head network connected in sequence. The backbone network extracts feature maps from the image and uses the built-in first C3K2_MSDA module to perform multi-scale hollow sliding window self-attention aggregation on the feature maps, adaptively fusing local details with the global context of a larger receptive field, and outputting multi-scale feature maps. The neck network upsamples the multi-scale feature maps and concatenates the upsampled deep feature maps with the shallow feature maps from the backbone network along the channel dimension to generate a fused feature map. The built-in second C3K2_MSDA module and EMA-AFF module are used to perform multi-scale feature aggregation again and generate a cross-scale fused feature map. The built-in DEC-Head module in the head network performs feature enhancement and target detection processing on the cross-scale fused feature map, and outputs the defect category probability and the corresponding bounding box position of each candidate region as the detection result. Design a loss function and use the loss function to optimize the defect detection model.

[0005] In one specific implementation scheme, the backbone network includes: The low-level features of the image are extracted through several layers of convolution, normalization and activation functions, resulting in feature maps of several scales that include shallow detail features and deeper semantic features. The first C3K2_MSDA module is used to perform multi-scale holed sliding window self-attention aggregation on the feature map, adaptively fusing local details with the global context of a larger receptive field, and outputting a multi-scale feature map.

[0006] In one specific implementation, the first C3K2_MSDA module includes: The query vector is obtained by linear mapping the feature map. Key-value vectors Value vector The feature map is divided into different heads along the channel direction, for the first... Each head with an expansion rate Perform a self-attention operation on a sliding window hole, the first The input features of each head are as follows: Its output is denoted as: ; Search for location within the sliding window. Centered on the query location, the set of keys and values ​​within the window is sampled using a hole-based approach, and its attention output is applied to the query location. The calculation is as follows: ; in, and This represents the set of keys and values ​​sampled within the sliding window. This indicates the transpose operation. It is the feature vector of the query location. It is the dimension of the key vector. and These are the height and width of the feature map, respectively. After all heads have completed the sliding window hole self-attention calculation, the output features of each head are... By concatenating information along the channel dimension and fusing information from different scales through a linear transformation layer, the aggregated output features are obtained as follows: ; in, The output for each head, This represents the number of sampling points within the window. This indicates a splicing operation in the channel direction. The final output is an enhanced feature map that integrates multi-scale semantic and structural information from different receptive fields. This indicates a subsequent linear transformation.

[0007] In a specific implementation, the step of upsampling the multi-scale feature map and concatenating the upsampled deep feature map with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map, and then using the built-in second C3K2_MSDA module and EMA-AFF module to perform multi-scale feature aggregation again and generate a cross-scale fused feature map includes: The neck network upsamples the multi-scale feature maps to improve the spatial resolution of the deep feature maps, making their size match the shallow feature maps output by the backbone network, and generating high-resolution deep feature maps with uniform size. The upsampled high-resolution deep feature map is concatenated with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map. The second C3K2_MSDA module captures local and global contextual information of different receptive fields through multi-scale dilated convolution, and combines a multi-head self-attention mechanism to adaptively assign weights to different features in the channel and spatial dimensions, thereby achieving cross-scale feature enhancement and aggregation, and generating an enhanced multi-scale feature map after processing the fused feature map.

[0008] In one specific implementation, the step of upsampling the multi-scale feature map and concatenating the upsampled deep feature map with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map, and then using the built-in second C3K2_MSDA module and EMA-AFF module to perform multi-scale feature aggregation again and generate a cross-scale fused feature map, further includes: The EMA-AFF module divides the enhanced multi-scale feature map into G sub-features along the channel direction, and processes the multi-scale features separately using a three-branch parallel structure; in the feature encoding stage, two Convolution extracts global spatial features of different dimensions through horizontal and vertical global average pooling. Convolution is used to capture local cross-channel interaction information, simultaneously obtaining local details and global semantic representations; in the feature aggregation stage, ... The feature vectors of the branches are subjected to two-dimensional global average pooling, and the processed features are then multiplied by the global spatial features through a reshaping operation. After processing by the EMA-AFF module, shallow features with deep features Obtain the initial attention weights across the channel-space. The channel mixing strategy is adopted to With original features To arrange alternately, and through The convolution process is performed on the groups, and then activated by the Sigmoid function. Generate refined feature weights as follows: ; in, This indicates a channel splicing operation. GC represents channel shuffling operation, and GC represents group convolution operation. This represents the Sigmoid activation function. This represents the feature map output by the second C3K2_MSDA module; The original features are weighted and summed using refined weights, and then... Convolution completes feature fusion, generating the final cross-scale fused feature map. : .

[0009] In one specific implementation scheme, the DEC-Head module built into the head network performs feature enhancement and target detection processing on the cross-scale fused feature map, and outputs the defect category probability and corresponding bounding box position of each candidate region as the detection result, including: The DEC-Head module will display the feature map. The number of channels in the feature map is compressed by using a channel dimensionality reduction layer, while the mean and variance within each channel are calculated to suppress the risk of overfitting and stabilize the training process. The dimensionality-reduced features are input into two 3×3 DEConv convolutions with shared weights for processing, fusing multi-scale contextual information and detail information to capture image edge, texture, and small object features with defects; After processing, multiple convolutional branches are integrated into a single equivalent convolution using reparameterization technology. After processing by classification and regression branches, the probability of defect category and bounding box of each candidate region are predicted. At the same time, the output scale is dynamically adjusted through the scale layer to output the detection results.

[0010] In one specific implementation scheme, designing the loss function and optimizing the defect detection model using the loss function includes: The MPDIOU loss function is introduced, and its calculation formula is as follows: ; ; The Euclidean distance between the diagonals of the predicted bounding box and the ground truth bounding box is used as... and This indicates the height and width of the two frames. and The intersection ratio between the two frames is indicated by... express, To improve the location-sensitive cross-union ratio; Based on MPDIOU, Inner-IOU and Wise-IoUv3 strategies are introduced, and the Inner-Wise-MPDIOU loss function is formed by combining the above strategies, which is defined as follows: ; ; ; ; The coordinates of the four points of the ground truth bounding box are used. , , , This indicates that the coordinates of the four points of the prediction box are used. , , , This indicates that the width and height of the predicted bounding box and the ground truth bounding box are respectively... , , , This indicates that the scaling factor for the prediction box uses... express, Used to dynamically adjust the contribution of low-quality samples to the gradient. This represents the area of ​​the intersection region between the predicted bounding box and the ground truth bounding box. This is the total area after weighting the predicted bounding box and the ground truth bounding box. This is the internal intersection-union ratio between the predicted bounding box and the ground truth bounding box.

[0011] In one specific implementation scheme, the design of the loss function and the optimization of the defect detection model using the loss function further includes: During model training, the predicted bounding boxes and the ground truth bounding boxes are input into the Inner-Wise-MPDIOU loss function to calculate the loss value. The gradient of the loss is then passed to the model parameters through backpropagation to update the model weights in order to minimize the loss function.

[0012] In a second aspect, this application provides an electronic device, the device including a processor and a memory; the memory stores a program, the program being loaded and executed by the processor to implement a photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies as described in the first aspect.

[0013] Thirdly, this application provides a computer-readable storage medium storing a program that, when executed by a processor, is used to implement a photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies as described in the first aspect.

[0014] By introducing the multi-scale dilated self-attention module C3K2_MSDA and the cross-layer multi-scale adaptive feature fusion module EMA-AFF into the backbone and neck networks, the interaction and fusion capabilities between features of different scales are enhanced. This allows the model to retain more detailed features while extracting high-level semantic information, thereby improving the representation accuracy of small target defects. Simultaneously, a lightweight module DEC-Head with a fusion reparameterization strategy is introduced into the detection head. Detail-enhancing convolutions improve the feature response intensity, further enhancing the model's ability to locate and identify minute defects. Through this structural synergistic optimization, the problem of insufficient representation capability of small target defects in existing photovoltaic cell defect detection methods during multi-scale feature extraction and fusion is effectively solved, achieving a balance between detection accuracy and stability.

[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0016] Figure 1 This is a model structure diagram of YOLOv11n in the embodiments of this application.

[0017] Figure 2 This is a flowchart illustrating the photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies in the embodiments of this application.

[0018] Figure 3 This is a model structure diagram of the photovoltaic cell defect detection model in the embodiments of this application.

[0019] Figure 4 This is a schematic diagram of the MSDA network structure in an embodiment of this application.

[0020] Figure 5 This is a schematic diagram of the structure of the C3K2_MSDA module in an embodiment of this application.

[0021] Figure 6 This is a schematic diagram of the EMA-AFF module in an embodiment of this application.

[0022] Figure 7 This is a schematic diagram of the DEC-Head module in an embodiment of this application.

[0023] Figure 8This is a sample example diagram of the photovoltaic cell defect dataset in the embodiments of this application.

[0024] Figure 9 This is a thermal comparison diagram of the defect detection model in this application embodiment after various improvements have been added sequentially.

[0025] Figure 10 This is a comparison chart of the detection accuracy of various defects before and after the improvement of the defect detection model in the embodiments of this application.

[0026] Figure 11 This is a block diagram of an electronic device for photovoltaic cell defect detection that integrates multi-scale features and reparameterization strategies, as described in this application. Detailed Implementation

[0027] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0028] Optionally, this application uses the photovoltaic cell defect detection method integrating multi-scale features and reparameterization strategy provided in various embodiments as an example for application in electronic devices. The electronic device is a terminal or server. The terminal can be a computer, tablet computer, etc. This embodiment does not limit the type of electronic device.

[0029] First, the model structure of the defect detection model in this application is YOLOv11, referring to... Figure 1 This is a diagram of the YOLOv11 model architecture. YOLOv11, released in 2024, is a recent research achievement that improves model performance through several enhancements. First, the C3k2 module replaces the C2F module, combining variable convolution kernels and channel separation strategies to extract multi-scale features. Second, a C2PSA module is added after SPPF to weight the feature maps in both channel and spatial dimensions, enabling the model to automatically focus on key regions in the image and filter out irrelevant details. Furthermore, the YOLOv11 detection head integrates lightweight depthwise separable convolution (DWConv) and anchor-free mechanisms. The former decomposes standard convolution into depthwise convolution and pointwise convolution, while the latter directly predicts the coordinates and class probabilities of bounding boxes at each pixel. These structural optimizations not only maintain YOLOv11's excellent detection accuracy but also significantly improve computational efficiency. In the YOLOv11 series, YOLOv11n is a lightweight model designed for small and lightweight tasks, aiming to meet the needs of edge computing devices or low-power scenarios. This application adopts this model for improvement.

[0030] Reference Figure 2This is a flowchart illustrating a photovoltaic cell defect detection method integrating multi-scale features and reparameterization strategies provided in an embodiment of this application. The method includes at least the following steps: Step S101: Obtain the image of the photovoltaic cell to be detected and preprocess the photovoltaic cell image.

[0031] In step S101, an image of the photovoltaic cell to be inspected is acquired and preprocessed. Specifically, a high-resolution image of the photovoltaic cell can be acquired using an industrial camera or a production line acquisition system to ensure that the image completely covers the effective inspection area of ​​the cell. Subsequently, the acquired image is preprocessed, including size normalization, brightness equalization, contrast enhancement, and noise suppression, to eliminate the influence of different acquisition conditions, improve image quality and feature stability, and thus provide standardized, clear, and consistent input data for subsequent defect detection.

[0032] Step S102: Input the preprocessed photovoltaic cell image into a preset defect detection model. The defect detection model includes a backbone network, a neck network, and a head network connected in sequence. The backbone network is used to extract the feature map of the image and use the built-in first C3K2_MSDA module to perform multi-scale hole sliding window self-attention aggregation on the feature map, adaptively fusing local details with the global context of a larger receptive field, and outputting a multi-scale feature map. The neck network is used to upsample the multi-scale feature map and splice the upsampled deep feature map with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map. The built-in second C3K2_MSDA module and EMA-AFF module are used to perform multi-scale feature aggregation again and generate a cross-scale fused feature map. The built-in DEC-Head module in the head network performs feature enhancement and target detection processing on the cross-scale fused feature map, and outputs the defect category probability and the corresponding bounding box position of each candidate region as the detection result.

[0033] In step S102, refer to Figure 3The defect detection model comprises a backbone network, a neck network, and a head network connected in sequence. After the preprocessed photovoltaic cell image is input into the defect detection model, it first passes through a series of standard convolutional layers to extract low-level visual features. Then, it enters the first C3K2_MSDA module of the backbone network for multi-scale feature extraction and attention enhancement. Further processing by SPPF (Fast Spatial Pyramid Pooling) and C2PSA (Channel Spatial Attention) modules serves as the input to the neck network. The neck network aggregates features at different scales through upsampling and concatenation, then reconstructs the multi-scale context through the second C3K2_MSDA module. Finally, it enters the EMA-AFF (Multi-Layer Multi-Scale Adaptive Feature Fusion) module for adaptive fusion, outputting a fused feature map for the detection head.

[0034] In photovoltaic cell defect detection, accurately identifying targets at different scales and understanding their context is crucial. YOLOv11 uses dilated convolutions to capture features at different scales, enhancing the model's receptive field and multi-scale processing capabilities. However, due to the lack of attention mechanisms, the network cannot effectively focus on important feature regions in the image, resulting in insufficient feature extraction. The MSDA module uses multi-scale dilated convolutions to effectively fuse features at different scales and performs self-attention within a sliding window to focus global and local contextual information, thereby improving the model's understanding and representation of key features at different scales. The MSDA network structure is as follows: Figure 4 As shown.

[0035] In implementation, the backbone network first extracts the low-level features of the image through several layers of convolution, normalization and activation functions, generating intermediate feature maps of several scales that include shallow detail features and deeper semantic features. Then, the first C3K2_MSDA module in the backbone network is used to perform multi-scale dilated sliding window self-attention aggregation on the intermediate feature maps, adaptively fusing local details with the global context of a larger receptive field, and outputting enhanced multi-scale feature maps.

[0036] Specifically, the query vector is obtained by linear mapping the feature map. Key-value vectors Value vector Then, the feature map is divided into different heads along the channel direction, for the first... Each head with an expansion rate Perform a self-attention operation on a sliding window hole, the first The input features of each head are as follows: Its output is denoted as: ; in, For the feature map of the th Each slice, the size of the convolution kernel in this application is [size missing]. Take the expansion rate The set of receptive fields in different heads, therefore, are respectively , , This allows for the modeling of dependencies between local and contextual features at different scales. Subsequently, the query location is used within a sliding window. Centered on this, the set of keys and values ​​within the window is sampled using a hole-based approach. For this query location, the attention output... The calculation is as follows: ; in, and This represents the set of keys and values ​​sampled within the sliding window. This indicates the transpose operation. It is the feature vector of the query location. It is the dimension of the key vector. and These represent the height and width of the feature map, respectively. This calculation process essentially involves weighted aggregation of features sampled within the sliding window, thereby enabling information exchange within the local neighborhood. Simultaneously, a diffusive sampling mechanism expands the receptive field to incorporate global contextual information.

[0037] After all heads have completed the sliding window hole self-attention calculation, the output features of each head are... By concatenating information along the channel dimension and fusing information from different scales through a linear transformation layer, the aggregated output features are obtained as follows: ; in, The output for each head, This represents the number of sampling points within the window. This indicates a splicing operation in the channel direction. The final output is an enhanced feature map that integrates multi-scale semantic and structural information from different receptive fields. This indicates a subsequent linear transformation (channel-wise linear mapping).

[0038] In this module, attention output Represents the position of each pixel in the feature map The weighted fusion result obtained through a self-attention mechanism within the sliding window is the information aggregation result within the neighborhood of that location. The model calculates the corresponding [data] for each location in the feature map. This results in a new feature map, which is the output of the attention head. Multiple attention heads perform the same calculations at different dilation rates, resulting in multiple output feature maps. These feature maps are then concatenated along the channel direction and subjected to a linear transformation to obtain the final output. .

[0039] Furthermore, preferably, the first C3K2_MSDA module outputs... Subsequently, fast spatial pyramid pooling is used to perform multi-scale pooling on the feature map to expand the receptive field and aggregate contextual information at different scales, thereby enhancing the ability to represent the context of defects of different sizes. Then, weights are assigned to the feature map in the channel dimension and spatial dimension. Through the joint operation of channel attention and spatial attention, the model automatically focuses on key regions in the image and suppresses irrelevant background details, thereby improving the signal-to-noise ratio of the backbone output features.

[0040] Backbone network outputs multi-scale feature maps Next, the neck network first performs upsampling, which increases the spatial resolution of the deep feature map to match its size with the shallow feature map output by the backbone network, thus generating a high-resolution deep feature map with uniform size. Then, the neck network concatenates the upsampled high-resolution deep feature map with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map. This fused feature map contains both shallow detail information and deep semantic information, giving the model rich representation capabilities in both spatial detail and global semantics. Subsequently, the fused feature map enters the second C3K2_MSDA module. This module captures local and global contextual information from different receptive fields through multi-scale dilated convolutions and, combined with a multi-head self-attention mechanism, adaptively assigns weights to different features along the channel and spatial dimensions, achieving cross-scale feature enhancement and aggregation. After processing by this module, the generated output feature map not only aggregates multi-scale semantic information but also retains shallow details and deep context, significantly improving the model's ability to perceive small target defects in photovoltaic cells. Similar to the first C3K2_MSDA module in the backbone network, the second C3K2_MSDA module not only aggregates multi-scale semantic information, but also combines the spatial details of shallow and deep features, further enhancing the model's ability to perceive small target defects in photovoltaic cells.

[0041] Reference Figure 5 The diagram shows the structure of the C3K2_MSDA module. This application introduces MSDA into C3K2 to form a novel multi-scale feature extraction module, C3K2_MSDA. Through multi-scale dilated convolution and multi-head self-attention, it can adaptively aggregate multi-scale features, enhance the feature extraction capability of the model, and improve detection accuracy and network robustness.

[0042] In implementation, the multi-scale feature map of the neck network, enhanced by the second C3K2_MSDA module, is input into the EMA-AFF (Multi-Scale Adaptive Feature Fusion) module. This module further fuses shallow and deep features, adaptively enhances key feature representations, mitigates information loss due to small target defects during feature fusion, and suppresses redundant information, thereby generating a high-quality multi-scale fused feature map. A schematic diagram of the EMA-AFF structure is shown below. Figure 6 As shown.

[0043] Specifically, the EMA-AFF module first divides the input feature map into G sub-features along the channel direction, and then uses a three-branch parallel structure to process the multi-scale features separately. In the feature encoding stage, two... Convolution extracts global spatial features of different dimensions through horizontal and vertical global average pooling. Convolution is used to capture local cross-channel interaction information, thereby simultaneously obtaining local details and global semantic representations. During the feature aggregation stage, [the following is applied]... The feature vectors of the branches are subjected to two-dimensional global average pooling, and the processed features are then multiplied by the global spatial features through a reshaping operation to enhance the contextual relevance of the feature maps in spatial location.

[0044] After EMA processing, shallow features with deep features Obtain the initial attention weights across the channel-space. This weight is used to reflect the importance of different channels and spatial locations. To further enhance the focus on key features, the EMA-AFF module employs a channel shuffle strategy, which... With original features To arrange alternately, and through Group convolution (GC) is used for convolution processing, followed by the Sigmoid activation function. Generate refined feature weights as follows: ; in, This indicates a channel splicing operation. GC represents channel shuffling operation, and GC represents group convolution operation. This represents the Sigmoid activation function. This represents the feature map output by the second C3K2_MSDA module.

[0045] Finally, the EMA-AFF module uses the refined weights to perform a weighted sum of the original features, and then... Convolution completes feature fusion, generating the final cross-scale fused feature map. : ; Through the above processing, the EMA-AFF module not only enhances the expressive power of key features, but also fully integrates information from different channels, improves the cross-channel information interaction capability, and strengthens the representation capability of spatial details and semantic information, thereby significantly improving the model's performance in small target defect detection.

[0046] Cross-scale fused feature maps output by the EMA-AFF module of the neck network The DEC-Head module, also known as the DEC-Head detection head, is fed into the head network. This module further enhances the features of the fused feature map and performs object detection processing, outputting the defect category probability and corresponding bounding box position for each candidate region, thereby achieving accurate localization and classification of photovoltaic cell defects. Specifically, DEC-Head utilizes Enhanced Details Convolution (DEConv) to integrate prior information and local gradient information, improving the perception of details such as defect edges and textures. Simultaneously, it reduces the model's computational cost and parameter complexity through a reparameterization strategy, ultimately outputting detection results for localization and classification.

[0047] The principle of DEConv is as follows: Figure 6 As shown, prior information is first encoded into the convolutional layer using center difference convolution (CDC) and angle difference convolution (ADC) to enhance the ability to perceive details of the target. Then, to strengthen edge representation, gradient information is extracted using horizontal difference convolution (HDC) and vertical difference convolution (VDC). Finally, the outputs of all branches are element-wise added and fused with the features after standard convolution.

[0048] Specifically, refer to Figure 7 This is a schematic diagram of the DEC-Head module. The DEC-Head module first processes the input feature map... The feature map's channel count is compressed using a channel reduction layer (1×1 grouped normalized convolution GN_Conv), while simultaneously calculating the mean and variance within each channel to suppress overfitting and stabilize the training process. Next, the dimensionality-reduced features are fed into two weighted 3×3 DEConv convolutions for processing. This fuses multi-scale contextual and detail information, capturing image edges, textures, and small defect features. After processing, reparameterization integrates multiple convolutional branches into a single equivalent convolution, preserving enhanced feature representation capabilities while reducing computational complexity. Finally, classification (Cls_Conv) and regression (Reg_Conv) branches predict the defect category probability and bounding box for each candidate region. A scale layer dynamically adjusts the output scale to accommodate defect targets of different sizes, ensuring the output detection results can be directly used for defect localization and classification.

[0049] In the design of the aforementioned defect detection model, by introducing a first C3K2_MSDA module and a second C3K2_MSDA module into the backbone and neck networks, the model can capture local and global information from different receptive fields using multi-scale dilated convolutions and adaptively aggregate these multi-scale features through a multi-head self-attention mechanism. This design significantly enhances the model's ability to understand cross-scale features, enabling more accurate extraction of key features when processing images containing both small target defects and larger structural features, improving the richness and accuracy of feature representation, and thus enhancing overall detection performance. The introduction of an EMA-AFF module into the neck network effectively integrates the detailed information of shallow features with the semantic information of deep features through a cross-layer multi-scale adaptive feature fusion strategy. In this process, the EMA-AFF module can suppress redundant information while preserving key detailed features, especially demonstrating good perception capabilities for densely distributed, small-sized defects such as broken grids and scratches. This not only alleviates the problem of small targets being easily lost during feature fusion but also reduces the model's missed detections and enhances its robustness in complex backgrounds. In the head network, the DEC-Head detection head is adopted. A lightweight detection head is constructed through a fusion reparameterization strategy, and detail enhancement convolution (DEConv) is combined to process the feature map, further improving the ability to capture details such as edges and textures in photovoltaic cell defect images. The DEC-Head module can achieve accurate localization and classification of defect targets while maintaining low model computational cost, improving localization accuracy and detection reliability. This design enables the entire model to quickly and accurately complete photovoltaic cell defect detection tasks in real-world production environments while maintaining lightweight design and high efficiency.

[0050] Step S103: Design the loss function and use the loss function to optimize the defect detection model.

[0051] In step S103, to optimize model performance and improve prediction accuracy, it is necessary to measure the difference between the output of the detection model and the true annotation, and use this to guide the update of model parameters. Therefore, to address the shortcomings of the YOLOv11n model in photovoltaic cell defect scenarios, an improved loss function, Inner-Wise-MPDIOU, is designed to more accurately optimize the model bounding box regression and improve the accuracy of defect detection.

[0052] Specifically, the YOLOv11n model uses the traditional CIOU loss function, which only considers the overlap between the predicted and ground truth bounding boxes and the distance between their center points, but does not take into account the relative position and shape differences of defects in the image. When the predicted bounding box does not match the actual defect shape, model optimization is limited, and detection accuracy decreases. To address this issue, this paper first introduces the MPDIOU loss function, whose calculation formula is as follows: ; ; The Euclidean distance between the diagonals of the predicted bounding box and the ground truth bounding box is used as... and This indicates the height and width of the two frames. and The intersection ratio between the two frames is indicated by... express, To improve the location-sensitive cross-union ratio (CUI). As shown in the formula, the MPDIOU loss function not only considers the overlap between the predicted and ground truth boxes as an important factor, but also takes into account their relative position and shape. This helps to more accurately describe the relative position of photovoltaic cell defects in the image, thereby improving the accuracy of defect detection.

[0053] To address the potential presence of low-quality bounding boxes (such as blurry or misaligned boxes) in photovoltaic cell defect datasets, this paper further introduces Inner-IOU and Wise-IoUv3 strategies on top of MPDIOU. Inner-IOU dynamically adjusts the scaling ratio based on the target size, accelerating bounding box regression; Wise-IoUv3 provides a gradient gain allocation strategy to suppress the negative impact of low-quality samples on the gradient, enabling the model to focus more on ordinary anchor boxes, thereby improving the overall detection performance.

[0054] Finally, the Inner-Wise-MPDIOU loss function, formed by combining the above strategies, is defined as follows: ; ; ; ; The coordinates of the four points of the ground truth bounding box are used. , , , This indicates that the coordinates of the four points of the prediction box are used. , , , This indicates that the width and height of the predicted bounding box and the ground truth bounding box are respectively... , , , This indicates that the scaling factor for the prediction box uses... express, Used to dynamically adjust the contribution of low-quality samples to the gradient. This represents the area of ​​the intersection region between the predicted bounding box and the ground truth bounding box. This is the total area after weighting the predicted bounding box and the ground truth bounding box. This is the internal intersection-union ratio between the predicted bounding box and the ground truth bounding box.

[0055] During model training, predicted bounding boxes and ground truth bounding boxes are input into the Inner-Wise-MPDIOU loss function. The loss value is calculated, and the gradient of the loss is passed to the model parameters through backpropagation to update the model weights to minimize the loss. This method allows the model to consider the overlap, position, shape, and low-quality sample influence of defect bounding boxes, achieving high-precision localization and classification of photovoltaic cell defects. By drawing inspiration from Inner-IOU and WIOUv3, MPDIOU is optimized into Inner-Wise-MPDIOU and combined with a dynamic gradient weight adjustment mechanism. This loss function effectively suppresses the dominant effect of low-quality samples (such as ambiguous labeling or localization bias) on gradient updates during training, reducing optimization bias in model training. This allows the model to focus more on ordinary anchor boxes and high-quality samples, improving the accuracy of bounding box regression. Furthermore, this design accelerates model convergence, improves training stability, and enhances the optimization capability for small targets and detailed features while maintaining global matching constraints, thus comprehensively improving the accuracy and robustness of photovoltaic cell defect detection.

[0056] In summary, to address the shortcomings of existing photovoltaic cell detection methods, such as insufficient multi-scale feature capture capabilities and the tendency for small-target defects to go undetected, this application proposes a detection algorithm that integrates multi-scale features with a reparameterization strategy. First, the C3K2_MSDA module is introduced into the backbone and neck network of YOLOv11n, combining multi-scale dilated convolution and multi-head self-attention mechanisms to improve the model's understanding of multi-scale features. Second, a cross-layer multi-scale adaptive feature fusion module, EMA-AFF, is proposed to mitigate the loss of small targets during feature fusion and reduce missed detections. Next, a reparameterization strategy is used to construct a lightweight detection head, DEC-Head, which, combined with detail enhancement convolution, enhances the ability to capture image details. Finally, drawing inspiration from Inner-IOU and WIOUv3, MPDIOU is optimized to Inner-Wise-MPDIOU to suppress the dominance of low-quality samples, alleviate optimization bias during model training, and improve model convergence.

[0057] In addition, to verify the performance of the model in this application, a technical effect verification experiment was conducted. This application selected the PVEL-AD photovoltaic cell defect dataset to verify the model's effectiveness. This dataset has large differences in defect scale, dense small targets, and numerous defect types, containing 36,543 images of various defects across 12 categories. This application filtered the images in the original dataset, selecting six different categories of defect images for training and verification: linear cracks (line_crack), star cracks (star_crack), scratches (scratch), black cores (black_core), thick lines (thick_line), and broken grids (finger). Relevant photovoltaic cell defect samples are shown below. Figure 8 As shown.

[0058] Among them, star-shaped cracks and scratches have relatively few samples, which will affect the final detection accuracy of the model. To address this, this paper uses enhancement methods such as geometric transformation, brightness and contrast adjustment, and random noise to expand the dataset, ultimately obtaining 13,809 images. The number of images in the training set, validation set, and test set are in a ratio of 7:2:1, respectively. The number of each defect category is shown in Table 1.

[0059] To evaluate the overall performance of the model, precision (P), recall (R), and mean average precision (mAP) were selected as evaluation metrics. Precision measures the classifier's ability to identify positive samples; recall assesses the classifier's coverage of positive examples; and mAP represents the average precision at different IOU values, used to measure the performance of the object detection algorithm. This paper selects the average precision value when IOU >= 0.5. The formulas for each evaluation metric are shown below: ; ; ; Where TP is the number of positive examples that are classified as positive; FP is the number of negative examples that are classified as positive; FN is the number of positive examples that are classified as negative; and N is the number of categories.

[0060] To verify the effectiveness of each improvement, an ablation strategy was adopted for comparative experiments. The gain performance of each improved module was analyzed by activating it one by one. The specific experimental results are shown in Table 2.

[0061] As shown in Table 2, all four improvements contributed to the improvement of the mAP50 index. First, adding the C3K2_MSDA module to YOLOv11n improved detection accuracy by 1.2%, indicating that the multi-scale dilated convolution and multi-head self-attention of the C3K2_MSDA module can adaptively aggregate features at different scales, better improving the understanding of defects at different abstract levels in photovoltaic cells. Experiment 2, building on Experiment 1, incorporated the EMA-AFF module to further optimize the feature extraction structure. A channel mixing strategy was used to fuse shallow and deep features, achieving synergistic enhancement of spatial details and semantic information, further improving detection accuracy by 0.5%. Experiment 3 combined C3K2_MSDA and DE... The Conv-Head detection heads are combined using a reparameterization strategy to merge different detection branches, reducing the number of parameters and computational complexity by 0.34M and 0.5G, respectively. Simultaneously, detail-enhancing convolutions are introduced to improve the ability to capture image details, resulting in a 1.3% improvement in detection accuracy compared to the baseline model. Experiment 5 applies C3K2_MSDA, EMA-AFF, and Inner-Wise-MPDIOU to the baseline model, using dynamic gradient weight adjustment to optimize the loss function. By suppressing the dominance of low-quality samples, optimization bias during model training is mitigated, further improving the accuracy of photovoltaic cell defect detection. Experiment 7 shows that applying all four modules simultaneously to the baseline model yields the best results, improving detection accuracy by 4.7%, while only increasing the number of parameters and computational complexity by 0.19M and 0.5G, respectively, demonstrating that the proposed method can efficiently detect defects in photovoltaic cells.

[0062] like Figure 9The heatmap shows the model after each improvement is added sequentially. Integrating C3K2_MSDA enhances the model's understanding of multi-scale information through multi-scale dilated convolution and multi-head self-attention. The EMA-AFF module effectively suppresses redundant information through cross-layer multi-scale adaptive feature fusion. DEConv-Head enhances the ability to capture image details, further highlighting key features and improving the model's detection accuracy. To further analyze the improved model's detection performance for each defect, Figure 10 The detection accuracy of various defects before and after the model improvement was plotted. The figures show that the improved model improves the detection accuracy of linear cracks and star-shaped cracks on multi-scale targets by 9% and 11%, respectively, and improves the accuracy of scratches and broken grids on small-scale targets by 5% and 1%, respectively. In the detection of black cores in complex backgrounds, the improved model effectively suppresses false positives by enhancing background perception. In summary, the model presented in this paper has stronger feature perception capabilities, effectively overcomes false positives and false negatives for small targets, and improves the overall detection performance of the model.

[0063] Figure 11 This is a block diagram of an electronic device provided in one embodiment of this application. The device includes at least a processor 401 and a memory 402.

[0064] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0065] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 is used to store at least one instruction, which is executed by the processor 401 to implement the photovoltaic cell defect detection method integrating multi-scale features and reparameterization strategies provided in the method embodiments of this application.

[0066] Optionally, this application also provides a computer-readable storage medium storing a program that is loaded and executed by a processor to implement the photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies described in the above method embodiments.

[0067] Optionally, this application also provides a computer product including a computer-readable storage medium storing a program, which is loaded and executed by a processor to implement the photovoltaic cell defect detection method integrating multi-scale features and reparameterization strategy described in the above method embodiments.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] The above embodiments merely illustrate several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for photovoltaic cell defect detection fusing multi-scale features with reparameterization strategy, characterized in that, The method includes: Acquire images of the photovoltaic cells to be detected and preprocess the photovoltaic cell images; The preprocessed photovoltaic cell image is input into a pre-defined defect detection model, which includes a backbone network, a neck network, and a head network connected in sequence. The backbone network extracts feature maps from the image and uses the built-in first C3K2_MSDA module to perform multi-scale hollow sliding window self-attention aggregation on the feature maps, adaptively fusing local details with the global context of a larger receptive field, and outputting multi-scale feature maps. The neck network upsamples the multi-scale feature maps and concatenates the upsampled deep feature maps with the shallow feature maps from the backbone network along the channel dimension to generate a fused feature map. The built-in second C3K2_MSDA module and EMA-AFF module are used to perform multi-scale feature aggregation again and generate a cross-scale fused feature map. The built-in DEC-Head module in the head network performs feature enhancement and target detection processing on the cross-scale fused feature map, and outputs the defect category probability and the corresponding bounding box position of each candidate region as the detection result. Design a loss function and use the loss function to optimize the defect detection model.

2. The method of photovoltaic cell defects detection fusing multi-scale features with reparameterization strategy according to claim 1, characterized in that, The backbone network includes: The low-level features of the image are extracted through several layers of convolution, normalization and activation functions, resulting in feature maps of several scales that include shallow detail features and deeper semantic features. The first C3K2_MSDA module is used to perform multi-scale holed sliding window self-attention aggregation on the feature map, adaptively fusing local details with the global context of a larger receptive field, and outputting a multi-scale feature map.

3. The method of photovoltaic cell defects detection fusing multi-scale features with reparameterization strategy according to claim 2, characterized in that, The first C3K2_MSDA module includes: The feature map is mapped to a query vector through a linear mapping , a key-value vector , a value vector , the feature map is divided into different heads in the channel direction, and a sliding window dilated self-attention operation is performed on the first head at an expansion rate , and the input features of the first head are , and the output is denoted as ； Querying a position within a sliding window centered, the set of keys and values within the window are sampled in a hollow fashion, with attention output for the query position is computed as follows: ； wherein, and denote the set of keys and values sampled within the sliding window, denotes the transpose operation, is the feature vector of the query location, is the dimension of the key vector, and are the height and width of the feature map, respectively; After the sliding window hole self-attention calculation of all the heads is completed, the output features of the heads The information of different scales is fused via a linear transformation layer after concatenation in the channel dimension, and the aggregated output features are as follows: ； wherein, is the output for each head, is the number of sampling points within the window, denotes a concatenation operation in the channel direction, is the final output of enhanced feature maps that fuse multi-scale semantic and structural information under different receptive fields, denotes a subsequent linear transformation.

4. The method of photovoltaic cell defects detection fusing multi-scale features with reparameterization strategy according to claim 1, characterized in that, The process of upsampling the multi-scale feature map and concatenating the upsampled deep feature map with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map, and then using the built-in second C3K2_MSDA module and EMA-AFF module to perform multi-scale feature aggregation again and generate a cross-scale fused feature map includes: The neck network upsamples the multi-scale feature maps to improve the spatial resolution of the deep feature maps, making their size match the shallow feature maps output by the backbone network, and generating high-resolution deep feature maps with uniform size. The upsampled high-resolution deep feature map is concatenated with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map. The second C3K2_MSDA module captures local and global contextual information of different receptive fields through multi-scale dilated convolution, and combines a multi-head self-attention mechanism to adaptively assign weights to different features in the channel and spatial dimensions, thereby achieving cross-scale feature enhancement and aggregation, and generating an enhanced multi-scale feature map after processing the fused feature map.

5. The method of photovoltaic cell defects detection fusing multi-scale features with reparameterization strategy according to claim 4, characterized in that, The process of upsampling the multi-scale feature map and concatenating the upsampled deep feature map with the shallow feature map from the backbone network along the channel dimension to generate a fused feature map, and then using the built-in second C3K2_MSDA module and EMA-AFF module to perform multi-scale feature aggregation again and generate a cross-scale fused feature map, further includes: The EMA-AFF module divides the enhanced multi-scale feature map into G sub-features along the channel direction, and processes the multi-scale features respectively by using a three-branch parallel structure; in the feature encoding stage, two Convolution extracts global spatial features of different dimensions through horizontal and vertical global average pooling, Convolution is used to capture local cross-channel interaction information, while obtaining local details and global semantic representation; in the feature aggregation stage, the feature vectors of the three branches are processed by two-dimensional global average pooling, and the processed features are dot multiplied with the global spatial features through reshaping operation; Branch feature vectors are processed by two-dimensional global average pooling, and the processed features are dot multiplied with the global spatial features through reshaping operation; After processing by the EMA-AFF module, shallow features with deep features Obtain the initial attention weights across the channel-space. The channel mixing strategy is adopted to With original features To arrange alternately, and through The convolution process is performed on the groups, and then activated by the Sigmoid function. Generate refined feature weights as follows: ； wherein, denotes a channel concatenation operation, denotes a channel shuffle operation, GC denotes a group convolution operation, denotes a Sigmoid activation function, denotes the feature map output by the second C3K2_MSDA module; The original features are weighted and summed using refined weights, and the result is passed through Convolution is performed to fuse the features to generate the final cross-scale fused feature map : 。 6. The method of photovoltaic cell defects detection fusing multi-scale features with reparameterization strategy according to claim 5, characterized in that, The DEC-Head module built into the head network performs feature enhancement and target detection processing on the cross-scale fused feature map, and outputs the defect category probability and corresponding bounding box position of each candidate region as the detection result, including: The DEC-Head module compresses the channel number of the feature map The channel number of the feature map is compressed by the channel reduction layer, and the mean and variance in each channel are calculated to suppress the risk of overfitting and stabilize the training process. The dimensionality-reduced features are input into two 3×3 DEConv convolutions with shared weights for processing, fusing multi-scale contextual information and detail information to capture image edge, texture, and small object features with defects; After processing, multiple convolutional branches are integrated into a single equivalent convolution using reparameterization technology. After processing by classification and regression branches, the probability of defect category and bounding box of each candidate region are predicted. At the same time, the output scale is dynamically adjusted through the scale layer to output the detection results.

7. The method of photovoltaic cell defects detection fusing multi-scale features with reparameterization strategy according to claim 1, characterized in that, The design of the loss function and the optimization of the defect detection model using the loss function include: The MPDIOU loss function is introduced, and its calculation formula is as follows: ； ； where the Euclidean distance between the diagonal of the predicted and ground truth boxes is denoted by and denote the height and width of the two boxes and denote the intersection over union between the two boxes denote, improved position information sensitive intersection over union; Based on MPDIOU, Inner-IOU and Wise-IoUv3 strategies are introduced, and the Inner-Wise-MPDIOU loss function is formed by combining the above strategies, which is defined as follows: ； ； ； ； wherein the coordinates of the four points of the ground truth bounding box are denoted by , , , denote the coordinates of the four points of the predicted bounding box , , , denote the width and height of the predicted bounding box and the ground truth bounding box respectively , , , denote the scaling factor of the predicted bounding box denotes is used to dynamically adjust the contribution of low-quality samples to the gradient, is the area of the intersection region of the predicted bounding box and the ground truth bounding box, is the total area of the predicted bounding box and the ground truth bounding box after weighting, is the IoU of the predicted bounding box and the ground truth bounding box.

8. The photovoltaic cell defect detection method integrating multi-scale features and reparameterization strategy according to claim 7, characterized in that, The design of the loss function and the optimization of the defect detection model using the loss function also include: During model training, the predicted bounding boxes and the ground truth bounding boxes are input into the Inner-Wise-MPDIOU loss function to calculate the loss value. The gradient of the loss is then passed to the model parameters through backpropagation to update the model weights in order to minimize the loss function.

9. An electronic device, characterized in that, The device includes a processor and a memory; the memory stores a program, which is loaded and executed by the processor to implement a photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a program that, when executed by a processor, is used to implement a photovoltaic cell defect detection method that integrates multi-scale features and reparameterization strategies as described in any one of claims 1 to 8.

Images