Solar cell panel defect detection method based on improved RT-DETR

Abstract

The invention relates to the technical field of solar cell panel defect detection, in particular to a solar cell panel defect detection method based on improved RT-DETR, comprising the following steps: preprocessing a collected solar cell panel defect image to obtain a data set for model training; according to the method, an original RT-DETR model is improved, an improved PABW-RTDETR model is obtained, and a backbone network module of the improved PABW-RTDETR model is provided with a C2f-AP module; inputting the preprocessed data set into the improved model, and carrying out multi-round iterative training on the improved PABW-RTDETR model; and detecting the surface of the solar cell panel by using the trained PABW-RTDETR model to obtain a defect type. According to the invention, the PABW-RTDETR model is provided, so that collaborative optimization of the defect detection precision and efficiency of the solar cell panel is realized; the C2f-AP backbone network enhances the multi-scale feature expression while reducing the calculation cost; the ASSA and the BIMAFPN are combined for use, so that the focusing capability of the model on key defects is effectively improved; and the dynamic loss function design improves the small target positioning precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of solar panel defect detection, and in particular relates to a solar panel defect detection method based on improved RT-DETR. BACKGROUND

[0002] In the production and transportation process of solar panels, various defects are inevitably caused due to the complexity of the process, external factors and the like. These defects may cause the power generation efficiency of the solar panel to decrease, and may also cause safety hazards or shorten the service life of the equipment. Therefore, it is particularly important to detect defects in solar panels.

[0003] At present, there are two kinds of target detection algorithms based on deep learning. One is a two-stage target detection method, which divides target detection into two stages of candidate region generation and classification regression. Common two-stage target detection algorithms include the R-CNN series of algorithms. The other is a one-stage target detection algorithm. This kind of method does not need to generate candidate regions, and directly predicts the category and position information of the target through regression, so the detection speed is usually faster. One-stage target detection algorithms include the YOLO series, the SSD series and the Anchor Free method. The Anchor Free series is different from other one-stage methods. This kind of method does not rely on the anchor frame mechanism, directly detects the target, avoids the problems of unbalanced positive and negative samples caused by the anchor frame, and has higher flexibility. Although the existing network can be used in solar panel defect detection, its performance is not ideal and there is still a lot of room for improvement. SUMMARY

[0004] The technical problem to be solved by the present application is that the existing defect detection method has low precision and high hardware requirements.

[0005] Therefore, the present application provides a solar panel defect detection method based on improved RT-DETR, which improves the average precision of solar panel defect detection by improving RT-DETR, and reduces the hardware deployment resource requirements.

[0006] The technical scheme adopted by the present application to solve the technical problem is:

[0007] A solar panel defect detection method based on improved RT-DETR, comprising the following steps,

[0008] S1, pre-processing the collected solar panel defect image to obtain a data set for model training;

[0009] S2, improvements are made to the original RT-DETR model to obtain an improved PABW-RTDETR model, wherein a backbone network module of the improved PABW-RTDETR model has a C2f-AP module;

[0010] S3, inputting the preprocessed data set into the improved model, performing multi-round iterative training on the improved PABW-RTDETR model;

[0011] S4, using the trained PABW-RTDETR model to detect the surface of the solar cell panel to obtain the defect type.

[0012] Further, in the step S2, the C2f-AP module is fused from an asymmetric padding AP and a C2f structure, the original Bottleneck is replaced with the AP module, the AP module adopts asymmetric padding convolution, adopts four groups of asymmetric padding parameters, realizes spatial feature enhancement, and adopts a divide-and-conquer strategy to divide the channel dimension into four subspaces for independent spatial feature learning.

[0013] Further, each subspace respectively performs a directional feature learning operation, which is represented as: Wherein, x' = Conv c1 (x) is an initial feature projection operation, P1 and P2 respectively correspond to a main path and an enhanced path after channel segmentation, represents a feature fusion process.

[0014] Further, the improved PABW-RTDETR model introduces an adaptive sparse self-attention module ASSA, adopts a double-branch structure of an SSA sparse branch and a DSA dense branch, the SSA sparse branch is responsible for filtering out pixel points with low query-key matching scores, avoiding the negative impact of these irrelevant points on feature aggregation and reducing noise interaction; the DSA dense branch ensures sufficient information flow in the network, so that the model can learn discriminative feature representations and extract key information in the image.

[0015] Further, in the step S2, under the double-branch attention mechanism, the attention update is: A = (ω1*SSA + ω2*DSA)V, wherein ω1, ω2 ∈ R 1 are two normalized weights for adaptively adjusting the double-branch features, the * symbol represents multiplication, and V represents the value.

[0016] Further, the improved PABW-RTDETR model further includes a bidirectional multi-branch auxiliary fusion pyramid network BIMAFPN, which fuses bidirectional weighted features of BiFPN and multi-branch auxiliary fusion of MAFPN to optimize cross-scale feature interaction.

[0017] Furthermore, in the Bidirectional Multi-Branch Assisted Fusion Pyramid Network (BIMAFPN), the feature fusion step includes:

[0018] Feature paths are enhanced by leveraging bidirectional cross-scale connectivity and weighted feature fusion of BIFPN;

[0019] In the bottom-up information transmission path, the UCF module extracts multi-scale features from the backbone network and performs preliminary assisted fusion in the superficial neck layer;

[0020] In the deep network of BIMAFPN, the CCF module is used to integrate multi-scale information and output features that simultaneously fuse information from four different layers.

[0021] Furthermore, the output after applying UCF is: P′ n =concat(δ(C(Conv(P)) n-1 ))),P n ,U(P′ n+1 )); δ is the SiLU activation function, C represents the 1×1 convolution that controls the number of channels, Conv is the 3×3 downsampling convolution, and the symbol U(·) represents the upsampling operation.

[0022] Furthermore, the output after applying CCF is as follows: P″ n =concat(δ(C(Conv(P′)) n-1 ))),δ(C(Conv(P″ n-1 ))),P′ n ,C(U(P′ n+1 Finally, the output layer P4 can fuse information from four different layers simultaneously.

[0023] Furthermore, in step S2, the WIoUv3 loss function module is used in the output layer to introduce an error-aware weight function and an inner mechanism to dynamically adjust the importance of samples and optimize the bounding box regression of small targets and irregular defects.

[0024] The beneficial effects of this invention are as follows: The solar panel defect detection method based on the improved RT-DETR has the following significant effects:

[0025] 1. By using a self-made dataset, the defect data of solar panels required for the experiment can meet the experimental environment, which is conducive to improving the accuracy of detection;

[0026] 2. Improve the backbone network of the RT-DETR model. C2f-AP constructs an asymmetric receptive field through asymmetric padding convolution, which enhances the ability to extract the directionality of multi-scale defect edges and textures. At the same time, it divides the channel subspace through a divide-and-conquer strategy, which improves the feature extraction efficiency while being lightweight.

[0027] 3. Introducing an adaptive sparse self-attention module ASSA, ASSA filters background redundant information through a double-branch attention mechanism, focuses on key defect features, and especially improves the recognition accuracy of small target defects in complex backgrounds, but the parameters increase slightly due to the introduction of attention calculation.

[0028] 4. Constructing a bidirectional multi-branch auxiliary fusion pyramid network BIMAFPN, BIMAFPN optimizes the cross-scale feature interaction path through UCF and CCF modules, enhances the feature complementarity of defects of different scales (such as micro scratches and large-area no electricity), and improves the feature expression ability, but the parameters increase slightly due to the increase of feature fusion branches.

[0029] 5. Replace the GIoU loss function with WIoUv3, WIoUv3 optimizes the boundary box regression through error perception weight and Inner mechanism.

[0030] 6. Combine the four improved modules, the performance improvement amplitude is greater than that of single module stacking, compared with the original model, the improved PABW-RTDETR model improves mAP50 by 3.1%, reduces the parameter size by 27.06%, and improves FPS by 16.09%. BRIEF DESCRIPTION OF DRAWINGS

[0031] The present application will be further described below in conjunction with the drawings and examples.

[0032] Figure 1 is a flowchart of an embodiment of the present application.

[0033] Figure 2 is an improved process diagram of RT-DETR of the present application.

[0034] Figure 3 is a structure diagram of the AP module in the present application.

[0035] Figure 4 is a structure diagram of the ASSA module in the present application.

[0036] Figure 5 is a structure diagram of the UCF module in the present application.

[0037] Figure 6 is a structure diagram of the CCF module in the present application.

[0038] Figure 7 is a structure diagram of the improved PABW-RTDETR model of the present application.

[0039] Figure 8 is a precision comparison curve diagram of the improved PABW-RTDETR model of the present application.

[0040] Figure 9 is the recall comparison curve diagram of the improved PABW-RTDETR model of the application.

[0041] Figure 10 is the P-R comparison curve diagram of the improved PABW-RTDETR model of the application.

[0042] Figure 11 is the F1 comparison curve diagram of the improved PABW-RTDETR model of the application. DETAILED DESCRIPTION

[0043] The application will now be described in further detail with reference to the drawings. These drawings are simplified schematic diagrams which show the basic structure of the application in a schematic manner only, and therefore only show the components relevant to the application.

[0044] In the description of the application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the application, unless otherwise specified and limited, the term "a plurality of" means two or more.

[0045] In the description of the application, it should be noted that unless otherwise specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the communication between the two elements inside. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0046] An improved RT-DETR-based solar panel defect detection method, as shown in Figure 1 includes the following steps:

[0047] S1, collect solar panel defect images, pre-process the images to obtain a data set for model training.

[0048] S11, collect the defect images of the solar panels in the field, all the original images are taken by a special camera, and the image resolution is 4043×643 pixels.

[0049] S12, after manual cropping of the original images, all the images are uniformly cropped into a square of 600×600 pixels, a total of 2043 high-quality image samples are included, and through classification labeling and data enhancement such as horizontal flipping and vertical flipping, a dataset is obtained.

[0050] S13, in order to ensure the rationality of data distribution and the reliability of model training, the labeled dataset is divided into training set, validation set and test set according to the ratio of 7:2:1.

[0051] S2, the original RT-DETR model is improved to obtain the improved PABW-RTDETR model.

[0052] S21, in the backbone network module, the asymmetric padding AP is fused with the C2f structure to form the C2f-AP module, the AP module adopts asymmetric padding convolution to simulate the spatial offset characteristics of the defect target and improve the response ability of the model to edge details and directional features. The C2f module in YOLOv8 has good channel fusion ability and parameter efficiency, and is suitable for embedding lightweight improved structure.

[0053] In the embodiment, in the step S21, as shown in the formula: Figure 3 , the original Bottleneck is replaced by the AP module, and spatial feature enhancement is realized through four groups of asymmetric padding parameters, for example, the padding parameters (2, 0, 2, 0) represent the number of padding pixels in the left, right, top and bottom directions respectively. The channel dimension is divided into four subspaces by using the divide-and-conquer strategy, and independent spatial feature learning is performed on each subspace. The mathematical form is:

[0054]

[0055] Wherein, x″=Conv c1 (x) is the initial feature projection operation, F C2f-AP (x) is the formula for replacing the Bottleneck module in C2f with the AP module, and the whole C2f acts on the result, P1 and P2 correspond to the main path and the enhanced path after channel segmentation respectively, and the channel in C2f is divided into two parts: one part is directly used as the main path; the other part is sent to the enhanced path composed of n AP Bottleneck. represents the feature fusion process.

[0056] The four subspace feature processing processes in the AP module are:

[0057] yk = Conv 1×1 (ZeroPad k (x))(k = 1, 2, 3, 4)

[0058] Then the spatial feature fusion is realized by cross-branch feature concatenation and 3x3 convolution, and the formula is as follows:

[0059] F AP (x) = x + Conv 3×3 (Concat(y1, y2, y3, y4))

[0060] In this way, the total process of the C2f-AP module replacement is divided into three stages:

[0061] 1. The initial feature projection stage generates the benchmark feature through 1x1 convolution and SiLU activation function:

[0062] x' = σ(BN(Conv 1×1 (x))) ∈ R B×2C×H×W

[0063] 2. The double-path feature (main path and enhanced path) learning stage divides x' into P1(x') ∈ R B×C×H×W and P2(x') ∈ R B×C×H×W , wherein the enhanced path is processed by n-level AP Bottleneck to obtain:

[0064]

[0065] 3. Finally, the feature mapping is completed through channel concatenation and 1x1 convolution:

[0066]

[0067] S22, an adaptive sparse self-attention module ASSA is introduced, which adopts a double-branch structure of SSA sparse branch and DSA dense branch. The SSA sparse branch filters out pixel points with low query-key matching scores to avoid the negative impact of irrelevant points on feature aggregation and reduce noise interaction. The DSA dense branch ensures sufficient information flow in the network, enabling the model to learn discriminative feature representations and extract key information from images.

[0068] In an embodiment, in the step S22, as shown in Figure 4 ,

[0069] For normalized features Figure X ∈ R H×W×C , first divide them into non-overlapping windows of size MxM to obtain the flattened representation of the ith window Then, the matrix of queries Q, keys K and values V is generated from X, as follows:

[0070] Q = XW Q K = XW K V = XW V

[0071] where linear projection matrices WQ, WK, WV∈R C×d are shared across all windows. The attention computation is as follows:

[0072]

[0073] where A denotes the estimated attention weights; B denotes the learnable relative position bias, and f(·) is the scoring function. And the weight computation of different "heads" is parallel, and is fused by concatenation and linear projection.

[0074] The standard dense self-attention mechanism (DSA) processes all query-key value pairs through a SoftMax layer to obtain attention scores, as follows:

[0075]

[0076] But not all query tokens are highly relevant to the corresponding tokens in the key. For this purpose, the sparse self-attention mechanism (SSA) is introduced, which uses a square ReLU layer to realize the attention sparsity, as follows:

[0077]

[0078] SSA can filter out negative correlation similarity and propagate effective information flow, but may cause information loss due to excessive sparsity. To balance the two, a dual-branch self-attention mechanism is proposed, whose attention update is as follows:

[0079] A = (ω1*SSA + ω2*DSA)V

[0080] where ω1, ω2∈R 1 are two normalized weights for adaptively adjusting the dual-branch features, and the symbol * denotes multiplication. The calculation formula is as follows:

[0081]

[0082] where {a1, a2} are learnable parameters.

[0083] S23, a bidirectional multi-branch auxiliary fusion pyramid network BIMAFPN is constructed, which fuses the bidirectional weighted features of BiFPN and the multi-branch auxiliary fusion of MAFPN to optimize the cross-scale feature interaction;

[0084] S231 leverages bidirectional cross-scale connectivity and weighted feature fusion of BIFPN to enhance feature paths.

[0085] The core idea of ​​BiFPN is to enhance feature paths through efficient bidirectional cross-scale connections and weighted feature fusion. The BiFPN structure is based on PAN (Pixel Aggregation Network), removing nodes with only a single input and adding a new aggregation path between the original input and output nodes. BiFPN's bidirectional cross-scale connections replace the bottom-up path with the UCF module in MAFPN and the top-down path with the CCF module in MAFPN. Through multiple stacking operations, more feature information is fused. The BiFPN network uses a fast weighted normalization fusion method, as shown in the following formula:

[0086]

[0087] Among them, I i O is the input feature, O is the output feature, and ω is the input feature. i and ω j These are the learnable weights, which are scaled to the range [0,1] using the ReLU activation function. ε = 0.0001 is a small value used to ensure stable output.

[0088] S232, such as Figure 5 As shown, in the bottom-up information transmission path, the UCF module extracts multi-scale features from the backbone network and performs preliminary assisted fusion in the superficial neck layer. The output after applying UCF is as follows:

[0089] P′ n =concat(δ(C(Conv(P)) n-1 ))),P n ,U(P′ n+1 ))

[0090] δ is the SiLU activation function, C represents a 1×1 convolution controlling the number of channels, Conv is a 3×3 downsampling convolution, the symbol U(·) represents an upsampling operation, and P n It represents Figure 7 In the PABW-RTDETR, the Backbone's P3, P4, P5, and n values ​​are 3, 4, and 5.

[0091] S233, in the deep networks of BIMAFPN, such as Figure 6 As shown, the CCF module is used to perform multi-scale information integration. The output results after applying CCF are as follows:

[0092] P″ n =concat(δ(C(Conv(P′)) n-1 ))),δ(C(Conv(P″n-1 n n+1

[0093] The final output layer P4 can simultaneously fuse information of four different layers, significantly improving the detection performance of the model on medium-sized targets and enhancing the fusion and utilization efficiency of the model on defect features of different scales.

[0094] In step S24, a WIoUv3 loss function module is used to introduce an error perception weight function and an Inner mechanism to dynamically adjust the importance of samples and optimize the boundary box regression of small targets and irregular defects.

[0095] In an embodiment, in the step S24, traditional IoU type losses such as GIoU, DIoU, CIoU, etc. have certain limitations in gradient distribution, which easily gives too strong gradient interference to low-quality samples or causes gradient saturation to high-quality samples. The WIoUv3 solves this problem by introducing an error perception weight function ω. The basic loss function form is:

[0096] L WIoU =ω·(1-IoU(b,b gt ))

[0097] Where b represents the predicted box, b gt represents the real box, IoU(b,b gt ) is the intersection over union between the two, and ω is a sample weight function generated by sequential evidence guidance, and the formula is:

[0098] ω=exp(-α·(δ-μ) 2 )

[0099] Where δ represents the prediction error, μ is the gradient barycenter setting position, and α controls the steepness of the gradient response range.

[0100] In an embodiment, in the step S24, the Inner mechanism is also introduced in the design of WIoU. Inner is used as an auxiliary gradient adjustment factor to measure the quality of the effective overlap area inside the predicted box, thereby enhancing the response ability of the loss to the actual area coverage quality. The scale factor ratio is set to 0.75.

[0101] S3, input the preprocessed data set into the improved model, and perform multi-round iterative training on the improved PABW-RTDETR model;

[0102] S4, using the trained PABW-RTDETR model to detect the surface of the solar cell panel to obtain the defect type.

[0103] ​​​The technical concept of the present application is: first, the C2f-AP module is proposed to optimize the backbone network, the C2f structure of YOLOv8 is combined with the asymmetric padding (AP) module to enhance the multi-scale defect feature extraction capability; second, the attention mechanism is improved, the adaptive sparse self-attention (ASSA) is used to replace the attention-based intrascale feature interaction (AIFI) to improve the focusing ability of the model on key defect features; third, a new type of feature pyramid network BIMAFPN (Bidirectional Multi-Branch Auxiliary Fusion Pyramid Network) is proposed to optimize the cross-scale feature interaction; finally, the loss function is replaced to improve the detection accuracy of small size and irregular defects.

[0104] In the embodiments, the improved RT-DETR of the present application for solar panel defect detection has the following advantages compared with the RT-DETR for solar panel defect detection in the prior art:

[0105] 1. Through the self-made data set, the data set covers the four most representative defects in the solar panel production process: no electricity, black spots, scratches, and cross hidden, so that the solar panel defect data required for the experiment meets the experimental environment, which is conducive to the improvement of detection accuracy;

[0106] 2. The backbone network of the improved RT-DETR model is improved, and the C2f-AP constructs an asymmetric receptive field through asymmetric padding convolution to enhance the directional extraction capability of multi-scale defect edges and textures, and at the same time, through the divide-and-conquer strategy to divide the channel subspace, the feature extraction efficiency is improved while the model is lightweight. Compared with the baseline model, the C2f-AP model parameters achieve a significant reduction of 32.9%, and the computational amount is reduced by 17.7%. In the core indicator mAP50 of detection performance, C2f-AP reaches 80.8%, successfully improving by 1.5 percentage points, and the detection effect is enhanced.

[0107] Transverse comparison with other model variants, each has advantages and disadvantages:

[0108] Although C2f-DTAB reaches the highest accuracy indicator of 83.4%, its computational amount is 57.3G, which has obvious redundancy;

[0109] The model parameters and computational amount of C2f-SHSA are relatively close to those of C2f-AP, which has certain lightweight potential, but in the mAP50 indicator of 80.0%, the improvement amplitude is the smallest among the compared models;

[0110] C2f-FAT, although reduced compared to the baseline, has limited amplitude and mAP50 gain is not outstanding, and the performance improvement effect is not obvious.

[0111] Considering the balance between model detection performance and computational cost, C2f-AP has a significant advantage in multi-dimensional comparison, and can effectively reduce the demand for hardware resources while ensuring good detection effect;

[0112] 3, introduce adaptive sparse self-attention module ASSA, ASSA filters background redundant information through double-branch attention mechanism, focuses on key defect features, and the recognition accuracy of small target defects is significantly improved in complex background, but the parameters increase slightly due to the introduction of attention calculation.

[0113] The original model has a lot of blue and cyan cold tones in the heat map of the non-electric region, and the internal features of the defect are not fully captured. The improved model RT-DETR+ASSA has a significant increase in the proportion of warm tones in the defect region, indicating that ASSA focuses more on key features of defects and filters out redundant information such as background. At this time, the confidence is increased to 0.90, indicating that the model is more accurate and confident in identifying the defect, verifying the role of ASSA in enhancing small target defect feature extraction and improving recognition accuracy.

[0114] Compared with the original model, the improved model has an mAP50 increase of 1.6%, an mAP50:95 increase of 3.5%, but the parameters increase to 20.71M;

[0115] 4, build a bidirectional multi-branch auxiliary fusion pyramid network BIMAFPN, BIMAFPN optimizes the cross-scale feature interaction path through UCF and CCF modules, enhances the complementary nature of features of different scales (such as small scratches and large areas of no electricity), and improves feature expression ability, but the parameters increase slightly due to the increase of feature fusion branches.

[0116] Comparative experiments verify the influence of different feature fusion networks on the performance of the RT-DETR model:

[0117] The original RT-DETR model as baseline achieved 79.3% mAP50 in the solar panel defect detection task, with a computational complexity of 57.0 GFLOPs.

[0118] After introducing the BIFPN feature fusion network, the model accuracy drops to 79.5%, mAP50 drops to 79.0%, and parameter quantity and computational quantity increase to 20.30M and 64.3G respectively, detection performance and efficiency are both damaged.

[0119] Although MAFPN improves mAP50 to 80.3%, its parameter count of 22.93M is the highest among all models, indicating insufficient lightweight design.

[0120] The BIMAPPN fusion solution performs better, achieving the highest accuracy of 84.0% and the most significant improvement with mAP50. It also has a smaller increase in parameters and computational load, achieving a better balance between performance and cost, and providing a more practical direction for improvement in this scenario.

[0121] 5. Replace the GIoU loss function with WIoUv3. WIoUv3 optimizes bounding box regression through error-aware weights and the Inner mechanism.

[0122] To confirm the effectiveness of the proposed Wise-Inner-WIOU loss function, tests were conducted using various ratio settings to adjust the size of the auxiliary bounding boxes. Comparative analyses were performed with established loss functions such as DIoU, CIoU, Wise-Inner-EIOU, and Wise-Inner-ShapeIoU. The model achieved the highest detection accuracy when the ratio of the Wise-Inner-WIOU loss function was 0.75.

[0123] 6. Combining the four improved modules results in a greater performance improvement than stacking a single module. Compared to the original model, the improved PABW-RTDETR model has a 3.1% increase in mAP50, a 27.06% reduction in parameter size, and a 16.09% increase in FPS.

[0124] 7. Ablation experiments show that using a lightweight C2f-AP-enhanced backbone network improves mAP50 and mAP50:95 by 1.5% and 2.9%, respectively. These improvements demonstrate more efficient feature extraction, faster detection speed, and reductions in parameters and GFLOPs by 32.9% and 17.7%, respectively. Compared to the baseline model, integrating the ASSA layer into the hybrid encoder improves mAP50 by 1.6% and mAP50:95 by 3.5%. Utilizing the BIMAFPN architecture as a cross-scale feature fusion network yields significant results, achieving an mAP50 of 81.5% through the combined effect of multiple improvements, with computational cost controlled at 49.6 GFLOPs, demonstrating the potential of module synergy. The proposed method, through optimized integration strategy, achieves 82.4% mAP50 and 39.2% mAP50:95 while maintaining 14.50M parameters and 49.6 GFLOPs of computation, representing a 3.1% improvement over the baseline model and the best performance among all comparative methods. Experiments show that each modification has a significant impact on the performance of the PABW-RTDETR model.

[0125] 8. Compared with various state-of-the-art object detection models, the PABW-RTDETR model shows significant improvement in performance indicators:

[0126] In terms of mAP50, PABW-RTDETR is 34.0%, 4.7%, 11.0%, 13.1%, 11.3%, 15.0%, and 14.1% higher than Faster-RCNN, SSD, YOLOv5, YOLOv6, YOLOv8, YOLOv10m, and YOLOv11, respectively. These substantial progress in mAP50 highlights the outstanding detection accuracy of PABW-RTDETR, especially for small solar panel defect objects. In addition, PABW-RTDETR model can process about 177 frames of images per second, exceeding the real-time detection standard. This fast inference time makes PABW-RTDETR an effective model for practical applications that require real-time performance. Compared with RT-DETR-R34 and RT-DETR-R50 models, mAP50 is increased by 3.0% and 2.6%, respectively. These results emphasize the effectiveness of the architecture optimization and design choices in PABW-RTDETR, which contribute to improving its detection accuracy. In addition, compared with the original RT-DETR model, PABW-RTDETR shows significant progress, with mAP50 and mAP50:95 indicators both increasing by 3.1%, indicating that performance has been improved under the four improvements. At the same time, F1 score and FPS are increased by 3.48% and 16.09%, respectively, emphasizing the balanced approach of the model in terms of accuracy and efficiency.

[0127] 9. To visually demonstrate the model's ability to identify solar panel defect detection objects, several evaluation indicators are visualized, Figure 8- Figure 11 The performance curves of PABW-RTDETR model for various defect object precision, recall, mAP50, and F1. Precision is the ratio of true positives to all positive predictions made by the model. Recall measures the proportion of true positive samples correctly identified as positive by the model. The expressions are:

[0128]

[0129] where TP refers to instances correctly identified as positive, FP refers to instances incorrectly identified as positive, and FN refers to instances incorrectly identified as negative. F1 score, as the harmonic mean between precision and recall, has the expression:

[0130]

[0131] Average precision (AP) is the average highest accuracy at different class recall levels. Mean average precision (mAP) calculates the average AP of all classes and provides an overall evaluation of the effectiveness of the model. Its expression is as follows:

[0132]

[0133] where S represents the total number of classes, and the denominator is the sum of the AP values of all classes.

[0134] The proposed PABW-RTDETR model realizes the synergistic optimization of solar panel defect detection precision and efficiency by fusing lightweight structure and attention mechanism. The C2f-AP backbone network reduces the computational cost while enhancing the multi-scale feature expression; the joint use of ASSA and BIMAFPN effectively improves the focusing ability of the model on key defects; the dynamic loss function design improves the small target positioning accuracy. On the self-built dataset, compared with the original RT-DETR, the mAP50 of the model increases by 3.1%, the parameter size reduces by 27.06%, and the FPS increases by 16.09%. Therefore, there is a good improvement in detection speed and detection accuracy. Future work will focus on improving the detection accuracy of mAP50:95 and exploring the deployment of the model on mobile devices, further promoting the intelligent development of photovoltaic quality inspection technology.

[0135] With the above ideal embodiments according to the present application as the inspiration, through the above description, relevant personnel can certainly make various changes and modifications without deviating from the scope of the technical idea of the present application. The technical scope of the present application is not limited to the contents in the specification, and must be determined by the scope of the claims.

Claims

1. A method for detecting defects in solar panels based on an improved RT-DETR, characterized in that, Includes the following steps, S1, preprocess the collected images of solar panel defects to obtain the dataset for model training; S2, the original RT-DETR model is improved to obtain the improved PABW-RTDETR model, and the backbone network module of the improved PABW-RTDETR model has a C2f-AP module. S3, input the preprocessed dataset into the improved model, and perform multiple rounds of iterative training on the improved PABW-RTDETR model; S4. The trained PABW-RTDETR model is used to detect defects on the surface of the solar panel and obtain the defect types.

2. The solar panel defect detection method based on improved RT-DETR according to claim 1, characterized in that, In step S2, the C2f-AP module is formed by fusing the asymmetric padding AP and the C2f structure, replacing the original Bottleneck with the AP module. The AP module uses asymmetric padding convolution and four sets of asymmetric padding parameters to enhance spatial features. A divide-and-conquer strategy is used to divide the channel dimension into four subspaces for independent spatial feature learning.

3. The solar panel defect detection method based on improved RT-DETR according to claim 1, characterized in that, Each subspace performs a targeted feature learning operation, represented as follows: Where, x′=Conv c1 (x) represents the initial feature projection operation, where P1 and P2 correspond to the main path and enhancement path after channel segmentation, respectively. This indicates the feature fusion process.

4. The solar panel defect detection method based on improved RT-DETR according to claim 1, characterized in that, The improved PABW-RTDETR model introduces an adaptive sparse self-attention module (ASSA), employing a dual-branch structure of SSA sparse branch and DSA dense branch. The SSA sparse branch filters out pixels with low query-key matching scores, preventing these irrelevant points from negatively impacting feature aggregation and reducing noise interactions. The DSA dense branch ensures sufficient information flow in the network, enabling the model to learn discriminative feature representations and extract key information from the image.

5. The solar panel defect detection method based on improved RT-DETR according to claim 4, characterized in that, In step S2, under the bi-branch attention mechanism, the attention is updated as: A = (ω1*SSA + ω2*DSA)V, where ω1, ω2 ∈ R. 1 These are two normalized weights used to adaptively adjust the dual-branch features. The asterisk (*) represents a multiplication operation, and V represents the value.

6. The solar panel defect detection method based on improved RT-DETR according to claim 4, characterized in that, The improved PABW-RTDETR model also includes a bidirectional multi-branch assisted fusion pyramid network BIMAFPN, which fuses the bidirectional weighted features of BiFPN with the multi-branch assisted fusion of MAFPN to optimize cross-scale feature interaction.

7. The solar panel defect detection method based on improved RT-DETR according to claim 6, characterized in that, In the Bidirectional Multi-Branch Assisted Fusion Pyramid Network (BIMAFPN), the feature fusion steps include: Feature paths are enhanced by leveraging bidirectional cross-scale connectivity and weighted feature fusion of BIFPN; In the bottom-up information transmission path, the UCF module extracts multi-scale features from the backbone network and performs preliminary assisted fusion in the superficial neck layer. In the deep network of BIMAFPN, the CCF module is used to integrate multi-scale information and output features that simultaneously fuse information from four different layers.

8. The solar panel defect detection method based on improved RT-DETR according to claim 7, characterized in that, The output after applying UCF is: P′ n =concat(δ(C(Conv(P)) n-1 ))),P n ,U(P′ n+1 )); δ is the SiLU activation function, C represents the 1×1 convolution that controls the number of channels, Conv is the 3×3 downsampling convolution, and the symbol U(·) represents the upsampling operation.

9. The solar panel defect detection method based on improved RT-DETR according to claim 7, characterized in that, The output after applying CCF is as follows: P″ n =concat(δ(C(Conv(P′)) n-1 ))),δ(C(Conv(P″ n-1 ))),P′ n ,C(U(P′ n+1 Finally, the output layer P4 can fuse information from four different layers simultaneously.

10. The solar panel defect detection method based on improved RT-DETR according to claim 1, characterized in that, In step S2, the WIoUv3 loss function module is used in the output layer to introduce an error-aware weight function and an inner mechanism to dynamically adjust the importance of samples and optimize the bounding box regression of small targets and irregular defects.

Images