Lightweight dual-path multi-scale photovoltaic cell defect detection method based on transformer

Abstract

This invention relates to the field of target detection technology in deep learning and computer vision, and discloses a lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer. The method includes: acquiring and preprocessing photovoltaic cell defect images; constructing an LDPM-DETR detection model, using RT-DETR-R18 as the baseline model, comprising an LDPMGNet dual-path backbone network, an efficient hybrid encoder, and a Transformer decoder. The efficient hybrid encoder includes an intra-scale feature interaction module MSLA-AIFI and a cross-scale feature fusion module EWMB-FPN. MSLA-AIFI removes explicit positional encoding while maintaining the original feedforward network and layer normalization structure, and replaces the standard multi-head self-attention with an MSLA module; training the LDPM-DETR detection model, and using the trained LDPM-DETR detection model to detect the photovoltaic cell defect images. Compared with existing technologies, this invention significantly reduces model complexity while improving the detection accuracy of multi-scale defects.

Description

Technical Field

[0001] This invention relates to the field of target detection technology in deep learning and computer vision, specifically to a lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer, applicable to industrial quality inspection and edge deployment scenarios in photovoltaic module production lines. Background Technology

[0002] As the core component of photovoltaic modules, the timely and accurate detection of defects in photovoltaic cells is crucial for ensuring module quality and stable system operation. EL imaging, which uses near-infrared images to reveal internal defects in cells, is an important method for defect detection.

[0003] Convolutional neural network (CNN)-based detection algorithms (such as the YOLO series and its improved versions) have achieved significant results in conventional defect identification. However, when dealing with defects with large spatial spans, discontinuous physical forms, and low background contrast (such as vertical misalignment, thick lines, and linear cracks), it is difficult to effectively model the long-range and discrete global morphology of the defects, and the global perception capability is limited.

[0004] RT-DETR, as the first Transformer architecture to achieve end-to-end real-time object detection, consists of a backbone network, an efficient hybrid encoder (including attention-based intra-scale feature interaction AIFI and convolution-based cross-scale feature fusion CCFF), and a Transformer decoder with an auxiliary prediction head. It effectively perceives the overall structure and spatial context of defects by leveraging the global modeling capabilities of the Transformer.

[0005] However, when RT-DETR (represented by RT-DETR-R18) is directly applied to photovoltaic cell defect detection, the following technical problems exist:

[0006] The RT-DETR series has a large number of parameters and high computational complexity. The RT-DETR-R18 has 19.9M parameters and 57.0 GFLOPs of computation, making it difficult to balance the detection efficiency and accuracy required for industrial applications and difficult to deploy in resource-constrained edge devices.

[0007] The backbone network lacks multi-scale adaptability. RT-DETR-R18 uses a ResNet-based backbone network, but its single-scale feature extraction paradigm and fixed-step downsampling strategy are difficult to adaptively represent defect targets of different scales, especially with limited accuracy in identifying fine-grained defects such as microcracks.

[0008] Intra-scale attention struggles to focus on minute defects. AIFI only deploys standard multi-head self-attention at the highest semantic layer S5, and its globally uniform distribution makes it difficult to allocate sufficient attention weights to minute defect regions. Furthermore, the single-scale modeling paradigm cannot simultaneously meet the dual needs of high-resolution image processing and defect representation across orders of magnitude.

[0009] The cross-scale feature fusion capability is insufficient. The scale of defects in photovoltaic cells varies greatly (from micro-cracks to large-area damage), and the dense conductive grids and welded structures on the surface generate a large number of high-frequency background textures. The visual features of defects such as broken grids, thick lines, and micro-cracks are weak. In this scenario, CCFF has insufficient cross-scale adaptability and anti-background interference capability. After inappropriate fusion, the weak signal is further attenuated, and the risk of missed detection increases. Summary of the Invention

[0010] Purpose of the invention: To address the problems existing in the prior art, this invention provides a lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer. Using RT-DETR as the baseline, it solves the problems of insufficient multi-scale representation ability of the backbone network, difficulty in focusing attention on small defects within the scale, and insufficient cross-scale fusion adaptability while effectively controlling the number of model parameters and computational overhead. Thus, it improves the detection accuracy of multi-scale defects while significantly reducing model complexity.

[0011] Technical solution: This invention provides a lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer, including:

[0012] Acquire images of photovoltaic cell defects to be processed and perform preprocessing;

[0013] An LDPM-DETR detection model is constructed, which uses RT-DETR-R18 as the baseline model and includes an LDPMGNet dual-path backbone network, an efficient hybrid encoder, and a Transformer decoder. The efficient hybrid encoder includes an intra-scale feature interaction module MSLA-AIFI and a cross-scale feature fusion module EWMB-FPN. MSLA-AIFI removes explicit positional encoding while maintaining the original feedforward network and layer normalization structure, and replaces the standard multi-head self-attention with the MSLA module.

[0014] The LDPM-DETR detection model is trained, and the trained LDPM-DETR detection model is used to detect defects in the photovoltaic cell images to be processed.

[0015] Furthermore, the LDPMGNet dual-path backbone network employs heterogeneous dual-path extraction of complementary features:

[0016] First, a shared input mechanism is set up. Before the dual-path differentiation, a unified shallow feature extraction module is set up to extract shallow features as the output feature S2 of the P2 layer. After the shared input, it is differentiated into two parallel paths: the first spatial path and the first semantic path.

[0017] The first spatial path is processed sequentially by the heterogeneous dual path and the efficient module EDCS-MRFB, Conv, and the heterogeneous dual path and the efficient module EDCS-MRFB before being output; the first semantic path is processed sequentially by HGBlock, DWConv, and HGBlock before being output; the dual path output features are fused by the multi-scale adaptive spatial attention gate MASAG module and used as the output features S3 of layer P3.

[0018] The output feature S3 of layer P3 passes through two parallel paths: the second spatial path and the second semantic path. The second spatial path processes the output features sequentially through Conv, the heterogeneous dual path, and the efficient module EDCS-MRFB. The second semantic path processes the output features sequentially through DWConv and HGBlock. After being fused by the multi-scale adaptive spatial attention gate MASAG module, it becomes the output feature S4 of layer P4.

[0019] The output feature S4 of layer P4 passes through two parallel paths: the third spatial path and the third semantic path. It has the same structure as the second spatial path and the second semantic path. After being fused by the multi-scale adaptive spatial attention gate MASAG module, it becomes the output feature S5 of layer P5.

[0020] Furthermore, the input features of the cross-scale feature fusion module EWMB-FPN are the output features S2, S3, and S4 of the P2, P3, and P4 layers of the LDPMGNet dual-path backbone network and the F5 output of the intra-scale feature interaction module MSLA-AIFI. It is an improvement based on the multi-branch auxiliary feature pyramid network MAFPN. In the auxiliary fusion stage, the weighted fusion operation of the bidirectional feature pyramid network BIFPN is introduced. The heterogeneous dual-path and high-efficiency module EDCS-MRFB is used to extract photovoltaic defect features. In the auxiliary fusion and the upsampling of the heterogeneous dual-path and high-efficiency module EDCS-MRFB, the high-efficiency upsampling module EUCB is introduced.

[0021] Furthermore, the heterogeneous dual-path and high-efficiency module EDCS-MRFB combines the CSP network with the directional channel offset multi-receptive field module DCS-MRFB, replacing Bottleneck in the CSP network with the directional channel offset multi-receptive field module DCS-MRFB. The directional channel offset multi-receptive field module DCS-MRFB integrates multi-receptive field convolution and a lightweight directional channel offset mechanism. After the input features are processed by multi-branch depthwise convolution, cross-channel information fusion is performed, and a directional channel offset DCS operation is added to maximize spatial feature interaction with zero parameters. The fused features are divided into four equal parts along the channel, and then cyclically offset along the height and width dimensions in the positive and negative directions and spliced. Finally, the DCS output is connected with the residual of the original input features to obtain the DCS-MRFB output.

[0022] Furthermore, the HGBlock contains multiple cascaded convolutional layers, and the output of each layer is concatenated with the original input in the channel dimension to achieve cross-layer feature reuse and aggregation. After aggregation, the high-dimensional features are transformed by two consecutive 1×1 convolutions to output features. Preferably, the gradient propagation is optimized by concatenating the output features with residual connections.

[0023] Furthermore, the multi-scale adaptive spatial attention gate (MASAG) module includes multi-scale fusion operation, spatial selection operation, spatial interaction and cross-modulation operation, and recalibration operation;

[0024] (1) Multi-scale fusion: The local branch processes the spatial path feature X with depthwise separable convolution and dilated convolution, and the global branch performs global pooling of the semantic path feature G along the channel dimension. The two branches are then spliced ​​and fused.

[0025] (2) Spatial selection: The feature U after splicing and fusing the two branches is projected into two sets of single-channel weight maps by convolution. After Softmax normalization, X and G are weighted element by element and residual connection is used to output features X' and G'.

[0026] (3) Spatial interaction and cross-modulation: The output features X' and G' are enhanced by cross-modulation of the two features with spatial weights activated by Sigmoid, and then multiplied element by element to output feature U';

[0027] (4) Recalibration: The output feature U' is generated by pointwise convolution and Sigmoid activation to generate attention map weights, which are then multiplied element-wise with the input X.

[0028] Furthermore, the MSLA module extracts local structural features of different receptive fields through parallel multi-scale deep convolution branches, and each branch is connected by residuals and activated by ReLU to output the features. At the same time, the multi-scale features of each branch are reshaped and global long-range dependencies are modeled using linear attention. The outputs of each attention head are concatenated along the channel dimension and linearly transformed to obtain the outputs of each branch. A learnable weight γᵢ is introduced to adaptively weight the features of each branch. After weighting, the features are concatenated by channels and fused by convolution to reshape the final MSLA output.

[0029] Furthermore, the high-efficiency upsampling module EUCB sequentially performs linear upsampling with a scaling factor of 2, 3×3 depthwise separable convolution, batch normalization, and ReLU activation, and adjusts the number of channels to match the target dimension through 1×1 convolution.

[0030] Beneficial effects:

[0031] 1. The backbone network designed in this invention employs a heterogeneous dual-path extraction of complementary features and achieves efficient extraction and adaptive fusion of multi-scale features through a three-stage progressive architecture. Furthermore, a unified shallow feature extraction module is set before the dual-path differentiation, allowing both paths to share the same set of preprocessing results, reducing shallow computational overhead while retaining rich feature information. The EDCS-MRFB module of the spatial path combines the CSP network concept with DCS-MRFB. Compared to directly stacking DCS-MRFB, it reduces computation and parameter count through cross-stage feature reuse. Its branch structure introduces an additional gradient path to alleviate the gradient vanishing problem, focusing on spatial multi-scale receptive field modeling. The semantic path realizes cross-layer feature reuse and aggregation, focusing on vertical fusion of hierarchical features and lightweight channel modulation. The two paths perform differentiated feature modeling from the spatial scale dimension and semantic hierarchical dimension, respectively, providing complementary feature representations with both multi-scale adaptability and semantic richness for subsequent networks.

[0032] 2. The multi-scale adaptive spatial attention gate MASAG fusion designed in the backbone network of this invention fuses and refines the dual-path features, effectively enhancing the perception of multi-scale defects in photovoltaic cells while maintaining a lightweight architecture.

[0033] 3. In addition, this invention designs the MSLA-AIFI scale feature interaction module based on the original RT-DETR-R18 baseline model. MSLA extracts local structural features of different receptive fields through parallel multi-scale deep convolution branches, and at the same time uses linear attention to model global long-range dependencies, so as to achieve the unity of multi-scale local representation and efficient global modeling.

[0034] 4. In cross-scale feature fusion, this invention draws on the multi-directional connectivity and assisted fusion concepts of the Multi-Branch Auxiliary Feature Pyramid Network (MAFPN) in its information flow architecture to enrich feature interactions, reduce computational overhead, and achieve efficient adaptive feature fusion. During the fusion process, EDCS-MRFB is used for photovoltaic defect feature extraction (configuring convolutional kernels of different scales to enhance multi-scale feature representation, and enhancing spatial representation through channel shuffling and directional offset). An efficient upsampling module (EUCB) is introduced into the assisted fusion and EDCS-MRFB upsampling processes, replacing standard convolutions with depthwise separable convolutions to significantly reduce computational overhead while ensuring feature enhancement. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall structure of the LDPM-DETR network of the present invention;

[0036] Figure 2 This is a schematic diagram of the LDPMGNet structure of the present invention;

[0037] Figure 3 This is a schematic diagram of the shared Stem structure of the present invention;

[0038] Figure 4 This is a schematic diagram of the EDCS-MRFB structure of the present invention;

[0039] Figure 5 This is a schematic diagram of the DCS-MRFB module structure of the present invention;

[0040] Figure 6 This is a schematic diagram of a DCS (Directional Channel Offset) structure;

[0041] Figure 7 This is a schematic diagram of the HGBlock structure of the present invention;

[0042] Figure 8 This is a schematic diagram of the MASAG structure of the present invention;

[0043] Figure 9 This is a schematic diagram of the MSLA-AIFI structure of the present invention;

[0044] Figure 10 This is a schematic diagram of the MSLA structure of the present invention;

[0045] Figure 11 This is a schematic diagram of the EUCB high-efficiency upsampling module structure of the present invention;

[0046] Figure 12 Precision and recall curves for RT-DETR-R18;

[0047] Figure 13The precision and recall curves of the LDPM-DETR of this invention are shown below;

[0048] Figure 14 This is a visualization of the detection results of the present invention;

[0049] Figure 15 This is a thermal visualization of the detection results of this invention. Detailed Implementation

[0050] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0051] This invention discloses a lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer, comprising the following steps:

[0052] Step 1: Obtain the defect image of the photovoltaic cell to be processed and perform preprocessing.

[0053] This embodiment constructs a balanced photovoltaic cell defect dataset based on the publicly available PVEL-AD dataset (by correcting and labeling each sample, removing extremely low-frequency categories, performing geometric and photometric enhancements on mid- and low-frequency categories, and introducing material defect categories), ultimately retaining 10 defect categories for training and testing.

[0054] The EL image of the photovoltaic cell to be inspected is preprocessed to 640×640.

[0055] Step 2: Construct the LDPM-DETR detection model, see [link / reference] Figure 1 The LDPM-DETR detection model uses RT-DETR-R18 as the baseline model and includes the LDPMGNet dual-path backbone network, an efficient hybrid encoder, and a Transformer decoder. The efficient hybrid encoder includes the intra-scale feature interaction module MSLA-AIFI and the cross-scale feature fusion module EWMB-FPN. MSLA-AIFI removes explicit position encoding while maintaining the original feedforward network and layer normalization structure, and replaces the standard multi-head self-attention with the MSLA module.

[0056] The LDPM-DETR detection model consists of three main parts: a backbone network, a high-efficiency hybrid encoder, and a Transformer decoder (see Figure 1).

[0057] The backbone network replaces the original ResNet-18 with the lightweight dual-path multi-scale gating network (LDPMGNet) proposed in this invention; the encoder replaces the original AIFI with the intra-scale feature interaction module MSLA-AIFI, which is an improvement on multi-scale linear attention (MSLA) proposed in this invention; and cross-scale feature fusion replaces the original CCFF with the efficient weighted multi-branch feature pyramid (EWMB-FPN) proposed in this invention. The core processes are as follows:

[0058] Step 2.1: Construct the LDPMGNet lightweight dual-path backbone network (see appendix) Figure 2 )

[0059] LDPMGNet employs a heterogeneous dual-path extraction of complementary features and achieves efficient extraction and adaptive fusion of multi-scale features through a three-stage progressive architecture.

[0060] (1) Shared Input Mechanism. To address the computational redundancy caused by the independent processing of shallow high-resolution features in the two branches, a unified shallow feature extraction module is set up before the two paths differentiate, allowing the two paths to share the same set of preprocessing results, which serve as the output feature S2 of the P2 layer of the backbone network. See Stem structure. Figure 3 First, a 3×3 convolution with a stride of 2 is used to achieve the first spatial downsampling. Then, the downsampled information is fed into the pooling path and the convolution path to extract features. The two features are concatenated by the channel dimension and then subjected to a convolution with a stride of 2 to complete the second downsampling. Finally, the output channel dimension is adjusted by a 1×1 convolution to quickly reduce the resolution by 4 times downsampling and fuse the convolution and pooling features, thereby reducing the shallow computational overhead while retaining rich feature information.

[0061] (2) After Stem, the network splits into two parallel paths with complementary functions:

[0062] The first spatial path processes data sequentially using the heterogeneous dual-path and the efficient module EDCS-MRFB, Conv, and then the heterogeneous dual-path and the efficient module EDCS-MRFB again before outputting the result. The core is the efficient directional channel offset multi-receptive field module DCS-MRFB (EDCS-MRFB). EDCS-MRFB combines the CSP network concept with DCS-MRFB. Compared to directly stacking DCS-MRFB, it reduces computation and parameter count through cross-stage feature reuse. Its branch structure introduces additional gradient paths to alleviate the gradient vanishing problem, focusing on spatial multi-scale receptive field modeling.

[0063] The first semantic path is processed sequentially by HGBlock, DWConv, and HGBlock before being output. HGBlock internally contains multiple cascaded convolutional layers; see [link to documentation]. Figure 7Each layer output is concatenated with the original input in the channel dimension to achieve cross-layer feature reuse and aggregation. After aggregation, the high-dimensional features are compressed and activated by "1×1 convolution to compress to half of the target channel and 1×1 convolution to restore to the target dimension" to complete the channel transformation. Optional residual connections optimize gradient propagation, focusing on the vertical fusion of hierarchical features and lightweight channel modulation.

[0064] The two paths model the features differently from the spatial scale dimension and the semantic level dimension, respectively, providing complementary feature representations for subsequent networks that have both multi-scale adaptability and semantic richness.

[0065] Among them, the directional channel offset multi-receptive field module DCS-MRFB (see Figures 5 and 6) integrates multi-receptive field convolution with a lightweight directional channel offset mechanism to achieve cross-scale and cross-channel feature enhancement with low computational cost.

[0066] First, the input features X∈R are convolved using a 1×1 pointwise convolution. (C×H×W) The channel is extended to a high-dimensional feature space, resulting in Fᵉ∈R (αC×H×W) (α is the channel expansion factor):

[0067] Formula (1) .

[0068] The extended features are fed in parallel into three depthwise convolutional branches configured with s×s, m×m, and l×l kernels (corresponding to small, medium, and large scales) to extract spatial features with different receptive fields:

[0069] Formula (2) .

[0070] After element-wise addition of the three branches, channel shuffling is performed to promote cross-channel information fusion: [Public...]

[0071] Equation (3) , formula (4) .

[0072] After channel shuffling, a Directional Channel Offset (DCS) operation is added to maximize spatial feature interaction with zero parameters: the features are divided into four equal parts along the channels, and then positive / negative cyclic offsets are performed along the height and width dimensions respectively, followed by stitching.

[0073] Formula (5) ;

[0074] Formula (6) .

[0075] Finally, the DCS output is connected to the original input residual to obtain the DCS-MRFB output.

[0076] DCS replaces complex hybrid shift strategies with directional offsets and removes learnable scaling parameters, relying solely on basic tensor operations to achieve zero-parameter spatial feature enhancement.

[0077] (3) Multi-scale adaptive spatial attention gate MASAG fusion (see appendix) Figure 8 To address the limitations of traditional element-wise addition / channel splicing methods, which use fixed weights and struggle to adaptively handle differences in defect scale and morphology, MASAG is introduced to fuse and refine dual-path features. This involves multi-scale fusion, spatial selection, spatial interaction and cross-modulation, and recalibration.

[0078] Multi-scale fusion: The local branch processes spatial path features X using depthwise separable convolutions and dilated convolutions, while the global branch performs global pooling on semantic path features G along the channel dimension. The two branches are then fused.

[0079] Formula (7) ;

[0080] Spatial selection: U is projected by a 1×1 convolution into two sets of single-channel weight maps, which are then normalized by Softmax and weighted element-wise for X and G, followed by residual connections.

[0081] Formula (8) ;

[0082] Formula (9) ;

[0083] Spatial interaction and cross-modulation: Two feature paths are cross-enhanced with spatial weights activated by Sigmoid (global to local context injection, local to global detail injection), followed by element-wise multiplication.

[0084] Formula (10) , formula (11) ;

[0085] Recalibration: Attention map weights are generated by pointwise convolution and sigmoid activation, and then multiplied element-wise with respect to the input.

[0086] Formula (12) .

[0087] The above design effectively enhances the ability to detect multi-scale defects in photovoltaic cells while maintaining a lightweight architecture.

[0088] The output features after integrating the first spatial path and the first semantic path are used as the output features S3 of the P3 layer.

[0089] (4) Based on this, it goes through two parallel paths: the second spatial path and the second semantic path; the second spatial path is Conv, heterogeneous dual path and efficient module EDCS-MRFB processed in sequence to output features; the second semantic path is DWConv and HGBlock processed in sequence to output features; after being fused by the multi-scale adaptive spatial attention gate MASAG module, it is used as the output feature S4 of layer P4. The output feature S4 of layer P4 goes through two parallel paths again: the third spatial path and the third semantic path, which have the same structure as the second spatial path and the second semantic path, and after being fused by the multi-scale adaptive spatial attention gate MASAG module, it is used as the output feature S5 of layer P5.

[0090] Step 2.2: Construct the MSLA-AIFI scale-based feature interaction module (see Figure 9 and...) Figure 10 )

[0091] MSLA-AIFI removes explicit positional encoding while maintaining the original feedforward network (FFN) and layer normalization (LN) structure, and replaces the standard multi-head self-attention with MSLA. MSLA extracts local structural features from different receptive fields through parallel multi-scale deep convolutional branches, while using linear attention to model global long-range dependencies, achieving a unification of multi-scale local representation and efficient global modeling.

[0092] (1) Multi-scale feature extraction. Given an input feature map X∈R (C×H×W) In this embodiment, the input feature is the output feature S5 of layer P5, which is reshaped into a sequence X′∈R. (N×C) (N=H×W), the feature map is evenly divided into four sub-feature maps along the channel dimension. These sub-feature maps are then input into depthwise convolutional branches with kernel sizes of 3×3, 5×5, 7×7, and 9×9 respectively (small kernels focus on minor defects such as broken gates and printing errors; medium kernels adapt to linear defects such as cracks; and large kernels cover large-area defects such as black cores). Each branch is then connected via residuals and activated by ReLU before outputting:

[0093] Formula (13):

[0094] Formula (14):

[0095] (2) Linear attention computation. After reshaping the multi-scale features of each branch, query, key, and value tensors are generated through linear projection. Efficient attention computation is performed in parallel on each attention head. The outputs of each head are concatenated along the channel dimension and linearly transformed to obtain the branch outputs:

[0096] Formula (15):

[0097] Formula (16):

[0098] Formula (17):

[0099] (3) Learnable weights adaptively weighted. Learnable weights γᵢ are introduced to adaptively weight the features of each branch to adjust the contribution of each scale. After weighting, the features are concatenated by channels and fused with a 1×1 convolution to reshape the final output of MSLA, which serves as an input F5 of the cross-scale feature fusion module EWMB-FPN.

[0100] Formula (18): .

[0101] Step 2.3: Construct the cross-scale feature fusion module EWMB-FPN (see appendix) Figure 1 and Figure 11 )

[0102] The cross-scale feature fusion module EWMB-FPN takes as input features the output features S2, S3, and S4 from layers P2, P3, and P4 of the LDPMGNet dual-path backbone network, and F5 from the intra-scale feature interaction module MSLA-AIFI. EWMB-FPN borrows the multi-directional connectivity and auxiliary fusion ideas of the Multi-Branch Auxiliary Feature Pyramid Network (MAFPN) in its information flow architecture to enrich feature interaction. To address the issue that directly porting MAFPN to the RT-DETR encoder would significantly increase the number of parameters, computational complexity, and inference time, a weighted fusion idea from the Bidirectional Feature Pyramid Network (BIFPN) is introduced in the auxiliary fusion stage. Learnable weights are used to normalize and weight the sum of features from different levels, and ReLU is used to apply a non-negativity constraint, normalizing and integrating the features according to the sum of weights. This achieves efficient adaptive feature fusion while avoiding the significant computational overhead of Softmax.

[0103] Formula (19):

[0104] In the formula, O is the output feature, Iᵢ is the i-th input feature, ε is the stability constant (usually taken as 0.0001), and ωᵢ is the i-th learnable weight.

[0105] Figure 1In this model, weighted fusion connecting the backbone network maintains shallow feature information and enhances small target detection capabilities through bidirectional connections; weighted fusion connecting the output layer enriches the gradient information of the output layer and preserves sufficient localization details to enhance spatial representation through multi-directional connections. Both connections use EDCS-MRFB for photovoltaic defect feature extraction (configuring convolutional kernels of different scales to enhance multi-scale feature expression, and enhancing spatial expression through channel shuffling and directional offset). To balance detection accuracy and computational efficiency, an efficient upsampling module (EUCB) is introduced in the upsampling of the auxiliary fusion and EDCS-MRFB modules: EUCB sequentially performs linear upsampling with a scaling factor of 2, 3×3 depthwise separable convolution, batch normalization, and ReLU activation, and adjusts the number of channels through 1×1 convolution to match the target dimension. Replacing standard convolution with depthwise separable convolution significantly reduces computational overhead while ensuring feature enhancement.

[0106] Step 3: Train the LDPM-DETR detection model and use the trained LDPM-DETR detection model to detect defects in the photovoltaic cell images to be processed.

[0107] This invention was validated on a photovoltaic cell defect dataset (constructed based on the publicly available PVEL-AD dataset, consisting of 10 defect classes) after class equalization and data augmentation. The main experimental environment was NVIDIA GeForce RTX4090 GPU, PyTorch 2.5.0, and CUDA 12.4. The training parameters were: image size 640×640, batch size 16, maximum training 300 epochs, patience value 50, AdamW optimizer, initial learning rate 0.0001, and momentum factor 0.9.

[0108] Training results: (1) Significantly lightweight with improved accuracy. The mAP50 of LDPM-DETR reached 90.6%, which is 0.2 percentage points higher than the baseline RT-DETR-R18 (90.4%). The number of parameters was reduced to 8.9M and the computational cost was reduced to 20.5 GFLOPs, which are only 44.7% and 36.0% of the baseline (19.9M, 57.0 GFLOPs) respectively (i.e., a reduction of 55.3% and 64.0% respectively).

[0109] Load the model weights obtained from the above training, input the photovoltaic cell EL image to be detected, and preprocess it at 640×640. Model forward inference: Backbone network (LDPMGNet) outputs multi-scale features → MSLA-AIFI performs multi-scale local and global interactions within the high semantic layer → EWMB-FPN completes cross-scale weighted fusion → Minimum uncertainty query selection generates the decoder's initial object query → Transformer decoder directly outputs the target category and bounding box coordinates through multiple iterations (end-to-end, no NMS post-processing required), and the confidence threshold and IoU threshold are adjustable.

[0110] Overlay detection bounding boxes onto the original image and label the defect categories and quantities. Output the detection result image and statistical information. (See [link]) Figures 13-15 The end-to-end FPS is 142.8. This means that while reducing both the number of parameters and computational load by more than 55%, a slight improvement in detection accuracy is still achieved, making it more suitable for resource-constrained industrial deployment scenarios.

[0111] Experimental verification:

[0112] (1) Leading compared to mainstream models (horizontal comparison). Compared with YOLO series (m / l specification) and RT-DETR series:

[0113]

[0114] The mAP50 of the LDPM-DETR module of this invention is 10.4 percentage points higher than that of YOLO13l, the best in the YOLO series, and the number of parameters and computational cost are reduced by 66.9% and 75.7% respectively. Compared with RT-DETR-R18, the number of parameters and computational cost are only about 44.7% and 36.0% (that is, the baseline is about 2.24 times and 2.78 times that of this invention). The accuracy is slightly higher than the two models in the RT-DETR series.

[0115] (2) Overall ablation experiment.

[0116] Using RT-DETR-R18 as a baseline, LDPMGNet, MSLA-AIFI, and EWMB-FPN were gradually introduced:

[0117]

[0118] (3) LDPMGNet Internal Ablation—Verifying the Optimal Combination of Heterogeneous Dual Paths. Verification was performed from three dimensions: single-branch, homogeneous dual-branch, and heterogeneous dual-branch.

[0119]

[0120] The results show that the heterogeneous dual-path design with EDCS-MRFB as the main spatial path and HGBlock as the semantic auxiliary path achieves the best mAP50 (90.5%), which is significantly better than other combinations, verifying the effectiveness of the dual-path heterogeneous design and MASAG asymmetric gating fusion.

[0121] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer, characterized in that, include: Acquire images of photovoltaic cell defects to be processed and perform preprocessing; An LDPM-DETR detection model is constructed, which uses RT-DETR-R18 as the baseline model and includes an LDPMGNet dual-path backbone network, an efficient hybrid encoder, and a Transformer decoder. The efficient hybrid encoder includes an intra-scale feature interaction module MSLA-AIFI and a cross-scale feature fusion module EWMB-FPN. MSLA-AIFI removes explicit positional encoding while maintaining the original feedforward network and layer normalization structure, and replaces the standard multi-head self-attention with the MSLA module. The LDPM-DETR detection model is trained, and the trained LDPM-DETR detection model is used to detect defects in the photovoltaic cell images to be processed.

2. The lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 1, characterized in that, The LDPMGNet dual-path backbone network uses heterogeneous dual-path extraction of complementary features: First, a shared input mechanism is set up. Before the dual-path differentiation, a unified shallow feature extraction module is set up to extract shallow features as the output feature S2 of the P2 layer. After the shared input, it is differentiated into two parallel paths: the first spatial path and the first semantic path. The first spatial path is processed sequentially by the heterogeneous dual path and the efficient module EDCS-MRFB, Conv, and the heterogeneous dual path and the efficient module EDCS-MRFB before being output; the first semantic path is processed sequentially by HGBlock, DWConv, and HGBlock before being output; the dual path output features are fused by the multi-scale adaptive spatial attention gate MASAG module and used as the output features S3 of layer P3. The output feature S3 of layer P3 passes through two parallel paths: the second spatial path and the second semantic path. The second spatial path processes the output features sequentially through Conv, the heterogeneous dual path, and the efficient module EDCS-MRFB. The second semantic path processes the output features sequentially through DWConv and HGBlock. After being fused by the multi-scale adaptive spatial attention gate MASAG module, it becomes the output feature S4 of layer P4. The output feature S4 of layer P4 passes through two parallel paths: the third spatial path and the third semantic path. It has the same structure as the second spatial path and the second semantic path. After being fused by the multi-scale adaptive spatial attention gate MASAG module, it becomes the output feature S5 of layer P5.

3. The lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 1, characterized in that, The input features of the cross-scale feature fusion module EWMB-FPN are the output features S2, S3, and S4 of the P2, P3, and P4 layers of the LDPMGNet dual-path backbone network and the F5 output of the intra-scale feature interaction module MSLA-AIFI. It is an improvement based on the multi-branch auxiliary feature pyramid network MAFPN. In the auxiliary fusion stage, the weighted fusion operation of the bidirectional feature pyramid network BIFPN is introduced. The heterogeneous dual-path and high-efficiency module EDCS-MRFB is used to extract photovoltaic defect features. The high-efficiency upsampling module EUCB is introduced in the auxiliary fusion and the upsampling of the heterogeneous dual-path and high-efficiency module EDCS-MRFB.

4. A lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 2 or 3, characterized in that, The heterogeneous dual-path and high-efficiency module EDCS-MRFB combines the CSP network with the directional channel offset multi-receptive field module DCS-MRFB, replacing Bottleneck in the CSP network with the directional channel offset multi-receptive field module DCS-MRFB. The directional channel offset multi-receptive field module DCS-MRFB integrates multi-receptive field convolution and a lightweight directional channel offset mechanism. After the input features are processed by multi-branch depthwise convolution, cross-channel information fusion is performed, and a directional channel offset DCS operation is added to maximize spatial feature interaction with zero parameters. The fused features are divided into four equal parts along the channels, and then cyclically offset along the height and width dimensions in the positive and negative directions and spliced. Finally, the DCS output is concatenated with the residual of the original input features to obtain the DCS-MRFB output.

5. A lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 2, characterized in that, The HGBlock contains multiple cascaded convolutional layers. The output of each layer is concatenated with the original input in the channel dimension to achieve cross-layer feature reuse and aggregation. After aggregation, the high-dimensional features are transformed by two consecutive 1×1 convolutions to output features. Preferably, the gradient propagation is optimized by concatenating the output features with residual connections.

6. A lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 2, characterized in that, The multi-scale adaptive spatial attention gate (MASAG) module includes multi-scale fusion operation, spatial selection operation, spatial interaction and cross-modulation operation, and recalibration operation. (1) Multi-scale fusion: The local branch processes the spatial path feature X with depthwise separable convolution and dilated convolution, and the global branch performs global pooling of the semantic path feature G along the channel dimension. The two branches are then spliced ​​and fused. (2) Spatial selection: The feature U after splicing and fusing the two branches is projected into two sets of single-channel weight maps by convolution. After Softmax normalization, X and G are weighted element by element and residual connection is used to output features X' and G'. (3) Spatial interaction and cross-modulation: The output features X' and G' are enhanced by cross-modulation of the two features with spatial weights activated by Sigmoid, and then multiplied element by element to output feature U'; (4) Recalibration: The output feature U' is generated by pointwise convolution and Sigmoid activation to generate attention map weights, which are then multiplied element-wise on the spatial path feature X.

7. A lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 1, characterized in that, The MSLA module extracts local structural features of different receptive fields through parallel multi-scale deep convolution branches, and each branch is connected by residuals and activated by ReLU to output the features. At the same time, the multi-scale features of each branch are reshaped and global long-range dependencies are modeled using linear attention. The outputs of each attention head are concatenated along the channel dimension and linearly transformed to obtain the outputs of each branch. A learnable weight γᵢ is introduced to adaptively weight the features of each branch. After weighting, the features are concatenated by channels and fused by convolution to reshape the final MSLA output.

8. A lightweight dual-path multi-scale photovoltaic cell defect detection method based on Transformer according to claim 3, characterized in that, The high-efficiency upsampling module EUCB sequentially performs linear upsampling with a scaling factor of 2, 3×3 depthwise separable convolution, batch normalization, and ReLU activation, and adjusts the number of channels to match the target dimension through 1×1 convolution.

Images