Real-time defect detection method for photovoltaic panels based on multi-scale feature fusion
By using a deep learning detection framework that integrates multi-scale features, the contradiction between multi-scale feature representation and anti-interference in complex backgrounds in photovoltaic panel defect detection is resolved, enabling efficient and accurate detection of minute defects and improving detection accuracy and speed.
Patent Information
- Application Number
- CN202611214263.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-11
- Publication Date
- 2026-11-10
AI Technical Summary
Existing photovoltaic panel defect detection technologies suffer from a contradiction between multi-scale feature representation, anti-interference capability in complex backgrounds, and real-time computing efficiency at the edge, making it difficult to efficiently detect minute defects and maintain high accuracy and efficiency in complex environments.
A deep learning detection framework based on multi-scale feature fusion is adopted, including a feature extraction backbone network, a multi-scale feature fusion neck network, and a decoupled detection head. Through a dilated spatial convolutional pooling pyramid module, a dynamically weighted bidirectional feature pyramid, and a spatial and channel decoupled attention module, combined with an improved marginal loss function, efficient detection of defects in photovoltaic panels is achieved.
It significantly improves the detection accuracy and anti-interference ability of micro-defects, reduces the false alarm rate, and increases the detection speed and positioning accuracy, thus meeting the real-time and high-efficiency requirements of photovoltaic panel defect detection.
Smart Images

Figure CN123071256A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and image processing technology, and particularly relates to a real-time defect detection method for photovoltaic panels based on multi-scale feature fusion. Background Technology
[0002] As one of the most representative renewable energy technologies, photovoltaic (PV) power generation is experiencing unprecedented explosive growth in installed capacity and grid share. As the core energy conversion unit of a PV power generation system, PV panels operate in extremely complex outdoor environments, such as deserts, plateaus, or coastal areas. They not only face natural aging risks such as sun exposure, rain, and wind erosion, but are also highly susceptible to damage from hail, bird strikes, and defects such as microcracks and hot spots caused by uneven thermal stress. If these defects are not detected and located in a timely and accurate manner, they can significantly reduce the energy conversion efficiency of the entire array and may even induce localized high temperatures, leading to component damage or fires, causing incalculable economic losses and safety hazards.
[0003] Therefore, efficient, intelligent, and real-time photovoltaic panel defect monitoring has become an important cornerstone for ensuring the safe operation and maintenance of photovoltaic power plants throughout their entire life cycle.
[0004] Early photovoltaic inspections relied primarily on manual inspections, requiring inspectors to walk between arrays using handheld infrared thermal imagers or high-magnification cameras. This method was not only labor-intensive and risky, but also prone to omissions and misjudgments when dealing with ultra-large-scale power plants due to limitations in field of vision and subjective judgment.
[0005] With the deep integration of UAV aerial remote sensing technology and computer vision algorithms, automatic target detection technology based on convolutional neural networks has gradually become the mainstream. Currently, the industry commonly adopts detection frameworks based on deep feature extraction, building target detection models capable of automatically identifying anomalies such as breakage, hot spots, and dirt through offline training on large-scale labeled photovoltaic defect datasets. These existing technologies have indeed demonstrated superior detection performance compared to traditional digital image processing algorithms under standard operating conditions, specifically for images with relatively constant target scale and clean backgrounds, representing a preliminary shift from human experience-driven to data-driven intelligence.
[0006] However, as photovoltaic (PV) operation and maintenance evolves towards greater precision and real-time capabilities, the inherent limitations of existing detection schemes based on conventional convolutional structures at the principle level have gradually exposed significant technical contradictions when dealing with complex and realistic outdoor application scenarios. Specifically, PV panel defects exhibit strong multi-scale characteristics. For example, large-area cover plate damage or severe shading can be clearly identified on low-resolution global feature maps, while microscopic cell cracks, localized solder joint oxidation, or tiny foreign objects obscuring the surface occupy very little space and have extremely weak texture features, often being hidden by complex cell grid lines and reflective backgrounds.
[0007] Existing deep neural networks, when acquiring deep, high-order semantic information through multi-layer downsampling operations, inevitably suffer from the loss of shallow, fine spatial information due to the continuous expansion of the receptive field. This results in severely insufficient robustness of the model when detecting small-scale defects. Conversely, if shallow and deep features are simply fused across layers to preserve spatial details, the shallow feature maps often contain a large amount of background noise signals, such as irregular reflections on glass surfaces or shadows from supports. This lack of discriminative feature fusion often introduces severe signal interference, leading to a sharp increase in the false alarm rate of the model under complex lighting conditions.
[0008] Existing technologies exhibit an irreconcilable negative correlation between "detection accuracy" and "inference speed." To improve the ability to capture multi-scale defects, existing improvement schemes often tend to construct deeper feature extraction backbone networks with a larger number of parameters, or introduce computationally extremely complex attention mechanisms and multi-level feature pyramid structures.
[0009] However, in real-time detection scenarios based on UAV edge computing, limited computing resources and extremely high real-time feedback requirements place almost stringent demands on the algorithm's inference latency. This non-linear increase in model complexity directly leads to a significant decrease in processing frame rate, making it impossible for the system to achieve synchronous defect localization and timely response during high-speed inspections. This deep contradiction between multi-scale feature representation depth, anti-interference capability in complex backgrounds, and edge real-time computing efficiency constitutes the core challenge for the current photovoltaic defect detection field to move towards high performance.
[0010] Therefore, this invention aims to provide a detection method that can efficiently fuse multi-scale features, accurately filter redundant noise, and achieve extremely high computational efficiency, in order to solve the technical problems existing in the prior art. Summary of the Invention
[0011] In response to the technical contradictions mentioned in the background art regarding the difficulty in capturing micro-defect features, severe interference from complex background reflections, and insufficient real-time computing efficiency at the edge side in the field of photovoltaic panel defect detection, this invention provides a real-time photovoltaic panel defect detection method based on multi-scale feature fusion.
[0012] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A real-time defect detection method for photovoltaic panels based on multi-scale feature fusion is implemented using a deep learning real-time detection framework consisting of a feature extraction backbone network, a multi-scale feature fusion neck network, and a decoupled detection head. The method includes the following steps: S1: Feature extraction backbone network is used to extract features from real-time acquired photovoltaic panel images to obtain feature maps of different resolutions. Hollow spatial convolutional pooling pyramid modules are embedded at the end of the feature maps of a specified scale to capture the local details and global contextual semantics of defects, resulting in a multi-scale feature map that integrates local details and global semantics.
[0013] In the feature extraction backbone network, an improved residual connection structure is adopted, replacing the traditional standard convolution with depthwise separable convolution to reduce the number of parameters. Specifically, the first layer of the feature extraction backbone network is set as a convolutional layer with a stride of 2 and a kernel size of 3×3, used to initially downsample the resolution of the input photovoltaic panel image, while increasing the number of channels to 32. In the subsequent feature extraction stages, five feature extraction stages with different downsampling ratios are set, corresponding to 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image size, respectively. To address the information loss caused by subtle defects such as hidden cracks or oxidation of small solder joints, dilated spatial convolution pooling pyramid modules are introduced at the end of the 3rd, 4th, and 5th feature extraction stages, respectively. The dilated spatial convolution pooling pyramid module consists of a 1×1 standard convolutional path, three 3×3 dilated convolutional paths with sampling rates of 6, 12, and 18, and a global average pooling path. The outputs of each path are concatenated along the channel dimension, and the number of channels is compressed to a preset dimension through a 1×1 convolutional layer. The introduction of dilated convolution expands the receptive field without increasing the number of parameters, enabling the model to capture both the local detail texture of defects and the global contextual semantics.
[0014] S2: Input the multi-scale feature map into the multi-scale feature fusion neck network, and perform bidirectional transmission and fusion of deep semantic information and shallow spatial information through a bidirectional feature pyramid structure. After the feature fusion node, the spatial and channel decoupling attention module enhances the features of the defect region.
[0015] The multi-scale feature fusion neck network includes a bidirectional feature pyramid structure based on dynamic weight adjustment. This structure transmits deep semantic information to shallow feature maps via a top-down path and simultaneously transmits shallow spatial information back to deep feature maps via a bottom-up path. Dynamic weight coefficients are introduced at the nodes where feature maps of different scales are fused. The calculation logic of the dynamic weight coefficients is as follows: For the feature map sequence to be fused, a global average pooling operation is first applied to each layer of feature maps to obtain a scalar descriptor. Then, the relative importance score of each layer is calculated through a bottleneck structure composed of two fully connected layers, and normalized exponential function is applied for normalization. The fused feature map is composed of the sum of the weighted products of each layer's feature map and its corresponding dynamic weight. This fusion method eliminates feature redundancy caused by simple addition or cascading operations and effectively suppresses background noise introduced by solar cell reflection or shadow occlusion.
[0016] To further enhance the model's ability to remove non-target interference signals in complex outdoor environments, a spatial and channel-decoupled attention module is embedded after each feature fusion layer of the neck network. This spatial and channel-decoupled attention module consists of parallel channel attention branches and spatial attention branches. The channel attention branch performs max pooling and average pooling on the feature map in the spatial dimension to extract two sets of feature vectors with a length equal to the number of channels. These two sets of vectors are then fed into a multilayer perceptron with shared weights. Finally, the output vectors are summed and processed by a sigmoid activation function to generate a channel weight map. The spatial attention branch performs max pooling and average pooling in the channel dimension to obtain two single-channel two-dimensional feature maps. Dimensionality reduction and feature extraction are then performed through convolutional layers to generate a spatial weight map. Finally, the input feature map is element-wise multiplied with both the channel weight map and the spatial weight map, thereby significantly enhancing the features of the defect region.
[0017] S3: The fused multi-scale features are processed by a decoupled detection head. The classification prediction path and regression prediction path are used to output the category confidence and bounding box position parameters of photovoltaic panel defects, respectively. The regression accuracy is optimized by combining the improved marginal loss function.
[0018] The decoupled detection head adopts an architecture that separates classification and regression tasks to improve the model's localization accuracy. The detection head can contain three prediction branches, each corresponding to a different feature map scale to cover photovoltaic defects of varying sizes. In each prediction branch, the feature map is first processed through two parallel 1×1 convolutional paths: one path predicts the target's class probability, and the other path predicts the target's bounding box offset. x,y,w,hThe model also includes the confidence score of the target. For the bounding box regression task, an improved marginal loss function is used for optimization. This loss function accelerates the convergence speed of the model for extremely slender defect bounding boxes by introducing an aspect ratio penalty term and a center point distance constraint.
[0019] S4: Post-process the prediction results output by the decoupled detection head, use the soft nonmaximum suppression algorithm to remove redundant boxes, and use symmetric quantization based on KL divergence and operator fusion to realize real-time inference and result output of photovoltaic panel defects.
[0020] Preferably, the specific process of step S1 is as follows: S11: Adopting an adaptive scaling strategy, the scaling factor is calculated based on the aspect ratio of the original photovoltaic panel image, and the long side is shrunk to a preset pixel threshold to obtain the standard input image. I ; S12: Downsampling and upsampling based on the first standard convolutional layer: The first layer of the backbone network has a stride of s=2 and a convolutional kernel. k A standard 3×3 convolutional layer is used for a standard input image. I Initial downsampling was performed, increasing the number of channels to 32, and the feature map was output. F 0; S13: Perform five-stage multi-level downsampling feature extraction: The backbone network has five downsampling stages, corresponding to the input image scales of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32. Each stage uses linear bottleneck inverted residuals to extract features, outputting five levels of feature maps: F 1. F 2. F 3. F 4. F 5; S14: In the feature map F 3. F 4. F The ASPP module, which is embedded at the 5th end, consists of 5 parallel paths and outputs an enhanced feature map after fusion. F aspp : S15: Output multi-scale feature maps with embedded ASPP enhancements F 1. F 2. F aspp3 , F aspp4 , F aspp5 The feature is fed into the multi-scale feature fusion neck network in step S2.
[0021] The preferred procedure for extracting features using linear bottleneck inverted residuals at each stage is as follows: 1×1 convolution for dimensionality increase: Fup =Conv 1×1 ( F in ); 3×3 depthwise separable convolution for feature extraction: F depth =DWConv 3×3 ( F up ); 1×1 convolution dimensionality reduction: F out =Conv 1×1 ( F depth ); Residual connection: F res = F in + F out ; Where DWConv is a depthwise separable convolution; F in This is the input feature map for the inverted residual module; F up This is the high-dimensional feature map after dimensionality upgrade; F depth This is the spatial feature map extracted by depthwise convolution; F out The output feature map after dimensionality reduction; F res This is the final output feature map after adding the residuals; Output 5-level feature maps: F 1. F 2. F 3. F 4. F 5.
[0022] Preferably, the calculation flow for the five paths in step S14 is as follows: 1×1 standard convolution path: ; Dilated convolution path: ; ; ; Global average pooling path: ; in, r The sampling rate is the dilated convolution rate; GlobalAvgPool is the global average pooling rate. Feature fusion and dimensionality reduction: The outputs of the five paths are concatenated along the channel dimension and compressed to the target number of channels via a 1×1 convolution. C out : ; Wherein, Concat: concatenation of channel dimensions; C out : Number of channels in the ASPP output feature map.
[0023] Preferably, the specific process of step S2 is as follows: S21: Define the input feature map F in1 ~ F in5 These are the 5-level features output from step S1; S22: Bidirectional feature pyramid fusion processing based on dynamic weight adjustment; S23: Output features for each scale of BiFPN Parallel enhancement of channel attention and spatial attention; S24: Outputs 5-level bidirectional fusion + attention-enhanced multi-scale features: F out1 , F out2 , F out3 , F out4 , F out5 .
[0024] Preferably, the specific process of step S22 is as follows: S221: Bidirectional path construction, including: Top-down approach: Deep, high-semantic features are upsampled via bilinear interpolation and then fused with shallow features after their resolution is aligned. Bottom-up approach: Shallow high-resolution features are downsampled by 3×3 convolutions with a stride of 2 and fused with deep features; S222: Multi-layer features involved in fusion F i1 , F i2 ,..., F in Calculate dynamic weights w i , and complete the weighted fusion.
[0025] Preferably, the specific process of step S23 is as follows: S231: Processed via channel attention branch; S232: Processed via spatial attention branch; S233: Residual connection outputs attention-enhanced feature maps.
[0026] Preferably, the specific process of step S3 is as follows: S31: Incorporating 5-level multi-scale features F out1 , F out2 , F out3 , F out4 , F out5 Assign to different detection branches according to resolution: ; in, C out Output the number of feature channels for step S2; S32: Decoupled detection head dual-path processing: for each scale feature F detk The classification prediction path and regression prediction path are executed in parallel, achieving decoupled learning of category and localization: Classification prediction path: Extract category features through multi-layer convolution and output the confidence distribution of defect categories; ; in, For the first k The class confidence vector of the scaling branch; Conv 3×3 A 3×3 standard convolutional layer; BN is a batch normalization layer; ReLU: modified linear unit activation function; It is a 1×1 convolutional layer with C output channels. cls This maps features to class scores; Softmax: the Softmax activation function; Regression prediction path: Extract localization features through multiple convolutional layers, and output the position and size offset of the bounding box. ; in, For the first k The bounding box prediction vector of the scale branch corresponds to (Δ x ,Δ y ,Δ w ,Δ h ); It is a 1×1 convolutional layer with 4 output channels; Convert the predicted offsets to true bounding box coordinates in the image coordinate system: ; in,( x,y,w,h) represents the decoded true bounding box coordinates, and the center coordinates are ( x,y ),Width w ,high h ;( x anchor , y anchor , w anchor , h anchor ) represents the coordinates and dimensions of the preset anchor frame; (Δ) x ,Δ y ,Δ w ,Δ h ) represents the offset of the regression path output; S33: To resolve the conflict between classification and regression tasks, a weighted combination of classification loss and improved marginal regression loss is used to optimize the detection head parameters. Classification loss: ; in, is the classification loss value; N is the number of samples involved in the loss calculation; α c Balance the weights for each category; Focusing factors for challenging samples; For the first i Sample No. c The prediction confidence of the class; For the first i Sample No. c The true label of the class; Improved marginal regression loss: ; CIoU is: ; in, To improve the marginal regression loss value; b pred b is the decoded predicted bounding box; gt Labels for actual defect bounding boxes; The squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box; c The length of the diagonal of the smallest bounding rectangle containing the two boxes; v The aspect ratio consistency factor; α The aspect ratio weighting coefficient; Marginal penalty weight; IoU min Minimum crossover ratio threshold; Total loss function: ; in, This represents the total loss value. βThese are the classification loss weight coefficients; S34: Output for each detection branch: Defect category confidence level p k ; Decoded bounding box coordinates ( x,y,w,h ); After nonmaximum suppression, the final defect detection results are obtained, including defect category, confidence level, and bounding box.
[0027] Preferably, the specific process of step S4 is as follows: S41: Convert the prediction results (p) output from each detection branch in step S3. k ,b k Aggregate to obtain a set of all candidate boxes. B ={ b i} and their corresponding confidence levels S ={ s i}; S42: Eliminating redundant boxes using soft nonmaximum suppression: Sort candidate boxes B by confidence level from high to low, select the box M with the highest confidence level, and then process the remaining boxes... b i Perform confidence decay: ; in, The first after decay i Each candidate bounding box confidence level; IoU(M, b i () represents the highest confidence bounding box M and the candidate bounding box. b i The intersection and union ratio; σ The attenuation coefficient; Repeat the iteration until the confidence scores of all bounding boxes are lower than the preset confidence threshold τ, thus obtaining the final set of retained detection boxes. B final ; S43: Symmetric quantization is used to compress the model weights and activation values from 32-bit floating-point numbers to 8-bit integers, and the distribution difference before and after quantization is minimized by KL divergence; S44: Merge consecutive operators in the model, including merging convolutional layers, batch normalization layers, and activation layers into a single operator: Original convolution output: y=W x+b ; Normalization: ; Equivalent weights and biases after fusion: ; ; Output after fusion: y relu =ReLU( W ′ x + b ′); in, W,b These represent the weights and biases of the original convolutional layer, respectively; γ ,β These are the scaling and offset parameters for the batch normalization layer, respectively. μ,σ 2 These are the mean and variance of the batch normalized layer, respectively. W′,b′ These are the weights and biases of the equivalent convolutional layer after fusion; S45: Outputs the preprocessed photovoltaic panel image, defect detection results, and post-processing result analysis. The post-processing result analysis maps the detection box back to the original image coordinate system and labels the defect category, confidence level, and location.
[0028] Preferably, the specific process of step S43 is as follows: S431: For floating-point data x Quantized to INT8: ; in, x FP32 data to be quantized; x q For quantized INT8 data; round( ) is the rounding function; s This is the quantization scaling factor; Dequantization restores floating-point values: ; in, x f This is the dequantized FP32 data; S432: Find the optimal quantization scaling factor s *, to make the quantized distribution P q Compared with the original distribution P f The KL divergence is minimal: ; The KL divergence is defined as: ; Among them, DKL ( ) represents the KL divergence; P f The probability distribution of the original FP32 data; P qThis represents the probability distribution of the quantized INT8 data. x i For the th in the distribution i Data points.
[0029] The beneficial effects of this invention include: 1. Significantly enhanced ability to capture minute defects and higher detection accuracy: By embedding the Spatial Convolutional Pooling Pyramid (ASPP) module into the feature extraction backbone network, the receptive field is greatly expanded without increasing the number of parameters. It can simultaneously capture local details of minute defects such as microcracks, micro-cracks, and solder joint oxidation in photovoltaic panels, as well as global semantic information of large-area hot spots and damage. This effectively solves the problem of loss of subtle features caused by downsampling in traditional networks, and greatly improves the detection rate of minute defects.
[0030] 2. Strong anti-interference capability against complex backgrounds and significantly reduced false alarm rate: The bidirectional feature pyramid with dynamic weight adjustment is adopted to achieve adaptive weighted fusion of deep semantic and shallow spatial information, which can automatically suppress background noise such as photovoltaic glass reflection, grid texture, and bracket shadow. At the same time, with the spatial and channel decoupled attention module, it can accurately focus on defect areas and weaken interference information, maintaining high stability under complex outdoor conditions such as strong light, dirt, and shading, and significantly reducing false detection and false alarm rates.
[0031] 3. A decoupled detection head is adopted, which separates defect classification and bounding box regression into independent learning paths to avoid mutual interference between tasks; and an improved marginal loss function is used to introduce center point distance constraints and aspect ratio consistency penalties, which makes the convergence of defects such as slender hidden cracks and linear scratches faster and the positioning more accurate, thus greatly improving the accuracy of bounding box regression.
[0032] 4. The backbone network adopts a linear bottleneck inverted residual structure and depthwise separable convolution, which significantly reduces the number of model parameters and computational cost. Combined with engineering optimizations such as KL divergence-based symmetric quantization and operator fusion, and memory reuse, the model is compressed from FP32 to INT8 fixed-point computation, which greatly improves the inference speed. Attached Figure Description
[0033] Figure 1 This is a flowchart illustrating the real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to the present invention. Figure 2 This is a schematic diagram of the overall architecture of the detection framework of the present invention; Figure 3 This is a schematic diagram of the structure of the hollow spatial convolutional pooling pyramid module of the present invention; Figure 4 This is a schematic diagram of the spatial and channel decoupling attention module of the present invention; Figure 5 This is a schematic diagram of the decoupled detection head of the present invention. Detailed Implementation
[0034] The following is in conjunction with the appendix Figures 1-5The present invention will be further described in detail below: Example 1 See Figures 1 to 5 As shown, a real-time defect detection method for photovoltaic panels based on multi-scale feature fusion is implemented using a deep learning real-time detection framework consisting of a feature extraction backbone network, a multi-scale feature fusion neck network, and a decoupled detection head. The feature extraction backbone network is responsible for extracting multi-dimensional physical features from the input high-definition photovoltaic image. The multi-scale feature fusion neck network enhances the model's representation ability for minor defects through the interaction of deep and shallow features. The decoupled detection head performs the final category determination and bounding box regression. The method includes the following steps: S1: The feature extraction backbone network is used to perform multi-level downsampling feature extraction on the real-time acquired photovoltaic panel images. Feature maps of different resolutions are obtained through the improved residual connection structure. A holed spatial convolutional pooling pyramid module is embedded at the end of the feature map at a specified scale to capture the local details and global contextual semantics of defects, resulting in a multi-scale feature map that integrates local details and global semantics.
[0035] When processing the input raw photovoltaic panel image, the feature extraction backbone network first employs a standard convolutional layer with a stride of 2 and a kernel size of 3×3. The main function of this layer is to perform initial feature downsampling, compressing the spatial dimension of the image to half its original size, while simultaneously expanding the channel dimension from RGB three channels to 32 channels. Following this layer, the feature extraction backbone network performs deep feature mining through an improved residual connection structure. This improved residual connection structure uses inverted residuals with a linear bottleneck. The specific computational flow is as follows: first, a 1×1 convolutional layer increases the number of channels in the input feature map to a preset dilation factor; then, a 3×3 depthwise separable convolution is applied for spatial feature extraction; finally, a 1×1 convolutional layer compresses the number of channels back to the target dimension. It is worth noting that after the final 1×1 dimensionality-reducing convolution, no nonlinear activation function is set in this embodiment to avoid information collapse during the projection of high-dimensional features into low-dimensional space. Depthwise separable convolution decomposes standard convolution into channel-wise convolution and point-wise convolution, which significantly reduces computational cost while maintaining the same receptive field.
[0036] The feature extraction backbone network 1 is divided into five stages, corresponding to downsampling rates of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image size, respectively. At the output ends of the third stage (1 / 8 scale), fourth stage (1 / 16 scale), and fifth stage (1 / 32 scale), a dilated spatial convolutional pooling pyramid module is embedded. Further, this dilated spatial convolutional pooling pyramid module consists of five parallel paths: the first path is a standard 1×1 convolutional path used to preserve the semantics of the original scale; the second, third, and fourth paths are 3×3 dilated convolutional paths with sampling rates of 6, 12, and 18, respectively, which multiply the feature capture range without increasing the weights by inserting zero values between the convolutional kernel parameters; the fifth path is a global average pooling path, responsible for obtaining global statistical information of the entire feature map. The outputs of these five paths are concatenated along the channel dimension, and a 1×1 convolutional layer is used to compress the number of channels to a preset feature thickness. This structure ensures that the network can simultaneously cover large-area hot spot defects and microcrack features on the photovoltaic panel.
[0037] S2: The multi-scale feature map is input into the multi-scale feature fusion neck network. The deep semantic information and shallow spatial information are transmitted and fused in both directions through a bidirectional feature pyramid structure based on dynamic weight adjustment. After the feature fusion node, a spatial and channel decoupling attention module is applied to enhance the features of the defect region.
[0038] In the multi-scale feature fusion neck network, a bidirectional feature pyramid structure based on dynamic weight adjustment is adopted. The core logic of this structure is that feature maps of different scales do not contribute equally to the detection of different types of defects. To achieve optimal feature fusion, dynamic weight coefficients are introduced at each fusion node. Before performing the fusion operation, for each layer of feature maps participating in the fusion, channel descriptors of each layer are first extracted through global average pooling. Then, these descriptors are fed into a bottleneck module consisting of two fully connected layers to calculate the relative importance score of each layer's feature map. These scores are processed by a normalized exponential function to ensure that the sum of all weights participating in the fusion is 1. The final fused feature map is obtained by performing element-wise multiplication of each layer's feature map with its corresponding dynamic weight and then summing the results. This mechanism effectively solves the feature redundancy problem of traditional feature pyramids when dealing with reflective interference from photovoltaic cells, and significantly improves the signal-to-noise ratio of the target features by suppressing the weights of noise-level layers.
[0039] At the output of each layer of the bidirectional feature pyramid structure, a spatial and channel-decoupled attention module is integrated. This module enhances defect regions through parallel channel attention branches and spatial attention branches. Specifically, the channel attention branch first performs global max pooling and global average pooling on the input feature map, generating two sets of one-dimensional feature vectors. These two sets of vectors then share a multilayer perceptron containing two hidden layers. The output of the multilayer perceptron is element-wise summed and then mapped to a weight range between 0 and 1 using the sigmoid function to adjust the response intensity of each channel of the feature map. Simultaneously, the spatial attention branch performs max pooling and average pooling on the feature map along the channel dimension, generating two single-channel two-dimensional spatial maps, which are then concatenated. To reduce computational cost and maintain the receptive field, this embodiment replaces the traditional 7×7 convolutional kernel with two cascaded 3×3 convolutional kernels. The generated spatial attention map is activated by the sigmoid function and then multiplied element-wise with the feature map. The channel attention branch and the spatial attention branch work together to enable the model to automatically focus on the geometrically anomalous areas on the photovoltaic panel, eliminating the interference of periodic grid lines in the background.
[0040] Because visible light images captured in high-altitude photovoltaic inspection scenarios by drones contain a large number of micro-scale defects (microcracks, micro-solder oxidation, fine scratches) + a small number of large-area shadows / dirt, the large and small defects are severely mixed; strong light reflection and strong noise in the cell grid texture.
[0041] This invention adds a scale prior mask constraint to the dynamic weighted fusion of bidirectional feature pyramid feature fusion nodes. By pre-calculating the defect pixel size distribution based on a photovoltaic defect dataset, three defect scale intervals (small, medium, and large) are defined, and a fixed-scale prior mask matrix is pre-generated. During the multi-feature weighted fusion at each layer of BiFPN, the original dynamic weights are multiplied element-wise with the scale prior mask. Feature weights matching the defect size at the current layer are amplified, while weights for mismatched scales are attenuated, achieving decoupling of scale features and eliminating cross-scale interference between large and small defects. The specific process is as follows: Statistical analysis of offline photovoltaic defect datasets, and construction of scale prior masks: Collect all defect ground truth bounding box labels from the photovoltaic annotation dataset. Calculate the area of a single defect pixel. The dataset contains all types of defects, including microcracks, solder joint oxidation, micro-scratches (small scale), localized hot spots, solder ribbon blackening (medium scale), whole component breakage, large-area hot spots, and shadow occlusion (large scale).
[0042] Defining three-level defect pixel scale threshold intervals: Combining photovoltaic image preprocessing with uniform long-side scaling to a standard input size of 640 pixels, statistically analyzing the total defect area distribution, and dividing fixed intervals: Small-scale defect S: This represents hidden cracks, minor solder joint oxidation, and fine scratches; corresponding to shallow, high-resolution feature maps. ; Mesoscale defect M: This represents localized hot spots, localized overheating of the solder strip, and small pieces of dirt; corresponding to the mid-layer feature map. ; Large-scale defects L: This represents a broken solar panel, continuous hot spots, and large areas of obstruction; corresponding to a deep, low-resolution feature map. ; Generate scale prior mask matrix The mask is a one-dimensional vector with a length equal to the number of feature layers involved in the fusion of the current BiFPN fusion node. The range of values for mask elements Value selection rules: Feature index matching the target scale at this level: mask value = 1.0 (preservation and amplification weights); Feature indices that do not match the current scale: mask value = 0.2 (suppression and attenuation weights); Masking Example: A shallow fusion node is only responsible for detecting small-scale defects. There are three layers of features involved in the fusion (shallow F2, middle F3, deep F4). Only F2 matches small-scale defects. Therefore, the small-scale prior mask is... The mesoscale mask corresponding to the mid-layer fusion node. Deep fusion node large-scale mask .
[0043] BiFPN hierarchy and scale mask binding: BiFPN outputs a total of 5 levels of features. Bind the corresponding scale mask according to the resolution: Shallow branching ( ): Binding small-scale prior masks Only the weights of minor defect features are strengthened; Middle-level branches ( ): Binding mesoscale prior mask It focuses on medium-sized localized hot spots; Deep branching ( ): Binding large-scale prior masks Priority should be given to preserving features of large-area damage and continuous hot spots.
[0044] BiFPN fusion nodes calculate the original unconstrained dynamic weights. (Real-time computation of network forward inference): For any fusion node, there exists Zhang Waiting to be Fusion Feature Map Original dynamic weight calculation process: For each input feature map Perform global average pooling (GAP) to compress the two-dimensional feature map into a one-dimensional channel descriptor. ; Will The input is fed into a two-layer fully connected bottleneck network, which outputs a single-layer feature importance score. ; For all scores Performing Softmax normalization yields the original unconstrained dynamic weights: ; The original weights only reflect the information content of the feature itself and do not distinguish the defect scale, which may lead to the problem that the weights of large-scale features squeeze out the features of small defects.
[0045] Scale-prior mask constraints are used to generate the final effective fusion weights. : The obtained original weight vector The scale prior mask vector bound to the current level Perform element-wise multiplication to fulfill the scale constraint: ; in: : No. The final fusion weights of layer features after scale prior constraints; : Mask The numerical value has a matching scale of 1.0 and an irrelevance scale of 0.2.
[0046] Secondary normalization after multiplication: To ensure that the sum of all weights is 1, for Perform normalization again: ; Example of constraint effect: Shallow small-scale fusion node, original dynamic weight calculation results: (The weights of features in the middle and deep layers are too high, causing minor interference defects); Binding mask Element-by-element multiplication yields Final weights after double normalization ; The weights of irrelevant scale features in the middle and deep layers, which were originally equal, were significantly reduced, while the weights of features that match minor defects in the shallow layer were significantly amplified, thus solving the problem of mutual suppression of cross-scale features.
[0047] Multi-feature weighted fusion based on scale-constrained weights: using the normalized final weights. The fused feature map is output by performing a channel-wise weighted summation on all input feature maps. : ; Features that match the current level of defect scale: high weight, complete preservation of defect texture and edge information; Features with mismatched scales (such as deep large-image features of shallow fusion nodes): weights are suppressed by the mask, and the large-area background reflection and grid line noise are simultaneously weakened; the mutual interference of defects of different sizes is isolated from the source of feature fusion.
[0048] Post-decoupling attention module scale noise secondary filtering: weighted fusion output The input is fed into a spatial-channel decoupled attention module as a supplementary optimization to the scale prior constraint: Channel attention branch: automatically suppresses useless channel responses corresponding to irrelevant scale features; Spatial attention branch: weakens background regions in the image that do not conform to the current layer scale, such as shallow branches filtering large areas of reflective patches and deep branches filtering fine grid line textures. The residual connection outputs the final enhanced feature. It is then fed into the subsequent decoupled detection head.
[0049] S3: The fused multi-scale features are processed by a decoupled detection head. The classification prediction path and regression prediction path are used to output the category confidence and bounding box position parameters of photovoltaic panel defects, respectively. The regression accuracy is optimized by combining the improved marginal loss function.
[0050] The decoupled detection head employs a prediction task separation architecture. Feature extraction results are input into three different prediction scale branches, corresponding to feature resolutions of 85×85, 42×42, and 21×21, respectively. Each branch contains a classification prediction path and a regression prediction path. The classification prediction path determines the category of defects within the predicted bounding box, while the regression prediction path performs refined offset prediction of the center coordinates and dimensions of the target bounding box. For extremely elongated shapes present in photovoltaic defects, such as specific solder strip offsets or long, strip-shaped scratches, an improved marginal loss function is introduced into the regression prediction path. This loss function, based on the traditional intersection-union loss, adds a penalty term for the distance between the center point of the predicted box and the ground truth box, and introduces a constraint term for aspect ratio consistency, thereby significantly accelerating the model's convergence speed under extreme aspect ratio conditions.
[0051] S4: Post-process the prediction results output by the decoupled detection head, use the soft nonmaximum suppression algorithm to remove redundant boxes, and use symmetric quantization processing based on KL divergence and operator fusion technology to realize real-time inference and result output of photovoltaic panel defects.
[0052] Considering the imbalance of positive and negative samples in the photovoltaic defect dataset and the difficulty in capturing minute defects, a dynamic weight allocation scheme was adopted in the loss function design. For targets with a pixel area smaller than 1024 square pixels, the resulting loss value was multiplied by a gain coefficient of 1.5, forcing the model to tilt towards minute defects during the optimization process. Furthermore, the data augmentation stage introduced specialized simulations for photovoltaic inspection scenarios, including simulating flare noise caused by direct sunlight at different altitudes, motion blur effects caused by drone vibration, and grid moiré interference at specific frequencies. This multi-dimensional noise injection strategy enhanced the model's robustness under real-world complex conditions.
[0053] In another embodiment, an adaptive scaling strategy is adopted in the image preprocessing stage during actual photovoltaic power station inspections. This strategy first reads the aspect ratio of the original image. If the original resolution is 5472×3648, a scaling factor is calculated to align the long side to 640 pixels, while the short side is scaled proportionally and padded with grayscale values to ensure the image enters the network without geometric distortion. In the post-processing stage of the detection head, the system employs a soft non-maximum suppression (Soft-NMS) algorithm. Unlike traditional hard thresholding, Soft-NMS uses a Gaussian decay function to gradually reduce the confidence of candidate boxes based on their overlap. For two candidate boxes with high overlap, if both are defect targets, Soft-NMS can effectively avoid classifying one as a redundant box, thereby reducing the false negative rate in defect-dense areas. The specific process of step S1 is as follows: S11: Preprocessing the photovoltaic panel image: Adaptive scaling strategy is adopted. The scaling factor is calculated according to the aspect ratio of the original photovoltaic panel image. The long side is shrunk to a preset pixel threshold (640 or 1280 pixels), and the short side is minimized and padded to avoid geometric distortion, thus obtaining the standard input image. I ∈ R H×W×3 H: preprocessed image height; W: preprocessed image width; 3: RGB three channels; S12: Downsampling and upsampling based on the first standard convolutional layer: The first layer of the backbone network has a stride of s=2 and a convolutional kernel. k A standard 3×3 convolutional layer is used for a standard input image. I Initial downsampling was performed, increasing the number of channels to 32, and the feature map was output. F 0: ; Among them, Conv 3×3,s=2 : Standard convolution with a 3×3 kernel and a stride of 2 s The convolution stride; b 0: First-layer convolutional bias term; σ: Non-linear activation function (ReLU); S13: Perform five-stage multi-level downsampling feature extraction: The backbone network has five downsampling stages, corresponding to the input image scales of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32. Each stage uses linear bottleneck inverted residuals to extract features, outputting five levels of feature maps: F 1. F 2. F 3. F 4. F 5.
[0054] The specific process for extracting features using linear bottleneck inverted residuals at each stage is as follows: 1×1 convolution for dimensionality increase: F up =Conv 1×1 ( F in ); 3×3 depthwise separable convolution for feature extraction: F depth =DWConv 3×3 ( F up ); 1×1 convolution dimensionality reduction (without activation function): F out =Conv 1×1 ( F depth ); Residual connection: If the input / output channels and dimensions are the same, F res = F in + F out ; Where DWConv is a depthwise separable convolution; Output 5-level feature maps: F 1. F 2. F 3. F 4. F 5. Corresponding scale: ; ; ; ; ; C1~C5: Number of feature map channels at each stage.
[0055] S14: In stages 3, 4, and 5 ( F 3.F 4. F 5) An ASPP module is embedded at the end, consisting of 5 parallel paths, which are fused to output an enhanced feature map. F aspp : The calculation process for the five paths in step S14 is as follows: 1×1 standard convolution path: ; Dilated convolution paths (sampling rates 6, 12, 18): ; ; ; Global average pooling path: ; in, r The dilated convolution sampling rate can be 6, 12, or 18; GlobalAvgPool is the global average pooling. Feature fusion and dimensionality reduction: The outputs of the five paths are concatenated along the channel dimension and then compressed to the target number of channels using a 1×1 convolution. C out : ; Wherein, Concat: concatenation of channel dimensions; C out The number of channels in the ASPP output feature map can be 256 or 512.
[0056] S15: Output multi-scale feature maps with embedded ASPP enhancements F 1. F 2. F aspp3 , F aspp4 , F aspp5 The feature is fed into the multi-scale feature fusion neck network in step S2.
[0057] Example 2 Based on Example 1, the specific process of step S2 is as follows: S21: Define the input feature maps (in descending resolution): ; in, F in1 ~ F in5 These are the 5-level features output from step S1; S22: Bidirectional Feature Pyramid (BiFPN) fusion processing based on dynamic weight adjustment: S221: Bidirectional path construction: Top-down approach: Deep, high-semantic features are upsampled using bilinear interpolation and then fused with shallow features after resolution alignment. ; in, For the first i The layer outputs feature maps from top to bottom, Upsample bilinear This is a bilinear upsampling operation; For the first i +1 layer, top-down path input feature map; Bottom-up approach: Shallow high-resolution features are downsampled using a 3×3 convolution with a stride of 2 and then fused with deep features: ; in, For the first i The layer outputs feature maps from the bottom up path; For the first i The input feature map is a bottom-up path in layer 1.
[0058] S222: Dynamic Weight Calculation and Feature Fusion: Multi-layer Features Participating in Fusion F i1 , F i2 ,..., F in Calculate dynamic weights w i Complete weighted fusion: scalar descriptors are generated using global average pooling: g i =GlobalAvgPool( F i )∈ R C×1×1 ; in, g i For the first i Global AvgPool is a global average pooling descriptor for layer features. It is a global average pooling operation that averages all spatial locations of the feature map and compresses it into single-pixel features. F i For the first i The layer input feature map; R is the real number space, and the feature values are 32-bit floating-point numbers; C Number of channels; Importance score of bottleneck fully connected layer computation: ; in, s i Feature importance score; Softmax normalization yields dynamic weights w i : ; ; Dynamic weighted feature fusion: ; in, This is a dynamically weighted fusion feature.
[0059] S23: Spatial and Channel Decoupled Attention Module (SCDA) Feature Enhancement: Output features of BiFPN at each scale Parallel channel attention and spatial attention enhancements are performed, with residual connections to ensure gradient stability. The specific process of step S23 is as follows: S231: Processed via Channel Attention Branch (CA): ; ; ; ; in, s c Channel attention weight map; s s Spatial attention weight map; S232: Processed via spatial attention branch (SA): ; ; ; ; ; S233: Residual connection output attention-enhanced feature map: ; in, For attention-enhanced feature maps; For element-wise multiplication; Activated for Sigmoid or ReLU.
[0060] S24: Outputs 5-level bidirectional fusion + attention-enhanced multi-scale features: F out1 ,F out2 , F out3 , F out4 , F out5 .
[0061] The specific process of step S3 is as follows: S31: Incorporating 5-level multi-scale features F out1 , F out2 , F out3 , F out4 , F out5 Assign to different detection branches according to resolution: ; in, C out The number of feature channels output for step S2 can be 256 or 512. Different branches correspond to the detection of defects of different sizes, including small defects, medium defects, and large defects. S32: Decoupled detection head dual-path processing: for each scale feature F detk The classification prediction path and regression prediction path are executed in parallel, achieving decoupled learning of category and localization: Classification prediction path (category confidence output): Extract category features through multi-layer convolution and output the defect category confidence distribution; ; in, For the first k The class confidence vector of the scaling branch; Conv 3×3 A 3×3 standard convolutional layer; BN is a batch normalization layer; ReLU: modified linear unit activation function; It is a 1×1 convolutional layer with C output channels. cls The feature is mapped to a class score; Softmax: The Softmax activation function normalizes the scores to a class probability distribution, satisfying... ; Regression prediction path (bounding box position parameter output): Extracts localization features through multiple convolutional layers, and outputs the position and size offset of the bounding box. ; in, For the first k The bounding box prediction vector of the scale branch corresponds to (Δ x ,Δ y,Δ w ,Δ h ); It is a 1×1 convolutional layer with 4 output channels, which maps features to bounding box offsets; Convert the predicted offsets to true bounding box coordinates in the image coordinate system: ; in,( x,y,w,h ) represents the decoded true bounding box coordinates, and the center coordinates are ( x,y ),Width w ,high h ;( x anchor , y anchor , w anchor , h anchor (Δ) represents the coordinates and dimensions of the preset anchor frame, predefined according to different scales, corresponding to the detection branches; x ,Δ y ,Δ w ,Δ h ) represents the offset of the regression path output, and is the normalized predicted value, derived from... supply; S33: To resolve the conflict between classification and regression tasks, a weighted combination of classification loss and improved marginal regression loss is used to optimize the detection head parameters. Classification loss (FocalLoss, which alleviates class imbalance): ; in, The classification loss value measures the difference between the predicted class and the true label; N is the number of samples involved in the loss calculation, which is the total number of positive samples and hard-to-bear samples. α c To balance the weights of different categories and alleviate the imbalance of defect categories, the background category has a lower weight. The focusing factor for difficult samples can be set to 2, while reducing the weight of easy samples. For the first i Sample No. c The prediction confidence of the class; For the first i Sample No. c The true label of the class, in one-hot encoded form, with 1 for the correct category and 0 for the rest; Improved Marginal Regression Loss (CIoUoss + Marginal Constraints): A marginal penalty term is introduced on top of the CIoU loss to improve the localization accuracy of the defect bounding box. ; CIoU is defined as: ; in, To improve the marginal regression loss value, measure the difference in localization between the predicted bounding box and the actual bounding box; b pred The decoded predicted bounding box is obtained by outputting and decoding the regression path; b gt The labels are the actual defect bounding boxes, provided by the data annotation; IoU is the intersection / union ratio, which is the intersection / union of the predicted box and the actual box; The center offset is measured by the squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box. c is the length of the diagonal of the smallest bounding rectangle containing the two boxes, and is the normalized center distance; v The aspect ratio consistency factor measures the difference in aspect ratio between the predicted bounding box and the ground truth bounding box. A higher value indicates a greater difference in aspect ratio. , w gt The width of the actual bounding box, and the width of the labeled defect bounding box, are provided by the data labels; h gt The height of the actual bounding box, and the height of the labeled defect bounding box, are provided by the data labels; w pred The predicted bounding box width is the width of the predicted bounding box output and decoded from the regression path of the detection head. h pred The height of the predicted bounding box is the output of the regression path of the detection head and the height of the predicted bounding box after decoding. α This is the aspect ratio weighting coefficient, used to balance the aspect ratio and positional error; it can be set to 0.5. The marginal penalty weight can be set to 0.1~0.5; IoU min Additional penalties are applied to samples with an IoU below the minimum intersection-union ratio (MIU) threshold to improve localization accuracy; Total loss function: ; in, This is the total loss value, used for backpropagation to optimize the detection head parameters; β This is the classification loss weight coefficient, which can be set to 0.5, used to balance the gradients of classification and regression tasks; S34: Output for each detection branch: Defect category confidence level p k ; Decoded bounding box coordinates ( x,y,w,h ); After nonmaximum suppression (NMS), the final defect detection results (category, confidence, bounding box) are obtained.
[0062] Example 3 Based on Example 1 or Example 2, the specific process of step S4 is as follows: S41: Aggregate multi-scale prediction results: Aggregate the prediction results (p) output from each detection branch in step S3. k ,b k Aggregate to obtain a set of all candidate boxes. B ={ b i} and their corresponding confidence levels S ={ s i}; b i =( x i ,y i ,w i ,h i ): No. i The coordinates and dimensions of each candidate box; s i =p i,c : No. i The confidence level of each candidate box corresponding to the defect category. c For the predicted category; S42: Eliminating redundant boxes using Soft Non-Maximum Suppression (Soft-NMS): Traditional NMS directly deletes overlapping boxes, easily missing dense defects; Soft-NMS replaces direct deletion by attenuating confidence levels, retaining defect boxes with high overlap but suboptimal confidence, making it more suitable for dense defect scenarios in photovoltaic panels. Sort candidate boxes B in descending order of confidence level, select the box M with the highest confidence level, and then sort the remaining boxes... b i Perform confidence decay: ; in, The first after decay i A candidate bounding box has a confidence level that, after decay, still exceeds a preset confidence threshold τ; the bounding box is retained. b i () represents the highest confidence bounding box M and the candidate bounding box. b i The intersection-over-union ratio is used to measure the degree of overlap between frames; σ The attenuation coefficient can be set to 0.5 to control the confidence attenuation rate. Repeat the iteration until the confidence level of all bounding boxes is lower than the preset confidence threshold τ, which can be taken from 0.001 to 0.3, to obtain the final set of retained detection boxes. B final ; S43: Symmetric Quantization Processing Based on KL Divergence: To adapt to the computing power limitations of embedded devices, symmetric quantization is used to compress the model weights and activation values from 32-bit floating-point numbers (FP32) to 8-bit integers (INT8). KL divergence is used to minimize the distribution difference before and after quantization, ensuring that the loss of detection accuracy is controllable. S431: For floating-point data x Quantized to INT8: ; in, x FP32 data to be quantized; x q The quantized INT8 data has a value range of [ 128,127];round( ) is a rounding function that maps floating-point values to INT8 integers; s The scaling factor is determined by KL divergence optimization, with an initial value of s = max(| x| ) / 127; Dequantization restores floating-point values: ; in x f This is the FP32 data after dequantization, and the approximate value after quantization error correction. S432: Scaling Factor Optimization Based on KL Divergence: The core of quantization is finding the optimal quantization scaling factor. s *, to make the quantized distribution P q Compared with the original distribution P f The KL divergence is minimal: ; The KL divergence is defined as: ; Among them, DKL ( KL divergence (relative entropy) is used to measure the difference between two probability distributions; the smaller the value, the closer the distributions are. P f The probability distribution of the original FP32 data is obtained from the statistical histogram of model weights or activation values; P q This represents the probability distribution of the quantized INT8 data. x i For the th in the distribution i Each data point corresponds to the center value of the quantization interval.
[0063] S44: Operator fusion optimization: To reduce memory access overhead and computational latency, continuous operators in the model are fused, including merging convolutional layers, batch normalization layers, and activation layers into a single operator, reducing memory read / write operations on intermediate features. Original convolution output: y=W x+b ; Normalization: ; Equivalent weights and biases after fusion: ; ; Output after fusion: y relu =ReLU( W ′ x + b ′); Where W and b are the original convolutional layer weights and biases, respectively, and are model parameters obtained from FP32 training; γ ,β These are the scaling and offset parameters for the batch normalization layer, which are learned during training. μ,σ 2 , respectively, are the mean and variance of the batch normalization layer, and are moving statistics on the training set; W′,b′ , respectively, are the weights and biases of the equivalent convolutional layer after fusion, and are the equivalent parameters of the fusion operator; S45: Output preprocessed photovoltaic panel image (scaling, normalization, format conversion), defect detection results, and post-processing result analysis: map the detection box back to the original image coordinate system, and label the defect category, confidence level, and location.
Claims
1. A real-time defect detection method for photovoltaic panels based on multi-scale feature fusion, characterized in that, The method is based on a deep learning real-time detection framework consisting of a feature extraction backbone network, a multi-scale feature fusion neck network, and a decoupled detection head, and includes the following steps: S1: Use the feature extraction backbone network to extract features from real-time acquired photovoltaic panel images, obtain feature maps of different resolutions, and embed a holed spatial convolutional pooling pyramid module at the end of the feature map of a specified scale to capture the local details and global contextual semantics of defects, thereby obtaining a multi-scale feature map that integrates local details and global semantics. S2: Input the multi-scale feature map into the multi-scale feature fusion neck network, and perform bidirectional transmission and fusion of deep semantic information and shallow spatial information through a bidirectional feature pyramid structure. After the feature fusion node, the spatial and channel decoupling attention module is used to enhance the features of the defect region. S3: The fused multi-scale features are processed by a decoupled detection head. The classification prediction path and regression prediction path are used to output the category confidence and bounding box position parameters of photovoltaic panel defects, respectively. The regression accuracy is optimized by combining the improved marginal loss function. S4: Post-process the prediction results output by the decoupled detection head, use the soft nonmaximum suppression algorithm to remove redundant boxes, and use symmetric quantization based on KL divergence and operator fusion to realize real-time inference and result output of photovoltaic panel defects.
2. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 1, characterized in that, The specific process of step S1 is as follows: S11: Adopting an adaptive scaling strategy, the scaling factor is calculated based on the aspect ratio of the original photovoltaic panel image, and the long side is shrunk to a preset pixel threshold to obtain the standard input image. I ; S12: Downsampling and upsampling based on the first standard convolutional layer: The first layer of the backbone network has a stride of s=2 and a convolutional kernel. k A standard 3×3 convolutional layer is used for a standard input image. I Initial downsampling was performed, increasing the number of channels to 32, and the feature map was output. F 0; S13: Perform five-stage multi-level downsampling feature extraction: The backbone network has five downsampling stages, corresponding to the input image scales of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32. Each stage uses linear bottleneck inverted residuals to extract features, outputting five levels of feature maps: F 1. F 2. F 3. F 4. F 5; S14: In the feature map F 3. F 4. F The ASPP module, which is embedded at the 5th end, consists of 5 parallel paths and outputs an enhanced feature map after fusion. F aspp : S15: Output multi-scale feature maps with embedded ASPP enhancements F 1. F 2. F aspp3 , F aspp4 , F aspp5 The feature is fed into the multi-scale feature fusion neck network in step S2.
3. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 2, characterized in that, The specific process for extracting features using linear bottleneck inverted residuals at each stage is as follows: 1×1 convolution for dimensionality increase: F up =Conv 1×1 ( F in ); 3×3 depthwise separable convolution for feature extraction: F depth =DWConv 3×3 ( F up ); 1×1 convolution dimensionality reduction: F out =Conv 1×1 ( F depth ); Residual connection: F res = F in + F out ; Where DWConv is a depthwise separable convolution; F in This is the input feature map for the inverted residual module; F up This is the high-dimensional feature map after dimensionality upgrade; F depth This is the spatial feature map extracted by depthwise convolution; F out The output feature map after dimensionality reduction; F res This is the final output feature map after adding the residuals; Output 5-level feature maps: F 1. F 2. F 3. F 4. F 5.
4. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 2, characterized in that, The calculation process for the five paths in step S14 is as follows: 1×1 standard convolution path: ; Dilated convolution path: ; ; ; Global average pooling path: ; in, r The sampling rate is the dilated convolution rate; GlobalAvgPool is the global average pooling rate. Feature fusion and dimensionality reduction: The outputs of the five paths are concatenated along the channel dimension and compressed to the target number of channels via a 1×1 convolution. C out : ; Wherein, Concat: concatenation of channel dimensions; C out : Number of channels in the ASPP output feature map.
5. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 1, characterized in that, The specific process of step S2 is as follows: S21: Define the input feature map F in1 ~ F in5 These are the 5-level features output from step S1; S22: Bidirectional feature pyramid fusion processing based on dynamic weight adjustment; S23: Output features for each scale of BiFPN Parallel enhancement of channel attention and spatial attention; S24: Outputs 5-level bidirectional fusion + attention-enhanced multi-scale features: F out1 , F out2 , F out3 , F out4 , F out5 .
6. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 5, characterized in that, The specific process of step S22 is as follows: S221: Bidirectional path construction, including: Top-down approach: Deep, high-semantic features are upsampled via bilinear interpolation and then fused with shallow features after their resolution is aligned. Bottom-up approach: Shallow high-resolution features are downsampled by 3×3 convolutions with a stride of 2 and fused with deep features; S222: Multi-layer features involved in fusion F i1 , F i2 ,..., F in Calculate dynamic weights w i , and complete the weighted fusion.
7. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 5, characterized in that, The specific process of step S23 is as follows: S231: Processed via channel attention branch; S232: Processed via spatial attention branch; S233: Residual connection outputs attention-enhanced feature maps.
8. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 5, characterized in that, The specific process of step S3 is as follows: S31: Incorporating 5-level multi-scale features F out1 , F out2 , F out3 , F out4 , F out5 Assign to different detection branches according to resolution: ; in, C out Output the number of feature channels for step S2; S32: Decoupled detection head dual-path processing: for each scale feature F detk The classification prediction path and regression prediction path are executed in parallel, achieving decoupled learning of category and localization: Classification prediction path: Extract category features through multi-layer convolution and output the confidence distribution of defect categories; ; in, For the first k The class confidence vector of the scaling branch; Conv 3×3 A 3×3 standard convolutional layer; BN is a batch normalization layer; ReLU: modified linear unit activation function; It is a 1×1 convolutional layer with C output channels. cls This maps features to class scores; Softmax: the Softmax activation function; Regression prediction path: Extract localization features through multiple convolutional layers, and output the position and size offset of the bounding box. ; in, For the first k The bounding box prediction vector of the scale branch corresponds to (Δ x ,Δ y ,Δ w ,Δ h ); It is a 1×1 convolutional layer with 4 output channels; Convert the predicted offsets to true bounding box coordinates in the image coordinate system: ; in,( x,y,w,h ) represents the decoded true bounding box coordinates, and the center coordinates are ( x,y ),Width w ,high h ;( x anchor , y anchor , w anchor , h anchor ) represents the coordinates and dimensions of the preset anchor frame; (Δ) x ,Δ y ,Δ w ,Δ h ) represents the offset of the regression path output; S33: To resolve the conflict between classification and regression tasks, a weighted combination of classification loss and improved marginal regression loss is used to optimize the detection head parameters. Classification loss: ; in, is the classification loss value; N is the number of samples involved in the loss calculation; α c Balance the weights for each category; Focusing factors for challenging samples; For the first i Sample No. c The prediction confidence of the class; For the first i Sample No. c The true label of the class; Improved marginal regression loss: ; CIoU is: ; in, To improve the marginal regression loss value; b pred b is the decoded predicted bounding box; gt Labels for actual defect bounding boxes; The squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box; c The length of the diagonal of the smallest bounding rectangle containing the two boxes; v The aspect ratio consistency factor; α The aspect ratio weighting coefficient; Marginal penalty weight; IoU min Minimum crossover ratio threshold; Total loss function: ; in, This represents the total loss value. β These are the classification loss weight coefficients; S34: Output for each detection branch: Defect category confidence level p k ; Decoded bounding box coordinates ( x,y,w,h ); After nonmaximum suppression, the final defect detection results are obtained, including defect category, confidence level, and bounding box.
9. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 8, characterized in that, The specific process of step S4 is as follows: S41: Convert the prediction results (p) output from each detection branch in step S3. k ,b k Aggregate to obtain a set of all candidate boxes. B ={ b i } and their corresponding confidence levels S ={ s i }; S42: Eliminating redundant boxes using soft nonmaximum suppression: Sort candidate boxes B by confidence level from high to low, select the box M with the highest confidence level, and then process the remaining boxes... b i Perform confidence decay: ; in, The first after decay i Each candidate bounding box confidence level; IoU(M, b i () represents the highest confidence bounding box M and the candidate bounding box. b i The intersection and union ratio; σ The attenuation coefficient; Repeat the iteration until the confidence scores of all bounding boxes are lower than the preset confidence threshold τ, thus obtaining the final set of retained detection boxes. B final ; S43: Symmetric quantization is used to compress the model weights and activation values from 32-bit floating-point numbers to 8-bit integers, and the distribution difference before and after quantization is minimized by KL divergence; S44: Merge consecutive operators in the model, including merging convolutional layers, batch normalization layers, and activation layers into a single operator: Original convolution output: y=W x+b ; Normalization: ; Equivalent weights and biases after fusion: ; ; Output after fusion: y relu =ReLU( W ′ x + b ′); in, W,b These represent the weights and biases of the original convolutional layer, respectively; γ ,β These are the scaling and offset parameters for the batch normalization layer, respectively. μ, σ 2 These are the mean and variance of the batch normalized layer, respectively. W′,b′ These are the weights and biases of the equivalent convolutional layer after fusion; S45: Outputs the preprocessed photovoltaic panel image, defect detection results, and post-processing result analysis. The post-processing result analysis maps the detection box back to the original image coordinate system and labels the defect category, confidence level, and location.
10. The real-time defect detection method for photovoltaic panels based on multi-scale feature fusion according to claim 9, characterized in that, The specific process of step S43 is as follows: S431: For floating-point data x Quantized to INT8: ; in, x FP32 data to be quantized; x q For quantized INT8 data; round( ) is the rounding function; s This is the quantization scaling factor; Dequantization restores floating-point values: ; in, x f This is the dequantized FP32 data; S432: Find the optimal quantization scaling factor s *, to make the quantized distribution P q Compared with the original distribution P f The KL divergence is minimal: ; The KL divergence is defined as: ; Among them, DKL ( ) represents the KL divergence; P f The probability distribution of the original FP32 data; P q This represents the probability distribution of the quantized INT8 data. x i For the th in the distribution i Data points.
