Multi-scale feature fusion photovoltaic panel defect detection method
Through multi-scale feature fusion technology, combined with deep learning and image processing methods, the problems of low efficiency, high cost and poor accuracy of photovoltaic panel defect detection are solved, and efficient and accurate detection and positioning are achieved in complex environments, reducing detection costs .
Patent Information
- Application Number
- CN202411216294.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-22
AI Technical Summary
The existing photovoltaic panel defect detection methods have low detection efficiency, high cost, poor accuracy and strong personnel dependence, making it difficult to effectively locate and identify defects in complex environments.
Using multi-scale feature fusion photovoltaic panel defect detection methods, including image acquisition, geometric correction, feature extraction, feature fusion and defect detection steps, feature extraction and fusion using CSPDarknet53 network and improved GFPN, combined with DySample dynamic upsampling and SPD volumes Product, the model parameters are optimized using adaptive non-maximum suppression and sliding weighted loss function based on exponential moving average.
It significantly improves the accuracy and efficiency of photovoltaic panel defect detection, enhances the robustness and generalization capabilities of the model, and can efficiently and accurately detect and locate defects in complex environments, reducing detection costs.
Smart Images

Figure CN119006438A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to defect detection technology in a photovoltaic system, and in particular to a photovoltaic panel defect detection method based on multi-scale feature fusion, which is particularly suitable for small-size defect detection of photovoltaic panels in complex environments. Background Art
[0002] Photovoltaic technology, as an important part of the renewable energy field, has made remarkable progress in recent years. As the core component of the photovoltaic power generation system, the quality of photovoltaic panels is directly related to the power generation efficiency and service life of the entire system. However, photovoltaic panels will inevitably have various defects in long-term operation, such as scratches, broken grids and dirt. These defects will not only affect the photoelectric conversion efficiency of photovoltaic panels, but may also cause rapid degradation of their performance and even cause safety accidents.
[0003] Existing photovoltaic defect detection methods are mainly divided into two categories: physical property detection and image detection. Physical property detection methods mainly detect defects by measuring the electrical and mechanical properties of photovoltaic panels. These methods mainly include: current detection, ultrasonic detection, resonance detection, etc. Traditional image detection methods image the surface of photovoltaic panels and detect defects using digital image processing technology. These methods include: electroluminescence imaging detection, visible light imaging detection and infrared thermal imaging detection methods.
[0004] However, there are many problems with the above traditional photovoltaic panel defect detection methods:
[0005] (1) Low detection efficiency: Traditional detection methods often require a lot of time for manual operation and data processing. For example, current detection and ultrasonic detection require measuring each photovoltaic panel one by one, which is time-consuming and laborious; while electroluminescence imaging and infrared thermal imaging require shooting in a specific environment, which takes a long time. In addition, the complexity of the detection equipment and the cumbersome operation further reduce the detection efficiency. (2) High detection cost of traditional methods: Both physical property detection methods and image detection methods require expensive equipment and professional operators. For example, ultrasonic detection and resonance detection require special instruments, while electroluminescence imaging and infrared thermal imaging require high-performance cameras and image processing systems. The purchase and maintenance costs of these devices are high, and they also require regular calibration and maintenance. In addition, the labor cost and time cost during the detection process cannot be ignored. (3) Poor accuracy: The accuracy of traditional detection methods is often limited by many factors. For example, current detection and resonance detection are sensitive to changes in the external environment and are easily interfered with; the detection results of visible light imaging and infrared thermal imaging are easily affected by lighting conditions and background noise, resulting in false detection and missed detection. In addition, traditional detection methods perform poorly when dealing with complex defects and small-sized defects, and the reliability and stability of the detection results need to be improved. (4) Strong dependence on personnel: Traditional detection methods often require professional technicians to operate and analyze data, which not only increases the dependence on personnel, but also leads to subjectivity and inconsistency in the detection results. The experience and skill levels of different operators will affect the accuracy and consistency of the detection results, and human errors in operation will also lead to unreliable detection results. In addition, the shortage of professionals and training costs also limit the widespread application of traditional detection methods.
[0006] These problems limit the environmental adaptability and use conditions of traditional photovoltaic panel defect detection methods, resulting in inaccurate positioning, low recognition efficiency, and high use costs of existing photovoltaic panel defect detection methods in complex environments. In order to overcome the above problems, image detection methods based on deep learning have gradually become a research hotspot in recent years. By utilizing deep learning technology, automated detection systems can achieve efficient, accurate, and low-cost photovoltaic defect detection and reduce dependence on professionals. These methods not only improve detection efficiency and accuracy, but also significantly reduce detection costs and human errors, providing strong support for efficient maintenance and management of photovoltaic systems.
[0007] Prior art CN 118100797 A discloses a photovoltaic module fault detection method, but the patent fails to systematically solve the complex problems in large-scale photovoltaic module detection. Summary of the invention
[0008] In view of the shortcomings of existing photovoltaic defect detection algorithms, such as low detection efficiency, high detection cost of traditional methods, poor accuracy and strong dependence on personnel, the present invention provides a photovoltaic panel defect detection method with multi-scale feature fusion, so as to effectively locate and identify defects of photovoltaic panels in complex environments, significantly improve the accuracy and real-time performance of detection, and reduce the detection cost.
[0009] To achieve the above object, the present invention adopts the following technical solutions:
[0010] A photovoltaic panel defect detection method based on multi-scale feature fusion, comprising:
[0011] 1) Photovoltaic panel image acquisition step: use high-precision cameras, surveillance cameras and other equipment to photograph photovoltaic panels. Photovoltaic panel image acquisition is a key step in the entire inspection process. By obtaining high-quality image data, it can provide a reliable foundation for subsequent defect detection.
[0012] 2) Data preprocessing step, geometric correction and data enhancement processing are performed on the collected photovoltaic panel images. The photovoltaic panel images are geometrically corrected using the image registration algorithm to correct the distortion in the image, and the collected photovoltaic panel images are rotated, scaled, flipped, and other enhanced using the image processing tool OpenCV library. Specifically, the image is geometrically transformed using known geometric calibration parameters to correct the straight lines and planes in the image. This process can eliminate perspective distortion in the image and ensure the geometric accuracy of the image. Use the functions in the OpenCV library to adjust the collected photovoltaic panel images to generate image data with different installation angles, different directions, and consistent sizes. This process can enhance the model's adaptability to photovoltaic panels in different directions and installation angles, and improve the model's detection performance and robustness.
[0013] 3) Feature extraction step: CSPDarknet53 network extracts features of photovoltaic panel images. Specifically, CSPDarknet53 introduces deep separable convolution and cross-cascade network. Deep separable convolution and cross-cascade network enable the model to process large-scale image data more efficiently, improve overall computing efficiency and speed, more accurately extract detailed features in photovoltaic panel images, and improve the accuracy and speed of model detection.
[0014] 4) Feature fusion step: Use the improved GFPN (Generalized Feature Pyramid Network, Generalized-FPN) to effectively fuse the extracted low-level feature information and high-level feature information. Specifically, by introducing DySample dynamic upsampling, the upsampling process is reconstructed from the perspective of point sampling, and the sampling point position is dynamically adjusted according to the content of the input feature map, which has higher flexibility and can better preserve the detail information in the input feature map. By replacing the traditional convolution layer with SPD convolution, combining the space-to-depth layer and the non-stride convolution layer, it is possible to retain fine-grained feature information and more important details, and finally achieve full fusion of the extracted features.
[0015] 5) Defect detection step: In the layered detection strategy, each layer of feature maps is responsible for detecting targets of different scales. By adding high-resolution layers to the detection head, the model's accuracy for small-scale defects is improved. In the post-processing stage, the adaptive non-maximum suppression method is used to dynamically adjust the intersection-over-union ratio threshold to adapt to changes in target density, thereby better retaining valuable detection results and improving the accuracy and recall rate of target detection.
[0016] 6) Parameter optimization step, define a sliding weighted loss function based on exponential moving average, and use this loss function to optimize the network parameters of the improved YOLOv8 model. The sliding weighted loss function based on exponential moving average can balance the influence of historical and current observation data while focusing on difficult samples during training, making the loss value smoother and more stable, and enhancing the robustness and generalization ability of the model.
[0017] The image geometric correction method is specifically described as follows:
[0018] The geometric correction method is an image processing technology that can effectively eliminate image distortion caused by shooting conditions and equipment factors. First, the original image is captured by a high-precision camera or surveillance camera. Then, the optical characteristics and position direction of the camera are calculated by calibrating the intrinsic and extrinsic parameters of the camera, and a distortion model is established. Next, according to the calibrated radial distortion coefficient and tangential distortion coefficient, the image is geometrically corrected, including the correction of radial distortion and tangential distortion. Finally, image registration is performed to ensure that multiple images are aligned in the same coordinate system, and perspective transformation correction is performed to make the oblique image look like it is shot vertically from directly above. Through these steps, the photovoltaic panel image is distortion-free and accurately aligned, providing reliable image data for subsequent defect detection and feature extraction.
[0019] The CSPDarknet53 network extracts the initial features of the photovoltaic panel image, and the specific contents are as follows:
[0020] CSPDarknet53 is a Darknet53 network combined with the Cross Stage Partial (CSP) mechanism. Darknet53 is a deep convolutional neural network that was originally used in the YOLOv3 target detection model. By introducing the CSP mechanism, CSPDarknet53 reduces the amount of computation and improves the performance of the network while maintaining efficient feature extraction capabilities.
[0021] 1) Input feature extraction: Extract the initial feature map from the preprocessed photovoltaic panel image. This feature map is obtained through the initial convolutional layer in the CSPDarknet53 network and contains the basic feature information of different regions in the image.
[0022] 2) CSP (Cross Stage Partial) module: The CSP module is applied to the initial feature map. The CSP module divides the feature map into two parts, one part is directly passed to the subsequent layer, and the other part is merged with the directly passed part after passing through multiple residual blocks. The introduction of the CSP module reduces repeated calculations and optimizes feature fusion, thereby effectively improving the network's computational efficiency and feature expression capabilities.
[0023] 3) Residual block: Residual blocks are used in the CSP module. The residual block consists of two consecutive convolutional layers and a skip connection, which can capture more feature details and reduce the gradient vanishing problem, thus improving the training effect and feature extraction ability of the model.
[0024] 4) Multi-scale feature output mechanism: The feature maps generated by the network at different levels capture information of different scales. The low-level feature maps contain low-level features with rich details (such as edges and textures), while the high-level feature maps contain high-level features with richer semantics (such as shapes and structures). The CSPDarknet53 network outputs feature maps at different levels, laying the foundation for the subsequent fusion of feature information of different scales.
[0025] The multi-scale feature fusion network based on GFPN is specifically as follows:
[0026] GFPN (Generalized Feature Pyramid Network, Generalized-FPN) fully exchanges feature information of different scales through a more efficient queen fusion mechanism and skip-layer connections. The queen fusion mechanism performs feature fusion at multiple levels to enhance the information interaction between feature maps of different scales. By using different channel dimension settings for feature maps of different scales, the balance between computational efficiency and detection accuracy is improved. In the feature fusion process, an efficient layer aggregation network mechanism is introduced. Through effective layer aggregation connections, the representation ability of feature maps is enhanced, and the efficiency and performance of feature fusion are improved.
[0027] The DySample dynamic upsampling is as follows:
[0028] Traditional upsampling methods use fixed rules for interpolation, ignoring the semantic information in the feature map. DySample dynamic upsampling reconstructs the upsampling process from the perspective of point sampling, that is, for each pixel in the image, its value and position are dynamically adjusted during the upsampling process to better preserve and utilize the details and features in the input image, so that the feature map can be interpolated in a more flexible way. The main process includes generating the initial sampling grid, generating offsets, pixel rearrangement, and resampling. It avoids the problem of detail loss in traditional methods and effectively improves the efficiency of the model and the ability to detect small-sized defects.
[0029] The specific contents of the SPD convolution are as follows:
[0030] SPD convolution aims to solve the problem that traditional stride convolution and pooling may be ignored when downsampling feature maps, resulting in the loss of important detail information. It includes space-to-depth layers and non-stride convolution layers. The space-to-depth layer divides the input feature map into multiple sub-feature maps and concatenates them in the channel dimension, retaining all pixel information in the channel dimension, and achieving downsampling without information loss. The non-stride convolution layer reduces the number of channels through learnable convolution kernels to ensure that as many discriminative features as possible are retained.
[0031] The adaptive non-maximum suppression method is specifically described as follows:
[0032] The adaptive non-maximum suppression method first calculates the target density around each detection box based on the number and distribution of detection boxes. Then, according to the calculated target density, the intersection-over-union ratio threshold of the corresponding prediction box is dynamically adjusted. A larger intersection-over-union ratio threshold is used when the target density is large, and a smaller intersection-over-union ratio threshold is used when the target density is small. When applying adaptive non-maximum suppression, each prediction box is compared and screened using its corresponding dynamic intersection-over-union ratio threshold. This can avoid excessive deletion of detection boxes in target-dense areas while maintaining high accuracy in target-sparse areas.
[0033] The sliding weighted loss function based on exponential moving average is specifically as follows:
[0034] The sliding weighted loss function emphasizes the importance of difficult samples by assigning higher weights to them, so that the model can pay more attention to these difficult samples during training. On this basis, the idea of exponential moving average is introduced to smooth and dynamically adjust the intersection-over-union ratio to solve the problem that the weight adjustment of difficult samples in the actual training process may be unstable due to fluctuations in the loss value.
[0035] Queen fusion mechanism,Queen fusion is a feature fusion method proposed in the generalized feature pyramid network (GFPN, Generalized-FPN). Function: Enable the network to capture features of different spatial scales and effectively fuse them, significantly improving the detection accuracy of small and large targets while maintaining computational efficiency. Queen fusion imitates the flexible movement of the queen pieces in chess, using 1x1 convolution, bilinear interpolation and maximum pooling operations to effectively integrate features from different spatial scales and semantic levels at each feature layer, thereby promoting the dense exchange of high-level semantic information and low-level spatial information, and enhancing the network's ability to detect targets of different scales.
[0036] The present invention can effectively improve the accuracy and efficiency of photovoltaic panel defect detection and improve the performance and life of the photovoltaic system through the above-mentioned technical means. The present invention improves the feature fusion efficiency and the detection capability of small-size defects by improving the feature fusion network and the loss function, while enhancing the robustness and generalization ability of the model. The system can realize efficient and accurate detection and positioning of photovoltaic panel defects in complex environments, significantly improve the accuracy and real-time performance of detection, and is suitable for automated monitoring and maintenance management of photovoltaic panel defects. Specifically, the following steps are included: photovoltaic panel image acquisition step, data preprocessing step, feature extraction step, feature fusion step, defect detection step and parameter optimization step.
[0037] Specifically, the present invention has the following advantages:
[0038] 1. Multi-scale feature fusion is more efficient
[0039] This patent adopts an improved GFPN method, which improves the fusion efficiency of multi-scale features by effectively fusing low-level features and high-level features, especially when dealing with defects of different scales, it can provide more accurate detection results. In the comparison document, the RAM-YOLOv8 algorithm used mainly focuses on the local adjustment of the model structure and lacks systematic multi-scale feature fusion optimization.
[0040] 2. Stronger robustness and generalization ability
[0041] This patent uses a sliding weighted loss function based on the idea of exponential moving average, so that the model can balance the influence of historical and current observation data while paying attention to difficult samples, thereby improving the robustness and generalization ability of the model. In addition, for photovoltaic panels in different directions and installation angles, the detection performance and robustness of the model are improved through data enhancement and geometric correction. The comparative documents rely on traditional training methods and lack similar enhancement methods, so they may not perform as well as this patent under complex backgrounds and different lighting conditions.
[0042] 3. Higher detection accuracy and efficiency
[0043] This patent uses a combination of DySample dynamic upsampling and SPD convolution in the feature fusion step to better preserve the detail information in the input feature map. Although the comparative document also introduces the Transformer mechanism, it is slightly inferior to this patent in terms of detail preservation and dynamic adjustment. In addition, the introduction of the Transformer mechanism will lead to high consumption of computing resources, and a large amount of training data is required to fully learn effective feature expressions.
[0044] 4. More adaptable non-maximum suppression method
[0045] The adaptive non-maximum suppression method in this patent can dynamically adjust the intersection-over-union ratio threshold according to the target density, improving the detection accuracy in dense target areas. The comparative document uses the traditional non-maximum suppression method, which fails to optimize in dense target areas and may perform poorly in dense defect detection scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 Flowchart of photovoltaic panel defect detection based on multi-scale feature fusion.
[0047] Figure 2 Schematic diagram of feature extraction.
[0048] Figure 3 Schematic diagram of feature fusion based on improved GFPN.
[0049] Figure 4 DySample dynamic upsampling.
[0050] Figure 5 SPD convolution structure diagram.
[0051] Figure 6 Sliding weighted loss function graph.
[0052] Figure 7 Architecture diagram of the detection network based on the improved YOLOv8 multi-scale feature fusion. DETAILED DESCRIPTION
[0053] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0054] Photovoltaic panel defect detection is a key technology in photovoltaic systems. The purpose of this invention is to improve the defect recognition efficiency of photovoltaic panels in photovoltaic modules and improve the efficiency and life of photovoltaic systems. The present invention adopts multi-scale feature fusion technology to more accurately identify and locate defects, avoiding the problems of low detection efficiency, high cost, poor accuracy and strong dependence on personnel in traditional methods.
[0055] This embodiment provides a photovoltaic panel defect detection method and system with multi-scale feature fusion, such as Figure 1 As shown, the specific steps include:
[0056] 1) Photovoltaic panel image acquisition steps:
[0057] Install high-precision cameras and surveillance cameras on stable brackets to ensure that they can cover the entire photovoltaic panel area. The camera should be installed at an appropriate height and angle to obtain the best viewing angle. Generally, high-precision cameras and surveillance cameras should be perpendicular to the surface of the photovoltaic panel or slightly tilted to reduce reflection interference. During the acquisition process, in order to ensure the comprehensiveness and coverage of the image, the photovoltaic panel needs to be photographed from multiple angles and directions. By adjusting the angle and position of the camera, the front, side and oblique angle images of the photovoltaic panel are obtained, so as to fully capture the surface features and possible defects of the photovoltaic panel.
[0058] 2) Data preprocessing steps:
[0059] In order to improve the quality of photovoltaic panel images and enhance the significance of photovoltaic panel surface defects, this embodiment uses an image geometric correction method to eliminate photovoltaic panel image distortion and uses the OpenCV library to perform data enhancement on photovoltaic panel images. The specific steps are as follows:
[0060] Calibration of camera internal and external parameters: First, use a high-precision camera or surveillance camera to take multiple images containing the calibration plate, and use these images to calculate the camera's internal and external parameters, including: focal length, principal point coordinates, rotation matrix, translation vector). The formula is as follows:
[0061] x=f·X / Z
[0062] y=f·Y / Z
[0063] Among them, x and y are the coordinates of the feature point in the imaging plane coordinate system; f represents the focal length of a device such as a camera or a surveillance camera; X, Y, and Z represent the three-dimensional coordinates of the feature point in the camera coordinate system.
[0064] u=(f x *x)+x 0
[0065] v=(f y ·y)+y 0
[0066] Among them, u and v are the coordinates of the feature point in the pixel plane coordinate system; f x and f y is the number of pixels per unit distance in the X-axis and Y-axis directions of the imaging plane; 0 and 0 is the coordinate of the origin O of the imaging plane coordinate system in the pixel coordinate system.
[0067] Establish a distortion model: Establish a distortion model based on the internal and external parameters obtained by camera calibration, including the correction of radial distortion and tangential distortion. The formula is as follows:
[0068] x c =x(1+k 1 r 2 +k 2 ·r 4 +k 3 ·r 6 )+2p 1 ·x·y+p 2 ·(r 2 +2 x 2 )
[0069] y c =y(1+k 1 r 2 +k 2 r 4 +k 3 r 6 )+2p 2 xy+p 1 ·(r 2 +2y 2 )
[0070] Among them, r 2 =x 2 +y 2 , x and y are the coordinates of the feature point in the imaging plane coordinate system; c and c is the coordinate of the corrected feature point in the imaging plane coordinate system; k 1 , k 2 , k 3 is the radial distortion coefficient, p 1 , p 2 is the tangential distortion coefficient, and r is the distance from the origin O of the imaging plane coordinate system to the feature point.
[0071] Image registration: Register the geometrically corrected images to ensure that multiple images are aligned in the same coordinate system. The perspective transformation correction formula is as follows:
[0072]
[0073] Among them, (u′, v′) is the pixel coordinates of the feature point after perspective transformation correction, (u, v) is the coordinates of the feature point in the pixel plane coordinate system. A, B, C, D, E, F, G, H are the parameters of the perspective transformation matrix.
[0074] Data enhancement: Perform data enhancement on the geometrically corrected and registered images to increase the diversity and robustness of the data. Data enhancement methods include image rotation, scaling, flipping, and cropping. The specific operation formula is as follows:
[0075] Rotation: Randomly rotate the image by a certain angle to generate images in different orientations.
[0076] I′(x, y)=I(x·cosθ)-y·sinθ, xsin·θ+ycos·θ)
[0077] Among them, I′(x,y) is the photovoltaic panel image after data enhancement, I(x,y) is the image after geometric correction and registration, and θ is the rotation angle.
[0078] Scaling: Randomly scale the image size to generate images of different sizes.
[0079] I′(x, y) = I(sx, sy)
[0080] Where s is the scaling factor.
[0081] Flip: Randomly flip the image to generate a mirror image.
[0082] I′(x,y)=I(Wx,y)
[0083] Where W is the image width.
[0084] 3) Feature extraction step:
[0085] In order to more accurately detect defects in photovoltaic panel images, the following Figure 2 The CSPDarknet53 network shown in the figure performs initial feature extraction. The network combines deep separable convolution and cross-cascade networks, allowing the network model to process large-scale image data more efficiently, improving the overall computing efficiency and speed, more accurately extracting detailed features in photovoltaic panel images, and improving detection accuracy and speed. The specific steps are as follows:
[0086] Input image preprocessing: First, the preprocessed photovoltaic panel image is input into the CSPDarknet53 network. The image size is uniformly adjusted to 256×256 pixels to meet the input requirements of the network.
[0087] Extract the initial features of the input image: Extract the initial feature map from the preprocessed photovoltaic panel image. This feature map is obtained through the initial convolution layer in the CSPDarknet53 network and contains the basic feature information of different regions in the image. The formula is as follows:
[0088] F init Conv(I)
[0089] Among them, F init is the initial feature map, Conv(·) represents the convolution operation, and I is the input image.
[0090] CSP module: The CSP module is applied to the initial feature map. The CSP module divides the feature map into two parts, one part is directly passed to the subsequent layer, and the other part is merged with the directly passed part after passing through multiple residual blocks. Its structural diagram is as follows:
[0091] F CSP =Concat(F direct , F res )
[0092] Among them, F direct is the feature map directly transferred, F res is the feature map processed by the residual block, and Concat(·) represents the concatenation operation of the feature map.
[0093] Residual block: Residual blocks are used in the CSP module. The residual block consists of two consecutive convolutional layers and a skip connection, which can capture more feature details and reduce the gradient vanishing problem. Its calculation formula is as follows:
[0094] Fresidual=ReLU(F init +Conv 2 (ReLU(Conv 1 (F init ))))
[0095] Among them, Conv 1 and Conv 2 They represent two convolution operations respectively, ReLU is the activation function, and Fresidual is the residual feature map.
[0096] Depthwise separable convolution: In the feature extraction process, depthwise separable convolution is used to reduce the amount of calculation and parameters. Its calculation is divided into two steps: depthwise convolution and pointwise convolution.
[0097] Depthwise Convolution:
[0098]
[0099] Among them, I is the input feature map, K is the convolution kernel, i, j are spatial position indexes, k is the channel index, and m is the convolution kernel size index.
[0100] Point-wise convolution:
[0101]
[0102] Among them, W is a 1×1 convolution kernel and l is the output channel index.
[0103] Multi-scale feature output mechanism: The feature maps generated by CSPDarknet53 at different levels capture information of different scales. The low-level feature maps contain low-level features with rich details (such as edges and textures), while the high-level feature maps contain high-level features with richer semantics (such as shapes and structures).
[0104] 4) Feature fusion steps:
[0105] Use Figure 3 The improved GFPN shown in FIG. performs effective feature fusion on the extracted low-level feature information and high-level feature information. Figure 4 The DySample dynamic upsampling shown reconstructs the upsampling process from the perspective of point sampling, dynamically adjusting the sampling point position based on the content of the input feature map to better preserve the detail information in the input feature map. The specific steps are as follows:
[0106] For the input feature map X∈C×H×W, an initial sampling grid is first generated, which is evenly distributed on the input feature map using bilinear interpolation. After that, the input feature map X passes through two linear layers, each of which projects the feature map X to a different dimension to generate a feature map of size 2gs2×H×W to generate an offset O, as follows:
[0107] O=linear(X)
[0108] The offset is then reshaped to 2×sH×sW by pixel shuffling. The dynamic range factor is generated by the input feature map through linear projection and sigmoid function, ranging from 0 to 0.5, and the formula is as follows:
[0109] O = 0.5sigmoid (linear 1 (X)·linear 2 (X)
[0110] Finally, the offset O is added to the initial sampling grid G to obtain the final sampling set S, and the input feature map X is resampled to obtain the sampled feature map.
[0111] Introduce Figure 5 The SPD convolution shown replaces the traditional convolution layer, which includes a spatial-to-depth layer and a non-strided convolution layer. The specific steps are as follows:
[0112] The spatial-to-depth layer achieves lossless downsampling by splitting the input feature map into multiple sub-feature maps and concatenating them in the channel dimension. For a given feature map X∈S×S×C1, the SPD layer slices it into multiple sub-feature maps. Assuming the scale factor is scale, the SPD layer splits X into scale 2 sub-feature maps, the size of each sub-feature map is Then, these sub-feature maps are concatenated in the channel dimension to obtain a new feature map X', whose size is By retaining all pixel information in the channel dimension, downsampling is achieved without information loss.
[0113] After the spatial-to-depth layer, a non-strided convolutional layer with a kernel size of 3×3 and a stride of 1 is added. The role of this convolutional layer is to reduce the number of channels through a learnable convolutional kernel to ensure that as many discriminative features as possible are retained.
[0114] Through the above feature fusion method, fine-grained feature information and more important details can be retained, and finally the extracted features can be fully fused, so that the model can more accurately detect and identify small defects on photovoltaic panels and show higher robustness in complex background and noise environments.
[0115] 5) Defect detection steps:
[0116] In the layered detection strategy, each layer of feature maps is responsible for detecting targets of different scales. By adding high-resolution layers to the detection head, the model's accuracy for small-scale defects is improved. In the post-processing stage, the adaptive non-maximum suppression method is used to dynamically adjust the intersection-over-union ratio threshold to adapt to changes in target density, thereby better retaining valuable detection results and improving the accuracy and recall rate of target detection.
[0117] The specific steps are as follows:
[0118] First, the target density around each detection box is calculated. For each prediction box b, a local area is defined, and the number of other prediction boxes d in the local area is calculated. The target density can be simply defined as the number of prediction boxes d in the local area. The density formula at the i-th object is defined as follows:
[0119]
[0120] Among them, d i represents the density of target i, and G is the set of real target boxes.
[0121] According to the calculated target density, dynamically adjust the intersection-over-union ratio threshold of the corresponding prediction box. When the target density is large, use a larger intersection-over-union ratio threshold. When the target density is small, use a smaller intersection-over-union ratio threshold. Threshold definition:
[0122] N M :=max(N t , d M )
[0123] Among them, N M represents the suppression threshold of M in adaptive non-maximum suppression, d M is the density of the target M, N t is a fixed non-maximum suppression threshold.
[0124] The adaptive non-maximum suppression algorithm is used to process the detection frame. The suppression strategy formula is as follows:
[0125]
[0126] Among them, s i is the detection box b i The score, f(iou(M, b i )) is a weight function based on overlap.
[0127] When applying the non-maximum suppression method, each prediction box is compared and screened using its corresponding dynamic IoU threshold. All prediction boxes are sorted from high to low according to confidence. Starting from the prediction box with the highest confidence, each prediction box is processed in turn. For the currently processed prediction box b, use its corresponding adaptive IoU threshold to filter out other prediction boxes whose IoU exceeds the threshold, retain the prediction boxes that meet the conditions, and repeat the above process until all prediction boxes are processed.
[0128] Through the above methods, a better balance can be achieved in defect detection of different densities, improving the overall detection performance and robustness.
[0129] 6) Parameter optimization steps:
[0130] Define a sliding weighted loss function based on exponential moving average, and use this loss function to optimize the network parameters of the improved YOLOv8 model. The sliding weighted loss function is improved based on the idea of exponential moving average. The sliding weighted loss function formula is as follows:
[0131]
[0132] By assigning higher weights to these boundary samples, the model can optimize these samples during training and make better use of them to train the network. However, in the actual training process, the weight adjustment of difficult samples using the sliding weighted loss function may be unstable due to fluctuations in the loss value. Therefore, the sliding weighted loss function is improved based on the idea of exponential moving average. For each update step t, it can be expressed as:
[0133] IOU t ′=d t ·(IOU t ′)+(1-d)·IOU t
[0134] Among them, d t is the decay factor of the current step, which is defined as:
[0135]
[0136] Here, decay is set to 0.999 and τ is set to 2000. At this time, the sliding weighted function f(x) based on the exponential moving average can be expressed as:
[0137]
[0138] Finally, the sliding weighted loss function calculation formula based on exponential moving average is as follows:
[0139] L = loss × f(x)
[0140] Among them, loss is the output value of the basic loss function.
[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Ordinary technicians in the relevant field can still modify or make equivalent substitutions to the specific implementation methods of the present invention with reference to the above embodiments. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention are within the scope of protection of the claims of the present invention to be approved.
Claims
1. A photovoltaic panel defect detection method based on multi-scale feature fusion, It is characterized in that The steps include: 1) Photovoltaic panel image acquisition step, using high-precision cameras and surveillance camera equipment to photograph photovoltaic panels; 2) Data preprocessing step, geometric correction and data enhancement processing are performed on the collected photovoltaic panel images; 3) Feature extraction step: CSPDarknet53 network extracts the features of photovoltaic panel images. Specifically, CSPDarknet53 introduces deep separable convolution and cross-cascade networks; 4) Feature fusion step: Use the improved GFPN to effectively fuse the extracted low-level feature information and high-level feature information, then reconstruct the upsampling process from the perspective of point sampling by introducing DySample dynamic upsampling, and dynamically adjust the sampling point position of the input feature map content; By replacing the traditional convolution layer with SPD convolution, combining the space-to-depth layer and the non-stride convolution layer, the extracted features are finally fully fused; The improved GFPN fully exchanges feature information of different scales through the queen fusion mechanism and skip layer connection. The queen fusion mechanism performs feature fusion at multiple levels, and introduces an efficient layer aggregation network mechanism in the feature fusion process; 5) Defect detection step. In the layered detection strategy, each layer of feature maps is responsible for detecting targets of different scales. By adding a high-resolution layer to the output to the detection head, in the post-processing stage, an adaptive non-maximum suppression method is used to dynamically adjust the intersection-over-union ratio threshold to adapt to changes in target density. 6) Parameter optimization step: define a sliding weighted loss function based on exponential moving average, and use this loss function to optimize the network parameters of the improved YOLOv8 model.
2. According to the photovoltaic panel defect detection method of multi-scale feature fusion according to claim 1, It is characterized in that The image geometric correction processing method of step (2) is specifically as follows: first, the original image is captured by a high-precision camera or a surveillance camera device, then, the optical characteristics and position direction of the camera are calculated by calibrating the intrinsic and extrinsic parameters of the camera, and a distortion model is established. Next, according to the calibrated radial distortion coefficient and tangential distortion coefficient, the image is geometrically corrected, including correction of radial distortion and tangential distortion. Finally, image registration is performed to ensure that multiple images are aligned in the same coordinate system, and perspective transformation correction is performed to make the obliquely shot image look like it is shot vertically from directly above.
3. According to the multi-scale feature fusion photovoltaic panel defect detection method of claim 1, It is characterized in that In step (3), the CSPDarknet53 network extracts features of the photovoltaic panel image. CSPDarknet53 is a Darknet53 network combined with the CSP mechanism. Darknet53 is a deep convolutional neural network. The specific contents are as follows: S1 Input feature extraction: Extract the initial feature map from the preprocessed photovoltaic panel image. The feature map is obtained through the initial convolution layer in the CSPDarknet53 network and contains the basic feature information of different regions in the image; S2 CSP module: Apply the CSP module on the initial feature map. The CSP module divides the feature map into two parts. One part is directly passed to the subsequent layer, and the other part is merged with the directly passed part after passing through multiple residual blocks; S3 residual block: A residual block is used in the CSP module. The residual block consists of two consecutive convolutional layers and a skip connection. S4 multi-scale feature output mechanism: The feature maps generated by the network at different levels capture information of different scales.
4. According to the photovoltaic panel defect detection method of multi-scale feature fusion according to claim 5, Features: The feature maps generated at different levels in step S4 include low-level feature maps and high-level feature maps, the low-level feature maps include edges and textures, and the high-level feature maps include shapes and structures.
5. The photovoltaic panel defect detection method based on multi-scale feature fusion according to claim 1, It is characterized in that The DySample dynamic upsampling in step (4) is specifically as follows: DySample dynamic upsampling reconstructs the upsampling process from the perspective of point sampling, that is, for each pixel in the image, its value and position will be dynamically adjusted during the upsampling process. The process includes generating an initial sampling grid, generating an offset, pixel rearrangement, and resampling.
6. The photovoltaic panel defect detection method based on multi-scale feature fusion according to claim 1, It is characterized in that The SPD convolution in step (4) is specifically as follows: it includes a space-to-depth layer and a non-stride convolution layer. The space-to-depth layer divides the input feature map into multiple sub-feature maps and concatenates them in the channel dimension, retaining all pixel information in the channel dimension, thereby achieving downsampling without information loss. The non-stride convolution layer reduces the number of channels through the learned convolution kernel.
7. The photovoltaic panel defect detection method based on multi-scale feature fusion according to claim 1, It is characterized in that The adaptive non-maximum suppression method in step (5) is specifically as follows: first, the target density around each detection box is calculated based on the number and distribution of the detection boxes, and then the intersection-and-union ratio threshold of the corresponding prediction box is dynamically adjusted according to the calculated target density. When the target density is large, a larger intersection-and-union ratio threshold is used, and when the target density is small, a smaller intersection-and-union ratio threshold is used.
8. The photovoltaic panel defect detection method based on multi-scale feature fusion according to claim 1, It is characterized in that The step (6) is based on a sliding weighted loss function of an exponential moving average, and the specific contents are as follows: The sliding weighted loss function emphasizes the importance of difficult samples by assigning higher weights to them.
9. The photovoltaic panel defect detection method based on multi-scale feature fusion according to claim 8, Features: The sliding weighted loss function formula is as follows:
Citation Information
Cited By
Lightweight photovoltaic cell panel defect detection method based on improved YOLOv8
CN119693712A
Lightweight photovoltaic panel defect detection method based on improved YOLOv8
CN119693712B
BC component performance detection method and device, electronic equipment and storage medium
CN121864020A
A method and device for detecting performance of a BC component, an electronic device, and a storage medium
CN121864020B
Photovoltaic panel multi-defect fusion detection method and system
CN121959248A
