Photovoltaic panel defect detection method based on lightweight YOLO

Abstract

The invention relates to the technical field of image processing, in particular to a photovoltaic panel defect detection method based on light-weight YOLO, and the method comprises the steps: collecting a data set of solar photovoltaic panel defects; performing data enhancement on the original data set of the solar photovoltaic panel; a PSDH detection head of the YOLOv8n network is constructed, and outputs of P3, P4 and P5 layers of the PSDH detection head are respectively subjected to 1 * 1 TNConv convolution, are cascaded through two 3 * 3 RCM modules and then respectively enter a ConvReg layer and a ConvCls layer to obtain bounding box loss and category loss; and finally, scaling the output features by using Scale. And evaluating the improved YOLOv8n network by using the evaluation index. According to the invention, the problem of YOLOv8n lightweight is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a photovoltaic panel defect detection method based on lightweight YOLO. Background Art

[0002] A photovoltaic panel is a device that converts solar energy into electrical energy. It is a thin sheet made of semiconductor material (such as silicon). When sunlight shines on the photovoltaic panel, photons are absorbed by the photovoltaic cells, causing electrons to be excited and form a current. This current is introduced into the inverter through the output terminals of the photovoltaic panel and finally converted into AC power for use in the power grid. Its application scenarios are very wide. During production and processing, slight defects will cause damage and failure during subsequent use. Therefore, defect detection of photovoltaic panels is very necessary during the production process.

[0003] The patent with publication number CN118587195A uses the Dyc_C2 module to replace the C2f module of the Backbone network; the Dyc_C2 module better captures the features of different scales and shapes, improves the detection accuracy and robustness; and uses the SPPF-Pro module to obtain global perspective information and reduce the impact of different scales, effectively improving the feature extraction capability of the backbone feature extraction network and enhancing the target recognition capability; however, this model is mainly for improving the model detection accuracy, but ignores the problem of lightweight. Due to limited hardware resources in certain environments, how to ensure that the detection accuracy is not affected under the premise of lightweight is an urgent problem to be solved; in addition, this method has not formed a detection system, and has not formed an effective monitoring mechanism for the missed detection problem in photovoltaic panels. Summary of the invention

[0004] In view of the shortcomings of the existing methods, the present invention designs a complete solar photovoltaic panel defect detection system, which can realize efficient and intelligent solar photovoltaic panel defect detection. At the same time, the lightweight YOLOv8n model greatly reduces the number of parameters, making it perfectly adaptable to edge computing devices such as low-power microcontrollers, solving the problem of running jams caused by excessive number of model parameters in the past.

[0005] The technical solution adopted by the present invention is: a photovoltaic panel defect detection method based on lightweight YOLO comprises the following steps:

[0006] Step 1: Collecting a dataset of solar photovoltaic panel defects;

[0007] As a preferred embodiment of the present invention, the defect categories include scratches, broken gates, silicon missing and breakage.

[0008] Step 2: Perform data enhancement on the original data set of solar photovoltaic panels;

[0009] As a preferred implementation of the present invention, data enhancement includes erasing, scaling, flipping and translating.

[0010] Step 3: Construct the PSDH detection head of the YOLOv8n network. The outputs of the P3, P4, and P5 layers of the PSDH detection head are respectively input into the first RCM module after 1*1 TN_Conv convolution. The output of the first RCM module is connected to the input of the second RCM module. The output of the second RCM module is respectively input into three Conv_Reg layers and three Conv_Cls layers. Finally, use Scale to scale the output features of the three Conv_Reg layers.

[0011] As a preferred embodiment of the present invention, the training stage of the RCM module includes four branches; wherein, the first branch is 1*1 convolution, BN operation, the second branch is 1*1 convolution, BN operation, K*K convolution, BN operation; the third branch is 1*1 convolution, BN operation, average pooling, BN operation; the fourth branch is K*K, BN operation; the four branches are added and then subjected to Nonlinearity operation.

[0012] As a preferred embodiment of the present invention, the reasoning of the RCM module includes K*K convolution and Nonlinearity operations.

[0013] Step 4: Use evaluation indicators to evaluate the improved YOLOv8n network;

[0014] As a preferred implementation of the present invention, the evaluation index includes parameter quantity, operation floating point number and average precision mean.

[0015] As a preferred embodiment of the present invention, a photovoltaic panel defect detection system based on lightweight YOLO includes: a controller, a position sensor, an image acquisition module and a photovoltaic panel detection module, wherein the position sensor, the image acquisition module and the photovoltaic panel detection module are electrically connected to the controller respectively; wherein,

[0016] The position sensor is used to obtain the position of the photovoltaic panel to be detected on the crawler conveyor belt;

[0017] The image acquisition module is used to obtain images of the photovoltaic panels;

[0018] The photovoltaic panel inspection module is used to identify the defect type of the photovoltaic panel;

[0019] The controller determines whether to conduct a secondary re-inspection based on the detection results of the photovoltaic panel detection module.

[0020] As a preferred embodiment of the present invention, it further includes: an information display module, and the controller displays the detection result on the information display module.

[0021] Beneficial effects of the present invention:

[0022] 1. The present invention makes lightweight improvements to YOLOv8n, significantly reduces the number of model parameters, and significantly reduces the model volume, so that it can be easily adapted to various edge computing devices at the site of solar photovoltaic power stations;

[0023] 2. Through the combination of EL photography and infrared thermal imaging technology, comprehensive detection of internal and surface defects of photovoltaic panels is achieved;

[0024] 3. Through re-inspection, the photovoltaic panels that the photovoltaic panel detection module initially judged as defective products can be verified twice, effectively avoiding misjudgment and missed judgment. Through the re-inspection operation, the system can automatically distinguish between finished products and defective products, thereby improving the reliability of the detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of the photovoltaic panel defect detection method based on lightweight YOLO of the present invention;

[0026] Figure 2 It is a schematic diagram of the structure of the PSDH module of the present invention;

[0027] Figure 3 It is a schematic diagram of the RCM structure of the present invention;

[0028] Figure 4 It is a connection diagram of the lightweight YOLO photovoltaic panel defect detection system of the present invention;

[0029] Figure 5 It is a schematic diagram of the positions of the modules of the photovoltaic panel of the present invention. DETAILED DESCRIPTION

[0030] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0031] like Figure 1 As shown, a photovoltaic panel defect detection method based on lightweight YOLO includes the following steps:

[0032] Step 1: Collecting a dataset of solar photovoltaic panel defects;

[0033] Defect categories include scratches, broken gates, missing silicon and breakage.

[0034] Step 2: Perform data enhancement on the original data set of solar photovoltaic panels;

[0035] The collected original photovoltaic panel images were erased, resized, flipped, and translated for data enhancement. The enhanced data set was obtained and divided into 2889 training data sets and 825 verification data sets in an 8:2 ratio. Data enhancement improves the generalization and expression capabilities of the model, enabling the model to make predictions more accurately and reduce overfitting.

[0036] Step 3: Build an improved YOLOv8n network model;

[0037] YOLOv8n uses multiple detection heads of different scales to detect targets of different sizes. These convolutional layers are used to extract features and predict target categories and positions. The parameters of the convolutional layers include convolution kernel weights and biases. With the increase in the number of convolutional layers and the combined effect of factors such as convolution kernel size and number of channels, the number of parameters will grow rapidly. A large number of parameters means that more computing resources are required in the inference phase. For example, when performing forward propagation calculations on the GPU, more video memory is required to store model parameters and intermediate calculation results. This is a big challenge for resource-constrained devices (such as some edge computing devices) and may cause the model to fail to run normally or run extremely slowly.

[0038] Therefore, the present invention designs a new lightweight detection head, a heavy parameter shared detection head PSDH (Heavy parameter shared detection head) composed of TN_Conv convolution and RCM modules, so that the YOLOv8n network model can compress the amount of parameters while maintaining the original accuracy of the model as much as possible;

[0039] The PSDH structure diagram is as follows Figure 2 As shown in the figure, the output results of the P3, P4, and P5 layers are respectively input into the first RCM module and then into the second RCM module after a 1*1 TN_Conv convolution. The size of the RCM module is 3*3. The second RCM module is respectively input into the first Conv_Reg layer and the first Conv_Cls layer, the second Conv_Reg layer and the second Conv_Cls layer, and the third Conv_Reg layer and the third Conv_Cls layer to obtain the bounding box loss and category loss; the bounding box loss is used to calculate the difference between the predicted bounding box and the true bounding box, and the category loss is used to measure the difference between the category distribution predicted by the model and the true label. Finally, Scale is used to scale the output features of the three Conv_Cls layers to meet the scale requirements of the large, medium, and small detection heads.

[0040] Use the TN_Conv layer to replace the original BN layer. The BN layer is very sensitive to the batch size during training. When the batch size is small, the calculated mean and variance may not represent the entire data distribution well, resulting in performance degradation. The TN_Conv layer is not limited by the batch size because it is normalized within the channel grouping and is independent of the batch size. The TN_Conv layer is more adaptable to the network architecture and can be applied more flexibly as long as there is a channel dimension. Therefore, in the case of small batches of data, the TN_Conv layer can still effectively normalize the data to maintain the stability and performance of the model.

[0041] In PSDH, the number of parameters can be greatly reduced by using convolution parameter sharing, which makes the model lighter, especially on resource-constrained devices. However, shared parameters may limit the expressiveness of the model, because different features may require different convolution kernels to capture complex patterns, and shared parameters may not be able to fully capture these differences. In order to make up for the negative impact of the convolution sharing operation adopted to achieve lightweighting as much as possible; the present invention adopts the RCM module (re-parameterization module). By introducing more learnable parameters, the network can more effectively extract features from the data, thereby compensating for the precision loss problem that may be caused by the lightweight model, and re-parameterization can greatly improve the parameter utilization rate, and there is no difference with ordinary convolution in the inference stage, which brings a lossless optimization solution to the model.

[0042] like Figure 3 The RCM module shown in a includes four branches in the branch during training. The first branch is 1*1 convolution and BN operation, the second branch is 1*1 convolution, BN operation, K*K convolution, BN operation; the third branch is 1*1 convolution, BN operation, average pooling, BN operation; the fourth branch is K*K, BN operation, where K=3; the four branches are added and then subjected to Nonlinearity operation; the four-branch combined convolution method is used to extract richer feature information, thereby enhancing the ability of the feature map to express multi-scale feature semantics at the end, and effectively alleviating the problem of multi-scale information loss in the deep detection head;

[0043] like Figure 3 As shown in b, during inference, the RCM module reparameterizes the four branch structures into a single-path structure of K*K convolution, and then performs the Nonlinearity operation; the reparameterization does not increase the additional computing cost during the inference stage, maintaining high-speed prediction while reducing the negative impact of the imbalance in the number of categories.

[0044] The problem of being unable to deploy a real-time detection system locally due to the large model in the past has been solved; the lightweight YOLOv8n model has greatly reduced the requirements for hardware resources. In the upgrade scenarios of some old solar photovoltaic power station monitoring equipment, the existing hardware can be used to achieve efficient defect detection function upgrades, avoiding the high cost investment caused by large-scale hardware replacement.

[0045] Step 4: Use indicators to evaluate the improved YOLOv8n network;

[0046] In order to test the performance of the improved model, the parameter size Params, the floating point size GFLOPs, and the average precision mAP are used as evaluation indicators of the model algorithm. The parameter size and floating point size determine the demand for the number of device memory channels and the pressure on the processor when working, and the average precision mAP determines the detection accuracy after deployment.

[0047] Table 1 Comparative experiment

[0048]

[0049] It can be seen from the data in Table 1 that based on the original YOLOv8n model, the accuracy is basically the same, but the number of parameters and the amount of calculation are reduced by 21.4% and 19.8% respectively compared with before. The number of parameters of the YOLOv8n network model is greatly reduced, and the model size is significantly reduced, making it easy to adapt to various edge computing devices at the solar photovoltaic power station site.

[0050] The problem of being unable to deploy a real-time detection system locally due to the large model has been solved. The lightweight YOLOv8n model has greatly reduced the requirements for hardware resources. In the upgrade scenarios of some old solar photovoltaic power station monitoring equipment, the existing hardware can be used to achieve efficient defect detection function upgrades, avoiding the high cost investment caused by large-scale hardware replacement.

[0051] like Figure 4 , a photovoltaic panel defect detection system based on lightweight YOLO, including: a controller, a position sensor, an image acquisition module and a photovoltaic panel detection module, the position sensor, the image acquisition module and the photovoltaic panel detection module are electrically connected to the controller respectively; wherein,

[0052] The position sensor is used to obtain the position of the photovoltaic panel to be detected on the crawler conveyor belt;

[0053] The image acquisition module is used to obtain images of the photovoltaic panels;

[0054] The photovoltaic panel inspection module is used to identify the defect type of the photovoltaic panel;

[0055] When the detection result of the photovoltaic panel detection module is defective, the controller performs a secondary re-inspection.

[0056] like Figure 5 The photovoltaic panel to be detected is placed on the crawler conveyor belt, and a fixing device is set on the crawler conveyor belt. The fixing device can be fine-tuned according to the size and shape of the photovoltaic panel to ensure that the photovoltaic panel can be firmly fixed on the crawler belt. In addition, the crawler belt surface is made of anti-slip material to improve the stability of the photovoltaic panel during transmission; when the photovoltaic panel is transported to the position of the position sensor through the crawler conveyor belt, the position sensor detects that the photovoltaic panel to be detected has arrived at the detection area, and sends an arrival signal to the controller. The controller sends an image acquisition signal to the image acquisition module and sends the acquired image to the controller. The controller sends the acquired image to the photovoltaic panel detection module The photovoltaic panel detection module detects the photovoltaic panel, outputs an abnormal value when a defect is detected, and triggers a secondary re-inspection; the controller uses the suction cup on the gripper of the robotic arm located at the detection exit to stably absorb the defective photovoltaic panel to be re-inspected, and re-places it in the detection area for secondary inspection. When the photovoltaic panel is detected as defective again, the gripper of the robotic arm absorbs the photovoltaic panel again and transfers it to the defect collection device for further analysis or processing; for photovoltaic panels that are confirmed to be intact after re-inspection, they will be guided by the robotic arm or automatic conveyor device and smoothly enter the finished product collection device to prepare for the subsequent packaging process.

[0057] The entire inspection process is highly automated, and the transportation, photography, inspection and result feedback of photovoltaic panels can be automatically completed without human intervention.

[0058] The image acquisition module uses a combination of EL camera equipment and infrared thermal imaging module. EL photography uses electroluminescence technology, which can take high-quality photos of photovoltaic panels in a dark room environment. In order to capture the tiny defects inside the photovoltaic panels, a high-resolution camera and a professional lens combination are selected; at the same time, a lighting system is set up in the dark room to ensure that a uniform and bright image can be obtained every time a photo is taken;

[0059] In order to ensure that the EL camera can work under the best conditions, a dark box can be built on the crawler conveyor belt. The EL camera is set in the dark box. The dark box can effectively shield external light and electromagnetic interference, thereby ensuring that the camera can capture pure light signals; the dark box uses highly light-shielding materials to make walls and tops to ensure that external light cannot enter the dark box; and an airtight door is set at the entrance of the dark box to prevent external light and air flow from affecting the interior of the dark room when the door is opened.

[0060] In addition, in order to ensure the stability of the temperature and humidity inside the dark box, the temperature and humidity inside the dark box are monitored in real time and adjusted as needed to ensure that the camera and photovoltaic panels are in the best working conditions for taking pictures and detection. Lighting is set up inside the dark box. The lighting uses a low-brightness, high-uniformity light source to ensure that uniform and bright images can be obtained when taking pictures.

[0061] The infrared imaging module improves the image acquisition capability of thermal defects inside photovoltaic panels. Infrared thermal imaging captures the temperature distribution on the surface of photovoltaic panels, thereby discovering internal thermal defects. These thermal defects are often caused by uneven resistance distribution or local overheating inside the photovoltaic panels. Through infrared thermal imaging technology, these defects can be discovered and located in time, providing support for subsequent repair and processing.

[0062] The photovoltaic panel detection module adopts the improved YOLOv8n network of the present invention, and can also adopt the existing photovoltaic defect detection model;

[0063] The controller uses an MCU microcontroller.

[0064] It also includes: an information display module, and the controller displays the test results on the information display module, so as to timely understand the quality status of the photovoltaic panel;

[0065] The system of the present invention can automatically distinguish between finished products and defective products, and mark or remove the defective products, without the need for on-site monitoring by production personnel, thus reducing labor costs.

[0066] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A photovoltaic panel defect detection method based on lightweight YOLO, characterized in that: The following steps are involved: Step 1: Collecting a dataset of solar photovoltaic panel defects; Step 2: Perform data enhancement on the original data set of solar photovoltaic panels; Step 3: Construct the PSDH detection head of the YOLOv8n network. The outputs of the P3, P4, and P5 layers of the PSDH detection head are respectively input into the first RCM module after 1*1 TN_Conv convolution. The output of the first RCM module is connected to the input of the second RCM module. The output of the second RCM module is respectively input into three Conv_Reg layers and three Conv_Cls layers. Finally, use Scale to scale the output features of the three Conv_Reg layers. Step 4: Use evaluation indicators to evaluate the improved YOLOv8n network.

2. The photovoltaic panel defect detection method based on lightweight YOLO according to claim 1 is characterized in that: The training phase of the RCM module includes four branches; the first branch is 1*1 convolution, BN operation, the second branch is 1*1 convolution, BN operation, K*K convolution, BN operation; the third branch is 1*1 convolution, BN operation, average pooling, BN operation; the fourth branch is K*K, BN operation; the four branches are added and then subjected to Nonlinearity operation.

3. The photovoltaic panel defect detection method based on lightweight YOLO according to claim 2 is characterized in that: The reasoning of the RCM module includes K*K convolution and Nonlinearity operations.

4. The photovoltaic panel defect detection method based on lightweight YOLO according to claim 1 is characterized in that: Defect categories include scratches, broken gates, missing silicon and breakage.

5. The photovoltaic panel defect detection method based on lightweight YOLO according to claim 1 is characterized in that: Data augmentation includes erasing, resizing, flipping, and translation.

6. The photovoltaic panel defect detection method based on lightweight YOLO according to claim 1 is characterized in that: Evaluation indicators include parameter quantity, floating point number of operations and average precision mean.

7. A system using the photovoltaic panel defect detection method based on lightweight YOLO according to claims 1-6, characterized in that: include: Controller, position sensor, image acquisition module and photovoltaic panel detection module; wherein, The position sensor is used to obtain the position of the photovoltaic panel to be detected on the crawler conveyor belt; The image acquisition module is used to obtain images of the photovoltaic panels; The photovoltaic panel inspection module is used to identify the defect type of the photovoltaic panel; The controller determines whether to conduct a secondary re-inspection based on the detection results of the photovoltaic panel detection module.

8. The system of photovoltaic panel defect detection method based on lightweight YOLO according to claim 7 is characterized in that: Also includes: Information display module, the controller displays the detection results on the information display module.

9. Photovoltaic panel defect detection system based on lightweight YOLO, characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the photovoltaic panel defect detection method based on lightweight YOLO as described in any one of claims 1 to 6.

10. A computer readable medium storing computer program code, characterized in that: When the computer program code is executed by a processor, the method for photovoltaic panel defect detection based on lightweight YOLO as described in any one of claims 1 to 6 is implemented.

Images