Defect detection method and device of photovoltaic module, medium and electronic equipment

Abstract

The invention belongs to the technical field of photovoltaic power generation, and relates to a defect detection method and device for a photovoltaic module, a medium and electronic equipment. The method comprises the following steps: acquiring a target image of a photovoltaic module, and acquiring annotation data of the target image; the target image is input into a target detection model, so that the target detection model outputs annotation data to obtain a trained target detection model, the trained target detection model is utilized to perform defect detection on the photovoltaic module, and the target detection model comprises a feature extraction module, a channel differentiation module and a feature fusion module. According to the invention, through training the improved target detection model, an effect of carrying out defect detection on the photovoltaic module by using the target detection model is realized. On the basis of identifying the defects of the photovoltaic module, the requirement of high defect identification speed is met, the defect detection accuracy is also improved, and an accurate and efficient defect detection method for the photovoltaic module is provided.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of photovoltaic power generation, and in particular, relates to a photovoltaic module defect detection method, a photovoltaic module defect detection device, a non-temporary computer-readable storage medium, and an electronic device. Background Art

[0002] At present, photovoltaic modules occupy an important position in the field of new energy. With the proposal of China's carbon neutrality concept, photovoltaic modules have been shipped in large quantities. However, the control of the quality of photovoltaic modules mainly comes from the judgment of machines, and the accuracy of the current judgment needs to be improved.

[0003] During the production process, various factors may cause large cracks, hidden cracks, scratches, cold solder joints, over-soldering, broken grids, black spots, pollution, broken corners, failures, and mixed grades in the cells. If the defects of photovoltaic modules are not accurately identified, the power of the photovoltaic modules will be reduced, which will lead to the downgrade of the modules. Therefore, how to accurately and efficiently identify the defects of photovoltaic modules and improve the yield of modules is not only of great research significance, but also an indispensable skill for module manufacturers. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a photovoltaic module defect detection method, a photovoltaic module defect detection device, a non-transitory computer-readable storage medium and an electronic device.

[0005] According to a first aspect of an embodiment of the present disclosure, a method for detecting defects in a photovoltaic module is provided, the method comprising: Acquire a target image of a photovoltaic assembly and acquire annotation data of the target image; The target image is input into a target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and the trained target detection model is used to perform defect detection on the photovoltaic module. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

[0006] Optionally, acquiring a target image of a photovoltaic assembly includes: A component image of the photovoltaic component is acquired, and image enhancement processing is performed on the component image to obtain a target image.

[0007] Optionally, inputting the target image into a target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model includes: Inputting the target image into a target detection model so that the target detection model performs multiple extraction processing on the target image to obtain a first feature vector; Using the feature extraction module to perform feature extraction processing on the first feature vector to obtain a second feature vector, and using the channel differentiation module to perform channel differentiation processing on the second feature vector to obtain a third feature vector; The feature fusion module is used to perform feature fusion processing on the third feature vector to obtain a fourth feature vector, so that the target detection model outputs the labeled data according to the fourth feature vector to obtain a trained target detection model.

[0008] Optionally, the using the feature extraction module to perform feature extraction processing on the first feature vector to obtain a second feature vector includes: Extracting a first feature layer of the first feature vector using the feature extraction module, and performing a linear transformation process on the first feature layer to obtain a second feature layer; The first feature layer and the second feature layer are concatenated to obtain a second feature vector.

[0009] Optionally, the using the channel differentiation module to perform channel differentiation processing on the second feature vector to obtain a third feature vector includes: Using the channel differentiation module to perform information compression processing on the second feature vector to obtain a fifth feature vector; Performing feature excitation processing on the fifth feature vector to obtain a third feature vector.

[0010] Optionally, the using the feature fusion module to perform feature fusion processing on the third feature vector to obtain a fourth feature vector includes: Using the feature fusion module to perform feature mapping processing on the third feature vector to obtain at least two sixth feature vectors; Perform feature concatenation processing on at least two of the sixth eigenvectors and the third eigenvector to obtain a fourth eigenvector.

[0011] Optionally, the target detection model includes: Tiny-YOLOv3.

[0012] According to a second aspect of an embodiment of the present disclosure, there is provided a photovoltaic module defect detection device, comprising: A data acquisition module is configured to acquire a target image of the photovoltaic assembly and acquire annotated data of the target image; The model training module is configured to input the target image into the target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and use the trained target detection model to perform defect detection on the photovoltaic component. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

[0013] According to a third aspect of an embodiment of the present disclosure, there is provided a non-temporary computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of any one of the methods described in the first aspect of the present disclosure are implemented.

[0014] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; The processor is configured to: execute the executable instructions to implement the steps of any one of the methods described in the first aspect of the present disclosure.

[0015] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects: In the method and device provided by the exemplary embodiment of the present disclosure, by training the improved target detection model, the effect of applying the target detection model to detect defects in photovoltaic modules is achieved. On the basis of identifying the defects of photovoltaic modules, the requirement of fast defect recognition speed is met, and the accuracy of defect detection is also improved, providing an accurate and efficient defect detection method for photovoltaic modules.

[0016] Other features and advantages of the present disclosure will be described in detail in the following detailed description.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings: Figure 1 The schematic diagram of the structure of Tiny-YOLOv3 in the related art is shown schematically; Figure 2 A schematic diagram of a process flow of a photovoltaic module defect detection method in an exemplary embodiment of the present disclosure is schematically shown; Figure 3 The schematic diagram of the model structure of the improved Tiny-YOLOv3 in the exemplary embodiment of the present disclosure is schematically shown; Figure 4 A schematic diagram of a process for training a target detection model in an exemplary embodiment of the present disclosure is schematically shown; Figure 5 A schematic diagram of a process of performing feature extraction processing on a first feature vector in an exemplary embodiment of the present disclosure is schematically shown; Figure 6 The schematic diagram schematically shows the principle of the Ghost module in the exemplary embodiment of the present disclosure; Figure 7 The schematic diagram of the structure of the Ghost module in the exemplary embodiment of the present disclosure is shown; Figure 8 A schematic diagram of a process of performing channel differentiation processing on a second feature vector in an exemplary embodiment of the present disclosure is schematically shown; Fig. 9 Schematically shows a schematic diagram of the principle of the SE module in the exemplary embodiment of the present disclosure; Fig.10 The structure diagram of the SE module in the exemplary embodiment of the present disclosure is schematically shown; Fig.11 A schematic flow chart of a method for performing feature fusion processing on a third feature vector in an exemplary embodiment of the present disclosure is schematically shown; Fig.12 Schematically shows a schematic diagram of the principle of the SPP structure in an exemplary embodiment of the present disclosure; Fig.13 The structural diagram of the SPP structure in the exemplary embodiment of the present disclosure is schematically shown; Fig.14 A schematic diagram of a process flow of a photovoltaic module defect detection method in an application scenario in an exemplary embodiment of the present disclosure is schematically shown; Fig.15 A schematic diagram of defects of a photovoltaic module in an application scenario in an exemplary embodiment of the present disclosure is schematically shown; Fig.16 A schematic structural diagram of a photovoltaic module defect detection device in an exemplary embodiment of the present disclosure is schematically shown; Fig.17 An electronic device for implementing a defect detection method for a photovoltaic module in an exemplary embodiment of the present disclosure is schematically shown; Fig.18 Another electronic device for implementing a defect detection method for a photovoltaic module in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0020] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0021] Today, EL (Electroluminescence) detection of photovoltaic modules uses traditional image processing methods, machine learning methods and deep learning methods.

[0022] The traditional defect detection image processing method uses industrial cameras to continuously scan the surface of photovoltaic modules. The image reflected by the light source on the surface of the photovoltaic module is collected by the industrial camera, converted into an image signal and transmitted to the computer, and the algorithm is used to determine whether there is a defect. However, in the actual application process, it mainly depends on the quality of the algorithm. Generally, although a good algorithm can make the detection accuracy better, the model is too large and not easy to deploy. Therefore, the improvement of the algorithm will gradually become the focus of research in the future.

[0023] Traditional machine learning algorithms often require manual design of models to extract features when the defect type changes or the data set is replaced. This method is complex to operate, has high restrictions, has a long periodicity and a low recognition yield.

[0024] The target detection algorithm based on deep learning is the current research focus. With the advent of big data and breakthroughs in hardware, deep learning technology has shown explosive development in the past few years, and therefore occupies an increasingly important position in the field of artificial intelligence. The field that best reflects intelligence in artificial intelligence is machine learning, and deep learning is an important branch of traditional machine learning. Deep learning mainly analyzes problems by simulating multi-layer neural networks in the human brain, combining the initial feature data to form a higher-level feature representation. Its fundamental idea is to learn the inherent laws and representation levels of sample data so that the model has better expression capabilities.

[0025] Since deep learning simulates the way human brain learns, it can replace humans in handling some tasks that require a lot of time to observe, especially in the manufacturing industry, where deep learning technology can be used to control product quality. Compared with traditional algorithms, the use of deep learning algorithms can better identify product types or defects more quickly and accurately. The algorithm has strong uniformity and applicability. Therefore, defect detection and identification of photovoltaic modules based on deep learning algorithms can help improve the yield of modules.

[0026] Among deep learning algorithms, the Tiny-YOLOv3 (a micro version of You Only Look Once version 3) algorithm has the advantage of fast recognition speed. However, the Tiny-YOLOv3 algorithm has problems of high missed detection rate and low accuracy in real-time detection when identifying defects in EL images.

[0027] Among them, Tiny-YOLOv3 removes some feature layers based on YOLOv3 in the YOLO (You Only Look Once) series of algorithms. The network structure of the model is relatively simple, but the input image cannot be fully extracted, which will lead to excessive loss of information, especially in the detection of tiny cracks, there are problems such as high missed detection rate and low accuracy.

[0028] Figure 1 The schematic diagram of the structure of Tiny-YOLOv3 in the related art is shown as follows: Figure 1 As shown in the figure, the number of convolutional layers in the Tiny-YOLOv3 algorithm network structure is greatly reduced, and no fully connected layer or residual layer structure is used. There are only convolutional layers and maximum pooling layers on the backbone network, so the structure and number of parameters of the model are relatively small.

[0029] The Tiny-YOLOv3 algorithm network is mainly composed of 13 CBL layers and 6 pooling layers. Among them, the CBL layer is mainly composed of three parts: convolution layer, batch normalization (Batch Normalization, BN) and activation layer (Leaky ReLU), and the pooling layer adopts the maximum pooling method, using a convolution kernel of 2×2 and a step size of 2. It should be noted that the step size of the maximum pooling between the 512 layer and the 1024 layer is 1, not 2.

[0030] The input image is divided into s×s grid cells (Tiny-YOLOv3 is a multi-scale prediction output with 2 layers of feature maps. Each layer of feature maps has s×s grids, 13×13 and 26×26 respectively). Assuming that the center of the object under test falls within the grid, then this grid is responsible for predicting the object under test. Each grid predicts B bounding boxes. Each bounding box not only predicts the position of the detected object, but also predicts the confidence value. At the same time, each grid also predicts the scores of C categories. Therefore, each grid cell needs to generate 3×(4+1+C) values.

[0031] The present invention provides a method for detecting defects in a photovoltaic module. Figure 2 is a flow chart of a photovoltaic module defect detection method according to an exemplary embodiment. Figure 2 As shown, the method may at least include the following steps: Step S210: Acquire a target image of a photovoltaic module and acquire annotation data of the target image.

[0032] Step S220. Input the target image into the target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and use the trained target detection model to perform defect detection on the photovoltaic module. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

[0033] In the exemplary embodiment of the present disclosure, by training the improved target detection model, the effect of applying the target detection model to detect defects in photovoltaic modules is achieved. On the basis of identifying the defects of photovoltaic modules, the requirement of fast defect recognition speed is met, and the accuracy of defect detection is also improved, providing an accurate and efficient defect detection method for photovoltaic modules.

[0034] The following is a detailed description of each step of the photovoltaic module defect detection method.

[0035] In step S210, a target image of a photovoltaic assembly is acquired, and annotation data of the target image is acquired.

[0036] In the exemplary embodiments of the present disclosure, a photovoltaic module is also referred to as a solar panel, which is a device that converts sunlight into electrical energy. It is mainly composed of solar cells, glass, a back panel, a frame, a junction box, and the like.

[0037] Photovoltaic modules do not produce pollution when working and are a clean renewable energy source. They are widely used in household rooftop power generation, large-scale photovoltaic power stations, solar lighting, portable power supplies and other fields. They are one of the important means to achieve a low-carbon economy and promote the transformation of energy structure.

[0038] In an optional embodiment, a component image of the photovoltaic component is acquired, and image enhancement processing is performed on the component image to obtain a target image.

[0039] The component image of the photovoltaic component may be an image showing the photovoltaic component having large cracks, hidden cracks, scratches, cold solder joints, over-soldering, broken grids, black spots, pollution, broken corners, failures, and mixed files.

[0040] When the number of component images of the photovoltaic module is insufficient, the component images can be rotated 90° or 180° and the brightness of the component images can be adjusted to perform image enhancement processing to obtain more target images. The component images and / or target images can be of a size of 416×416×3 or other sizes, which is not particularly limited in this exemplary embodiment.

[0041] The annotation data of the target image may be used to mark whether the target image has defects, and if so, whether there are four types of defects, namely short circuit, hidden crack, fragment and black spot.

[0042] In step S220, the target image is input into the target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and the trained target detection model is used to perform defect detection on the photovoltaic module. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

[0043] In an exemplary embodiment of the present disclosure, after acquiring the target image and the annotation data, the target image can be input into the target detection model so that the target detection model outputs the annotation data to obtain a trained target detection model.

[0044] In an optional embodiment, the target detection model includes: Tiny-YOLOv3.

[0045] Tiny-YOLOv3 is a lightweight, real-time object detection model, which is a version of the YOLO family of models. The YOLO model is widely used in object detection tasks due to its fast speed and high accuracy. Tiny-YOLOv3 is a simplification of YOLOv3 to reduce the complexity and computation of the model, so that it can run on resource-constrained devices, such as mobile devices or embedded systems.

[0046] The main features of Tiny-YOLOv3 include: Architecture simplification: Tiny-YOLOv3 simplifies the network structure of YOLOv3, reducing the number of convolutional layers and the number of filters, thereby reducing the complexity and computational requirements of the model.

[0047] Real-time detection: Despite the smaller model, Tiny-YOLOv3 is still able to achieve real-time object detection, which is very important for application scenarios that require fast response.

[0048] Multi-scale detection: Tiny-YOLOv3 retains the multi-scale detection feature of YOLOv3 and can detect objects of different sizes.

[0049] Balance between performance and speed: Tiny-YOLOv3 significantly improves the detection speed while maintaining a certain detection accuracy, making it suitable for deployment in environments with limited computing resources.

[0050] Easy to deploy: Due to its small model size and low computational requirements, Tiny-YOLOv3 can be easily deployed on various hardware platforms.

[0051] Considering that although Tiny-YOLOv3 is much smaller than YOLOv3 in model size, it is still not mature enough to run on low-end machines, and the accuracy rate has also dropped significantly, and the mean average precision (mAP) has also decreased. The main reason for the decline is that the backbone network of Tiny-YOLOv3 is shallow and the level of abstraction is low, which makes it difficult to extract higher-level image semantic features. Therefore, adding the Ghost module (feature extraction module), SE module (channel differentiation module) and SPP structure (feature fusion module) to Tiny-YOLOv3 can better improve the model size, accuracy and mAP value.

[0052] Figure 3 The schematic diagram of the improved Tiny-YOLOv3 model structure is shown in FIG. Figure 3 As shown in the figure, since the maximum pooling operation only selects one of the four adjacent pixels, this will lead to the waste of the remaining three pixels, resulting in feature loss during network transmission. Therefore, this model uses a convolution layer instead of a pooling layer, and uses a convolution kernel of 3×3 and a step size of 2 for convolution operations. In this way, after convolution, the same effect as pooling can be achieved, that is, the size of the image will be reduced to half of the original size, but more feature information will be retained than the pooling layer.

[0053] In an alternative embodiment, Figure 4 A flow chart of a method for training a target detection model is shown, such as Figure 4 As shown, the method may at least include the following steps: In step S410, the target image is input into the target detection model, so that the target detection model performs multiple extraction processing on the target image to obtain a first feature vector.

[0054] Figure 3 The improved Tiny-YOLOv3 model includes a combination structure consisting of 5 CBL layers and 5 convolutional layers, and a combination structure consisting of one CBL layer and one maximum pooling layer. These two combination structures can perform multiple feature extraction processes on the target image to obtain the first feature vector.

[0055] In step S420, a feature extraction module is used to perform feature extraction processing on the first feature vector to obtain a second feature vector, and a channel differentiation module is used to perform channel differentiation processing on the second feature vector to obtain a third feature vector.

[0056] In an alternative embodiment, Figure 5 A schematic flow chart of a method for extracting features from a first feature vector is shown, such as Figure 5As shown, the method may at least include the following steps: in step S510, a first feature layer of a first feature vector is extracted by using a feature extraction module, and a linear transformation is performed on the first feature layer to obtain a second feature layer.

[0057] Due to the limited computing resources and memory in embedded devices, adding the Ghost module to Tiny-YOLOv3 can reduce the computational consumption of the neural network.

[0058] Ghost Net is also called a lightweight deep convolutional model architecture. Both redundant and important information in the feature layer can ensure a comprehensive understanding of the input data. There are many similarities between the feature layers, and these similarities are like illusions of each other. Considering that redundant information in the feature layer may also be an important part of a successful model, some lightweight network models will remove the redundant part, but Ghost Net does not remove the redundant information, but uses low-cost computing to obtain the redundant part.

[0059] The Ghost module is an important part of Ghost Net. The Ghost module is introduced in Tiny-YOLOv3. When only part of the real feature layer is generated by convolution operation for a certain feature layer as the first feature layer, the remaining feature layer (phantom feature layer) is obtained by linear transformation of the real first feature layer as the second feature layer. The linear transformation here means that the thickness of the convolution kernel in the 3×3 convolution is 1, while the thickness of the convolution kernel of the ordinary convolution is consistent with the number of channels of the input feature map. This not only avoids a large number of operations but also reduces a lot of parameters at the same time.

[0060] In step S520, the first feature layer and the second feature layer are concatenated to obtain a second feature vector.

[0061] Furthermore, the first feature layer and the second feature layer are concatenated together to form a complete feature layer as a second feature vector.

[0062] Figure 6 The schematic diagram of the Ghost module is shown in FIG. Figure 6 As shown in the figure, linear transformation refers to the depthwise convolution using s×s convolution and a convolution kernel thickness of 1, while the convolution kernel thickness of traditional convolution is the same as the number of input channels.

[0063] Figure 7 The schematic diagram of the structure of the Ghost module is shown in FIG. Figure 7 As shown in the figure, the input of the Ghost module is h (height) × w (width) × c (number of channels), and the output is h'×w'×n (number of convolution kernels), with a convolution kernel size of k.

[0064] The computational cost of ordinary convolution is ; The computational cost of the Ghost module is ; where s represents the number of components in the output layer.

[0065] The compression ratio is .

[0066] Since s << c and , it shows that the convolution of the Ghost module has much lower computational cost than ordinary convolution.

[0067] In an alternative embodiment, Figure 8 shows a schematic flowchart of a method for performing channel differentiation on a second feature vector, as Figure 8 shown. The method may at least include the following steps: In step S810, the channel differentiation module is used to perform information compression processing on the second feature vector to obtain a fifth feature vector.

[0068] The main function of the Squeeze-and-Excitation blocks (SE) is to enable the model to focus on the channel features of more valuable information during the transmission of the feature map, and solve the loss problem caused by different weights of different channels of the feature map.

[0069] During the convolution operation of the feature map, it is generally considered that each channel plays an equally important role. However, in solving practical problems, it is found that there are also significant differences between channels. Therefore, adding the SE module allows the model to autonomously learn the different roles between channels. At the same time, strengthen the more useful channels and compress the channels of useless information. In addition, the SE module can be easily loaded into the existing network model framework without disturbing the original main structure of the network.

[0070] Fig. 9 shows a schematic diagram of the principle of the SE module, as Fig. 9 shown. The SE module mainly performs two important operations: Squeeze and Excitation.

[0071] Fig.10 shows a schematic diagram of the structure of the SE module, as Fig.10 shown. Squeeze is to better establish the dependency relationship between channels. The global average pooling method is used to compress the information of the H×W dimension on each channel into a number. In this way, the matrix of H×W×C can be compressed to 1×1×C as the fifth feature vector, and the information of each channel is represented by the global average pooling value of that channel.

[0072] In step S820, feature excitation processing is performed on the fifth eigenvector to obtain a third eigenvector.

[0073] Through the excitation method, it is possible to use compressed information, that is, the fifth eigenvector, to model the channel relationship and capture the spatial correlation, thereby improving the expressive power of the network.

[0074] The third eigenvector of the output can be understood as a weight relationship of channel analysis, with more important channels given greater weights. After being stimulated to the second fully connected layer, the output is the weight value activated by Sigmoid (S-shaped growth curve), and then the weight coefficients calculated for different channels are multiplied by the matrix of the corresponding channel of the feature map that has undergone convolution operation, thereby realizing the re-modification of the feature channel and updating the feature map.

[0075] Since the SE module uses a fully connected layer, although the amount of calculation has increased, it is not much different from the entire network, so it has little impact on the real-time performance of the algorithm, and the accuracy is significantly improved.

[0076] In step S430, the feature fusion module is used to perform feature fusion processing on the third feature vector to obtain a fourth feature vector, so that the target detection model outputs the labeled data according to the fourth feature vector to obtain a trained target detection model.

[0077] Fig.11 A schematic flow chart of a method for performing feature fusion processing on the third feature vector is shown, such as Fig.11 As shown, the method may at least include the following steps: In step S1110, a feature fusion module is used to perform feature mapping processing on the third feature vector to obtain at least two sixth feature vectors.

[0078] Spatial Pyramid Pooling (SPP) is a pooling method that maps feature maps to multiple dimensional spaces and fuses these multiple receptive fields. It allows the network structure to adapt to image inputs of different sizes, and then obtains feature maps of fixed size. It can also effectively extract multi-scale feature information of the target artifact.

[0079] Fig.12 The schematic diagram of the SPP structure is shown in Fig.12 As shown in Figure 1, the SPP structure in Tiny-YOLOv3 mainly realizes the fusion of feature maps of local features and global features. This method can perform three different maximum pooling operations on the local third feature vector to obtain a feature map as the sixth feature vector.

[0080] In step S1120, feature concatenation processing is performed on at least two sixth eigenvectors and the third eigenvector to obtain a fourth eigenvector.

[0081] Furthermore, the sixth eigenvector and the third eigenvector are concatenated to obtain the fourth eigenvector, so that the field of view of the convolution kernel is wider and the expressive ability of the feature map is more sufficient, thereby improving the mAP value.

[0082] Fig.13 A schematic diagram of the structure of the SPP structure is shown in FIG. Fig.13 As shown in the figure, the SPP structure is mainly composed of four parallel mechanisms, namely, maximum pooling layers with convolution kernel sizes of 5×5, 9×9, and 13×13, and a skip connection layer.

[0083] Assuming that the size of the feature map input is 512×512, the SPP structure can transform the feature map mapping into a feature matrix of (13×13+9×9+5×5+1×1)×512, and then expand it into a one-dimensional matrix to form a fixed-size feature vector, thereby achieving feature fusion of different scales.

[0084] When using the fourth eigenvector to train the target detection model, the confidence loss, classification loss and positioning loss can be used as the loss function. The specific formula is as follows:

[0085] in, Represents the weight coefficient of the position coordinate error in the entire loss function, which is set to 5; Indicates the weight coefficient of the entire loss function that does not include the object confidence error loss, which is set to 0.5; Indicates that the jth detection box of the i-th cell is responsible for predicting the target object. If there is a target, the value is 1, otherwise the value is 0; Indicates that when there is no target to be detected in the jth border of the i-th network, if it does not exist, it is 1, otherwise it is 0; Indicates whether the center point of the object falls within the grid i, if yes, it is 1, otherwise it is 0; and Represent the predicted value and true value of the confidence in the i-th grid respectively.

[0086] After obtaining the trained target detection model, the trained target detection model can be applied to perform defect detection on photovoltaic modules.

[0087] The defect detection method of the photovoltaic module in the embodiment of the present disclosure is described in detail below in conjunction with an application scenario.

[0088] Fig.14A schematic diagram of a defect detection method for photovoltaic modules in an application scenario is shown, such as Fig.14 As shown in the figure, the experimental operating system is Ubuntu 18.04 (an operating system), the CPU (Central Processing Unit) model is Intel(R) Xeon(R) CPU E5-2678v3@2.50GHZ, the GPU (Graphic Processing Unit) model is NVIDIA GeForce RTX 2080TI 11GB, the software dependency package is CUDA10.2+cuDNN7.6, and the deep learning framework is Pytorch1.6.0.

[0089] The format of the EL defect dataset follows the Pascal VOC dataset format. In this experiment, there are a total of 2400 EL defect images after amplification. The dataset is divided into training set, validation set and test set according to the ratio of 8:1:1. When traversing the training set once is considered as one iteration, there are 500 iterations in total.

[0090] Since memory overflow may occur during training, batch training is adopted. A batch contains 32 images during training. When the batch size is too small, the model converges difficultly, and when the batch size is too large, insufficient video memory is caused. The initial learning rate is set to 0.001, because the initial learning rate determines the speed of weight update. If it is set too large, it will exceed the optimal value. If it is set too small, the weight update speed will slow down. The momentum is generally set to 0.9, mainly to make the speed of stochastic gradient descent reach the optimal value. The weight decay rate is set to 0.0005 by default to avoid overfitting of the model.

[0091] Photovoltaic modules can be divided into defect-free and defective. Defects include large cracks, hidden cracks, scratches, cold solder joints, over-soldering, broken grids, black spots, pollution, broken corners, failures and mixed grades. This study focuses on the four categories of short circuits, hidden cracks, fragments and black spots.

[0092] Fig.15 A schematic diagram of defects of photovoltaic modules in application scenarios is shown, such as Fig.15 As shown, short circuit 1510 is an invalid black block with no current flowing through, broken fragment 1520 is a crack found in the battery cell and appears as a black block, hidden crack 1530 is a fine line crack that appears on the main grid line or edge of the battery cell due to collision and pressure, and cold solder joint 1540 is a rectangular shadow formed by poor welding.

[0093] Table 1 shows the comparison results of defect detection of different algorithms:

[0094] Table 1 Compared with the improved Tiny-YOLOv3 algorithm, although the mAP value and accuracy have been slightly improved, the model size is too large and is not suitable for use on embedded devices. The improved Tiny-YOLOv3 algorithm accuracy is 6 percentage points higher than that of Tiny-YOLOv3, and the mAP value is also increased by 1.9 percentage points, indicating that the missed detection rate of the improved Tiny-YOLOv3 has been significantly reduced, and it can be better used in visual inspection for fast and accurate detection.

[0095] The performance of the detection model includes not only the accuracy, mAP value and size of the model, but also the detection speed and floating-point operations. Under the same test environment, the improved Tiny-YOLOv3 model is compared with other models. Compared with Tiny-YOLOv3, the detection speed of the improved model is 10.25fps lower, but it is still higher than YOLOv3 and meets the real-time requirements. Compared with YOLOv3, the floating-point operations are reduced by 109.17GFLOPs to only 9.13GFLOPs, which means that the model is relatively simple and helps to reduce the consumption of device resources.

[0096] First, EL defect images are collected by collecting data sets on site. The more data sets collected, the more helpful it is for subsequent algorithm recognition. The collected PV module defect images are generally divided into training sets, verification sets, and test sets.

[0097] Then, the relevant training defect images are carried out according to the designed network structure. The designed network structure mainly adds three network modules, namely SPP structure, SE module and Ghost module, to the feature extraction network of Tiny-YOLOv3, and replaces the pooling layer with the convolution layer. The improved network structure has significant improvements in average precision mean, accuracy and network model size.

[0098] The model weights trained on the training set are introduced into the test set for testing to verify the feasibility of the algorithm. By combining the Tiny-YOLOv3 algorithm with three modules for photovoltaic module defect detection, end-to-end detection can be achieved without image preprocessing compared to existing methods, and photovoltaic module defect detection can be performed accurately and efficiently.

[0099] In the exemplary embodiment of the present disclosure, improvements are made on the basis of Tiny-YOLOv3 to achieve the effect of more quickly and accurately identifying the type of defects in EL images, which not only meets the requirement of fast recognition speed, but also forms the advantage of high detection accuracy.

[0100] In addition, in an exemplary embodiment of the present disclosure, a defect detection device for a photovoltaic module is also provided. Fig.16 The schematic diagram of the structure of the defect detection device of the photovoltaic module is shown as follows: Figure 8 As shown, the photovoltaic module defect detection device 1600 may include: a data acquisition module 1610 and a model training module 1620. Among them: The data acquisition module 1610 is configured to acquire a target image of a photovoltaic assembly and acquire annotation data of the target image; The model training module 1620 is configured to input the target image into the target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and use the trained target detection model to perform defect detection on the photovoltaic module. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

[0101] In an exemplary embodiment of the present invention, the data acquisition module 1610 is configured to: A component image of the photovoltaic component is acquired, and image enhancement processing is performed on the component image to obtain a target image.

[0102] In an exemplary embodiment of the present invention, the model training module 1620 is configured to: Inputting the target image into a target detection model so that the target detection model performs multiple extraction processing on the target image to obtain a first feature vector; Using the feature extraction module to perform feature extraction processing on the first feature vector to obtain a second feature vector, and using the channel differentiation module to perform channel differentiation processing on the second feature vector to obtain a third feature vector; The feature fusion module is used to perform feature fusion processing on the third feature vector to obtain a fourth feature vector, so that the target detection model outputs the labeled data according to the fourth feature vector to obtain a trained target detection model.

[0103] In an exemplary embodiment of the present invention, the model training module 1620 is configured to: Extracting a first feature layer of the first feature vector using the feature extraction module, and performing a linear transformation process on the first feature layer to obtain a second feature layer; The first feature layer and the second feature layer are concatenated to obtain a second feature vector.

[0104] In an exemplary embodiment of the present invention, the model training module 1620 is configured to: Using the channel differentiation module to perform information compression processing on the second feature vector to obtain a fifth feature vector; Performing feature excitation processing on the fifth feature vector to obtain a third feature vector.

[0105] In an exemplary embodiment of the present invention, the model training module 1620 is configured to: Using the feature fusion module to perform feature mapping processing on the third feature vector to obtain at least two sixth feature vectors; Perform feature concatenation processing on at least two of the sixth eigenvectors and the third eigenvector to obtain a fourth eigenvector.

[0106] In an exemplary embodiment of the present invention, the target detection model includes: Tiny-YOLOv3.

[0107] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0108] Fig.17 FIG. 1 is a block diagram of an electronic device 1700 according to an exemplary embodiment. Fig.17 As shown, the electronic device 1700 may include: a processor 1701 , a memory 1702 . The electronic device 1700 may also include one or more of a multimedia component 1703 , an input / output (I / O) interface 1704 , and a communication component 1705 .

[0109] The processor 1701 is used to control the overall operation of the electronic device 1700 to complete all or part of the steps in the above-mentioned photovoltaic module defect detection method. The memory 1702 is used to store various types of data to support the operation of the electronic device 1700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 1700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 1702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 1703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone, which is used to receive external audio signals. The received audio signal may be further stored in the memory 1702 or sent through the communication component 1705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 1704 provides an interface between the processor 1701 and other interface modules, and the above-mentioned other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 1705 is used for wired or wireless communication between the electronic device 1700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 1705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0110] In an exemplary embodiment, the electronic device 1700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned photovoltaic module defect detection method.

[0111] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned photovoltaic module defect detection method are implemented. For example, the computer-readable storage medium can be the above-mentioned memory 1702 including program instructions, and the above-mentioned program instructions can be executed by the processor 1701 of the electronic device 1700 to complete the above-mentioned photovoltaic module defect detection method.

[0112] Fig.18 1 is a block diagram of an electronic device 1800 according to an exemplary embodiment. For example, the electronic device 1800 may be provided as a server. Fig.18 The electronic device 1800 includes a processor 1822, which may be one or more, and a memory 1832 for storing a computer program executable by the processor 1822. The computer program stored in the memory 1832 may include one or more modules, each corresponding to a set of instructions. In addition, the processor 1822 may be configured to execute the computer program to perform the above-mentioned photovoltaic module defect detection method.

[0113] In addition, the electronic device 1800 may further include a power supply component 1826 and a communication component 1850, wherein the power supply component 1826 may be configured to perform power management of the electronic device 1800, and the communication component 1850 may be configured to implement communication of the electronic device 1800, for example, wired or wireless communication. In addition, the electronic device 1800 may further include an input / output (I / O) interface 1858. The electronic device 1800 may operate based on an operating system stored in the memory 1832.

[0114] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned photovoltaic module defect detection method are implemented. For example, the non-transitory computer-readable storage medium can be the above-mentioned memory 1832 including program instructions, and the above-mentioned program instructions can be executed by the processor 1822 of the electronic device 1800 to complete the above-mentioned photovoltaic module defect detection method.

[0115] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned photovoltaic component defect detection method when executed by the programmable device.

[0116] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0117] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0118] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for detecting defects in a photovoltaic module, characterized in that: The method comprises: Acquire a target image of a photovoltaic assembly and acquire annotation data of the target image; The target image is input into a target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and the trained target detection model is used to perform defect detection on the photovoltaic module. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

2. The photovoltaic module defect detection method according to claim 1, characterized in that: The step of acquiring a target image of a photovoltaic assembly comprises: A component image of the photovoltaic component is acquired, and image enhancement processing is performed on the component image to obtain a target image.

3. The photovoltaic module defect detection method according to claim 1, characterized in that: The step of inputting the target image into a target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model includes: Inputting the target image into a target detection model so that the target detection model performs multiple extraction processing on the target image to obtain a first feature vector; Using the feature extraction module to perform feature extraction processing on the first feature vector to obtain a second feature vector, and using the channel differentiation module to perform channel differentiation processing on the second feature vector to obtain a third feature vector; The feature fusion module is used to perform feature fusion processing on the third feature vector to obtain a fourth feature vector, so that the target detection model outputs the labeled data according to the fourth feature vector to obtain a trained target detection model.

4. The photovoltaic module defect detection method according to claim 3, characterized in that: The using the feature extraction module to perform feature extraction processing on the first feature vector to obtain a second feature vector includes: Extracting a first feature layer of the first feature vector using the feature extraction module, and performing a linear transformation process on the first feature layer to obtain a second feature layer; The first feature layer and the second feature layer are concatenated to obtain a second feature vector.

5. The photovoltaic module defect detection method according to claim 3, characterized in that: The using the channel differentiation module to perform channel differentiation processing on the second feature vector to obtain a third feature vector includes: Using the channel differentiation module to perform information compression processing on the second feature vector to obtain a fifth feature vector; Performing feature excitation processing on the fifth feature vector to obtain a third feature vector.

6. The photovoltaic module defect detection method according to claim 3, characterized in that: The step of using the feature fusion module to perform feature fusion processing on the third feature vector to obtain a fourth feature vector includes: Using the feature fusion module to perform feature mapping processing on the third feature vector to obtain at least two sixth feature vectors; Perform feature concatenation processing on at least two of the sixth eigenvectors and the third eigenvector to obtain a fourth eigenvector.

7. The photovoltaic module defect detection method according to claim 1, characterized in that: The target detection model includes: Tiny-YOLOv3.

8. A photovoltaic module defect detection device, characterized in that: include: A data acquisition module is configured to acquire a target image of the photovoltaic assembly and acquire annotated data of the target image; The model training module is configured to input the target image into the target detection model so that the target detection model outputs the labeled data to obtain a trained target detection model, and use the trained target detection model to perform defect detection on the photovoltaic component. The target detection model includes a feature extraction module, a channel differentiation module and a feature fusion module.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images