Method and device for detecting defects of photovoltaic cells based on improved yolov7 network model

Abstract

The application relates to a method and device for detecting defects of a photovoltaic cell based on an improved Yolov7 network model, and the method comprises the following steps: collecting a photovoltaic cell image under an electroluminescence condition; using a pre-trained improved Yolov7 network model to identify the photovoltaic cell image to obtain a defect identification accuracy result and a defect position identification result; wherein the improved Yolov7 network model comprises a Backbone network, a Neck network and a Head network, the Backbone network uses a lightweight ODM_block layer with a reverse residual structure and full-dimensional dynamic convolution, the Neck network is added with a full-dimensional dynamic convolution ODConv layer, and the Head network uses a Rep module. The application can efficiently and accurately detect cell defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of target detection, and in particular to a method and device for detecting defects of a photovoltaic cell based on an improved Yolov7 network model. BACKGROUND

[0002] A photovoltaic cell is a device that converts solar energy into electrical energy, usually composed of multiple crystalline silicon wafers, each of which is referred to as a cell piece. However, when defects exist on the surface of the cell piece, these defects will interfere with the normal operation of the photovoltaic cell and reduce its power generation efficiency. If the quality of the photovoltaic cell production product is to be improved and economic benefits are to be increased, photovoltaic cell surface defect detection must be performed.

[0003] In order to detect the above-mentioned defects, the current photovoltaic industry commonly used defect image acquisition technologies include electroluminescence technology, photoluminescence technology, and infrared penetration technology, etc. The infrared penetration technology selects a 1300nm near-infrared light source and illuminates the silicon wafer on one side of the camera. The near-infrared light of this band is totally reflected between the special crystal bond structure of the silicon material. If there is a hidden crack on the silicon wafer, the near-infrared light will directly penetrate the silicon wafer, so the imaging effect at the defect is darker than that of the normal area, and only hidden crack defects can be detected, without the ability to detect other defects. The photoluminescence technology requires the imaging field of view to be limited to a thin line across the width of the cell piece, and a laser is used to irradiate the thin line, so that the camera can collect the PL image. This method has a high requirement for the environment for collecting images and is expensive. The electroluminescence technology stimulates the photovoltaic cell by applying a forward bias, causing it to radiate light outward. The electroluminescence spectrum peak is at 1150nm, so a short-wave infrared camera is needed to image the radiated infrared light, and there are problems such as low contrast, blurred edges, and unclear texture. However, this method has the advantages of being able to detect a variety of hidden defects that cannot be observed by the human eye without contacting the cell piece, fast detection speed, and easy implementation.

[0004] Currently, it is common to use target detection to detect cell defects. However, due to the multi-scale characteristics of photovoltaic cell defects, the current deep neural network has deficiencies in detecting small targets, especially as the network training progresses and continues to downsample, the microscopic defect features such as cracks and broken grids tend to disappear; in addition, the computing power is limited in industrial applications, and lightening is needed, so it is necessary to make improvements. SUMMARY

[0005] The technical problem to be solved by the present application is to overcome the defects of the prior art and provide a method for detecting defects of a photovoltaic cell based on an improved Yolov7 network model, which can efficiently and accurately detect cell defects.

[0006] In order to solve the above technical problems, the technical scheme of the present application is: a method for detecting defects of a photovoltaic cell based on an improved Yolov7 network model, comprising:

[0007] Collecting a photovoltaic cell image under electroluminescence conditions;

[0008] Using a pre-trained improved Yolov7 network model to identify the photovoltaic cell image to obtain a defect identification accuracy result and a defect position identification result; wherein,

[0009] The improved Yolov7 network model comprises a Backbone network, a Neck network and a Head network, the Backbone network uses a lightweight ODM_block layer with a reverse residual structure and full-dimensional dynamic convolution, the Neck network is added with a full-dimensional dynamic convolution ODConv layer, and the Head network uses a Rep module.

[0010] Further, the method for training the improved Yolov7 network model comprises:

[0011] First, perform a data enhancement operation on the images in the image training data set, the data enhancement operation comprising flipping, changing brightness, cropping and translation;

[0012] Then, input the image training data set into the improved Yolov7 network model, change the optimizer to AdamW and train.

[0013] Further, the activation function in the CHB layer of the Backbone network is replaced with H-Swish.

[0014] Further, the ODM_block layer comprises an input module, a 1x1 convolution transformation module, a 3x3 depth separable convolution module, an attention mechanism SE module, an ODConv module, a 1x1 convolution transformation module and an output module connected in sequence, and the input module is also directly connected to the output module.

[0015] Further, the working method of the depth separable convolution module comprises:

[0016] First, perform a depthwise convolution operation, and then perform a pointwise convolution.

[0017] Further, the attention mechanism SE module comprises two stages of Squeeze and Excitation; wherein,

[0018] In the Squeeze stage, global average pooling is used to pool the input feature map in the channel dimension, so that the input feature map is changed from a three-dimensional tensor of h x w x c to a two-dimensional tensor of 1 x 1 x c;

[0019] In the Excitation stage, a multi-layer perceptron (MLP) is used to learn the importance weights of each channel and perform a weighted sum operation with the input feature map to obtain the output feature map, the MLP is composed of two 1x1 convolution layers and uses h-swish and sigmoid functions for activation.

[0020] Further, the specific implementation of the ODConv module is as follows:

[0021] y=(α w1 ⊙α f1 ⊙α c1 ⊙α s1 ⊙W1+…+α wn ⊙α fn ⊙α cn ⊙α sn ⊙W n )*x

[0022] wherein, respectively represent the input feature and the output feature alpha wi represents the attention scalar of the convolution kernel W i . alpha si e R k*k , and represent three newly introduced attentions, respectively used to calculate the spatial dimension, the input channel dimension and the output channel dimension of the kernel space of the convolution kernel W i ; and represents the multiplication operation along the different dimensions of the kernel space.

[0023] Further, the Neck network is also added with a feature fusion module and an attention mechanism SE module.

[0024] The application also provides a device for detecting defects of a photovoltaic cell based on an improved Yolov7 network model.

[0025] The acquisition module is configured to acquire a photovoltaic cell image under electroluminescence conditions.

[0026] The defect detection module is configured to identify the photovoltaic cell image by using the pre-trained improved Yolov7 network model to obtain a defect recognition accuracy result and a defect position recognition result.

[0027] The output module is configured to output the defect recognition accuracy result and the defect position recognition result.

[0028] The improved Yolov7 network model comprises a Backbone network, a Neck network and a Head network, the Backbone network uses a lightweight ODM_block layer with a reverse residual structure and full-dimensional dynamic convolution, the Neck network is added with a full-dimensional dynamic convolution ODConv layer, and the Head network uses a Rep module.

[0029] After the above technical scheme is adopted, the application has the following beneficial effects:

[0030] (1) The Backbone network is improved by using the ODM_block layer, thereby reducing the parameter quantity of the neural network and improving the model inference speed;

[0031] (2) The ODConv module is used to provide better performance to capture rich context information and better learn the micro defect features;

[0032] (3) The FPN in the application is a pyramid structure with feature fusion, and more defect features can be obtained;

[0033] (4) The Rep module is added in the Head network, the network structure is optimized by using the reparameterization method;

[0034] (5) The H-Swish function is used to improve the model recognition accuracy, and AdamW is used as the optimizer to optimize the training effect. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 It is a flowchart of the method for detecting photovoltaic cell defects based on the improved Yolov7 network model of the application;

[0036] Figure 2 It is a structure diagram of the improved Yolov7 network model of the application;

[0037] Figure 3 It is a structure diagram of the ODM_block layer of the application;

[0038] Figure 4 It is a structure diagram of the attention mechanism SE module of the application;

[0039] Figure 5 It is a structure diagram of the ODConv layer of the application;

[0040] Figure 6 It is a result output schematic diagram of the application;

[0041] Figure 7 It is a structure diagram of the device for detecting photovoltaic cell defects based on the improved Yolov7 network model of the application. DETAILED DESCRIPTION

[0042] In order to make the content of the present application easier to be clearly understood, the present application will be further described in detail below according to specific embodiments and in conjunction with the accompanying drawings.

[0043] Embodiment one

[0044] As Figure 1 shown, a method for detecting defects of a photovoltaic cell based on an improved Yolov7 network model, comprising:

[0045] S1, collecting an image of the photovoltaic cell under electroluminescence conditions;

[0046] S2, using a pre-trained improved Yolov7 network model to identify the image of the photovoltaic cell, to obtain a defect recognition accuracy result and a defect position recognition result;

[0047] S3, outputting the defect recognition accuracy result and the defect position recognition result;

[0048] Among them, the improved Yolov7 network model includes a Backbone network, a Neck network and a Head network, the Backbone network uses a lightweight ODM_block layer with a reverse residual structure and a full-dimensional dynamic convolution, the Neck network adds a full-dimensional dynamic convolution ODConv layer, which improves the accuracy of small targets, reduces the network operation time, the Head network uses a Rep module, which eliminates the residual in ResNet and the splicing in DenseNet by using the identity connection therein, and provides more gradient diversity for different feature mappings.

[0049] Specifically, the output is the accuracy of the defect and the position and probability of the defect, Figure 6 and a result output schematic diagram.

[0050] In this embodiment, according to the classification of the training data set, the defect detection type can be divided into several categories which are the same as the classification of the training data set, and exemplary, the defect categories can be divided into: crack, broken grid, black core, misalignment, thick line, scratch, fragment, broken corner, horizontal dislocation, transverse dislocation, printing error and short circuit defect.

[0051] In this embodiment, the method for training the improved Yolov7 network model comprises:

[0052] First, perform data enhancement operation on the images in the image training data set, the data enhancement operation includes flipping, changing brightness, cropping and translation, etc.

[0053] Then, input the image training data set into the improved Yolov7 network model, change the optimizer to AdamW and train.

[0054] Specifically, as Figure 2As shown, the improvement of the Backbone network can be but is not limited to replacing all layers of the Backbone network except the first layer with ODM_block layers, because Yolov7 does not have a c3 structure and it is not easy to directly modify the original network, therefore, it is better to replace all layers. The first layer CBS of the Backbone network is replaced with CBH, and CBS is replaced with Conv, BN and H-swish structure, where CBS is a Conv layer, a BN layer and a Silu layer.

[0055] H-swish is a high-performance neural network activation function that can improve network performance. It is a nonlinear function, and its function curve is the same as that of RELU, but it has no upper bound. However, it brings stronger regularization effect when the lower boundary x slowly tends to 0 from the left half axis. At the same time, it is continuous and derivable everywhere, and easy to train. The specific implementation is as follows:

[0056]

[0057] In one embodiment, Figure 3 As shown, the ODM_block layer can be but is not limited to including an input module, a 1x1 convolution transformation module, a 3x3 depth separable convolution module, an attention mechanism SE module, an ODConv module, a 1x1 convolution transformation module and an output module connected in sequence, and the input module is also directly connected to the output module. Specifically, the input is divided into two paths, one of which is an inverted bottleneck layer to realize an inverted residual structure, and the other of which is a 1x1 convolution transformation on the input feature map to expand the number of channels. Next, a 3x3 depth separable convolution is performed on the input feature map, followed by a Squeeze-and-Excitation (SE) module and an ODConv module. Since the ODConv module can better select useful features, after learning useful features using the SE module, the ODConv module can further strengthen these features by weighting, thereby optimizing the performance of the network. Finally, at the end of the ODM_block layer, a 1x1 convolution transformation is applied again to reduce the number of channels and the size of the feature map, so as to further compress the calculation amount of the ODM_block layer, and the feature map is fused by using the inverted residual structure to retain the information of the main channel and enhance the nonlinear representation ability of the model.

[0058] Since the use of ODConv module and SE module in the ODM_block layer will increase the parameter amount and reduce the operation rate, it is determined by ablation experiment which places to select how to use the two attention mechanisms in part of the layers, therefore, the ODConv module is enabled by the parameter use_odconv, and the SE module is enabled by the parameter use_se.

[0059] An attention module is initialized in the ODConv of the Backbone network, which is used to calculate the attention distribution of channels, filters, spaces and convolution kernels, and a weight parameter, which contains a set of kernel_num randomly initialized two-dimensional convolution kernels. The ODConv performs attention weighting on the input feature map and the convolution kernel, and then performs convolution operation.

[0060] In an embodiment, the working method of the depth separable convolution module includes:

[0061] First, the depthwise convolution operation is performed, and then the pointwise convolution is performed to reduce the amount of calculation. In the depthwise convolution stage, the convolution kernel size is 3*3, and the depth separable convolution separates the spatial convolution and the channel convolution, which is independent operation on the channel; in the pointwise convolution stage, each channel feature map is convolved with a 1*1 convolution kernel, so as to introduce the dependency between channels into the feature map. At the same time, BatchNormalization and h-swish activation function are added to improve the expression ability of the model.

[0062] In an embodiment, as shown in Figure 4 The attention mechanism SE module includes Squeeze and Excitation two stages; wherein,

[0063] In the Squeeze stage, global average pooling is used to pool the input feature map in the channel dimension, which changes the input feature map from a three-dimensional tensor of h* w* c to a two-dimensional tensor of 1*1* c;

[0064] In the Excitation stage, a multi-layer perceptron MLP is used to learn the importance weight of each channel, and the input feature map is weighted and operated to obtain the output feature map, and the MLP is composed of two 1*1 convolution layers, which are activated by h-swish and sigmoid functions.

[0065] Wherein, Figure 4 Global pooling step refers to the global average pooling operation, FC step refers to 1*1 convolution, FC+ReLU+FC+Sigmoid step refers to the Excitation stage, and FC+ReLU+FC refers to the MLP process. The output size of the multi-layer perceptron MLP is channels / 16. The N1C / 16 matrix is generated by the squeeze operation, and the corresponding channel is enhanced by the activated weight through the excitation operation.

[0066] Figure 5 The structure diagram of the ODConv module, the ODConv module, for a convolution kernel, assigns different attention values to the convolution parameters on the spatial dimension, the convolution filter for different input channels, the convolution filter for different output channels, and n overall convolution kernels. The four types of attention are complementary. By gradually multiplying different attention along the dimensions of position, channel, filter, and kernel, the convolution operation will have differences in each dimension for the input, providing better performance to capture rich contextual information. Therefore, ODCOnv can greatly improve the feature extraction capability of convolution; more importantly, ODConv works better with fewer convolution kernels. The specific implementation of the ODConv module is as follows:

[0067] y = (a w1 ⊙a f1 ⊙a c1 ⊙a s1 ⊙W1+…+a wn ⊙a fn ⊙a cn ⊙a sn ⊙W n )*x

[0068] wherein, respectively represent the input feature and the output feature a wi ∈R represents the attention scalar of the convolution kernel W i . a si ∈R k*k , and represent three newly introduced attentions, respectively used to calculate the spatial dimension, the input channel dimension, and the output channel dimension of the kernel space of the convolution kernel W i ; ⊙ represents the multiplication operation along the different dimensions of the kernel space.

[0069] In one embodiment, the Rep module includes two parts: train and deploy, wherein the train part includes three branches. The uppermost branch uses a 3x3 convolution to extract features; the middle branch uses a 1x1 convolution to smooth the features; and the last branch is an Identity, which does not perform convolution operation and directly maps the features to the next layer of network. After the train part completes feature extraction and fusion, the deploy part outputs the processing result in a 3x3 convolution manner, and converts the 1x1 convolution and Identity into a 3x3 convolution according to the reparameterization algorithm of the train part. In model training, the weights of the three branches are superimposed and fused to obtain the weights of the deploy part, which are used for feature extraction and weighting. Finally, the three branches are fused into a line, and the output part only has a 3x3 convolution layer, and the weight of the 3x3 convolution layer is the weighted sum of the weights of the three branches.

[0070] Embodiment Two

[0071] As shown in the figure, the difference between this embodiment and Embodiment One is that the Neck network is additionally provided with a feature fusion module and an attention mechanism SE module. Figure 2

[0072] Based on the specific structure of the improved Yolov7 network model in this embodiment, as shown in the figure, the working process of the improved Yolov7 network model is as follows: Figure 2

[0073] After the image with an input size of 640x640x3 is extracted by the improved Backbone network, feature maps with sizes of 80x80x64, 40x40x128 and 20x20x256 are output respectively and sent to the Neck network. After the feature fusion module, a new improved feature layer is obtained by using the attention mechanism SE module to enhance the expression, so as to obtain a new feature map. The Neck network contains an improved feature pyramid FPN and an improved path aggregation structure PAN. The FPN is used to transfer semantic information from the upper layer to the lower layer in the network, and the PAN is used to transfer positioning information from the lower layer to the upper layer, so as to fuse the information of different network layers in the Backbone network and improve the detection capability. Finally, the three layers of feature maps are respectively predicted after being processed by the re-parameterized convolution.

[0074] When performing feature fusion, due to the different feature scales and channel numbers, the method of pooling is used to unify the feature channel number. By using a 1x1 convolution kernel, the channel number of the feature layer is adjusted by controlling the number of the convolution kernel, a new feature layer is formed, and the feature layers with consistent scale and channel number are directly fused to obtain a new feature fusion P. After the fused features are adjusted by using the attention mechanism SE module, the attention is adjusted to enhance the expression ability of the feature map. SE contains Squeeze and Excitation. Squeeze operation encodes all spatial features in a channel into a global feature, and then uses Sigmoid activation function and ReLU activation function to correct the channel feature encoding information, to obtain an activation value S. The activation value S is multiplied by the feature fusion information P to obtain the final improved feature layer.

[0075] ​​The method for improving small-scale target detection accuracy by less calculation amount in the embodiment improves the feature loss problem existing in the traditional pyramid module. The pyramid module of the invention provides auxiliary features required for small target detection by supplementing the lost feature information due to the forward process of the backbone network; at the same time, the feature fusion stage does not need additional learning parameters, all feature information is extracted from the original backbone network and integrated through scale transformation. The pyramid module has the characteristics of fast, efficient and accurate in implementation, does not increase the additional parameter learning burden, can significantly improve the performance of small target detection task, and reduces the calculation cost.

[0076] Embodiment three

[0077] As shown in Figure 7 A device for detecting defects of a photovoltaic cell based on an improved Yolov7 network model, comprising:

[0078] A collection module for collecting photovoltaic cell images under electroluminescence conditions;

[0079] A defect detection module for identifying the photovoltaic cell images using a pre-trained improved Yolov7 network model to obtain defect recognition accuracy results and defect position recognition results;

[0080] An output module for outputting the defect recognition accuracy results and the defect position recognition results; wherein

[0081] The improved Yolov7 network model includes a Backbone network, a Neck network and a Head network, the Backbone network uses a lightweight ODM_block layer with a reverse residual structure and full-dimensional dynamic convolution, the Neck network adds a full-dimensional dynamic convolution ODConv layer, and the Head network uses a Rep module.

[0082] The collection module can use a camera, and the defect detection module and the output module can be located in the upper computer of PyQt5.

[0083] Based on the above ideal embodiments according to the invention, through the above description, relevant personnel can make various changes and modifications without deviating from the scope of the invention. The technical scope of the invention is not limited to the contents of the specification, and the technical scope must be determined according to the scope of the claims.

Claims

1. A method for detecting defects in photovoltaic cells based on an improved Yolov7 network model, characterized in that, include: Image acquisition of photovoltaic cells under electroluminescence conditions; The photovoltaic cell image was identified using a pre-trained improved Yolov7 network model, yielding defect identification accuracy and defect location results; wherein, The improved Yolov7 network model includes a Backbone network, a Neck network, and a Head network. The Backbone network uses a lightweight ODM_block layer with an inverted residual structure and full-dimensional dynamic convolution. The Neck network adds a full-dimensional dynamic convolution ODConv layer. The Head network uses a Rep module. The ODM_block layer includes an input module, a 1×1 convolution transformation module, a 3×3 depthwise separable convolution module, an attention mechanism (SE) module, an ODConv module, a 1×1 convolution transformation module, and an output module connected in sequence. The input module is also directly connected to the output module.

2. The method for detecting photovoltaic cell defects based on an improved Yolov7 network model according to claim 1, characterized in that, Methods for training improved Yolov7 network models include: First, data augmentation operations are performed on the images in the image training dataset. These data augmentation operations include flipping, changing brightness, cropping, and translation. Then, input the image training dataset into the improved Yolov7 network model, change the optimizer to AdamW, and train it.

3. The method for detecting photovoltaic cell defects based on an improved Yolov7 network model according to claim 1, characterized in that, The activation function in the CHB layer of the Backbone network is replaced with H-Swish.

4. The method for detecting photovoltaic cell defects based on an improved Yolov7 network model according to claim 1, characterized in that, The working method of the depthwise separable convolutional module includes: First perform depthwise convolution, then perform pointwise convolution.

5. The method for detecting photovoltaic cell defects based on an improved Yolov7 network model according to claim 1, characterized in that, The attention mechanism SE module comprises two phases: Squeeze and Excitation; wherein, In the Squeeze stage, global average pooling is used to pool the input feature map in the channel dimension, transforming the input feature map from a three-dimensional tensor of h×w×c into a two-dimensional tensor of 1×1×c. In the excitation stage, a multilayer perceptron (MLP) is used to learn the importance weights of each channel, and these weights are then weighted and summed with the input feature map to obtain the output feature map. The MLP consists of two 1×1 convolutional layers, activated using the h-swish and sigmoid functions.

6. The method for detecting photovoltaic cell defects based on an improved Yolov7 network model according to claim 1, characterized in that, The specific implementation of the ODConv module is as follows: in, , , representing input features and output features respectively. Represents the convolution kernel Attention scalar , and This represents three newly introduced attention points, each used to compute the convolution kernel. The spatial dimensions of the kernel space, the input channel dimension, and the output channel dimension; This represents multiplication operations along different dimensions of the kernel space.

7. The method for detecting photovoltaic cell defects based on an improved Yolov7 network model according to claim 1, characterized in that, The Neck network also includes a feature fusion module and an attention mechanism (SE) module.

8. A device for detecting defects in photovoltaic cells based on an improved Yolov7 network model, characterized in that, include: The acquisition module is used to acquire images of photovoltaic cells under electroluminescence conditions; The defect detection module is used to identify the photovoltaic cell image using a pre-trained improved Yolov7 network model, and obtain the defect identification accuracy result and the defect location identification result. The output module is used to output the defect identification accuracy result and the defect location identification result; wherein, The improved Yolov7 network model includes a Backbone network, a Neck network, and a Head network. The Backbone network uses a lightweight ODM_block layer with an inverted residual structure and full-dimensional dynamic convolution. The Neck network adds a full-dimensional dynamic convolution ODConv layer. The Head network uses a Rep module. The ODM_block layer includes an input module, a 1×1 convolution transformation module, a 3×3 depthwise separable convolution module, an attention mechanism (SE) module, an ODConv module, a 1×1 convolution transformation module, and an output module connected in sequence. The input module is also directly connected to the output module.

Images