A method for cutting photovoltaic module pictures based on mask generation

Abstract

The present invention discloses a method for cutting photovoltaic module images based on mask generation, and the method steps are: constructing an image data set; image preprocessing; generating a mask based on UNet; generating a binary image; separating candidate cell units; generating cell units; obtaining a coordinate frame containing any number of cell units through a set frame, and then extracting the corresponding cell unit image from the original photovoltaic module image according to the coordinate frame. The beneficial effects of the present invention are as follows: the present invention effectively improves the cutting accuracy of the cell units, alleviates the problems of poor robustness and low generalization of the method caused by imaging problems; can effectively improve the generalization ability of the model; the cell unit separation method can effectively solve the problem of inaccurate and incomplete cell unit segmentation due to poor image quality, and improve the cutting accuracy; it can be applied not only to full-cell, half-cell, and shingled photovoltaic modules, but also to any image with similar image features.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic module defect detection, and in particular to a photovoltaic module image cutting method based on mask generation. Background Art

[0002] Photovoltaic module defect detection is a recognized difficulty in the photovoltaic industry. Before the rise of deep learning, PV module defect detection was completed by quality inspectors through visual inspection. In recent years, with the development of artificial intelligence technology, especially deep learning technology represented by convolutional neural networks, more and more photovoltaic manufacturers have introduced artificial intelligence-based defect detection technology.

[0003] The implementation of this technology has an unavoidable difficulty, that is, photovoltaic modules are usually composed of tens of millions of pixels, and defects often only occupy a very small part of the module image, making it very difficult to detect them directly. The current common practice is to first cut the module image into multiple units, and then perform defect detection on each unit separately, and finally summarize the detection results of each unit to achieve a more accurate detection effect.

[0004] At present, there are two main methods for cutting photovoltaic module images in the industry, namely the average cutting method and the cutting method based on edge detection.

[0005] The average cut method, as the name suggests, involves averaging the PV module image to create several smaller image units. While simple to operate, this method significantly increases the difficulty of subsequent defect detection. For example, due to the randomness of defect location and size, if the average cut unit only includes a portion of the defect, it will miss important features for identifying that type of defect. Furthermore, PV modules often have redundant, non-functional areas outside the cell area, often represented by black borders. This area, mixed with the normal cell area, can easily interfere with detection results.

[0006] Cutting methods based on edge detection typically use edge detection and basic image pre- and post-processing methods from traditional image processing to extract cell units from PV module images. This method also has significant limitations. It can only effectively cut PV modules with clear images and minimal appearance changes. In actual industrial inspection scenarios, image quality and machine stability are often not effectively guaranteed, which can easily lead to over- or under-segmentation, resulting in loss of cell unit integrity and greatly increasing the difficulty of PV module defect detection.

[0007] Therefore, to address the above problems, a photovoltaic module image cutting method based on mask generation is proposed. Summary of the Invention

[0008] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a photovoltaic module image cutting method based on mask generation.

[0009] A photovoltaic module image cutting method based on mask generation, the method steps are as follows:

[0010] Step 1: Construct image dataset;

[0011] Step 2: Image preprocessing;

[0012] Step 3: Generate UNet-based mask;

[0013] Step 4: Generate binary image;

[0014] Step 5: Separate candidate cell units;

[0015] Step 6: Generate battery cell units;

[0016] Step 7: Obtain a coordinate frame containing any number of cell units through a collection frame, and then extract the corresponding cell unit image from the original photovoltaic module image based on the coordinate frame.

[0017] The steps for constructing the image dataset in step 1 are as follows: obtaining several photovoltaic module images with the cell edge lines marked, and dividing them into a training set and a test set.

[0018] The image preprocessing method in step 2 is: scaling the image while maintaining the original image aspect ratio, and then dividing the image by 255 to normalize its pixels to between 0 and 1 as the input data of the network.

[0019] The binary image generation method in step 4 is as follows: input the test image into the trained UNet model to obtain the predicted image, and obtain the binary image of the test image by threshold truncation; assuming that the predicted image obtained by the trained UNet model for the test image X(i, j) is M(i, j), the truncation threshold is t, and the pixels where M(i, j)>t are set to 1, that is, the area composed of pixel 1 is the battery cell area (target area), and the pixels where M(i, j)>t are set to 0, that is, the area composed of pixel 0 is the background area, and finally a binary image is obtained.

[0020] The cell unit separation method in step 5 is as follows:

[0021] Assuming that the photovoltaic module contains m rows and n columns of cell units, its height is H and width is W, the cell height h can be roughly obtained r=H / (n+1); Set the convolution kernel size of the erosion operation, then perform N erosion operations on the binary image, and then calculate the pixel sum of each row and column of the resulting binary image Mp. Suppose the pixel sums of the i-th row and j-th column are and Then when l_p i When ratio_w×hr is less than, all pixel values in row i are 0. j When <ratio_h×hr, all pixel values in the jth column are 0; finally, the optimized binary image is obtained.

[0022] The method for generating the battery cell in step 6 is as follows:

[0023] Connected regions are extracted from the optimized binary image. Each connected region is a candidate battery cell unit. Next, the width and height features of each battery cell unit are extracted: w r and h r ; Set the threshold intervals [a, b] and [c, d] that meet the width and height of the battery cell unit to determine whether the battery cell unit is retained; if it exceeds the threshold range, remove the candidate battery cell unit; if it meets the threshold range, retain it, and use the external rectangular frame of each battery cell to represent its battery cell unit, and finally obtain several set frames J containing valid battery cell units; then determine whether the number of set frames J is equal to the total number of battery cell units; if so, end this step; if not, use the template frame to complete the missing battery cell units so that the final number of set frames is equal to the total number of battery cell units.

[0024] The operation of completing the battery cell unit with the template frame is as follows: construct the template frame using the average width h_m and average width w_m of the collection frame J; secondly, sort the collection frame J in each row and column, and use the spacing between adjacent frames to locate the missing battery cell unit frame row by row; finally, use the template frame to complete the position of the missing battery cell unit to obtain the final collection frame containing all battery cells.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. The UNet-based mask generation method of the present invention effectively improves the cutting accuracy of battery cell units and alleviates the problems of poor robustness and low generalization of the method caused by imaging and other problems; its brightness enhancement method can effectively improve the generalization ability of the model.

[0027] 2. The cell unit separation method of the present invention can effectively solve the problem of inaccurate and incomplete cell unit separation caused by poor image quality (such as large area cracks in the cell), thereby improving the cutting accuracy;

[0028] 3. The method of the present invention can be applied not only to full-cell, half-cell, and shingled photovoltaic modules, but also to any image with similar image features. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a flow chart of the method of the present invention;

[0030] Figure 2 The image mask distribution map obtained by the average cutting method in the prior art;

[0031] Figure 3 The image mask distribution map obtained by the present invention;

[0032] Figure 4 This is an image mask distribution diagram of the full-sheet photovoltaic cell panel of the present invention;

[0033] Figure 5 for Figure 4 The segmentation diagram of the battery cell unit;

[0034] Figure 6 This is an image mask distribution diagram of the half-sheet photovoltaic cell panel of the present invention;

[0035] Figure 7 for Figure 6 The segmentation diagram of the battery cell unit;

[0036] Figure 8 This is an image mask distribution diagram of the laminated photovoltaic cell panel of the present invention;

[0037] Figure 9 for Figure 8 Segmented image of the battery cell. DETAILED DESCRIPTION

[0038] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0039] like Figure 1 and combined Figures 2 to 9 As shown, a photovoltaic module image cutting method based on mask generation, the method steps are as follows:

[0040] Step 1: Construct an image dataset; obtain several photovoltaic module images with labeled cell edge lines and divide them into a training set and a test set;

[0041] Step 2: Image preprocessing method; maintain the original image aspect ratio for image scaling, then divide the image by 255 to normalize its pixels to between 0 and 1 as the input data of the network.

[0042] Step 3: Mask generation method based on UNet; an online random data augmentation strategy is used to train UNet. It is worth noting that in addition to conventional data augmentation techniques such as image translation and image filtering, this paper also proposes an adaptive brightness enhancement method to ensure that the pixel interval of the enhanced image remains within the pixel interval of the original image, thereby improving the generalization ability of the model. The adaptive brightness enhancement method is as follows:

[0043] Assume that the input image is X(i, j), the pixel value at (i, j) represents x ij , the corresponding enhanced pixel value is y ij , the image X pixel value range is [x min ,x max ], then:

[0044] y ij =(1+λ)×x ij ,

[0045] in

[0046] Where λ is a randomly generated value in the range of [-0.1, 0.1].

[0047] Step 4: Generate a binary image; input the test image into the trained UNet model to obtain a predicted image, and obtain a binary image of the test image through threshold truncation; assuming that the predicted image obtained by the trained UNet model for the test image X(i, j) is M(i, j), the truncation threshold is t, set the pixels where M(i, j)>t to 1, that is, the area composed of pixel 1 is the battery cell area (target area), set the pixels where M(i, j)>t to 0, that is, the area composed of pixel 0 is the background area, and finally obtain a binary image; the truncation threshold t is obtained through cross-validation.

[0048] Step 5: Separate candidate cell units. Unlike the general cell unit separation operation based on morphology and binarization, an improved cell unit separation method is proposed. It can better deal with the problem of inaccurate and incomplete cell unit segmentation caused by poor image quality (such as large cracks in the cell), making cell separation more robust. The specific operations of the improved cell unit separation method are as follows:

[0049] Assuming that the photovoltaic module contains m rows and n columns of cell units, its height is H and width is W, the cell height h can be roughly obtained r =H / (n+1); Next, set the convolution kernel size of the erosion operation, and then perform N erosion operations on the binary image. Then calculate the pixel sum of each row and column of the resulting binary image Mp. Let the pixel sums of the i-th row and j-th column be and Then when l_p i When ratio_w×hr is less than, all pixel values in row i are 0. j When <ratio_h×hr, all pixel values in the jth column are 0; finally, the optimized binary image is obtained.

[0050] Step 6: Generate battery cell units; extract connected regions in the optimized binary image, and each connected region is a candidate battery cell unit; then, extract the width and height features of each battery cell unit: w r and h r ; Set the threshold intervals [a, b] and [c, d] that meet the width and height of the battery cell unit to determine whether the battery cell unit is retained; if it exceeds the threshold range, remove the candidate battery cell unit; if it meets the threshold range, retain it, and use the external rectangular frame of each battery cell to represent its battery cell unit, and finally obtain several set frames J containing valid battery cell units; then determine whether the number of set frames J is equal to the total number of battery cell units; if so, end this step; if not, use the template frame to complete the missing battery cell units so that the final number of set frames is equal to the total number of battery cell units; the template frame completion method for the battery cell unit is as follows: construct a template frame, use the average width h of the set frame J m and the average width w m To construct a template frame; sort the collection frame J in each row and column, and use the spacing between adjacent frames to locate the missing position of the battery cell unit frame row by row; finally, use the template frame to supplement the position of the missing battery cell unit to obtain the final collection frame containing all battery cells.

[0051] Step 7: Obtain a coordinate frame containing any number of cell units through the collection frame, and then extract the corresponding cell unit image from the original photovoltaic module image according to the coordinate frame, that is, obtain the final cut cell unit collection.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1. The UNet-based mask generation method of the present invention effectively improves the cutting accuracy of battery cell units and alleviates the problems of poor robustness and low generalization of the method caused by imaging and other problems; its brightness enhancement method can effectively improve the generalization ability of the model.

[0054] 2. The cell unit separation method of the present invention can effectively solve the problem of inaccurate and incomplete cell unit separation caused by poor image quality (such as large area cracks in the cell), thereby improving the cutting accuracy;

[0055] 3. The method of the present invention can be applied not only to full-cell, half-cell, and shingled photovoltaic modules, but also to any image with similar image features.

[0056] Experimental steps:

[0057] (1) Obtain 300 photovoltaic module images and their corresponding labels; the training set consists of 200 photovoltaic module images and the test set contains 100 photovoltaic module images; the photovoltaic module consists of 6*24 cell units, where 6 represents the number of rows and 24 represents the number of columns.

[0058] (2) Reduce the length and width of the image by the same ratio, then divide the resulting image by 255 and normalize its pixel values to between 0 and 1;

[0059] (3) Obtain the trained model. First, select the image enhancement method to train the UNet network model online. Then, use cross entropy as the network loss function to calculate the network error, and use the stochastic gradient descent method to continuously update the network parameters to obtain the optimal network model.

[0060] (4) Input the test image into the mask M(i, j) obtained by the trained model, use a threshold of 0.5 to obtain a binary image, judge the area where M(i, j)>0.5 as the battery cell area (target area) and fill it with 1, judge the area where M(i, j)>0.5 as the background area and fill it with 0.

[0061] (6) According to the size of the gap between the cell units, determine the size of the structural element and the number of corrosion times in the corrosion operation, and act on the binary image; then calculate the rough height h of the cell unit r , where h r =Height of component image / 7; if the sum of pixels in each row is less than ratio_w×hr, then all pixel values in the row are 0; if the sum of pixels in each column is less than ratio_w×hr, then all pixel values in the column are 0, and finally the optimized binary image is obtained;

[0062] (7) One by one, traverse the candidate battery cells in the optimized binary image, and then find the external rectangular frame of the candidate battery cell unit, and record the width of the frame as w r , the height of the box is h r , and calculate w r / h r and h r / h r ; Secondly, set the threshold intervals [a,b] and [c,d] that meet the width and height of the battery cell, which will also meet w r / h r In [a,b] and h r / h rThe frames between [c, d] are retained, and the rest of the frames are removed, and finally the set of battery cell frames corresponding to the second mask is obtained, which is recorded as the first set frame. Finally, it is determined whether the number of the first set frames is the total number of battery cell units. If not, the average width and height of the frames in the first set frame are calculated as the width and height of the template frame. Then, the first set frame is sorted row by row and column by column. The spacing between adjacent frames is used to locate the missing position of the battery cell frame row by row, and the position is supplemented by the template frame. If it is satisfied, the first set frame is the final set of battery cell frames. Finally, a set frame containing all battery cells is obtained.

[0063] (8) Generate a sub-image containing any number of cell units; first sort each row and column according to the set frame obtained in step 6, then, according to the target task, obtain the coordinate frame containing any number of cell units, and then crop the photovoltaic module image to obtain the final sub-image set.

[0064] Figure 2 and Figure 3 This is a comparison of the image mask distribution using the average cutting method and the image cutting method proposed in this embodiment. IOU is used for evaluation. IOU is equivalent to the result obtained by dividing the overlapping part of two regions by the combined part of the two regions. The comparison results are shown in Table 1:

[0065] Table 1 Comparison of processing results of the average cutting method and the image cutting method of this embodiment

[0066] Evaluation Metrics Average Cut Method Method of the present invention IOU 0.71 0.98

[0067] As can be seen from Table 1, compared with the average cutting method, the method proposed in this embodiment can completely separate the battery cell units, the IOU is close to 1, and the cutting success rate can reach 100% in 5,000 pictures.

[0068] The method proposed in this embodiment is used to cut the image of the full-sheet photovoltaic cell panel. The obtained image cutting results and cell unit are respectively Figure 4 and Figure 5 ;

[0069] The method proposed in this embodiment is used to cut the image of a half-sheet photovoltaic cell panel. The obtained image cutting results and cell units are respectively Figure 6 and Figure 7 ;

[0070] The method proposed in this embodiment is used to cut the image of a half-sheet photovoltaic cell panel. The obtained image cutting results and cell units are respectively Figure 8 and Figure 9 .

[0071] The above description is only a preferred embodiment of the invention and does not limit the patent scope of the invention. Any equivalent structure or equivalent process transformation made by using the contents of the invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A photovoltaic module image cutting method based on mask generation, characterized by: The method steps are: Step 1: Construct image dataset; Step 2: Image preprocessing; Step 3: Generate UNet-based mask; Step 4: Generate binary image; Step 5: Separate candidate cell units; Step 6: Generate battery cell units; Step 7: Obtain a coordinate frame containing any number of cell units through the collection frame, and then extract the corresponding cell unit image from the original photovoltaic module image based on the coordinate frame; The cell unit separation method in step 5 is as follows: Assume that the photovoltaic module contains m rows and n columns of cell units, with a height of H and a width of W. The cell height h r =H / (n+1); set the convolution kernel size of the erosion operation, then perform N erosion operations on the binary image, and then calculate the pixel sum of each row and column of the binary image Mp obtained; let the pixel sum of the i-th row and the j-th column be and , then when When , all pixel values in row i are 0, When , all pixel values in the jth column are 0; finally, the optimized binary image is obtained.

2. The photovoltaic module image cutting method based on mask generation according to claim 1, characterized in that: The steps for constructing the image dataset in step 1 are as follows: obtaining a number of photovoltaic module images with the cell edge lines marked, and dividing them into a training set and a test set.

3. The photovoltaic module image cutting method based on mask generation according to claim 1, characterized in that: The image preprocessing method in step 2 is: scaling the image while maintaining the original image aspect ratio, and then dividing the image by 255 to normalize its pixels to between 0 and 1 as the input data of the network.

4. The photovoltaic module image cutting method based on mask generation according to claim 1, characterized in that: The binary image generation method in step 4 is as follows: input the test image into the trained UNet model to obtain the predicted image, and obtain the binary image of the test image by threshold truncation; assuming that the predicted image obtained by the trained UNet model for the test image X(i, j) is M(i, j), the truncation threshold is t, The pixels of 1 are set to 1, that is, the area composed of pixel 1 is the battery area. The pixels of are set to 0, that is, the area composed of pixel 0 is the background area, and finally a binary image is obtained.

5. The photovoltaic module image cutting method based on mask generation according to claim 1, characterized in that: The method for generating the battery cell in step 6 is as follows: Connected regions are extracted from the optimized binary image. Each connected region is a candidate battery cell unit. Next, the width and height features of each battery cell unit are extracted: w r and h r ; Set the threshold intervals [a, b] and [c, d] that meet the width and height of the battery cell unit to determine whether the battery cell unit is retained; if it exceeds the threshold range, remove the candidate battery cell unit; if it meets the threshold range, retain it, and use the external rectangular frame of each battery cell to represent its battery cell unit, and finally obtain several set frames J containing valid battery cell units; then determine whether the number of set frames J is equal to the total number of battery cell units; if so, end this step; if not, use the template frame to complete the missing battery cell units so that the final number of set frames is equal to the total number of battery cell units.

6. The photovoltaic module image cutting method based on mask generation according to claim 1, characterized in that: The steps for completing the template frame of the battery cell unit are as follows: construct the template frame, use the average width h of the set frame J m and the average width w m To construct a template frame; sort the collection frame J in each row and column, and use the spacing between adjacent frames to locate the missing position of the battery cell unit frame row by row; finally, use the template frame to supplement the position of the missing battery cell unit to obtain the final collection frame containing all battery cells.

Images