Photovoltaic module detection method, device, equipment and medium

Abstract

The invention discloses a photovoltaic module detection method, device, equipment and medium, and relates to the technical field of photovoltaic string, and the method comprises the steps: segmenting a visible light photovoltaic image to obtain a photovoltaic string mask image; extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask pattern, and extracting a plurality of defect string images from the plurality of photovoltaic string images; graying the defect string images, and determining an upper bound threshold and a lower bound threshold of each defect string image based on the gray histogram; thresholding the defect string image based on an upper bound threshold and a lower bound threshold to obtain a thresholding image, and determining a photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholding image; and classifying the photovoltaic modules in the photovoltaic module edge frame based on the module types by using a lightweight classification network to determine defective modules, frame-selecting the defective modules in the visible light photovoltaic image, and marking the module types in the defective module frame. The photovoltaic string can be rapidly detected, and the generalization of string detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic strings, and in particular to a photovoltaic component detection method, device, equipment and medium. Background Art

[0002] At present, in the daily operation of photovoltaic power stations, due to environmental factors and defects in the components themselves, photovoltaic modules may have various problems, resulting in performance degradation and even safety hazards. Among them, shadows, bird droppings, shadows, and foreign matter such as dust are very common defects. These defects will not only reduce the output power of photovoltaic modules, but may also cause local overheating, which will further damage the modules and affect the overall operating efficiency and safety of photovoltaic power stations.

[0003] With the development of drone technology and image processing technology, photovoltaic module defect detection methods based on visible light images have gradually attracted attention. Visible light images have the advantages of being easy to obtain, relatively low cost, and not affected by ambient temperature. By combining drone inspection technology and advanced image processing algorithms, large-scale and rapid inspections of photovoltaic power stations can be achieved, and defects in photovoltaic modules can be discovered in a timely manner.

[0004] However, the existing hot spot detection methods for photovoltaic power station strings still have the following shortcomings: (1) The detection speed is slow due to the large number of components; (2) When using traditional image processing methods to achieve component segmentation, the method has poor generalization. Specifically, due to the diverse environmental conditions of photovoltaic power stations, traditional image processing methods often rely on fixed thresholds and rules, which makes them show great limitations under different lighting conditions, background complexity and camera angles. These methods are difficult to adapt to various complex situations when faced with ever-changing scenes. Although based on three methods: straight line detection, threshold segmentation, and equidistant segmentation, the threshold segmentation method uses a 99.46% success rate. However, when the scene changes, the threshold needs to be readjusted, and the generalization is insufficient.

[0005] In summary, how to quickly detect photovoltaic strings and improve the generalization of string detection is a problem that needs to be solved urgently. Summary of the invention

[0006] In view of this, the object of the present invention is to provide a photovoltaic module detection method, device, equipment and medium, which can quickly detect photovoltaic strings and improve the generalization of string detection. The specific scheme is as follows:

[0007] In a first aspect, the present application discloses a photovoltaic module detection method, comprising:

[0008] The semantic segmentation model is used to segment the visible light photovoltaic image to obtain the photovoltaic string mask map;

[0009] Extracting a number of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extracting a number of defective string images from the photovoltaic string images using a target detection model;

[0010] Graying the defective string image to obtain a grayscale histogram, and determining an upper threshold and a lower threshold corresponding to each defective string image based on the grayscale histogram;

[0011] Thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image;

[0012] A lightweight classification network is used to classify the photovoltaic components in the photovoltaic component edge frame based on component types to determine defective components, and the defective components are framed in the visible light photovoltaic image, and the component type is marked in the defective component frame.

[0013] The step of extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image includes:

[0014] Determine each photovoltaic string edge frame in the photovoltaic string mask image based on a contour detection algorithm;

[0015] Determine the minimum adjacent parallelogram corresponding to each photovoltaic string edge frame, and filter the minimum adjacent parallelograms whose area is smaller than a preset area;

[0016] Based on the photovoltaic string edge frame corresponding to the remaining minimum adjacent parallelogram, a plurality of photovoltaic string images are extracted from the visible light photovoltaic image.

[0017] Wherein, determining the upper threshold and the lower threshold corresponding to each defect group string image based on the grayscale histogram includes:

[0018] Performing polynomial fitting on the grayscale histogram to obtain a target polynomial, and determining a maximum value set and a minimum value set corresponding to the target polynomial;

[0019] Determine the maximum value in the maximum value set, and use two minimum values ​​adjacent to the maximum value as the upper threshold and the lower threshold corresponding to the defect group string image;

[0020] The upper threshold and the lower threshold are adjusted based on the number of pixels corresponding to the upper threshold and the lower threshold respectively.

[0021] The adjusting the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold, respectively, includes:

[0022] Determine whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number;

[0023] If so, adjust the upper threshold and / or the lower threshold to other adjacent minimum values, and jump to the step of determining whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number, until the number of pixels corresponding to the upper threshold and the lower threshold is not greater than the target number.

[0024] The step of thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image comprises:

[0025] Thresholding the defect group string image based on the upper threshold and the lower threshold to obtain a thresholded image;

[0026] Determining a photovoltaic module edge frame of each photovoltaic module in the defective string image using a contour detection algorithm and based on the thresholded image;

[0027] If the number of edge frames of the photovoltaic assembly is 0, the upper threshold is lowered, and the process jumps to the step of thresholding the defective string image based on the upper threshold and the lower threshold until the number of edge frames of the photovoltaic assembly is not 0.

[0028] Wherein, after thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic component edge frame of each photovoltaic component in the defective string image based on the thresholded image, it also includes:

[0029] Each photovoltaic component edge frame is screened based on a preset range, and the photovoltaic component edge frames that do not meet the preset range are removed; the preset range includes a standard area range, a standard aspect ratio range, a standard length range and a standard width range.

[0030] Wherein, before the photovoltaic components in the photovoltaic component edge frame are classified based on component type by using a lightweight classification network to determine defective components, the method further includes:

[0031] If the target area connected to the photovoltaic component edge frame does not exceed the image range and there is no other photovoltaic component edge frame, the photovoltaic component edge frame is translated to the target area.

[0032] In a second aspect, the present application discloses a photovoltaic module detection device, comprising:

[0033] A segmentation module is used to segment the visible light photovoltaic image using a semantic segmentation model to obtain a photovoltaic string mask image;

[0034] An extraction module, configured to extract a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and to extract a plurality of defective string images from the plurality of photovoltaic string images using a target detection model;

[0035] A threshold determination module, used to grayscale the defective string image to obtain a grayscale histogram, and determine an upper threshold and a lower threshold corresponding to each defective string image based on the grayscale histogram;

[0036] a candidate frame determination module, configured to threshold the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determine a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image;

[0037] A classification module is used to classify the photovoltaic components in the photovoltaic component edge frame based on component type using a lightweight classification network to determine defective components, select the defective components in the visible light photovoltaic image, and mark the component type in the defective component frame.

[0038] In a third aspect, the present application discloses an electronic device, comprising:

[0039] Memory, used to store computer programs;

[0040] The processor is used to execute the computer program to implement the photovoltaic component detection method disclosed above.

[0041] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the photovoltaic component detection method disclosed above is implemented.

[0042] It can be seen that the present application uses a semantic segmentation model to segment a visible light photovoltaic image to obtain a photovoltaic string mask map; extracts a number of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask map, and uses a target detection model to extract a number of defective string images from the photovoltaic string images; grayscales the defective string images to obtain a grayscale histogram, and determines the upper threshold and lower threshold corresponding to each defective string image based on the grayscale histogram; thresholds the defective string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determines the photovoltaic component edge frames of each photovoltaic component in the defective string image based on the thresholded image; uses a lightweight classification network to classify the photovoltaic components in the photovoltaic component edge frame based on the component type to determine the defective components, selects the defective components in the visible light photovoltaic image, and marks the component type in the defective component frame. It can be seen that the present application utilizes a semantic segmentation model and a target detection model to extract defective string images from visible light photovoltaic images, and the subsequent processing process only needs to process the defective string images instead of the entire visible light photovoltaic images, thereby improving the processing speed; the present application determines the upper and lower thresholds corresponding to the defective string images based on the grayscale histogram of the defective string images, rather than pre-setting them, so the threshold setting is more universal and can improve generalization; in summary, the present application can quickly detect photovoltaic strings and improve the generalization of string detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A flow chart of a photovoltaic module detection method disclosed in this application;

[0045] Figure 2 A visible light photovoltaic image disclosed in this application;

[0046] Figure 3 A photovoltaic string mask diagram disclosed in this application;

[0047] Figure 4 A string image consisting of 6 photovoltaic string images disclosed in this application;

[0048] Figure 5 A group string image consisting of two defect group string images disclosed in the present application;

[0049] Figure 6A thresholded image disclosed in the present application;

[0050] Figure 7 A flowchart of a specific photovoltaic module detection method disclosed in this application;

[0051] Figure 8 A schematic diagram of missed detection adjustment disclosed in this application;

[0052] Fig. 9 A schematic diagram of a photovoltaic string after single missed detection adjustment disclosed in the present application;

[0053] Fig.10 A visible light photovoltaic image after annotating a defective component disclosed in the present application;

[0054] Fig.11 Annotate an image of a partially defective component disclosed in the present application;

[0055] Fig.12 This is a schematic diagram of the structure of a photovoltaic module detection device disclosed in this application;

[0056] Fig.13 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Existing methods for detecting hot spots in photovoltaic power station strings still have the following shortcomings: (1) The number of components is large and the detection speed is slow; (2) When using traditional image processing methods to achieve component segmentation, the method has poor generalization. Specifically, due to the diverse environmental conditions of photovoltaic power stations, traditional image processing methods often rely on fixed thresholds and rules, which makes them show great limitations under different lighting conditions, background complexity and camera angles. These methods are difficult to adapt to various complex situations when faced with ever-changing scenes. Although based on three methods: straight line detection, threshold segmentation, and equidistant segmentation, the threshold segmentation method uses a 99.46% success rate. However, when the scene changes, the threshold needs to be readjusted, and the generalization is insufficient.

[0059] To this end, an embodiment of the present application proposes a photovoltaic module detection solution that can quickly detect photovoltaic strings and improve the generalization of string detection.

[0060] The present application discloses a photovoltaic module detection method. Figure 1 As shown, the method includes:

[0061] Step S11: Segment the visible light photovoltaic image using a semantic segmentation model to obtain a photovoltaic string mask image.

[0062] In this embodiment, the method of segmenting the visible light photovoltaic image using the semantic segmentation model to obtain the photovoltaic string mask image includes: segmenting the visible light photovoltaic image using the semantic segmentation model updated based on the loss function and the gradient descent method to obtain the photovoltaic string mask image; the loss function is a loss function composed of a linear combination of the cross entropy loss and the feature matching loss. Figure 2 As shown, it is a visible light photovoltaic image. Figure 3 The figure shows a photovoltaic string mask diagram.

[0063] It should be pointed out that it is necessary to first train the semantic segmentation network (UNet network) to obtain the optimal semantic segmentation model, and then use the semantic segmentation model for segmentation. Specifically, the semantic segmentation network is obtained by using the EfficientNet-B7 network and adding the MCF module at the connection between the encoder and the decoder. The MCF module effectively saves computing resources while expanding the receptive field to capture multi-scale spatial context information, thereby improving the segmentation accuracy of the UNet network; among them, MCF, also known as the MOM connector framework, is a technology based on network services.

[0064] It should be pointed out that the training process is as follows: obtain the visible light zoom image to be trained, annotate the photovoltaic strings in the visible light zoom image of the photovoltaic power station (including the location information of the photovoltaic strings), and make a segmented data set. In order to reduce the risk of overfitting and enable the model to better adapt to different scenes and lighting conditions, data augmentation is used to increase data diversity; input the samples into the UNet network in batches, perform feature extraction and fusion layer by layer in the network, and finally obtain a comprehensive feature fusion result; perform a softmax operation (normalization operation) on the final feature fusion result to obtain the image segmentation result predicted by the UNet network; use a linear combination of cross entropy loss and feature matching loss to form the loss function of the predicted image segmentation result and the standard segmentation map, which is defined as follows:

[0065] ;

[0066] is the coefficient; is the cross entropy loss; is the feature matching loss.

[0067] Considering that the photovoltaic string identification in this application is a binary classification task, the cross entropy loss function formula is as follows:

[0068] ;

[0069] in, Indicates the total number of pixels; Indicates The probability that a pixel point is predicted to be a cluster area; To be the first The probability that a pixel is predicted to be background;

[0070] use It represents the unlabeled (annotated) samples, that is, the samples that are not labeled when the photovoltaic strings are labeled in the visible light zoom image of the photovoltaic power station. The definition of feature matching loss is as follows:

[0071] ;

[0072] in, represents the predicted segmentation map (i.e., the PV string mask map); Represents the actual segmentation map (i.e., the PV string mask map); represents the marked sample, that is, the sample marked when the photovoltaic strings are marked in the visible light zoom diagram of the photovoltaic power station; Represents the image stitching operation, which stitches the input image and the corresponding mask image; Represents a labeled sample dataset; represents the intermediate representation of the discriminator network at layer k; Represents the norm.

[0073] Then, the gradient descent method is used to update the model parameters to obtain the optimal model parameters, and the optimal model parameters are used to segment the photovoltaic strings and output the final mask image.

[0074] It should be pointed out that the required photovoltaic images can be collected by drones.

[0075] Step S12: extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extracting a plurality of defective string images from the plurality of photovoltaic string images using a target detection model.

[0076] In this embodiment, the step of extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image includes: determining each photovoltaic string edge frame in the photovoltaic string mask image based on a contour detection algorithm; determining the minimum adjacent parallelogram corresponding to each photovoltaic string edge frame, and filtering the minimum adjacent parallelograms whose area is smaller than a preset area; and extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string edge frames corresponding to the remaining minimum adjacent parallelograms. Figure 4 As shown, a string image consisting of 6 photovoltaic string images is shown; see Figure 5 As shown, a string image is composed of two defect string images.

[0077] It should be noted that the convex hull algorithm and the rotating calcaneal method are used to determine the minimum adjacent parallelogram corresponding to each photovoltaic string edge frame.

[0078] It should be pointed out that the target detection model is also pre-trained. Specifically, the defects in the photovoltaic strings are annotated; the annotated data is input into the YOLOv5 network, and the feature map is generated after passing through the backbone network (backbone), neck network (neck) and head network (head) in sequence, and then the prediction box is obtained through detection box decoding and non-maximum suppression (NMS). Then, these predicted boxes are compared with the annotated real boxes to calculate the classification loss and positioning loss; based on the loss function, the network weights are adjusted using an optimization algorithm to minimize the loss; the optimal model parameters are obtained, and the confidence threshold is appropriately lowered to obtain a model with a lower missed detection rate, so as to obtain photovoltaic strings that may have defects. Figure 5 It shows from Figure 4 The two photovoltaic strings that may have defects are extracted from the image, and each photovoltaic string that may have defects is a defective string image. Figure 5 This is to facilitate the display of two defect group images together.

[0079] Step S13: graying the defective string image to obtain a grayscale histogram, and determining an upper threshold and a lower threshold corresponding to each defective string image based on the grayscale histogram.

[0080] In this embodiment, grayscale conversion and threshold determination are performed one defect group string image at a time, and each defect image has a corresponding grayscale histogram and an upper threshold and a lower threshold.

[0081] In this embodiment, the upper threshold and the lower threshold are not predetermined in advance, but are temporarily calculated and determined by the grayscale histogram. Therefore, it is applicable to various images, and the determination of the threshold is more generalized.

[0082] Step S14: thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image.

[0083] See also Figure 6 Shown is a thresholded image.

[0084] In this embodiment, the thresholded image makes it easier to determine the edge frame of the photovoltaic module.

[0085] Step S15: using a lightweight classification network to classify the photovoltaic components in the photovoltaic component edge frame based on component type to determine defective components, and select the defective components in the visible light photovoltaic image, and mark the component type in the defective component frame.

[0086] In this embodiment, the photovoltaic component edge frame is used to crop the component and realize perspective transformation; the obtained photovoltaic component image is input into the improved ShufflenetV2 for classification, which can be specifically divided into five component types: normal, bird droppings, occlusion, shadow and dust, and the component edge frame and component type whose results are not normal are stored in the corresponding dictionary set; the dictionary set storing the detection results is traversed, the defective component is framed in the original image and the component type is indicated in the frame, specifically, in the aforementioned process, the position of the photovoltaic component edge frame corresponding to the defective component in the defective string image is determined, the position of the defective string image in the photovoltaic string image is determined, and the position of the photovoltaic string image in the photovoltaic string mask image and the visible light photovoltaic image is determined, thereby determining the position of the photovoltaic component edge frame corresponding to the defective component in the visible light photovoltaic image, so as to frame the defective component in the original image and indicate the defect type in the frame.

[0087] It should be pointed out that in the improved ShufflenetV2, the PRelu (Parametric RectifiedLinear Unit) activation function (Relu with parameters) is used to replace the Relu nonlinear excitation function, so that the model can have better adaptability and better gradient flow, and can enable the network to obtain higher accuracy and faster convergence speed; at the same time, the channel and space dual attention mechanism CBAM (Convolutional Block AttentionModule) module is added between each module to improve the expression ability of features.

[0088] It should be pointed out that the lightweight classification model is also a pre-trained model. The training process is as follows: by cropping the components and implementing perspective transformation, the obtained photovoltaic component images are input into the improved ShufflenetV2 for classification. Specifically, they can be divided into five categories: normal, bird droppings, occlusion, shadow and dust. The edge frames and defect types of components that are not normal are stored in the corresponding collection. The segmented components are grouped into five categories: normal, bird droppings, occlusion, shadow and dust. The number of samples is increased by data augmentation. The images are input into the network in batches, and the improved ShuffleNetV2 generates the final feature fusion result by layer-by-layer feature extraction and fusion. The final feature fusion result is subjected to softmax operation (normalization operation) to obtain the prediction result. The loss is calculated based on the prediction result, and the parameters of the model are updated according to the result of the loss function using the gradient descent method to minimize the loss function and obtain the optimal model, so that the optimal model can be used to judge the type of photovoltaic components in the future.

[0089] In summary, this application uses a semantic segmentation network to extract the string area in the visible light photovoltaic image taken by the drone, and uses the segmented strings through the target detection network to pre-screen the defective strings in the visible light photovoltaic image, and identify the strings that may have defects. Then, a photovoltaic component segmentation algorithm based on adaptive dual thresholds and translation adjustment is used to achieve component segmentation of defective strings in visible light photovoltaic images. Finally, defective components are detected and identified based on a lightweight classification network to determine the existence and type of defects, and the defective components and defect types are displayed in the picture.

[0090] It can be seen that the present application uses a semantic segmentation model to segment a visible light photovoltaic image to obtain a photovoltaic string mask map; extracts a number of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask map, and uses a target detection model to extract a number of defective string images from the photovoltaic string images; grayscales the defective string images to obtain a grayscale histogram, and determines the upper threshold and lower threshold corresponding to each defective string image based on the grayscale histogram; thresholds the defective string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determines the photovoltaic component edge frames of each photovoltaic component in the defective string image based on the thresholded image; uses a lightweight classification network to classify the photovoltaic components in the photovoltaic component edge frame based on the component type to determine the defective components, selects the defective components in the visible light photovoltaic image, and marks the component type in the defective component frame. It can be seen that the present application utilizes a semantic segmentation model and a target detection model to extract defective string images from visible light photovoltaic images, and the subsequent processing process only needs to process the defective string images instead of the entire visible light photovoltaic images, thereby improving the processing speed; the present application determines the upper and lower thresholds corresponding to the defective string images based on the grayscale histogram of the defective string images, rather than pre-setting them, so the threshold setting is more universal and can improve generalization; in summary, the present application can quickly detect photovoltaic strings and improve the generalization of string detection.

[0091] The embodiment of the present application discloses a specific photovoltaic module detection method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Figure 7 As shown, specifically including:

[0092] Step S21: Segment the visible light photovoltaic image using a semantic segmentation model to obtain a photovoltaic string mask image.

[0093] Step S22: extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extracting a plurality of defective string images from the plurality of photovoltaic string images using a target detection model.

[0094] Step S23: grayscale the defect group string image to obtain a grayscale histogram, perform polynomial fitting on the grayscale histogram to obtain a target polynomial, and determine a maximum value set and a minimum value set corresponding to the target polynomial; determine the maximum value in the maximum value set, and use the two minimum values ​​adjacent to the maximum value as the upper threshold and lower threshold corresponding to the defect group string image, respectively.

[0095] In this embodiment, a photovoltaic component segmentation algorithm based on adaptive dual thresholds and translation adjustment realizes the component segmentation of defective strings in visible light photovoltaic images; the dual thresholds are the upper threshold and the lower threshold, and the translation adjustment refers to shifting the strings at other positions to the string positions where the strings are not detected to solve the problem of missed detection.

[0096] In this embodiment, the defect string image is grayed using a weighted value method, and a grayscale histogram is statistically calculated. The grayscale value calculation formula is as follows:

[0097] ;

[0098] Among them, Y represents the grayscale value, and RGB represents the colors of the three channels of red, green, and blue.

[0099] The least squares method is used to perform polynomial fitting on the grayscale histogram, and the argrelextrema function in the SciPy library is used to find the maximum value set and the minimum value set in the grayscale value array; the maximum value among the maximum values ​​is obtained, and the minimum value adjacent to its (maximum value) subscript (the subscript represents the pixel point and position) in the minimum value list is found as the upper and lower bounds of thresholding (that is, the upper threshold and the lower threshold).

[0100] Step S24: adjusting the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold respectively.

[0101] In this embodiment, the adjusting of the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold respectively includes: judging whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number; if so, adjusting the upper threshold and / or the lower threshold to other adjacent minimum values, and jumping to the step of judging whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number, until the number of pixels corresponding to the upper threshold and the lower threshold is not greater than the target number. It should be pointed out that other adjacent minimum values ​​are selected based on the distance from near to far.

[0102] Step S25: thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image.

[0103] In this embodiment, after the upper threshold is determined according to the pixels, the upper threshold is not fixed. In the process of determining the edge frame of the photovoltaic component, the upper threshold is also changed according to the number of edge frames. Specifically, the defective group string image is thresholded based on the upper threshold and the lower threshold to obtain a thresholded image, and the photovoltaic component edge frame of each photovoltaic component in the defective group string image is determined based on the thresholded image, including: thresholding the defective group string image based on the upper threshold and the lower threshold to obtain a thresholded image; using a contour detection algorithm and based on the thresholded image, determining the photovoltaic component edge frame of each photovoltaic component in the defective group string image; if the number of photovoltaic component edge frames is 0, lowering the upper threshold, and jumping to the step of thresholding the defective group string image based on the upper threshold and the lower threshold until the number of photovoltaic component edge frames is not 0.

[0104] It should be noted that, if the number of edge frames of the photovoltaic module is still 0 after jumping four times, that is, after the edge frame is determined five times, the image is recorded and the jumping stops.

[0105] In this embodiment, after obtaining the thresholded image, screening is performed based on the area aspect ratio of the photovoltaic component edge frame. Specifically, the defective group string image is thresholded based on the upper threshold and the lower threshold to obtain the thresholded image, and the photovoltaic component edge frames of each photovoltaic component in the defective group string image are determined based on the thresholded image. It also includes: screening each photovoltaic component edge frame based on a preset range, and removing the photovoltaic component edge frames that do not meet the preset range; the preset range includes a standard area range, a standard aspect ratio range, a standard length range and a standard width range.

[0106] The specific screening formula is as follows:

[0107] ;

[0108] ;

[0109] in, Indicates the actual width of the PV panel edge frame, in units of (pixels); Indicates the target width of the PV panel edge frame, in units of ; Indicates the actual length of the PV panel edge frame, in units of ; Indicates the target length of the PV panel edge frame, in units of ; Indicates the edge frame area of ​​the photovoltaic module, in units of .

[0110] In the present embodiment, if a missed detection occurs in the string, it is compensated by translation adjustment measures. Specifically, before the photovoltaic components in the photovoltaic component edge frame are classified based on the component type by using a lightweight classification network to determine the defective components, it also includes: if the target area connected to the photovoltaic component edge frame does not exceed the image range and there are no other photovoltaic component edge frames, the photovoltaic component edge frame is translated to the target area.

[0111] It should be pointed out that the steps of translation adjustment are as follows: traverse the existing edge boxes to determine whether there are other edge boxes above, below, left and right or whether they exceed the image range; calculate the interval between group strings based on adjacent edge boxes; when a component is not detected, calculate the slope of the minimum adjacent parallelogram side of the adjacent components, the side length and the interval between components, obtain a new edge box through translation and add it to the edge box set.

[0112] See also Figure 8 As shown, it is a schematic diagram of missed detection adjustment; the image on the left side of the figure is a single defective string image with the edge frame of the photovoltaic module determined, there are missed detection parts in the figure (that is, the photovoltaic modules not framed by the edge frame of the photovoltaic modules), and the missed detection parts are shown in the upper right image of the figure (the photovoltaic modules in the third and fourth columns of the second row of three rows of photovoltaic modules are the parts not framed by the edge frame of the photovoltaic modules, that is, the missed detection parts), and the arrow in the figure is to translate the non-missed detection part on the lower side to the missed detection part on the upper side to obtain the translated image on the lower right side (the non-missed detection part currently used is the photovoltaic modules in the third and fourth columns of the third row of three rows of photovoltaic modules); it should be pointed out that although it is translated, the edge frame of the photovoltaic module of the non-missed detection part before the translation is still retained.

[0113] See also Fig. 9 As shown, it is a schematic diagram of a photovoltaic string after a single missed detection adjustment; in the figure, each photovoltaic module is framed by the edge of the photovoltaic module without omission.

[0114] Step S26: using a lightweight classification network to classify the photovoltaic components in the photovoltaic component edge frame based on component type to determine defective components, and select the defective components in the visible light photovoltaic image, and mark the component type in the defective component frame.

[0115] In this embodiment, the defective component is selected in the visible light photovoltaic image, and the component type is marked in the defective component frame. Fig.10 and Fig.11 As shown, Fig.10 , which is a visible light photovoltaic image after defective components are marked, obtained after the defective components are framed in the visible light photovoltaic image and the component type is marked in the defective component frame. Fig.11For Fig.10 The marked images of some defective components are displayed after the defect marked parts are cut out from the visible light photovoltaic image; Fig.11 In the two rows of photovoltaic modules, the photovoltaic modules in the third column of the first row and the third column of the second row have white spots, which are defective parts, and these two photovoltaic modules are framed. Fig.10 Not obvious, special use Fig.11 Details display.

[0116] It can be seen that the present application uses a semantic segmentation model to segment a visible light photovoltaic image to obtain a photovoltaic string mask map; extracts a number of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask map, and uses a target detection model to extract a number of defective string images from the photovoltaic string images; grayscales the defective string images to obtain a grayscale histogram, performs polynomial fitting on the grayscale histogram to obtain a target polynomial, and determines the maximum value set and minimum value set corresponding to the target polynomial; determines the maximum value in the maximum value set, and uses the two minimum values ​​adjacent to the maximum value as the defective string images, respectively. The upper threshold and lower threshold corresponding to the image are respectively adjusted based on the number of pixels corresponding to the upper threshold and the lower threshold; the defective string image is thresholded based on the upper threshold and the lower threshold to obtain a thresholded image, and the photovoltaic component edge frame of each photovoltaic component in the defective string image is determined based on the thresholded image; the photovoltaic components in the photovoltaic component edge frame are classified based on the component type using a lightweight classification network to determine the defective components, and the defective components are selected in the visible light photovoltaic image, and the component type is marked in the defective component frame. It can be seen that the present application determines the upper threshold and the lower threshold according to the maximum and minimum values, and the upper threshold and the lower threshold can be adjusted by the number of pixels. The upper threshold can also be adjusted according to the number of edge frames later, which can be applied to the dual threshold setting of different images, making the dual threshold setting more generalized, thereby improving the generalization of string detection.

[0117] Accordingly, the present application also discloses a photovoltaic module detection device, see Fig.12 As shown, the device comprises:

[0118] A segmentation module 11 is used to segment the visible light photovoltaic image using a semantic segmentation model to obtain a photovoltaic string mask image;

[0119] An extraction module 12 is used to extract a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and to extract a plurality of defective string images from the plurality of photovoltaic string images using a target detection model;

[0120] A threshold determination module 13, configured to grayscale the defective string image to obtain a grayscale histogram, and determine an upper threshold and a lower threshold corresponding to each defective string image based on the grayscale histogram;

[0121] A candidate frame determination module 14 is used to threshold the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determine the photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image;

[0122] The classification module 15 is used to classify the photovoltaic components in the photovoltaic component edge frame based on component type using a lightweight classification network to determine defective components, select the defective components in the visible light photovoltaic image, and mark the component type in the defective component frame.

[0123] Among them, the more specific working process of each of the above modules can refer to the corresponding content disclosed in the above embodiments, which will not be repeated here.

[0124] It can be seen that the present application uses a semantic segmentation model to segment a visible light photovoltaic image to obtain a photovoltaic string mask map; extracts a number of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask map, and uses a target detection model to extract a number of defective string images from the photovoltaic string images; grayscales the defective string images to obtain a grayscale histogram, and determines the upper threshold and lower threshold corresponding to each defective string image based on the grayscale histogram; thresholds the defective string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determines the photovoltaic component edge frames of each photovoltaic component in the defective string image based on the thresholded image; uses a lightweight classification network to classify the photovoltaic components in the photovoltaic component edge frame based on the component type to determine the defective components, selects the defective components in the visible light photovoltaic image, and marks the component type in the defective component frame. It can be seen that the present application utilizes a semantic segmentation model and a target detection model to extract defective string images from visible light photovoltaic images, and the subsequent processing process only needs to process the defective string images instead of the entire visible light photovoltaic images, thereby improving the processing speed; the present application determines the upper and lower thresholds corresponding to the defective string images based on the grayscale histogram of the defective string images, rather than pre-setting them, so the threshold setting is more universal and can improve generalization; in summary, the present application can quickly detect photovoltaic strings and improve the generalization of string detection.

[0125] Furthermore, an embodiment of the present application also provides an electronic device. Fig.13 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0126] Fig.13 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26, and a communication bus 27. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the photovoltaic module detection method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0127] In this embodiment, the power supply 26 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 24 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0128] In addition, the memory 22, as a carrier for storing resources, may be a read-only memory, a random access memory, a disk or an optical disk, etc., and the resources stored thereon may include a computer program 221, and the storage method may be temporary storage or permanent storage. In addition to including a computer program that can be used to complete the photovoltaic module detection method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 221 may further include a computer program that can be used to complete other specific tasks.

[0129] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the photovoltaic module detection method disclosed above is implemented.

[0130] The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, which will not be repeated here.

[0131] The various embodiments in this application are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0132] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0134] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0135] The above is a detailed introduction to a photovoltaic module detection method, device, equipment, and storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A photovoltaic module detection method, characterized in that: include: The semantic segmentation model is used to segment the visible light photovoltaic image to obtain the photovoltaic string mask map; Extracting a number of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extracting a number of defective string images from the photovoltaic string images using a target detection model; Graying the defect group string image to obtain a grayscale histogram, and determining an upper threshold and a lower threshold corresponding to each defect group string image based on the grayscale histogram; Thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image; A lightweight classification network is used to classify the photovoltaic components in the photovoltaic component edge frame based on component types to determine defective components, and the defective components are framed in the visible light photovoltaic image, and the component type is marked in the defective component frame.

2. The photovoltaic module detection method according to claim 1, characterized in that: The step of extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image comprises: Determine each photovoltaic string edge frame in the photovoltaic string mask image based on a contour detection algorithm; Determine the minimum adjacent parallelogram corresponding to each photovoltaic string edge frame, and filter the minimum adjacent parallelograms whose area is smaller than a preset area; Based on the photovoltaic string edge frame corresponding to the remaining minimum adjacent parallelogram, a plurality of photovoltaic string images are extracted from the visible light photovoltaic image.

3. The photovoltaic module detection method according to claim 1, characterized in that: The step of determining the upper threshold and the lower threshold corresponding to each defect group string image based on the grayscale histogram includes: Performing polynomial fitting on the grayscale histogram to obtain a target polynomial, and determining a maximum value set and a minimum value set corresponding to the target polynomial; Determine the maximum value in the maximum value set, and use two minimum values ​​adjacent to the maximum value as the upper threshold and the lower threshold corresponding to the defect group string image; The upper threshold and the lower threshold are adjusted based on the number of pixels corresponding to the upper threshold and the lower threshold respectively.

4. The photovoltaic module detection method according to claim 3, characterized in that: The adjusting the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold, respectively, comprises: Determine whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number; If so, adjust the upper threshold and / or the lower threshold to other adjacent minimum values, and jump to the step of determining whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number, until the number of pixels corresponding to the upper threshold and the lower threshold is not greater than the target number.

5. The photovoltaic module detection method according to claim 1, characterized in that: The step of thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image includes: Thresholding the defect group string image based on the upper threshold and the lower threshold to obtain a thresholded image; Determining a photovoltaic module edge frame of each photovoltaic module in the defective string image using a contour detection algorithm and based on the thresholded image; If the number of edge frames of the photovoltaic assembly is 0, the upper threshold is lowered, and the process jumps to the step of thresholding the defective string image based on the upper threshold and the lower threshold until the number of edge frames of the photovoltaic assembly is not 0.

6. The photovoltaic module detection method according to claim 1, characterized in that: After thresholding the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic component edge frame of each photovoltaic component in the defective string image based on the thresholded image, the method further includes: Each photovoltaic component edge frame is screened based on a preset range, and the photovoltaic component edge frames that do not meet the preset range are removed; the preset range includes a standard area range, a standard aspect ratio range, a standard length range and a standard width range.

7. The photovoltaic module detection method according to any one of claims 1 to 6, characterized in that: Before the photovoltaic components in the photovoltaic component edge frame are classified based on component type by using a lightweight classification network to determine defective components, the method further includes: If the target area connected to the photovoltaic component edge frame does not exceed the image range and there is no other photovoltaic component edge frame, the photovoltaic component edge frame is translated to the target area.

8. A photovoltaic module detection device, characterized in that: include: A segmentation module is used to segment the visible light photovoltaic image using a semantic segmentation model to obtain a photovoltaic string mask image; An extraction module, configured to extract a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and to extract a plurality of defective string images from the plurality of photovoltaic string images using a target detection model; A threshold determination module, used to grayscale the defective string image to obtain a grayscale histogram, and determine an upper threshold and a lower threshold corresponding to each defective string image based on the grayscale histogram; a candidate frame determination module, configured to threshold the defective string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determine a photovoltaic module edge frame of each photovoltaic module in the defective string image based on the thresholded image; A classification module is used to classify the photovoltaic components in the photovoltaic component edge frame based on component type using a lightweight classification network to determine defective components, select the defective components in the visible light photovoltaic image, and mark the component type in the defective component frame.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the photovoltaic component detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program; wherein, when the computer program is executed by a processor, the photovoltaic component detection method according to any one of claims 1 to 7 is implemented.

Images