A photovoltaic module detection method, device, equipment and medium

Abstract

The application discloses a photovoltaic module detection method, device, equipment and medium, and relates to the technical field of photovoltaic module strings, and comprises the following steps: segmenting a visible light photovoltaic image to obtain a photovoltaic module string mask image; extracting a plurality of photovoltaic module string images from the visible light photovoltaic image based on the photovoltaic module string mask image, and extracting a plurality of defect module string images from the plurality of photovoltaic module string images; grayizing the defect module string images, determining the upper threshold and the lower threshold of each defect module string image based on a gray histogram; thresholding the defect module string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic module edge frame of each photovoltaic module in the defect module string image based on the thresholded image; classifying the photovoltaic module in the photovoltaic module edge frame based on the module type by using a lightweight classification network to determine a defect module, framing the defect module in the visible light photovoltaic image, and labeling the module type in the defect module frame. The photovoltaic module string can be quickly detected, and the generalization of the module string detection is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photovoltaic module strings, and in particular to a photovoltaic module detection method, device, equipment and medium. BACKGROUND

[0002] Currently, in the daily operation of photovoltaic power stations, due to environmental factors and defects of the modules themselves, photovoltaic modules may have various problems, leading to performance degradation or even safety hazards. Among them, shadow blocking, bird droppings, shadows and foreign matter such as dust are very common defects. These defects not only reduce the output power of photovoltaic modules, but also may cause local overheating, thereby further damaging the modules and affecting the overall operation efficiency and safety of the photovoltaic power station.

[0003] With the development of unmanned aerial vehicle technology and image processing technology, photovoltaic module defect detection methods based on visible light images have gradually attracted attention. Visible light images have the advantages of easy acquisition, relatively low cost, and are not affected by environmental temperature. By combining unmanned aerial vehicle inspection technology and advanced image processing algorithms, large-scale and rapid inspection of photovoltaic power stations can be realized, and defects in photovoltaic modules can be found in a timely manner.

[0004] However, the existing photovoltaic power station module string hot spot detection method still has the following shortcomings: (1) the number of modules is large and the detection speed is slow; (2) when using traditional image processing methods to realize module segmentation, the method has poor generalization, specifically, due to the diverse environmental conditions of photovoltaic power stations, traditional image processing methods often rely on fixed thresholds and rules, which makes them have great limitations under different lighting conditions, background complexity and camera angles. When these methods face a variety of changing scenes, they are difficult to adapt to various complex situations, although based on three methods of straight line detection, threshold segmentation and equidistant segmentation, the threshold segmentation method achieves a success rate of 99.46%. However, when the scene changes, the threshold needs to be adjusted, and the generalization is insufficient.

[0005] In summary, how to quickly detect photovoltaic module strings and improve the generalization of module string detection is a problem that needs to be solved at present. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a photovoltaic module detection method, device, equipment and medium, which can quickly detect photovoltaic module strings and improve the generalization of module string detection, and the specific scheme is as follows:

[0007] In a first aspect, the present application discloses a photovoltaic module detection method, comprising:

[0008] segmenting the visible light photovoltaic image using a semantic segmentation model to obtain a photovoltaic module string mask image;

[0009] extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask map, and extracting a plurality of defect string images from the plurality of photovoltaic string images by using a target detection model;

[0010] graying the defect string images to obtain a gray histogram, and determining an upper threshold value and a lower threshold value corresponding to each of the defect string images based on the gray histogram;

[0011] thresholding the defect string images based on the upper threshold value and the lower threshold value to obtain a thresholded image, and determining photovoltaic module edge frames of photovoltaic modules in the defect string images based on the thresholded image;

[0012] classifying the photovoltaic modules in the photovoltaic module edge frames based on module types by using a lightweight classification network to determine defect modules, and framing the defect modules in the visible light photovoltaic image and labeling module types in the defect module frames.

[0013] The method comprises the following steps:

[0014] determining an edge frame of each photovoltaic string in the photovoltaic string mask map based on a contour detection algorithm;

[0015] determining a minimum adjacent parallelogram corresponding to each photovoltaic string edge frame, and filtering the minimum adjacent parallelograms with an area less than a preset area;

[0016] extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask map, and extracting a plurality of defect string images from the plurality of photovoltaic string images by using a target detection model;

[0017] The method comprises the following steps:

[0018] performing polynomial fitting on the gray histogram to obtain a target polynomial, and determining a maximum value set and a minimum value set corresponding to the target polynomial;

[0019] determining a maximum value in the maximum value set, and taking two minimum values adjacent to the maximum value as an upper threshold value and a lower threshold value corresponding to each of the defect string images, respectively;

[0020] adjusting the upper threshold value and the lower threshold value based on the number of pixels corresponding to the upper threshold value and the lower threshold value, respectively.

[0021] The adjusting the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold respectively comprises:

[0022] determining whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than a target number;

[0023] if yes, adjusting the upper threshold and / or the lower threshold to other adjacent minimum values, and jumping to the step of determining whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number until the number of pixels corresponding to the upper threshold and the lower threshold is not greater than the target number.

[0024] The thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image comprises:

[0025] thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholded image;

[0026] determining the photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image by using a contour detection algorithm;

[0027] if the number of photovoltaic module edge frames is 0, reducing the upper threshold, and jumping to the step of thresholding the defect string image based on the upper threshold and the lower threshold until the number of photovoltaic module edge frames is not 0.

[0028] The thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image further comprises:

[0029] screening each photovoltaic module edge frame based on a preset range, and removing the photovoltaic module edge frame that does not satisfy the preset range; the preset range comprises a standard area range, a standard length-width ratio range, a standard length range, and a standard width range.

[0030] The classifying the photovoltaic module in the photovoltaic module edge frame based on the module type by using the lightweight classification network to determine the defective module further comprises:

[0031] if a target region connected with the photovoltaic module edge frame does not exceed an image range, and there is no other photovoltaic module edge frame, the photovoltaic module edge frame is translated to the target region.

[0032] In a second aspect, the present application discloses a photovoltaic module detection device, comprising:

[0033] A segmentation module is configured to segment a visible light photovoltaic image by using a semantic segmentation model to obtain a photovoltaic string mask image.

[0034] An extraction module is configured to extract a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extract a plurality of defective string images from the plurality of photovoltaic string images by using a target detection model.

[0035] A threshold determination module is configured to perform grayscale processing on the defective string images to obtain a grayscale histogram, and determine an upper threshold and a lower threshold corresponding to each defective string image based on the grayscale histogram.

[0036] A candidate box determination module is configured to perform thresholding on the defective string images based on the upper threshold and the lower threshold to obtain a thresholding image, and determine a photovoltaic module edge box of each photovoltaic module in the defective string image based on the thresholding image.

[0037] A classification module is configured to classify the photovoltaic modules in the photovoltaic module edge box based on a module type by using a lightweight classification network to determine a defective module, frame the defective module in the visible light photovoltaic image, and label the module type in the defective module frame.

[0038] In a third aspect, the present application discloses an electronic device, comprising:

[0039] A memory is configured to save a computer program.

[0040] A processor is configured to execute the computer program to implement the photovoltaic module detection method disclosed above.

[0041] In a fourth aspect, the present application discloses a computer readable storage medium configured to save a computer program, wherein the computer program is executed by a processor to implement the photovoltaic module detection method disclosed above.

[0042] It can be seen that the application utilizes a semantic segmentation model to segment a visible light photovoltaic image to obtain a photovoltaic string mask image; extracts a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, extracts a plurality of defect string images from the plurality of photovoltaic string images by utilizing a target detection model; performs gray-scale processing on the defect string images to obtain a gray histogram, and determines an upper threshold and a lower threshold corresponding to each of the defect string images based on the gray histogram; performs thresholding on the defect string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determines photovoltaic module edge frames of each photovoltaic module in the defect string image based on the thresholded image; classifies the photovoltaic modules in the photovoltaic module edge frames based on module types by utilizing a lightweight classification network to determine defect modules, and frames the defect modules in the visible light photovoltaic image and labels the module types in the defect module frames. It can be seen that the application extracts defect string images from a visible light photovoltaic image by utilizing a semantic segmentation model and a target detection model, and subsequent processing only needs to process the defect string images instead of the entire visible light photovoltaic image, thereby improving processing speed; the application determines the upper threshold and the lower threshold corresponding to the defect string images based on the gray histogram of the defect string images instead of pre-setting, so that the threshold setting is more universal and can improve generalization; in summary, the application can quickly detect photovoltaic strings and improve the generalization of string detection. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0044] Figure 1 A photovoltaic module detection method flowchart disclosed by the present application;

[0045] Figure 2 A visible light photovoltaic image disclosed by the present application;

[0046] Figure 3 A photovoltaic string mask image disclosed by the present application;

[0047] Figure 4 A string image composed of six photovoltaic string images disclosed by the present application;

[0048] Figure 5 A string image composed of two defect string images disclosed by the present application;

[0049] Figure 6A threshold image disclosed by the present application;

[0050] Figure 7 A specific photovoltaic module detection method flow chart disclosed by the present application;

[0051] Figure 8 A missed detection adjustment schematic diagram disclosed by the present application;

[0052] Figure 9 A single missed detection adjustment photovoltaic module string schematic diagram disclosed by the present application;

[0053] Figure 10 A visible light photovoltaic image of a defective module marked by the present application;

[0054] Figure 11 A partial defective module marking image disclosed by the present application;

[0055] Figure 12 A photovoltaic module detection device structure schematic diagram disclosed by the present application;

[0056] Figure 13 An electronic device structure diagram disclosed by the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] The existing photovoltaic power station module string hot spot detection method still has the following deficiencies: (1) the number of modules is large and the detection speed is slow; (2) when using traditional image processing methods to realize module segmentation, the method has poor generalization, specifically, due to the diversity of photovoltaic power station environmental conditions, traditional image processing methods often rely on fixed threshold and rules, which makes them have great limitations under different light conditions, background complexity and camera angles. When these methods face various changing scenes, they are difficult to adapt to various complex situations. Although based on the three methods of straight line detection, threshold segmentation and equidistant segmentation, the threshold segmentation method achieves a success rate of 99.46%. However, when the scene changes, the threshold needs to be adjusted again, and the generalization is insufficient.

[0059] Therefore, the present application embodiment proposes a photovoltaic module detection scheme, which can quickly detect photovoltaic module strings and improve the generalization of module string detection.

[0060] The present application embodiment discloses a photovoltaic module detection method, as followsFigure 1 As shown in the method comprises:

[0061] Step S11: segmenting the visible light photovoltaic image by using the semantic segmentation model to obtain a photovoltaic string mask graph.

[0062] In this embodiment, the photovoltaic string mask graph obtained by segmenting the visible light photovoltaic image by using the semantic segmentation model comprises: segmenting the visible light photovoltaic image by using the semantic segmentation model obtained by updating based on the loss function and the gradient descent method to obtain a photovoltaic string mask graph; the loss function is a loss function composed of a linear combination of cross-entropy loss and feature matching loss. Wherein, referring to Figure 2 As shown in the visible light photovoltaic image, see Figure 3 As shown in the photovoltaic string mask graph.

[0063] It should be pointed out that first, the semantic segmentation network (UNet network) needs to be trained to obtain the optimal semantic segmentation model, and then segmented by using the semantic segmentation model. Specifically, the semantic segmentation network is obtained by using the EfficientNet-B7 network and adding the MCF module at the connection between the encoder and the decoder. The MCF module effectively saves the computing resources while expanding the receptive field to capture multi-scale spatial context information, thereby improving the segmentation accuracy of the UNet network; wherein MCF, i.e. MOM connector framework, is a network service based technology.

[0064] It should be pointed out that the training process is as follows: obtaining a visible light zoom image to be trained, labeling the photovoltaic string in the visible light zoom image of the photovoltaic power station (including labeling the position information of the photovoltaic string), making a segmentation data set, in order to reduce the risk of overfitting and make the model better adapt to different scenes and lighting conditions, data augmentation is used to increase data diversity; input the sample in batches into the UNet network, and perform feature extraction and fusion layer by layer in the network to finally obtain the comprehensive feature fusion result; performing softmax operation (normalization operation) on the final feature fusion result to obtain the image segmentation result predicted by the UNet network; using a linear combination of cross-entropy loss and feature matching loss to constitute the loss function of the predicted image segmentation result and the standard segmentation graph, which is defined as follows:

[0065] ;

[0066] is the coefficient; is the cross-entropy loss; is the feature matching loss.

[0067] Considering that the photovoltaic string recognition of the present application is a binary classification task, the cross-entropy loss function formula is as follows:

[0068] ;

[0069] in, Indicates the total number of pixels; Indicates the first The probability that a pixel is predicted to be a string region; To make the first The probability of predicting each pixel as background;

[0070] use This refers to unlabeled (annotated) samples, which are the samples that were not labeled when annotating the photovoltaic strings in the visible light zoom image of the photovoltaic power plant mentioned above. The definition of Feature Matching Loss is as follows:

[0071] ;

[0072] in, This represents the predicted segmentation diagram (i.e., the photovoltaic string mask diagram). This represents the actual segmentation diagram (i.e., the photovoltaic string mask diagram). This refers to the labeled sample, which is the sample labeled when the photovoltaic strings are labeled in the visible light zoom image of the photovoltaic power station mentioned above; This indicates an image stitching operation, which stitches the input image and the corresponding mask image together; This represents a labeled sample dataset; This represents the intermediate representation of the discriminator network at layer k. Represents the norm.

[0073] Then, the model parameters are updated using gradient descent to obtain the optimal model parameters. The photovoltaic strings are then segmented using the optimal model parameters, and the final mask image is output.

[0074] It should be noted that the required photovoltaic images can be collected using drones.

[0075] Step S12: Based on the photovoltaic string mask, extract several photovoltaic string images from the visible light photovoltaic image, and use a target detection model to extract several defect string images from the several photovoltaic string images.

[0076] In this embodiment, the step of extracting several photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask includes: determining the edge box of each photovoltaic string in the photovoltaic string mask based on a contour detection algorithm; determining the minimum adjacent parallelogram corresponding to each photovoltaic string edge box, and filtering out the minimum adjacent parallelograms with an area smaller than a preset area; and extracting several photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string edge boxes corresponding to the remaining minimum adjacent parallelograms. See alsoFigure 4 As shown in FIG. 6, it is a group string image composed of six photovoltaic string images; see Figure 5 As shown in FIG. 7, it is a group string image composed of two defect string images.

[0077] It should be pointed out that the convex hull algorithm and the rotation jamming method are used to determine the minimum adjacent parallelogram corresponding to each photovoltaic string edge frame.

[0078] It should be pointed out that the target detection model is also pre-trained, specifically, the defects in the photovoltaic string are labeled; the labeled data is input into the YOLOv5 network, and after passing through the backbone network, neck network and head network in turn, the feature map is generated, and then the prediction box is obtained through the detection box decoding and non maximum suppression (NMS, non maximum suppression). Then, the prediction box is compared with the labeled true box to calculate the classification loss and positioning loss; based on the loss function, the network weight is adjusted using the optimization algorithm to minimize the loss; the optimal model parameters are obtained, and the confidence threshold is appropriately reduced to obtain a model with lower missed detection, so as to obtain photovoltaic strings that may have defects, specifically Figure 5 Two photovoltaic strings that may have defects extracted from Figure 4 Each photovoltaic string that may have defects is a defect string image, Figure 5 In order to facilitate the display of the two defect string images together.

[0079] Step S13: grayscale the defect string image to obtain a grayscale histogram, and determine the upper threshold and lower threshold corresponding to each defect string image based on the grayscale histogram.

[0080] In this embodiment, the grayscale and threshold determination are performed on one defect string image at a time, and each defect image has a corresponding grayscale histogram and upper threshold and lower threshold.

[0081] In this embodiment, the upper threshold and lower threshold are not predetermined in advance, but are temporarily calculated and determined through the grayscale histogram, so they are suitable for various images, and the determination of the threshold is more general.

[0082] Step S14: thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholding image, and determining the photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholding image.

[0083] As shown in FIG. 8, it is a thresholding image. Figure 6

[0084] ​In this embodiment, the thresholded image is more convenient to determine the edge frame of the photovoltaic module.

[0085] Step S15: classifying the photovoltaic module in the photovoltaic module edge frame based on the module type by using the lightweight classification network to determine the defective module, and framing the defective module in the visible light photovoltaic image and labeling the module type in the defective module frame.

[0086] In this embodiment, the module is cropped by the photovoltaic module edge frame, and perspective transformation is realized; the obtained photovoltaic module picture is input into the improved ShufflenetV2 for classification, which can be specifically classified into five types of module types: normal, bird droppings, occlusion, shadow and dust, and the module edge frame and module type other than normal are stored in the corresponding dictionary set; the dictionary set storing the detection results is traversed, the defective module is framed in the original image and the module type is marked in the frame, specifically, the position of the defective module corresponding to the photovoltaic module edge frame in the defective module string image is determined in the foregoing process, the position of the defective module string image in the photovoltaic module string image is determined, the position of the photovoltaic module string image in the photovoltaic module string mask image and the visible light photovoltaic image is determined, and thus the position of the defective module corresponding to the photovoltaic module edge frame in the visible light photovoltaic image is determined, so as to frame the defective module in the original image and mark the defect type in the frame.

[0087] It should be noted that in the improved ShufflenetV2, the PRelu (Parametric Rectified Linear Unit) activation function (Parametric Rectified Linear Unit) is used to replace the Relu nonlinear excitation function, so that the model has better adaptability and better gradient flow, and the network can have higher accuracy and faster convergence speed; at the same time, the channel and spatial dual attention mechanism CBAM (Convolutional Block Attention Module) module is added between each module to improve the expression ability of the features.

[0088] It should be pointed out that the lightweight classification model is also a pre-trained model, and the training process is as follows: the components are cropped and perspective transformation is realized, the obtained photovoltaic component pictures are input into the improved ShufflenetV2 for classification, which can be divided into five categories: normal, bird droppings, occlusion, shadow and dust, and the results of the components other than normal are stored in the corresponding set. The segmented components are grouped into five categories: normal, bird droppings, occlusion, shadow and dust, and the number of samples is increased by data augmentation. The images are input into the network in batches, and the improved ShuffleNetV2 extracts and fuses features layer by layer to generate the final feature fusion result. The final feature fusion result is subjected to softmax operation (normalization operation) to obtain the prediction result; the loss is calculated according to the prediction result, and the gradient descent method is used to update the parameters of the model according to the result of the loss function to minimize the loss function, and the optimal model is obtained to judge the type of photovoltaic component in the subsequent process.

[0089] In summary, the visible light photovoltaic image photographed by the unmanned aerial vehicle is extracted by the semantic segmentation network to obtain the string area in the visible light photovoltaic image, and the segmented string is pre-screened by the target detection network to obtain the defect string in the visible light photovoltaic image, and the string with possible defects is determined. Then, the photovoltaic component segmentation algorithm based on adaptive double threshold and translation adjustment is used to realize the component segmentation of the defect string in the visible light photovoltaic image. Finally, the lightweight classification network is used to realize the detection and identification of the defect component, to determine the existence and type of the defect, and to display the defect component and the defect type in the picture.

[0090] It can be seen that the application utilizes the semantic segmentation model to segment the visible light photovoltaic image to obtain a photovoltaic string mask image; extracts a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, extracts a plurality of defective string images from the plurality of photovoltaic string images by utilizing a target detection model; performs gray-scale processing on the defective string images to obtain a gray histogram, and determines an upper threshold value and a lower threshold value corresponding to each of the defective string images based on the gray histogram; performs thresholding on the defective string images based on the upper threshold value and the lower threshold value to obtain a thresholded image, and determines photovoltaic module edge frames of each photovoltaic module in the defective string image based on the thresholded image; classifies the photovoltaic modules in the photovoltaic module edge frames based on module types by utilizing a lightweight classification network to determine defective modules, and frames the defective modules in the visible light photovoltaic image and labels the module types in the defective module frames. It can be seen that the application utilizes the semantic segmentation model and the target detection model to extract defective string images from the visible light photovoltaic image, and subsequent processing only needs to process the defective string images instead of the entire visible light photovoltaic image, thereby improving the processing speed; the application determines the upper threshold value and the lower threshold value corresponding to the defective string image based on the gray histogram of the defective string image instead of pre-setting, so that the threshold setting is more universal and the generalization is improved. In summary, the application can quickly detect photovoltaic strings and improve the generalization of string detection.

[0091] The embodiment of the application discloses a specific photovoltaic module detection method. Compared with the previous embodiment, the technical solution is further described and optimized. Referring to Figure 7 The embodiment specifically comprises the following steps:

[0092] Step S21: utilizing a semantic segmentation model to segment a visible light photovoltaic image to obtain a photovoltaic string mask image.

[0093] Step S22: extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extracting a plurality of defective string images from the plurality of photovoltaic string images by utilizing a target detection model.

[0094] Step S23: performing gray-scale processing on the defective string images to obtain a gray histogram, performing polynomial fitting on the gray histogram to obtain a target polynomial, and determining a maximum value set and a minimum value set corresponding to the target polynomial; determining a maximum value in the maximum value set, and taking two minimum values adjacent to the maximum value as an upper threshold value and a lower threshold value corresponding to the defective string image.

[0095] In this embodiment, the adaptive double-threshold and translation adjustment-based photovoltaic module segmentation algorithm is used to realize the module segmentation of the defect string in the visible light photovoltaic image. The double threshold is the upper threshold and the lower threshold, and the translation adjustment refers to translating the string at other positions to the string position where the string is not detected to solve the missed detection situation.

[0096] In this embodiment, the weighted value method is used to perform gray scale on the defect string image, and the gray scale histogram is counted. The gray scale value calculation formula is as follows:

[0097]

[0098] Wherein, Y represents the gray scale value, and RGB represents the color representing the red, green, and blue channels.

[0099] The least square method is used to perform polynomial fitting on the gray scale histogram, and the argrelextrema function in the SciPy library is used to find the maximum value set and the minimum value set in the gray scale value array; the maximum value in the maximum value is obtained, and the minimum value adjacent to the index (index representing the pixel point and position) in the minimum value list is found as the upper and lower thresholds (i.e. the upper threshold and the lower threshold) of thresholding.

[0100] Step S24: adjusting the upper threshold and the lower threshold based on the pixel number corresponding to the upper threshold and the lower threshold, respectively.

[0101] In this embodiment, the upper threshold and the lower threshold are adjusted based on the pixel number corresponding to the upper threshold and the lower threshold, respectively, which includes: judging whether the pixel number corresponding to the upper threshold and / or the lower threshold is greater than a target number; if yes, adjusting the upper threshold and / or the lower threshold to other adjacent minimum values, and jumping to the step of judging whether the pixel number corresponding to the upper threshold and / or the lower threshold is greater than the target number until the pixel number corresponding to the upper threshold and the lower threshold is not greater than the target number. It should be pointed out that the other adjacent minimum values are selected in the order from near to far.

[0102] Step S25: thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image.

[0103] ​In this embodiment, after determining the upper threshold based on pixels, the upper threshold is not fixed. During the process of determining the photovoltaic module edge boxes, the upper threshold is also changed according to the number of edge boxes. Specifically, the step of thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholded image, and determining the photovoltaic module edge boxes of each photovoltaic module in the defect string image based on the thresholded image, includes: thresholding the defect string image based on the upper threshold and the lower threshold to obtain a thresholded image; using a contour detection algorithm and based on the thresholded image to determine the photovoltaic module edge boxes of each photovoltaic module in the defect string image; if the number of photovoltaic module edge boxes is 0, the upper threshold is lowered, and the process jumps to the step of thresholding the defect string image based on the upper threshold and the lower threshold until the number of photovoltaic module edge boxes is not 0.

[0104] It should be noted that if the number of photovoltaic module edge boxes is still 0 after four jumps, that is, after the edge boxes are determined five times, the image will be recorded and the jumping will stop.

[0105] In this embodiment, after obtaining the thresholded image, filtering is performed based on the area-to-width ratio of the photovoltaic module edge frame. Specifically, after thresholding the defect string image based on the upper and lower thresholds to obtain the thresholded image, and determining the photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image, the method further includes: filtering each photovoltaic module edge frame based on a preset range, and removing the photovoltaic module edge frames that do not meet the preset range; the preset range includes a standard area range, a standard aspect ratio range, a standard length range, and a standard width range.

[0106] The specific filtering formula is as follows:

[0107] ;

[0108] ;

[0109] in, This indicates the actual width of the photovoltaic module's edge frame, in units of... (pixels); Indicates the target width of the photovoltaic module's edge frame, in units of ; This indicates the actual length of the photovoltaic module's edge frame, in units of... ; Indicates the target length of the photovoltaic module edge frame, in units of ; This indicates the area of ​​the photovoltaic module's edge frame, in units of... .

[0110] In this embodiment, if the group string is missed, the missing is compensated by translation adjustment. Specifically, before the light-weight classification network is used to classify the photovoltaic components in the photovoltaic component edge frame based on the component type to determine the defective component, it further includes: if the target area connected with the photovoltaic component edge frame does not exceed the image range, and there is no other photovoltaic component edge frame, the photovoltaic component edge frame is translated to the target area.

[0111] It should be noted that the steps of translation adjustment are as follows: traverse the existing edge frame to determine whether there are other edge frames above, below, left and right or beyond the image range; calculate the interval between the group strings according to the adjacent edge frames; when the component is not detected, calculate the slope and length of the minimum adjacent parallelogram side of the adjacent components and the interval between the components, and obtain a new edge frame by translation and add it to the edge frame set.

[0112] Referring to Figure 8 , it is a missing adjustment diagram; the left image in the figure is a single defective group string image of the photovoltaic component edge frame, and there is a missing part (i.e. the photovoltaic component not framed by the photovoltaic component edge frame) in the figure. The upper right image in the figure shows the missing part (the third and fourth columns of photovoltaic components in the second row of the three rows of photovoltaic components are not framed by the photovoltaic component edge frame, i.e. the missing part), and the arrow in the figure is to translate the lower non-missing part to the upper missing part to obtain the translated image on the right lower side (the current non-missing part is the third and fourth columns of photovoltaic components in the third row of the three rows of photovoltaic components). It should be noted that although it is translation, the photovoltaic component edge frame of the non-missing part before translation is still retained.

[0113] Referring to Figure 9 , it is a single missing adjustment photovoltaic group string diagram; each photovoltaic component in the figure is framed by a photovoltaic component edge frame, and there is no omission.

[0114] Step S26: using a light-weight classification network to classify the photovoltaic components in the photovoltaic component edge frame based on the component type to determine the defective component, and framing the defective component in the visible light photovoltaic image, and labeling the component type in the defective component frame.

[0115] In this embodiment, after the defective component is framed in the visible light photovoltaic image and the component type is labeled in the defective component frame, the visible light photovoltaic image after labeling of the defective component is obtained, as shown in Figure 10 and Figure 11 . Figure 10 Figure 11 ​From Figure 10 This is a partial image of a defective component, showing the defect-annotated portion after cropping the defect-annotated part from a mid-visible photovoltaic image. Figure 11 In the two rows of photovoltaic modules, the modules in the third column of the first row and the third column of the second row have white spots, which are defects. These two modules are outlined. Figure 10 The middle is not obvious, special use Figure 11 Detailed display.

[0116] As can be seen, this application uses a semantic segmentation model to segment visible light photovoltaic images to obtain a photovoltaic string mask; based on the photovoltaic string mask, it extracts several photovoltaic string images from the visible light photovoltaic images, and uses a target detection model to extract several defect string images from the several photovoltaic string images; it converts the defect string images to grayscale to obtain a grayscale histogram, performs polynomial fitting on the grayscale histogram to obtain a target polynomial, and determines the set of maxima and the set of minima corresponding to the target polynomial; it determines the maximum value in the set of maxima, and uses the two minima adjacent to the maximum value as the defect string images respectively. The method involves setting upper and lower thresholds based on corresponding upper and lower thresholds; adjusting the upper and lower thresholds based on the number of pixels corresponding to the upper and lower thresholds respectively; thresholding the defect cluster image based on the upper and lower thresholds to obtain a thresholded image; determining the photovoltaic component edge boxes of each photovoltaic component in the defect cluster image based on the thresholded image; classifying the photovoltaic components in the photovoltaic component edge boxes based on component type using a lightweight classification network to identify defective components; selecting the defective components in the visible light photovoltaic image and labeling the component type in the defective component box. Therefore, this application determines the upper and lower thresholds based on the maximum and minimum values, and adjusts the upper and lower thresholds based on the number of pixels. Furthermore, the upper threshold can be adjusted based on the number of edge boxes, making it applicable to the setting of dual thresholds for different images, thus improving the generalization of cluster detection.

[0117] Accordingly, this application also discloses a photovoltaic module testing device, see [link to relevant documentation]. Figure 12 As shown, the device includes:

[0118] Segmentation module 11 is used to segment visible light photovoltaic images using a semantic segmentation model to obtain photovoltaic string mask images;

[0119] Extraction module 12 is used to extract several photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and to extract several defect string images from the several photovoltaic string images using a target detection model;

[0120] The threshold determination module 13 is configured to perform gray-scale processing on the defect group string image to obtain a gray histogram, and determine an upper threshold and a lower threshold corresponding to each defect group string image based on the gray histogram;

[0121] The candidate box determination module 14 is configured to perform thresholding on the defect group string image based on the upper threshold and the lower threshold to obtain a thresholding image, and determine a photovoltaic module edge box of each photovoltaic module in the defect group string image based on the thresholding image.

[0122] The classification module 15 is configured to classify the photovoltaic module in the photovoltaic module edge box based on the module type by using a lightweight classification network to determine a defective module, and frame the defective module in the visible light photovoltaic image and mark the module type in the defective module frame.

[0123] The working processes of the above modules are more specifically described in the corresponding content disclosed in the foregoing embodiments, and thus will not be described here again.

[0124] As can be seen, the semantic segmentation model is used to segment the visible light photovoltaic image to obtain a photovoltaic module string mask image; a plurality of photovoltaic module string images are extracted from the visible light photovoltaic image based on the photovoltaic module string mask image, and a plurality of defect group string images are extracted from the plurality of photovoltaic module string images by using a target detection model; a gray histogram is obtained by performing gray-scale processing on the defect group string image, and an upper threshold and a lower threshold corresponding to each defect group string image are determined based on the gray histogram; thresholding is performed on the defect group string image based on the upper threshold and the lower threshold to obtain a thresholding image, and a photovoltaic module edge box of each photovoltaic module in the defect group string image is determined based on the thresholding image; the photovoltaic module in the photovoltaic module edge box is classified based on the module type by using a lightweight classification network to determine a defective module, and the defective module is framed in the visible light photovoltaic image and the module type is marked in the defective module frame. As can be seen, the semantic segmentation model and the target detection model are used to extract the defect group string image from the visible light photovoltaic image, and the subsequent processing process only needs to process the defect group string image instead of the entire visible light photovoltaic image, so that the processing speed is improved; the upper threshold and the lower threshold corresponding to the defect group string image are determined based on the gray histogram of the defect group string image instead of being pre-set, so that the setting of the threshold is more universal and the generalization is improved. In summary, the photovoltaic module string can be quickly detected and the generalization of the module string detection is improved.

[0125] Further, the embodiment of the present application also provides an electronic device. Figure 13 FIG. 1 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure should not be considered as any limitation on the use range of the present application.

[0126] Figure 13 A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a display screen 23, an input / output interface 24, a communication interface 25, a power supply 26 and a communication bus 27. The memory 22 is configured to store a computer program, and the processor 21 is configured to load and execute the computer program to implement the related steps in the photovoltaic module detection method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the embodiments of the present application can be specifically an electronic computer.

[0127] In the embodiments of the present application, the power supply 26 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 25 is capable of creating a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 25 can be any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input / output interface 24 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0128] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include a computer program 221, and the storage mode can be temporary storage or permanent storage. In addition to the computer program capable of completing the photovoltaic module detection method executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 221 can further include a computer program capable of completing other specific work.

[0129] Further, the embodiments of the present application further disclose a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the photovoltaic module detection method disclosed above.

[0130] The specific steps of the method can refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0131] The embodiments in the present application are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can refer to the method part.

[0132] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without referring to a specific sequence of operations for implementing the functions. The order of various illustrative blocks, modules, circuits, and steps may be re-arranged or otherwise implemented without departing from the spirit of the application, which is defined by the appended claims.

[0133] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0134] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are more especially used for the purpose of description and are not limiting. In addition, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusions, such that a process, method, article, or apparatus that comprises a list of elements does not include those elements solely, but can also include other elements not expressly listed, or also include inherent elements of such process, method, article, or apparatus. Without further limitation, an element preceded by "comprises a" does not, without more constraints, foreclose the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element.

[0135] The above provides a photovoltaic module detection method, device, equipment, and storage medium. The principles and implementation manners of the application are described by using specific examples. The above example is only used to help understand the method and core idea of the application. Meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation manner and application range can be changed. The above description should not be understood as limiting the application.

Claims

1. A photovoltaic module inspection method characterized by, The method comprises the following steps: segmenting a visible light photovoltaic image by using a semantic segmentation model to obtain a photovoltaic string mask image; extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image, and extracting a plurality of defect string images from the plurality of photovoltaic string images by using a target detection model; graying the defect string images to obtain a gray histogram, and determining an upper threshold and a lower threshold corresponding to each of the defect string images based on the gray histogram; thresholding the defect string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image; classifying the photovoltaic modules in the photovoltaic module edge frame based on the type of the photovoltaic modules by using a lightweight classification network to determine a defective module, and framing the defective module in the visible light photovoltaic image and labeling the type of the defective module in the defective module frame.

2. The photovoltaic module inspection method of claim 1, wherein, The method of extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string mask image comprises: determining an edge frame of each photovoltaic string in the photovoltaic string mask image based on a contour detection algorithm; determining a minimum adjacent parallelogram corresponding to each of the photovoltaic string edge frames, and filtering the minimum adjacent parallelograms with an area less than a preset area; extracting a plurality of photovoltaic string images from the visible light photovoltaic image based on the photovoltaic string edge frames corresponding to the remaining minimum adjacent parallelograms.

3. The photovoltaic module inspection method of claim 1, wherein The method of determining an upper threshold and a lower threshold corresponding to each of the defect string images based on the gray histogram comprises: performing a polynomial fitting on the gray histogram to obtain a target polynomial, and determining a maximum value set and a minimum value set corresponding to the target polynomial; determining a maximum value in the maximum value set, and taking two minimum values adjacent to the maximum value as the upper threshold and the lower threshold corresponding to each of the defect string images, respectively; adjusting the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold, respectively.

4. The photovoltaic module inspection method of claim 3, wherein, The method of adjusting the upper threshold and the lower threshold based on the number of pixels corresponding to the upper threshold and the lower threshold, respectively, comprises: determining whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than a target number; if yes, adjusting the upper threshold and / or the lower threshold to another adjacent minimum value, and jumping to the step of determining whether the number of pixels corresponding to the upper threshold and / or the lower threshold is greater than the target number until the number of pixels corresponding to the upper threshold and the lower threshold is not greater than the target number.

5. The photovoltaic module inspection method of claim 1, wherein, The method of thresholding the defect string images based on the upper threshold and the lower threshold to obtain a thresholded image, and determining a photovoltaic module edge frame of each photovoltaic module in the defect string image based on the thresholded image, comprises: thresholding the defect string images based on the upper threshold and the lower threshold to obtain a thresholded image; determining, by using a contour detection algorithm and based on the thresholded image, a photovoltaic module edge frame of each photovoltaic module in the defective module string image; if the number of photovoltaic module edge frames is 0, reducing the upper threshold value, and jumping to the step of thresholding the defective module string image based on the upper threshold value and the lower threshold value until the number of photovoltaic module edge frames is not 0.

6. The photovoltaic module inspection method of claim 1, wherein, After the step of thresholding the defective module string image based on the upper threshold value and the lower threshold value to obtain a thresholded image, and determining, based on the thresholded image, a photovoltaic module edge frame of each photovoltaic module in the defective module string image, the method further comprises: screening each photovoltaic module edge frame based on a preset range, and removing the photovoltaic module edge frame that does not satisfy the preset range; the preset range includes a standard area range, a standard aspect ratio range, a standard length range, and a standard width range.

7. The photovoltaic module inspection method according to any one of claims 1 to 6, characterized by, Before the step of classifying the photovoltaic modules in the photovoltaic module edge frame based on the module type by using a lightweight classification network to determine defective modules, the method further comprises: if a target region connected with the photovoltaic module edge frame does not exceed the image range, and there is no other photovoltaic module edge frame, then translating the photovoltaic module edge frame to the target region.

8. A photovoltaic module inspection apparatus characterized by comprising: comprises: a segmentation module configured to segment a visible light photovoltaic image by using a semantic segmentation model to obtain a photovoltaic module string mask image; an extraction module configured to extract a plurality of photovoltaic module string images from the visible light photovoltaic image based on the photovoltaic module string mask image, and extract a plurality of defective module string images from the plurality of photovoltaic module string images by using a target detection model; a threshold determination module configured to perform grayscale processing on the defective module string images to obtain a grayscale histogram, and determine an upper threshold value and a lower threshold value corresponding to each defective module string image based on the grayscale histogram; a candidate frame determination module configured to threshold the defective module string image based on the upper threshold value and the lower threshold value to obtain a thresholded image, and determine a photovoltaic module edge frame of each photovoltaic module in the defective module string image based on the thresholded image; a classification module configured to classify the photovoltaic modules in the photovoltaic module edge frame based on the module type by using a lightweight classification network to determine defective modules, and frame the defective modules in the visible light photovoltaic image, and label the module type in the defective module frame.

9. An electronic device, comprising: comprises: a memory configured to save a computer program; a processor configured to execute the computer program to implement the photovoltaic module detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, a memory configured to save a computer program; wherein the computer program is executed by a processor to implement the photovoltaic module detection method according to any one of claims 1 to 7.

Images