A method, device for determining a photovoltaic panel defect detection model and defect detection

Abstract

The present invention discloses a method, device for determining a photovoltaic panel defect detection model and a defect detection method. In the method for determining the photovoltaic panel defect detection model, the initial edge detection network is decoupled into two relatively independent network modules, namely, feature encoding and feature decoding, which simplifies the design of the edge detection network; by adding short residual connections to the initial feature encoding network and the initial feature decoding network respectively, the forward propagation and fusion of features are promoted, effectively improving the utilization rate and resolution of features; further, by setting a long residual connection between the target feature encoding network and the target feature decoding network, a fusion channel between low-layer high-resolution features and high-layer features is established, strengthening the feature extraction ability of the high-layer network without changing the original network structure. At the same time, since the residual connection does not generate new parameters, the performance of the model is improved while consuming the same amount of computing resources.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a photovoltaic panel defect detection model determination and defect detection method and device. Background Art

[0002] The traditional method of photovoltaic panel defect detection mainly adopts manual screening, which is not only slow but also very inefficient. Another method is to use infrared thermal imagers and other detection equipment to obtain relevant photovoltaic panel images, and then use manual screening and identification to determine the defect location. The judgment and location of defects still cannot be separated from human subjective participation, and the application is very limited. In recent years, deep learning technology has developed rapidly, and image processing algorithms based on deep neural networks have been endowed with powerful image feature extraction capabilities, which have also been applied to photovoltaic panel defect detection.

[0003] However, at present, edge detection based on convolutional neural networks has problems such as complex design, low feature utilization, difficult structural design, and large amount of parameter redundant calculation. In terms of detection effect, there are problems such as edge lines are not fine enough, positioning accuracy is not enough, and edge details are easily lost in complex scenes. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a photovoltaic panel defect detection model determination and defect detection method and device to solve the technical problems in the prior art of edge detection based on convolutional neural networks, such as complex design, low feature utilization, difficult structural design, and large amount of parameter redundant calculation, which result in insufficient fineness of edge lines, insufficient positioning accuracy, and easy loss of edge details in complex scenarios.

[0005] The technical solution proposed by the present invention is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for determining a photovoltaic panel defect detection model, which includes: obtaining a photovoltaic panel image data set and an initial edge detection network, wherein the initial edge detection network includes an initial feature encoding network and an initial feature decoding network, and the initial feature encoding network is connected to the initial feature decoding network; short residual connections are respectively set inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network; long residual connections are set between the target feature encoding network and the target feature decoding network to obtain a target edge detection network; the photovoltaic panel image data set is input into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained.

[0007] In combination with the first aspect, in a possible implementation manner of the first aspect, the initial feature encoding network includes a data preprocessing module and at least one feature processing sub-module; the initial feature decoding network includes at least one upsampling module, and each upsampling module includes a feature extraction module, a transposed convolution upsampling module, and a feature fusion processing module, wherein the transposed convolution upsampling module is respectively connected to the feature extraction module and the feature fusion processing module.

[0008] In combination with the first aspect, in another possible implementation manner of the first aspect, short residual connections are respectively set inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network, including: setting the short residual connection at the input end and the output end of each feature processing sub-module in the initial feature encoding network to obtain the target feature encoding network; setting the short residual connection at the input end and the output end of each feature extraction module in the initial feature decoding network to obtain the target feature decoding network.

[0009] In combination with the first aspect, in yet another possible implementation manner of the first aspect, a long residual connection is set between the target feature encoding network and the target feature decoding network to obtain a target edge detection network, including: setting the long residual connection at the output end of the target feature processing sub-module in the target feature encoding network and the input end of the target feature fusion processing module in the target feature decoding network to obtain the target edge detection network, and the feature size of the target feature processing sub-module is the same as that of the target feature fusion processing module.

[0010] In combination with the first aspect, in yet another possible implementation manner of the first aspect, the photovoltaic panel image dataset is input into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained, including: obtaining a loss function; inputting the photovoltaic panel image dataset into the target edge detection network and training it through the loss function to obtain the photovoltaic panel defect detection model.

[0011] In a second aspect, an embodiment of the present invention provides a method for detecting photovoltaic panel defects. The method for detecting photovoltaic panel defects includes: obtaining an image of a photovoltaic panel to be detected; inputting the image of the photovoltaic panel to be detected into a photovoltaic panel defect detection model to obtain a photovoltaic panel edge detection probability map, where the photovoltaic panel defect detection model is determined according to the method for determining a photovoltaic panel defect detection model as described in the first aspect and any one of the first aspect of the embodiments of the present invention; based on the photovoltaic panel edge detection probability map, determining a defect area in the image of the photovoltaic panel to be detected through a preset processing method.

[0012] In combination with the second aspect, in a possible implementation manner of the second aspect, based on the photovoltaic panel edge detection probability map, through a preset processing method, the defect area in the photovoltaic panel image to be detected is determined, including: determining a binary edge map based on the photovoltaic panel edge detection probability map; analyzing the connected regions in the binary edge map to obtain the line contour of the photovoltaic panel edge detection probability map; performing rectangular fitting on the line contour to obtain the original line information of the photovoltaic panel image to be detected; and based on the original line information, through background difference method and region area analysis method processing, determining the defect area in the photovoltaic panel image to be detected.

[0013] In a third aspect, an embodiment of the present invention provides a photovoltaic panel defect detection model determination device, and the photovoltaic panel defect detection model determination device includes: a first acquisition module, configured to acquire a photovoltaic panel image data set and an initial edge detection network, where the initial edge detection network includes an initial feature encoding network and an initial feature decoding network, and the initial feature encoding network is connected to the initial feature decoding network; a first setting module, configured to respectively set short residual connections inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network; a second setting module, configured to set a long residual connection between the target feature encoding network and the target feature decoding network to obtain a target edge detection network; and a training module, configured to input the photovoltaic panel image data set into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained.

[0014] In a fourth aspect, an embodiment of the present invention provides a photovoltaic panel defect detection device, and the photovoltaic panel defect detection device includes: a second acquisition module, configured to acquire a photovoltaic panel image to be detected; an input module, configured to input the photovoltaic panel image to be detected into a photovoltaic panel defect detection model to obtain a photovoltaic panel edge detection probability map, where the photovoltaic panel defect detection model is obtained according to the photovoltaic panel defect detection model determination method described in the first aspect and any one of the first aspect of the embodiments of the present invention; and a processing module, configured to determine the defect area in the photovoltaic panel image to be detected based on the photovoltaic panel edge detection probability map through a preset processing method.

[0015] In a fifth aspect, an embodiment of the present invention provides an electronic device, including: a memory and a processor, where the memory and the processor are communicatively connected to each other, the memory stores a computer program, and the processor executes the computer program to execute the photovoltaic panel defect detection model determination method described in the first aspect and any one of the first aspect of the embodiments of the present invention, or the photovoltaic panel defect detection method described in the second aspect and any one of the second aspect of the embodiments of the present invention.

[0016] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program for causing a computer to execute the method for determining a photovoltaic panel defect detection model as described in the first aspect and any one of the first aspects of the embodiments of the present invention, or the method for detecting a photovoltaic panel defect as described in the second aspect and any one of the second aspects of the embodiments of the present invention.

[0017] The technical solution provided by the present invention has the following effects:

[0018] The method for determining a photovoltaic panel defect detection model provided by the embodiment of the present invention decouples an initial edge detection network into two relatively independent network modules, namely, feature encoding and feature decoding, which simplifies the edge detection network design; by adding short residual connections to the initial feature encoding network and the initial feature decoding network respectively, the forward propagation and fusion of features are promoted, effectively improving the utilization rate and resolution of features; further, by setting a long residual connection between the target feature encoding network and the target feature decoding network, a fusion channel for low-level high-resolution features and high-level features is established, strengthening the feature extraction ability of the high-level network without changing the original network structure. At the same time, since the residual connection does not generate new parameters, the performance of the model is improved while consuming the same computing resources.

[0019] The method for detecting a photovoltaic panel defect provided by the embodiment of the present invention can directly obtain a photovoltaic panel edge detection probability map of an image of a photovoltaic panel to be detected through the photovoltaic panel defect detection model obtained by the method for determining a photovoltaic panel defect detection model as described in the embodiment of the present invention. Further, by analyzing and processing the photovoltaic panel edge detection probability map, the defect area in the image of the photovoltaic panel to be detected can be determined. Therefore, by implementing the present invention, the accuracy of photovoltaic panel defect identification and the effectiveness of detection are improved. Description of the Drawings

[0020] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 is a flowchart of a method for determining a photovoltaic panel defect detection model provided by an embodiment of the present invention;

[0022] Figure 2 is a schematic structural diagram of a target edge detection network provided by an embodiment of the present invention;

[0023] Figure 3Schematic diagram of input and output changes of a sub-module provided according to an embodiment of the present invention;

[0024] Figure 4 Flowchart of a method for detecting defects in a photovoltaic panel provided according to an embodiment of the present invention;

[0025] Figure 5 Block diagram of the structure of a device for determining a photovoltaic panel defect detection model provided according to an embodiment of the present invention;

[0026] Figure 6 Block diagram of the structure of a device for detecting defects in a photovoltaic panel provided according to an embodiment of the present invention;

[0027] Figure 7 Schematic diagram of the structure of a computer-readable storage medium provided according to an embodiment of the present invention;

[0028] Figure 8 Schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. Detailed implementation manners

[0029] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] An embodiment of the present invention provides a method for determining a photovoltaic panel defect detection model, as Figure 1 shown, the method includes the following steps:

[0032] Step 101: Obtain a photovoltaic panel image data set and an initial edge detection network.

[0033] Among them, the initial edge detection network may include an initial feature encoding network and an initial feature decoding network, and the initial feature encoding network is connected to the initial feature decoding network.

[0034] Specifically, the initial feature encoding network is obtained by cropping the original convolutional neural network (VGG16) to remove the backend fully connected layer, average pooling layer, and classification (softmax) layer, etc.

[0035] Furthermore, a photovoltaic panel image dataset is obtained.

[0036] First, collect normal photovoltaic panel images and perform manual edge annotation to construct a training sample set;

[0037] Secondly, use random cropping, flipping, scale transformation, horizontal / vertical mirroring to perform marked sample data augmentation, increase the scale of the training data, and obtain a photovoltaic panel image dataset.

[0038] Step 102: Set short residual connections inside the initial feature encoding network and the initial feature decoding network respectively to obtain a target feature encoding network and a target feature decoding network.

[0039] Specifically, the short residual connection is used to process intermediate layer features.

[0040] By setting short residual connections, a feature transfer channel between high-level and low-level layers can be established to promote the forward propagation of features and information fusion.

[0041] Step 103: Set a long residual connection between the target feature encoding network and the target feature decoding network to obtain a target edge detection network.

[0042] Specifically, by setting a long residual connection between the target feature encoding network and the target feature decoding network, the feature extraction ability of the high-level network can be strengthened without changing the original network structure, and the feature information fusion between the encoding-decoding network structures can be promoted.

[0043] Step 104: Input the photovoltaic panel image dataset into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained.

[0044] First, initialize the overall parameters of the target edge detection network and load the pre-trained weights into the target edge detection network to achieve feature transfer of large-scale neural networks, reduce the actual training cost and the risk of overfitting.

[0045] Secondly, use the photovoltaic panel image dataset to train the target edge detection network and continuously adjust and optimize the target edge detection network with a smaller learning rate until a photovoltaic panel defect detection model that meets the conditions is obtained.

[0046] The method for determining a photovoltaic panel defect detection model provided by an embodiment of the present invention decouples an initial edge detection network into two relatively independent network modules, namely, feature encoding and feature decoding, which simplifies the design of the edge detection network; by adding short residual connections to the initial feature encoding network and the initial feature decoding network respectively, the forward propagation and fusion of features are promoted, effectively improving the utilization rate and resolution of features; further, by setting a long residual connection between the target feature encoding network and the target feature decoding network, a fusion channel between low-level high-resolution features and high-level features is established, strengthening the feature extraction ability of the high-level network without changing the original network structure. At the same time, since the residual connection does not generate new parameters, the performance of the model is improved while consuming the same amount of computing resources.

[0047] As an optional implementation manner of an embodiment of the present invention, the initial feature encoding network includes a data preprocessing module and at least one feature processing sub-module; the initial feature decoding network includes at least one upsampling module, and each upsampling module includes a feature extraction module, a transposed convolution upsampling module, and a feature fusion processing module, wherein the transposed convolution upsampling module is respectively connected to the feature extraction module and the feature fusion processing module.

[0048] Specifically, as Figure 2 shown, the initial feature encoding network includes a data preprocessing module and three feature processing sub-modules (sub-module 1, sub-module 2, and sub-module 3); the initial feature decoding network is mainly used for upsampling high-level abstract features and restoring the feature size and resolution, and a stacked upsampling structure is designed inside the initial feature decoding network for segmentally restoring the feature size. Among them, each upsampling module includes convolution processing (feature extraction module 1, feature extraction module 2, and feature extraction module 3), transposed convolution upsampling (upsampling 1, upsampling 2, and upsampling 3), and a feature fusion processing module ⊕.

[0049] Among them, in each upsampling module, the transposed convolution upsampling module is respectively connected to the feature extraction module and the feature fusion processing module.

[0050] As an optional implementation manner of an embodiment of the present invention, step 102 includes: setting the short residual connection at the input end and the output end of each feature processing sub-module in the initial feature encoding network to obtain the target feature encoding network; setting the short residual connection at the input end and the output end of each feature extraction module in the initial feature decoding network to obtain the target feature decoding network.

[0051] Specifically, the initial feature encoding network is composed of multiple sub-modules, and formally, X L can be defined as the convolution output in the L-th sub-module, and HL (·) is the non - linear transformation in the corresponding sub - module, including convolution, activation, and batch normalization, etc. Therefore, the non - linear output in each sub - module can be expressed as the following relational expression (1):

[0052] X L =H L (X L-1 ) (1)

[0053] Furthermore, as Figure 2 shown, short residual connections are respectively set at the input end and the output end of sub - module 1, sub - module 2, and sub - module 3.

[0054] Specifically, when adopting the residual structure, in each sub - module, the input and output are added element - by - element through the residual connection to promote feature fusion. When the residual connection is defined as, the output can be expressed as the following relational expression (2):

[0055] X L =H L (X L-1 )+W S X L-1 (2)

[0056] Among them, the specific schematic diagram of the above process is as Figure 3 shown.

[0057] Furthermore, as Figure 2 shown, short residual connections are respectively set at the input end and the output end of feature extraction module 1, feature extraction module 2, and feature extraction module 3.

[0058] As an optional implementation manner of an embodiment of the present invention, step 103 includes: setting the long residual connection at the output end of the target feature processing sub - module in the target feature encoding network and the input end of the target feature fusion processing module in the target feature decoding network to obtain the target edge detection network.

[0059] Among them, the feature size of the target feature processing sub - module is the same as that of the target feature fusion processing module.

[0060] As Figure 2 shown, a long residual connection is added between the output end of sub - module 1 and the input end of the second feature fusion processing module.

[0061] Specifically, the output of sub-module 1 in the target feature encoding network passes through sub-module 2 and sub-module 3, that is, after two times of convolutional pooling, it is output to the target feature decoding network, and then through feature extraction module 1, feature extraction module 2, upsampling 1 and upsampling 2, that is, after two times of deconvolutional pooling, it is input into the second feature fusion processing module. That is, the feature size of sub-module 1 is equal to the feature size of the second feature fusion processing module. Therefore, a long residual connection is added between the output end of sub-module 1 and the input end of the second feature fusion processing module.

[0062] As an optional implementation manner of the embodiment of the present invention, step 104 includes: obtaining a loss function; inputting the photovoltaic panel image data set into the target edge detection network, and training through the loss function to obtain the photovoltaic panel defect detection model.

[0063] Specifically, since the proportion of non-edge pixels in the input image is much larger than the proportion of edge pixels, it has a typical property of unbalanced positive and negative sample distribution during training. Therefore, a bias coefficient is added to the loss function during training to dynamically adjust the edge pixel loss value according to the pixel proportion. The loss function is defined as shown in the following relational expression (3):

[0064]

[0065] In the formula: represents the bias coefficient, and its size can be dynamically adjusted according to the proportion of edge and non-edge pixels in each sample image. Among them, the hyperparameter λ can adjust the size of the bias coefficient in different data sets to adapt to the change of pixel proportion; σ(·) represents the sigmoid function; represents the loss value calculated between the predicted edge map and the true label; Y = (y j , j = 1, 2,..., |Y|, y j ∈ *0, 1(), represents the manually annotated edge map; represents the edge probability map predicted by the network; Y + represents the non-edge pixels in the manually annotated edge map; Y - represents the edge pixels in the manually annotated edge map.

[0066] Further, input the photovoltaic panel image data set into the target edge detection network, and train through the above loss function to obtain the corresponding photovoltaic panel defect detection model.

[0067] The embodiment of the present invention provides a method for detecting photovoltaic panel defects, as Figure 4 shown, the method includes the following steps:

[0068] Step 201: Obtain the photovoltaic panel image to be detected.

[0069] Specifically, directly obtain the image of the photovoltaic panel to be detected.

[0070] Step 202: Input the image of the photovoltaic panel to be detected into the photovoltaic panel defect detection model to obtain the photovoltaic panel edge detection probability map.

[0071] Among them, the photovoltaic panel defect detection model is obtained according to the photovoltaic panel defect detection model determination method described in the embodiments of the present invention.

[0072] Specifically, input the image of the photovoltaic panel to be detected into the photovoltaic panel defect detection model. Through the processing of this photovoltaic panel defect detection model, the corresponding photovoltaic panel edge detection probability map can be output.

[0073] Step 203: Based on the photovoltaic panel edge detection probability map, through a preset processing method, determine the defect area in the image of the photovoltaic panel to be detected.

[0074] Specifically, by processing and analyzing the obtained photovoltaic panel edge detection probability map, the defect area in the image of the photovoltaic panel to be detected can be determined.

[0075] The photovoltaic panel defect detection method provided by the embodiments of the present invention can directly obtain the photovoltaic panel edge detection probability map of the image of the photovoltaic panel to be detected through the photovoltaic panel defect detection model obtained according to the photovoltaic panel defect detection model determination method described in the embodiments of the present invention. Further, by analyzing and processing this photovoltaic panel edge detection probability map, the defect area in the image of the photovoltaic panel to be detected can be determined. Therefore, by implementing the present invention, the accuracy of photovoltaic panel defect recognition and the effectiveness of detection are improved.

[0076] As an optional implementation manner of the embodiments of the present invention, step 203 includes: determining a binary edge map based on the photovoltaic panel edge detection probability map; analyzing the connected domains in the binary edge map to obtain the line contour of the photovoltaic panel edge detection probability map; performing rectangular fitting on the line contour to obtain the original line information of the image of the photovoltaic panel to be detected; and based on the original line information, through background difference method and region area analysis method processing, determine the defect area in the image of the photovoltaic panel to be detected.

[0077] First, by setting a threshold, process the photovoltaic panel edge detection probability map to obtain a binary edge map.

[0078] Secondly, analyze the connected domains in the edge map to obtain the line contour of the edge detection map.

[0079] Then, perform rectangular fitting on the analyzed edge lines to obtain the original line information of the photovoltaic panel.

[0080] Finally, the edge information of other regions of the photovoltaic panel is obtained through the background difference method, and a threshold is set through regional area analysis to determine whether the region is a defective region.

[0081] An embodiment of the present invention further provides a device for determining a photovoltaic panel defect detection model, as Figure 5 shown. The device includes:

[0082] A first acquisition module 301, configured to acquire a photovoltaic panel image dataset and an initial edge detection network. The initial edge detection network includes an initial feature encoding network and an initial feature decoding network, and the initial feature encoding network is connected to the initial feature decoding network; for detailed content, refer to the relevant description of step 101 in the above method embodiment.

[0083] A first setting module 302, configured to respectively set short residual connections inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network; for detailed content, refer to the relevant description of step 102 in the above method embodiment.

[0084] A second setting module 303, configured to set a long residual connection between the target feature encoding network and the target feature decoding network to obtain a target edge detection network; for detailed content, refer to the relevant description of step 103 in the above method embodiment.

[0085] A training module 304, configured to input the photovoltaic panel image dataset into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained; for detailed content, refer to the relevant description of step 104 in the above method embodiment.

[0086] The device for determining a photovoltaic panel defect detection model provided by the embodiment of the present invention decouples the initial edge detection network into two relatively independent network modules of feature encoding and feature decoding, simplifying the design of the edge detection network; by respectively adding short residual connections in the initial feature encoding network and the initial feature decoding network, the forward propagation and fusion of features are promoted, effectively improving the utilization rate and resolution of features; further, by setting a long residual connection between the target feature encoding network and the target feature decoding network, a fusion channel for low-layer high-resolution features and high-layer features is established, strengthening the feature extraction ability of the high-layer network without changing the original network structure. At the same time, since the residual connection does not generate new parameters, the performance of the model is improved while consuming the same computing resources.

[0087] As an alternative implementation manner of the embodiment of the present invention, the initial feature encoding network includes a data preprocessing module and at least one feature processing sub-module; the initial feature decoding network includes at least one upsampling module, and each upsampling module includes a feature extraction module, a deconvolution upsampling module, and a feature fusion processing module, wherein the deconvolution upsampling module is respectively connected to the feature extraction module and the feature fusion processing module.

[0088] As an alternative implementation manner of the embodiment of the present invention, the first setting module includes: a first setting sub-module, configured to set the short residual connection at the input end and the output end of each feature processing sub-module in the initial feature encoding network to obtain the target feature encoding network; a second setting sub-module, configured to set the short residual connection at the input end and the output end of each feature extraction module in the initial feature decoding network to obtain the target feature decoding network.

[0089] As an alternative implementation manner of the embodiment of the present invention, the second setting module includes: a third setting sub-module, configured to set the long residual connection at the output end of the target feature processing sub-module in the target feature encoding network and the input end of the target feature fusion processing module in the target feature decoding network to obtain the target edge detection network, and the feature size of the target feature processing sub-module is the same as that of the target feature fusion processing module.

[0090] As an alternative implementation manner of the embodiment of the present invention, the training module includes: an acquisition sub-module, configured to acquire a loss function; a training sub-module, configured to input the photovoltaic panel image dataset into the target edge detection network and obtain the photovoltaic panel defect detection model after training by the loss function.

[0091] For the detailed function description of the photovoltaic panel defect detection model determination device provided by the embodiment of the present invention, refer to the description of the photovoltaic panel defect detection model determination method in the above embodiment.

[0092] The embodiment of the present invention further provides a photovoltaic panel defect detection device, as Figure 6 shown, the device includes:

[0093] A second acquisition module 401, configured to acquire a photovoltaic panel image to be detected; for the detailed content, refer to the relevant description of step 201 in the above method embodiment.

[0094] An input module 402 is configured to input the photovoltaic panel image to be detected into a photovoltaic panel defect detection model to obtain a photovoltaic panel edge detection probability map, where the photovoltaic panel defect detection model is obtained according to the method for determining a photovoltaic panel defect detection model as described in the embodiments of the present invention. For detailed content, refer to the relevant description of step 202 in the above method embodiments.

[0095] A processing module 403 is configured to determine a defect area in the photovoltaic panel image to be detected based on the photovoltaic panel edge detection probability map through a preset processing method. For detailed content, refer to the relevant description of step 203 in the above method embodiments.

[0096] The photovoltaic panel defect detection device provided by the embodiments of the present invention can directly obtain a photovoltaic panel edge detection probability map of the photovoltaic panel image to be detected through the photovoltaic panel defect detection model obtained according to the method for determining a photovoltaic panel defect detection model as described in the embodiments of the present invention. Further, by analyzing and processing the photovoltaic panel edge detection probability map, the defect area in the photovoltaic panel image to be detected can be determined. Therefore, by implementing the present invention, the accuracy of photovoltaic panel defect identification and the effectiveness of detection are improved.

[0097] As an optional implementation manner of the embodiments of the present invention, the processing module includes: a determination sub-module configured to determine a binarized edge map based on the photovoltaic panel edge detection probability map; an analysis sub-module configured to analyze the connected domains in the binarized edge map to obtain a line contour of the photovoltaic panel edge detection probability map; a fitting sub-module configured to perform rectangular fitting on the line contour to obtain the original line information of the photovoltaic panel image to be detected; and a processing sub-module configured to determine the defect area in the photovoltaic panel image to be detected based on the original line information through background difference method and region area analysis method.

[0098] For a detailed description of the functions of the photovoltaic panel defect detection device provided by the embodiments of the present invention, refer to the description of the photovoltaic panel defect detection method in the above embodiments.

[0099] The embodiments of the present invention further provide a storage medium, such as Figure 7As shown in the figure, a computer program 501 is stored thereon. When the program is executed by a processor, it implements the steps of the photovoltaic panel defect detection model determination method or the photovoltaic panel defect detection method in the above embodiments. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0100] Those skilled in the art can understand that to implement all or part of the processes in the above embodiment methods, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0101] The embodiment of the present invention also provides an electronic device, as Figure 8 shown in the figure. The electronic device can include a processor 61 and a memory 62. Among them, the processor 61 and the memory 62 can be connected through a bus or other means. Figure 8 Taking the connection through the bus as an example.

[0102] The processor 61 can be a central processing unit (CPU). The processor 61 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or a combination of the above types of chips.

[0103] The memory 62, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the corresponding program instructions / modules in the embodiments of the present invention. The processor 61 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory 62, that is, to implement the photovoltaic panel defect detection model determination method or the photovoltaic panel defect detection method in the above method embodiments.

[0104] The memory 62 may include a program storage area and a data storage area. Among them, the program storage area can store an operating device and application programs required for at least one function; the data storage area can store data created by the processor 61, etc. In addition, the memory 62 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 62 may optionally include a memory remotely set relative to the processor 61, and these remote memories can be connected to the processor 61 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The one or more modules are stored in the memory 62 and, when executed by the processor 61, execute the photovoltaic panel defect detection model determination method or the photovoltaic panel defect detection method in the Figures 1-4 embodiments shown.

[0106] Specific details of the above electronic device can be understood by referring to the corresponding relevant descriptions and effects in the Figures 1 to 4 embodiments shown, and will not be elaborated here.

[0107] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A method for determining a photovoltaic panel defect detection model, characterized in that, The method includes: Obtaining a photovoltaic panel image dataset and an initial edge detection network. The initial edge detection network includes an initial feature encoding network and an initial feature decoding network. The initial feature encoding network is connected to the initial feature decoding network. The initial feature encoding network is obtained by cropping the original convolutional neural network VGG16 to remove the backend fully connected layer, average pooling layer, and classification layer. The initial feature decoding network is used to upsample high-level abstract features and restore the feature size and resolution. A stacked upsampling structure is designed inside the initial feature decoding network for segmentally restoring the feature size. The initial feature decoding network includes at least one upsampling module. Each upsampling module includes a feature extraction module, a transposed convolution upsampling module, and a feature fusion processing module. In each upsampling module, the transposed convolution upsampling module is respectively connected to the feature extraction module and the feature fusion processing module; Short residual connections are respectively set inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network. The short residual connection is used to process intermediate layer features; A long residual connection is set between the target feature encoding network and the target feature decoding network to obtain a target edge detection network; Inputting the photovoltaic panel image dataset into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained; Wherein, inputting the photovoltaic panel image dataset into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained, includes: Obtaining a loss function; Inputting the photovoltaic panel image dataset into the target edge detection network and training it through the loss function to obtain the photovoltaic panel defect detection model; Wherein, the loss function is shown in the following relational expression: Wherein: , representing a bias coefficient, whose size is dynamically adjusted according to the ratio of edge and non-edge pixels in each sample image, where the hyperparameter is used to adjust the size of the bias coefficient in different datasets to adapt to the change of pixel ratio; represents the sigmoid function; represents the loss value calculated between the predicted edge map and the ground truth label; , representing the manually annotated edge map; , representing the edge probability map predicted by the network; represents the non-edge pixels in the manually annotated edge map; represents the edge pixels in the manually annotated edge map.

2. The method according to claim 1, wherein The initial feature encoding network includes a data preprocessing module and at least one feature processing sub-module.

3. The method according to claim 2, wherein Respectively setting short residual connections inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network, includes: Setting the short residual connection at the input end and output end of each feature processing sub-module in the initial feature encoding network to obtain the target feature encoding network; Setting the short residual connection at the input end and output end of each feature extraction module in the initial feature decoding network to obtain the target feature decoding network.

4. The method according to claim 3, characterized in that, Setting a long residual connection between the target feature encoding network and the target feature decoding network to obtain a target edge detection network, includes: Setting the long residual connection at the output end of the target feature processing sub-module in the target feature encoding network and the input end of the target feature fusion processing module in the target feature decoding network to obtain the target edge detection network. The feature size of the target feature processing sub-module is the same as the feature size of the target feature fusion processing module.

5. The method according to claim 4, characterized in that Input the photovoltaic panel image dataset into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained, including: Obtain a loss function; Input the photovoltaic panel image dataset into the target edge detection network and obtain the photovoltaic panel defect detection model through training by the loss function.

6. A method for detecting defects in a photovoltaic panel, characterized in that, The method includes: Obtain a photovoltaic panel image to be detected; Input the photovoltaic panel image to be detected into the photovoltaic panel defect detection model to obtain a photovoltaic panel edge detection probability map, where the photovoltaic panel defect detection model is determined according to the method for determining a photovoltaic panel defect detection model as described in any one of claims 1-5; Based on the photovoltaic panel edge detection probability map, determine the defect area in the photovoltaic panel image to be detected through a preset processing method.

7. The method according to claim 6, wherein Based on the photovoltaic panel edge detection probability map, determine the defect area in the photovoltaic panel image to be detected through a preset processing method, including: Determine a binary edge map based on the photovoltaic panel edge detection probability map; Analyze the connected regions in the binary edge map to obtain the line contour of the photovoltaic panel edge detection probability map; Perform rectangular fitting on the line contour to obtain the original line information of the photovoltaic panel image to be detected; Based on the original line information, determine the defect area in the photovoltaic panel image to be detected through background difference method and regional area analysis method.

8. A device for determining a photovoltaic panel defect detection model, characterized in that, The device includes: A first acquisition module, configured to acquire a photovoltaic panel image dataset and an initial edge detection network. The initial edge detection network includes an initial feature encoding network and an initial feature decoding network. The initial feature encoding network is connected to the initial feature decoding network. The initial feature encoding network is obtained by cropping the original convolutional neural network VGG16 to remove the backend fully connected layer, average pooling layer, and classification layer. The initial feature decoding network is used to upsample high-level abstract features and restore the feature size and resolution. A stacked upsampling structure is designed inside the initial feature decoding network for segmentally restoring the feature size. The initial feature decoding network includes at least one upsampling module, and each upsampling module includes a feature extraction module, a deconvolution upsampling module, and a feature fusion processing module. In each upsampling module, the deconvolution upsampling module is respectively connected to the feature extraction module and the feature fusion processing module; A first setting module, configured to respectively set short residual connections inside the initial feature encoding network and the initial feature decoding network to obtain a target feature encoding network and a target feature decoding network. The short residual connection is used to process intermediate layer features; A second setting module, configured to set a long residual connection between the target feature encoding network and the target feature decoding network to obtain a target edge detection network; A training module, configured to input the photovoltaic panel image dataset into the target edge detection network for training until a photovoltaic panel defect detection model that meets the conditions is obtained; Among them, the training module includes: An acquisition sub-module, configured to acquire a loss function; A training sub-module, configured to input the photovoltaic panel image dataset into the target edge detection network, and obtain the photovoltaic panel defect detection model after training with the loss function; wherein, the loss function is shown in the following relational expression: In the formula: , representing the bias coefficient, whose size is dynamically adjusted according to the ratio of edge and non-edge pixels in each sample image, where the hyperparameter is used to adjust the size of the bias coefficient in different datasets to adapt to the change of pixel ratio; represents the sigmoid function; represents the loss value calculated between the predicted edge map and the ground truth label; , representing the manually annotated edge map; , representing the edge probability map predicted by the network; represents the non-edge pixels in the manually annotated edge map; represents the edge pixels in the manually annotated edge map.

9. A photovoltaic panel defect detection device, characterized in that, The device includes: A second acquisition module, configured to acquire a photovoltaic panel image to be detected; An input module, configured to input the photovoltaic panel image to be detected into the photovoltaic panel defect detection model to obtain a photovoltaic panel edge detection probability map, and the photovoltaic panel defect detection model is determined according to the photovoltaic panel defect detection model determination method described in any one of claims 1-5; A processing module, configured to determine a defect area in the photovoltaic panel image to be detected based on the photovoltaic panel edge detection probability map through a preset processing method.

10. An electronic device, characterized in that, It includes: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores a computer program, and the processor executes the computer program to execute the photovoltaic panel defect detection model determination method described in any one of claims 1 to 5, or the photovoltaic panel defect detection method described in claim 6 or 7.

Images