Photovoltaic module hot spot defect detection method based on multi-scale feature graph inference network

Through the cross-embedding mapping mechanism and multi-scale feature fusion of multi-scale feature map inference network, the problem of poor robustness of thermal spot defect detection in photovoltaic power plants is solved, and the effect of high accuracy and rapid detection is achieved.

CN114283137BActive Publication Date: 2024-11-01HEBEI UNIV OF TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111596547.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-24
Publication Date
2024-11-01
Estimated Expiration
2041-12-24

AI Technical Summary

Technical Problem

The detection of hot spot defects in photovoltaic power plants is poorly robust, and is greatly affected by light intensity, shooting angle and complex terrain. The size of hot spot defects is small, making it difficult to accurately identify in an open environment.

Method used

Design a thermal spot defect detection method for photovoltaic modules based on multi-scale feature map inference network, map different scale features to graph space through cross-embedding mapping mechanism, combine semantics and spatial information, and adopt multi-scale feature fusion and global inference strategies to improve detection robustness.

Benefits of technology

The accuracy and robustness of thermal spot defect detection of photovoltaic modules has been significantly improved, and the detection accuracy has been improved to 91.08%, avoiding the inefficiency of traditional manual feature extraction, and having strong robustness and rapid detection capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114283137B_ABST
    Figure CN114283137B_ABST
Patent Text Reader

Abstract

The present invention discloses a photovoltaic module hot spot defect detection method based on a multi-scale feature map reasoning network. The detection method is based on an improved YOLOv4 neural network model. A self-designed multi-scale feature map reasoning module suitable for photovoltaic power station hot spot defect detection is added to the feature fusion part of the original model, guiding the multi-scale feature fusion and highlighting the defect location area to suppress the complex background features in the image, effectively improving the recognition ability of photovoltaic module hot spot defects. The detection method of the present invention combines deep learning technology with image processing technology, not only avoiding the inefficiency and uncertainty of traditional manual feature extraction, but also the detection process has strong robustness, significantly improving the detection accuracy and speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of defect detection, and in particular to a visual detection method for hot spot defects in photovoltaic power stations, and in particular to a hot spot defect detection method for photovoltaic modules based on a multi-scale feature graph inference network. Background Art

[0002] Photovoltaic power generation has been widely used due to its advantages of being pollution-free, renewable, universal, flexible, storable, safe and reliable. With the continuous expansion of the scale of photovoltaic power stations, photovoltaic power generation systems are widely used in a wide range of places, including large-scale ground photovoltaic power stations, roofs of residential and commercial buildings, photovoltaic street lights, etc. In these places, it is inevitable that buildings, shades, chimneys, dust, clouds, etc. will block photovoltaic modules and form hot spot defects. The generation of hot spot defects will reduce the efficiency of the photovoltaic system, shorten the service life of photovoltaic modules, and even cause fires. Therefore, timely and accurate identification of hot spot defects is crucial to improving the operating safety of photovoltaic power plants and reducing downtime.

[0003] Since most photovoltaic power stations are located in scenes with complex terrain and changing environments, the captured hot spot infrared images will inevitably be affected by light intensity, shooting angle changes, and complex terrain. In addition, the hot spot defect size is small, so it is challenging to train a robust artificial intelligence model in such a highly open environment. A deep learning network is designed to be applied to the real-time detection of hot spot defects in solar photovoltaic modules, thereby ensuring the safe and stable operation of photovoltaic power stations. Summary of the invention

[0004] In order to solve the problem of poor robustness of hot spot defect detectors in infrared images of photovoltaic power stations at different heights. The present invention proposes a hot spot defect detection method for photovoltaic modules based on a multi-scale feature graph inference network. This detection method designs a new cross-embedding mapping mechanism to map features of different scales in the coordinate space to the graph space. Then, the cross-embedding graph inference module is used to combine the strongest semantic information and rich spatial information, and to complete the reasoning of interactions and global information at different scales at the same time. The model integrates spatial features and semantic features to obtain globally relevant information. By adopting multi-scale feature fusion and global reasoning strategies, the common information of deep convolutional features is fully mined, and it has significant robustness on photovoltaic infrared image datasets at different heights.

[0005] The technical solution adopted by the present invention to solve the above problem is to design a photovoltaic module hot spot defect detection method based on a multi-scale feature graph inference network, characterized in that the specific steps of the method are:

[0006] Step 1: Obtaining the dataset

[0007] 1.1 Original image acquisition: An infrared camera is mounted on the drone to take photos of the front of the photovoltaic module at different flight altitudes;

[0008] 1.2 Dataset preparation: Normalize the size of the photos obtained in step 1.1, and filter out images containing hot spot defects to obtain a hot spot defect image library;

[0009] 1.3 Manually use LabelImg to label the hot spot defect type of the defect area of ​​each image in the hot spot defect image library to obtain a hot spot defect image dataset; randomly extract the hot spot defect image dataset, and use no less than 60% of it as a training sample set, and the rest as a verification sample set;

[0010] Step 2: Training of multi-scale feature inference network detector

[0011] 2.1 Preprocessing of training sample sets

[0012] The training sample set is preprocessed using Mosaic data enhancement;

[0013] 2.2 Parameter settings

[0014] Initialize all weight values, bias values, batch normalization scale factor values, set the initial learning rate and batch_size of the detector, input the initialized parameter data into the detector, and set the initial learning rate of the detector to 0.001;

[0015] 2.3 Detector Training

[0016] The preprocessed training sample set is input into the detector after the initialization parameters are set. The detector is obtained by replacing the PANet module in the Yolov4 network model with a multi-scale feature map inference network module. In the detector, the preprocessed training sample set is firstly subjected to feature extraction and multi-scale fusion in combination with the multi-scale feature map inference network module. Then, the feature maps extracted from the last three layers are refined and fused through convolution layers to reduce the dimension. The tensor is predicted using a classification regression network to obtain the predicted values ​​of the position, defect category and confidence. The generated predicted value is compared with the labeled information to generate a loss value. Then, back propagation is performed to update the parameters of the backbone network and the classification regression network until the loss value meets the preset value and the training of the detector parameters is completed.

[0017] 2.4 Detector Test

[0018] Input the verification sample set into the detector that has completed parameter training in step 2.3 to obtain the tensor prediction value of the verification sample set; compare its tensor prediction value with the annotation information to verify the reliability of the detector, which is used to monitor whether the detector is overfitting, so as to decide whether to terminate the training and readjust the parameters; when the prediction accuracy of hot spot defects at different heights in the verification sample set is greater than 80%, the detector is verified to be reliable;

[0019] Step 3: PV module hot spot defect detection

[0020] The photo of the front side of the photovoltaic module to be inspected taken by the infrared camera is subjected to the same size normalization operation as in step 1.2 of the first step, and then input into the detector verified as reliable in step 2 to obtain the hot spot defect tensor information of the infrared image of the photovoltaic module to be inspected, including the defect location, defect category and confidence.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: the detection method of the present invention is based on the improved YOLO v4 neural network model, and a self-designed multi-scale feature map reasoning module suitable for hot spot defect detection in photovoltaic power stations is added to the feature fusion part of the original model, guiding the multi-scale feature fusion and highlighting the complex background features in the defect location area to suppress the image, effectively improving the recognition ability of hot spot defects of photovoltaic components. The detection method of the present invention combines deep learning technology with image processing technology, which not only avoids the inefficiency and uncertainty of traditional manual feature extraction, but also has strong robustness in the detection process; the original YOLOv4 network model is used to detect hot spot defects of photovoltaic components, and the average recognition rate on images taken at a height of 28 meters is 86.83%, while the detection method of the present invention uses the initial setting learning rate of 0.001, and the average accuracy of the model reaches 91.08%, which significantly improves the detection accuracy and speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 The present invention is a flow chart of an embodiment of the detection method.

[0024] Figure 2 A schematic diagram of the structure and principle of a multi-scale feature inference network detector according to an embodiment of the detection method of the present invention.

[0025] Figure 3It is a schematic diagram of the structure and principle of a cross-embedded graph reasoning module of a multi-scale feature graph reasoning network module of a multi-scale feature reasoning network detector of an embodiment of the detection method of the present invention.

[0026] Figure 4 It is the infrared image of the photovoltaic module to be inspected.

[0027] Figure 5 The detection method of the present invention is used to Figure 4 The result diagram of the detection on the image in . DETAILED DESCRIPTION

[0028] The present invention is further described below in conjunction with the embodiments and drawings.

[0029] The present invention provides a photovoltaic module hot spot defect detection method based on a multi-scale feature graph inference network (hereinafter referred to as the detection method), characterized in that the specific steps of the method are:

[0030] Step 1: Obtaining the dataset

[0031] 1.1 Original image acquisition: An infrared camera is mounted on the drone to take photos of the front of the photovoltaic module at different flight altitudes;

[0032] 1.2 Dataset preparation: Normalize the size of the photos obtained in step 1.1, and filter out images containing hot spot defects to obtain a hot spot defect image library;

[0033] 1.3 Manually use LabelImg to annotate the defect area of ​​each image in the hot spot defect image library with the hot spot defect type label to obtain the hot spot defect image dataset; randomly extract the hot spot defect image dataset, and use no less than 60% of it as the training sample set and the rest as the verification sample set.

[0034] Step 2: Training of Multi-Scale Feature Inference Network Detector (Detector for short)

[0035] 2.1 Preprocessing of training sample sets

[0036] The training sample set is preprocessed using Mosaic data enhancement;

[0037] 2.2 Parameter settings

[0038] Initialize all weight values, bias values, batch normalization scale factor values, set the initial learning rate and batch_size of the detector, input the initialized parameter data into the detector, and set the initial learning rate of the detector to 0.001;

[0039] 2.3 Detector Training

[0040] The preprocessed training sample set is input into the detector after the initialization parameters are set. The detector is obtained by replacing the PANet module (multi-scale fusion module) in the Yolov4 network model with a multi-scale feature map inference network module. In the detector, the preprocessed training sample set is firstly subjected to feature extraction and multi-scale fusion in combination with the multi-scale feature map inference network module. Then, the feature map obtained by the last three layers of extraction is refined and fused through the convolution layer to reduce the dimension. The tensor is predicted using the classification regression network to obtain the predicted value of the position, defect category and confidence. The generated predicted value is compared with the labeled information to generate the loss value, and then back propagation is performed to update the parameters of the backbone network and the classification regression network until the loss value meets the preset value and the training of the detector parameters is completed.

[0041] 2.4 Detector Test

[0042] The verification sample set is input into the detector that has completed parameter training in step 2.3 to obtain the tensor prediction value of the verification sample set; the tensor prediction value is compared with the annotation information to verify the reliability of the detector, which is used to monitor whether the detector is overfitting, so as to decide whether it is necessary to terminate the training and readjust the parameters; when the prediction accuracy of hot spot defects at different heights in the verification sample set is greater than 80%, the detector is verified to be reliable.

[0043] Step 3: PV module hot spot defect detection

[0044] The photo of the front side of the photovoltaic module to be inspected taken by the infrared camera is subjected to the same size normalization operation as in step 1.2 of the first step, and then input into the detector verified as reliable in step 2 to obtain the hot spot defect tensor information of the infrared image of the photovoltaic module to be inspected, including the defect location, defect category and confidence.

[0045] Working principle of the multi-scale feature map inference network module: The goal of the multi-scale feature map inference network module is to find a transformation function that can effectively aggregate rich spatial and semantic features and infer global information. Formally given a backbone network multi-scale feature list (C3, C4, C5), the feature fusion and inference process is defined as:

[0046] P5=dbl(spp(dbl(C3))) (1)

[0047] P4=G(dbl(C4),up(dbl(P5))) (2)

[0048] P3=G(dbl(C3),up(dbl(P5))) (3)

[0049] Where dbl represents the DBL (DarknetConvolution BatchNomalization-LeakyReLU, two-dimensional convolution operation-normalization-nonlinear activation function) operation, spp represents the spatial pyramid pooling operation, up represents the upsampling operation, and G(·) is the operation of the cross-embedded graph reasoning module.

[0050] Working principle of the cross-embedded graph reasoning module: The cross-embedded graph reasoning module aims to achieve complementary fusion of semantic features and spatial features and global relationship reasoning. The specific process is as follows:

[0051] First, the coordinate space with detailed spatial features and features with strong semantic information From the coordinate space Ω, cross embedding is performed into the same graph space H, where L = W × H, W, H, and C are the width, height, and number of channels of the feature graph, respectively. Thus, the joint distribution f(X1, X2) of the two scales and the constructed graph G = (V, E) are obtained. The constructed graph combines the two features of X1 and X2, where the number of nodes V and the edges E representing the relationship between nodes are defined.

[0052] The features in the construction graph G = (V, E) are transformed into features in the new coordinate space. The corresponding feature nodes are determined by the learnable mapping weights. The generation process of each node in the feature node V in the new coordinate space is as follows:

[0053]

[0054]

[0055] The mapping matrices that can be learned are and B1 and B2 can be obtained by training X1 and X2 respectively. and denote the row vectors of B1 and B2 respectively, and are row vectors and The jth element in ; N is the node dimension mapped to, which is equal to C / 2. Among them, are the column vectors of the input features X1 and X2 respectively. Through the operation, we can get the i-th node of the two features X1 and X2 respectively and

[0056] The obtained graph structure can be updated by defining a single-layer graph convolutional network GCN(·) to update the relationship between nodes and edges. In the definition of graph convolution, the way to update the node graph is as shown in formula (6), where A gThe adjacency matrix representing the edge relationships is randomly initialized, and the weights W are learned by gradient descent during training. g , I represents the unit matrix. is the node feature matrix in the new coordinate space, and the convolution of the graph is expressed as:

[0057] Z=((IA g )V)W g (6)

[0058] In order to embed graph convolutional networks more conveniently, a one-dimensional convolution operation Conv1D(·) is designed to implement the graph update process as shown in formula (7):

[0059] Z = GCN(V) = Conv1D(Conv1D(V) T ) T (7)

[0060] Finally, the output features of the single-layer graph convolutional network GCN are mapped back to the original coordinate space. By updating the node feature matrix Z, the goal is to establish a mapping function to map the updated features into the features Y in the coordinate space:

[0061] Y=g(Z) (8)

[0062] Similar to the graph space mapping function, a linear mapping g(·) is used to implement the mapping relationship as shown in formula (9), where the inverse mapping matrix D is as shown in formula (10):

[0063]

[0064]

[0065] Among them, use d i Represents the row vector of the inverse mapping matrix D, d ij is the row vector d i The jth element in z j is the column vector of the node feature matrix Z, y i is the row vector of feature Y mapped to the coordinate space. According to the matrix operation, the inverse mapping relationship is constructed using the equation. After mapping the information back to the coordinate space, a residual connection operation is added to improve the versatility of the module. The final output O of the cross-embedded graph reasoning module is as follows:

[0066] O=Y+X1 (11)

[0067] Example 1

[0068] This embodiment provides a method for detecting hot spot defects of photovoltaic modules based on a multi-scale feature graph inference network, characterized in that the specific steps of the method are:

[0069] Step 1: Obtaining the dataset

[0070] 1.1 Original image acquisition: An infrared camera is mounted on a drone, and the infrared camera is used to take photos of the front of the photovoltaic module at different flight altitudes. In this embodiment, infrared images at four altitudes of 28 meters, 35 meters, 47 meters, and 60 meters are taken.

[0071] 1.2 Dataset preparation: The photos obtained in step 1.1 are normalized to the same size. In this embodiment, the infrared image size is 640×512. 1500 defect images are selected at each height to obtain a hot spot defect image library of 6000 images.

[0072] 1.3 Manually use LabelImg to annotate the hot spot defect type label of the defect area of ​​each image in the hot spot defect image library to obtain a hot spot defect image dataset; randomly extract the hot spot defect image dataset, use 80% as the training sample set, and the remaining 20% ​​as the verification sample set.

[0073] Step 2: Training of multi-scale feature inference network detector

[0074] 2.1 Preprocessing of training sample sets

[0075] The training sample set is preprocessed using Mosaic data enhancement;

[0076] Mosaic data enhancement uses random scaling, random cropping, and random arrangement to stitch images together, which works well for detecting small objects. Then the resolution of the original input image is adjusted, and the image is adaptively scaled according to the aspect ratio of the image.

[0077] 2.2 Parameter settings

[0078] Initialize all weight values, bias values, batch normalization scale factor values, set the initial learning rate and batch_size of the detector, input the initialized parameter data into the detector, and set the initial learning rate of the detector to 0.001;

[0079] The detector parameters are initialized as follows: the maximum number of iterations (epoch) is set to 200, the learning rate is set to 0.001 for the first 50 epochs, and the learning rate is set to 0.0001 for the next 150 epochs, the learning rate reduction factor is 0.1, and the weight decay of the regularization term is 0.0005.

[0080] 2.3 Detector Training

[0081] The preprocessed training sample set is input into the detector after the initialization parameters are set. The detector is obtained by replacing the PANet module (multi-scale fusion module) in the Yolov4 network model with a multi-scale feature map inference network module. In the detector, the preprocessed training sample set is firstly subjected to feature extraction and multi-scale fusion in combination with the multi-scale feature map inference network module. Then, the feature map obtained by the last three layers of extraction is refined and fused through the convolution layer to reduce the dimension. The tensor is predicted using the classification regression network to obtain the predicted value of the position, defect category and confidence. The generated predicted value is compared with the labeled information to generate the loss value, and then back propagation is performed to update the parameters of the backbone network and the classification regression network until the loss value meets the preset value and the training of the detector parameters is completed.

[0082] 2.4 Detector Test

[0083] The verification sample set is input into the detector that has completed parameter training in step 2.3 to obtain the tensor prediction value of the verification sample set; the tensor prediction value is compared with the annotation information to obtain the accuracy of hot spot defects at different heights, which is greater than 85%.

[0084] Step 3: PV module hot spot defect detection

[0085] The photo of the front side of the photovoltaic module to be inspected taken by the infrared camera is subjected to the same size normalization operation as in step 1.2 of the first step, and then input into the detector verified as reliable in step 2 to obtain the hot spot defect tensor information of the infrared image of the photovoltaic module to be inspected, including the defect location, defect category and confidence.

[0086] The multi-scale feature inference network detector adopts an improved Yolov4 neural network model with high detection accuracy and fast recognition speed. Specifically, the detector model structure is as follows: Figure 2 As shown in the figure, it mainly consists of four parts: data enhancement, feature extraction, feature fusion and classification regression. The data enhancement part uses Mosaic data enhancement to splice the infrared images of the photovoltaic power station through random scaling, random cropping and random arrangement to improve the detection ability of small targets. In terms of feature extraction, the infrared image of the photovoltaic power station is input into the CSPDarknet53 backbone network, and the feature map is extracted from it through operations such as convolution and pooling. The feature map can be shared for subsequent feature fusion. In the feature fusion part, a multi-scale feature map inference network module is used to generate pooled feature vectors of different fixed sizes, enhance the expression ability of features, and have a good effect on the detection of the same object at different sizes. The classification regression part uses GIOU_Loss as the loss function of the Bounding box, which effectively solves the problem of non-overlapping bounding boxes and improves the speed and accuracy of prediction box regression.

[0087] The feature extraction stage of the detector follows the CSPDarknet53 structure of the Yolov4 neural network model. CSPDarknet53 adds a CSP module (cross-stage local module) on the basis of Darket53. The CSP module consists of a convolutional layer and a residual structure Concat in the Resnet network. Darket53 has a total of 53 convolution layers. Excluding the last FC (fully connected layer, which is actually implemented by 1x1 convolution), a total of 52 convolutions are used as the main network. These 52 convolutional layers are composed of a convolution kernel with 32 filters, and then 5 groups of repeated residual units resblock_body (these 5 groups of residual units, each unit consists of a separate convolutional layer and a group of repeated convolutional layers, and the repeated convolutional layers are repeated 1, 2, 8, 8, and 4 times respectively; in each repeated convolutional layer, a 1x1 convolution operation is performed first, and then a 3x3 convolution operation is performed. The number of filters is halved and then restored, a total of 52 layers.

[0088] The multi-scale feature map inference network module integrates strong semantic features into spatial features using the cross-embedded graph inference module. The multi-scale feature map inference network module can retain the best spatial detail features of small defects and introduce strong semantic information to achieve global context reasoning. In the multi-scale feature map inference network, the cross-embedded graph inference module plays a key role in feature refinement. The multi-scale feature map inference network combines sufficient spatial and semantic information of small defects in the simplest and most effective way, eliminating unnecessary information fusion. Through the integration and global reasoning of the strongest semantic features and spatial features of small defects, the multi-scale feature map inference network outperforms PANet in hotspot defect detection. In order to avoid invalid network connections and improve feature utilization, the small size characteristics of hotspot defects are analyzed. Using the cross-embedded graph inference module, the strongest semantic information is integrated and inferred for and respectively. The multi-scale feature map inference network effectively refines the feature distribution and enhances the robustness of the network using global features.

[0089] The test results of the present invention show that the present invention adopts the improved Yolov4 model and applies it to the hot spot defect detection of photovoltaic power stations, which can realize rapid intelligent photovoltaic power station inspection.

[0090] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

[0091] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A photovoltaic module hot spot defect detection method based on a multi-scale feature graph inference network, characterized in that the specific steps of the method are: Step 1: Obtaining the dataset 1.1 Original image acquisition: An infrared camera is mounted on the drone to take photos of the front of the photovoltaic module at different flight altitudes; 1.2 Dataset preparation: Normalize the size of the photos obtained in step 1.1, and filter out images containing hot spot defects to obtain a hot spot defect image library; 1.3 Manually use LabelImg to label the hot spot defect type of the defect area of ​​each image in the hot spot defect image library to obtain a hot spot defect image dataset; randomly extract the hot spot defect image dataset, and use no less than 60% of it as a training sample set, and the rest as a verification sample set; Step 2: Training of multi-scale feature inference network detector 2.1 Preprocessing of training sample sets The training sample set is preprocessed using Mosaic data enhancement; 2.2 Parameter settings Initialize all weight values, bias values, batch normalization scale factor values, set the initial learning rate and batch_size of the detector, input the initialized parameter data into the detector, and set the initial learning rate of the detector to 0.001; 2.3 Detector Training The preprocessed training sample set is input into the detector after the initialization parameters are set. The detector is obtained by replacing the PANet module in the Yolov4 network model with a multi-scale feature map inference network module. In the detector, the preprocessed training sample set is firstly subjected to feature extraction and multi-scale fusion in combination with the multi-scale feature map inference network module. Then, the feature maps extracted from the last three layers are refined and fused through convolution layers to reduce the dimension. The tensor is predicted using a classification regression network to obtain the predicted values ​​of the position, defect category and confidence. The generated predicted value is compared with the labeled information to generate a loss value. Then, back propagation is performed to update the parameters of the backbone network and the classification regression network until the loss value meets the preset value and the training of the detector parameters is completed. 2.4 Detector Test Input the verification sample set into the detector that has completed parameter training in step 2.3 to obtain the tensor prediction value of the verification sample set; compare its tensor prediction value with the annotation information to verify the reliability of the detector, which is used to monitor whether the detector is overfitting, so as to decide whether to terminate the training and readjust the parameters; when the prediction accuracy of hot spot defects at different heights in the verification sample set is greater than 80%, the detector is verified to be reliable; Step 3: PV module hot spot defect detection The photo of the front of the photovoltaic module to be inspected taken by the infrared camera is subjected to the same size normalization operation as in step 1.2 of the first step, and then input into the detector verified as reliable in step 2 to obtain the hot spot defect tensor information of the infrared image of the photovoltaic module to be inspected, including the defect location, defect category and confidence level; Working principle of the multi-scale feature map inference network module: The goal of the multi-scale feature map inference network module is to find a transformation function that can effectively aggregate rich spatial and semantic features and infer global information; formally given a backbone network multi-scale feature list (C3, C4, C5), the feature fusion and inference process is defined as: P5=dbl(spp(dbl(C3))) (1) P4=G(dbl(C4),up(dbl(P5))) (2) P3=G(dbl(C3),up(dbl(P5))) (3) Where dbl represents the DBL operation, spp represents the spatial pyramid pooling operation; up represents the upsampling operation, and G(·) is the operation of the cross-embedding graph reasoning module; Working principle of the cross-embedded graph reasoning module: The cross-embedded graph reasoning module aims to achieve complementary fusion of semantic features and spatial features and global relationship reasoning. The specific process is as follows: First, the coordinate space with detailed spatial features and features with strong semantic information From the coordinate space Ω, we cross-embed into the same graph space H, where L = W × H, W, H, and C are the width, height, and number of channels of the feature map, respectively; thus we get the joint distribution f(X 1 , X 2 ) and construct a graph G = (V, E), which combines X 1 and X 2 Two features, where the number of nodes V and the edges E representing the nodes and node relationships are defined; The features in the construction graph G = (V, E) are transformed into features in the new coordinate space. The corresponding feature nodes are determined by the learnable mapping weights. The generation process of each node in the feature node V in the new coordinate space is as follows: The mapping matrices that can be learned are and B 1 and B 2 X 1 and X 2 Through training, and Respectively represent B 1 and B 2 The row vector of and are row vectors and The jth element in ; N is the node dimension mapped to, which is equal to C / 2; where They are the input features X 1 and X 2 The column vector of 1 and X 2 The i-th node and The obtained graph structure is updated by defining a single-layer graph convolutional network GCN(·) to update the relationship between nodes and edges. In the definition of graph convolution, the way to update the node graph is as shown in formula (6), where A g The adjacency matrix representing the edge relationships is randomly initialized, and the weights W are learned by gradient descent during training. g , I represents the unit matrix; is the node feature matrix in the new coordinate space, and the convolution of the graph is expressed as: Z=((I-A g )V)W g (6) In order to embed graph convolutional networks more conveniently, a one-dimensional convolution operation Conv1D(·) is designed to implement the graph update process as shown in formula (7): Z=GCN(V)=Conv1D(Conv1D(V) T ) T (7) Finally, the output features of the single-layer graph convolutional network GCN are mapped back to the original coordinate space. By updating the node feature matrix Z, the goal is to establish a mapping function to map the updated features into the features Y in the coordinate space: Y=g(Z) (8) Similar to the graph space mapping function, a linear mapping g(·) is used to implement the mapping relationship as shown in formula (9), where the inverse mapping matrix D is as shown in formula (10): Among them, use d i Represents the row vector of the inverse mapping matrix D, d ij is the row vector d i The jth element in z j is the column vector of the node feature matrix Z, y i is the row vector of feature Y mapped to the coordinate space; according to the matrix operation, the inverse mapping relationship is constructed using the equation; after mapping the information back to the coordinate space, a residual connection operation is added to improve the versatility of the module. The final output O of the cross-embedded graph reasoning module is as follows: O=Y+X 1 (11)。 2. According to claim 1, the photovoltaic module hot spot defect detection method based on the multi-scale feature graph inference network, It is characterized in that In the first step, an infrared camera was used to take photos of the front of the photovoltaic panels when the drone was flying at four different altitudes: 28 meters, 35 meters, 47 meters, and 60 meters.

3. According to claim 1, the photovoltaic module hot spot defect detection method based on the multi-scale feature graph inference network, It is characterized in that In the first step, the number of defect images in the hot spot defect image library is 6,000.

4. The photovoltaic module hot spot defect detection method based on a multi-scale feature graph inference network according to claim 1, It is characterized in that In the first step, the hot spot defect image dataset is randomly extracted, 80% of which is used as the training sample set and the remaining 20% ​​is used as the verification sample set.

Citation Information

Patent Citations

  • Visual defect detection method of deep convolutional neural network

    CN110610475A

  • Lithium battery defect classification method based on multi-scale convolution feature fusion network

    CN111553433A

  • Non-uniform texture small defect detection method based on improved Faster R-CNN model

    CN111598861A

  • Photovoltaic defect detection method based on Yolov4 and thermal infrared image

    CN113538391A

  • Method for detecting image target in smart home environment

    WO2021244079A1