Photovoltaic module hot spot detection method and system

Abstract

The invention provides a photovoltaic module hot spot detection method and system, and the method comprises the implementation steps: obtaining a training and testing sample set; constructing an improved YOLOv8 target detection network model and carrying out iterative training on the improved YOLOv8 target detection network model; and obtaining a hot spot detection result of the photovoltaic module. According to the method, the visible light feature extraction branches and the infrared light feature extraction branches which are arranged in parallel are used for replacing a trunk feature extraction network in YOLOv8, and the target detection network model is iteratively trained and the hot spot detection result of the photovoltaic module is obtained. The trunk feature extraction network carries out feature extraction on each visible light image and each infrared light image, the cross-modal feature fusion network carries out feature fusion on the extracted visible light feature maps and infrared light feature maps, and then a detection result is obtained through feature fusion. The defects of complex detection process and high computing resource consumption caused by respective detection of a visible light image and an infrared light image in the prior art are avoided, and the detection efficiency is effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of photovoltaic module detection and relates to a photovoltaic module hot spot detection method and system. BACKGROUND

[0002] As an important part of clean energy, the power generation efficiency of photovoltaic power generation is highly dependent on the cleanliness and health status of photovoltaic panels. However, photovoltaic modules are exposed to outdoor environments for a long time, and are prone to accumulate pollutants such as dust and bird droppings, which can cause "hot spot" effects due to shading, resulting in local overheating of the panel, accelerated power decay, and even the risk of burning out.

[0003] In order to ensure the normal operation of photovoltaic power stations, regular inspections of photovoltaic systems are required. Traditional manual inspections cannot timely check the shading conditions, and it is difficult to judge the hot spots of photovoltaic modules, so it is not possible to replace photovoltaic modules with hot spots in a timely manner. Existing hot spot detection generally uses robots, for example, patent application No. CN116124219A discloses a distributed photovoltaic module shading and hot spot detection method and system, which proposes a distributed photovoltaic module shading and hot spot detection method and system. The detection robot moves according to the guidance of the path guidance unit distributed among the distributed photovoltaic modules; during the movement of the detection robot or after the detection robot moves to the predetermined position, the visible light image and thermal image of the photovoltaic module are collected by the image acquisition device of the detection robot; the analysis unit detects the shading and hot spot information by detecting the shading and hot spot of the visible light image and thermal image through the shading and hot spot detection model. This invention obtains the visible light image and thermal image of the photovoltaic module, and detects the shading and hot spot information of the photovoltaic module by means of the shading and hot spot detection model, which facilitates subsequent accurate maintenance and solves the problem of low efficiency of photovoltaic shading and hot spot detection. However, because the model used is an APSeg-YOLOv5 architecture shading detection neural network model and a hot spot detection neural network model, the detection process is complex and the computational resources are consumed, resulting in low detection efficiency, which cannot meet the real-time requirements of large photovoltaic power stations. SUMMARY

[0004] The purpose of the present application is to overcome the defects of the prior art and provide a photovoltaic module hot spot detection method and system to solve the technical problem of low detection efficiency in the prior art.

[0005] To achieve the above purpose, the technical solution adopted by the present application includes:

[0006] A photovoltaic module hot spot detection method, comprising the following steps:

[0007] (1) Obtain a training sample set and a test sample set:

[0008] Get including Visible light and infrared images containing hot spots from photovoltaic modules were captured, and the images were analyzed. Mark the hot spots in the image, and then... The training sample set consists of visible light images and infrared images along with their labels. The remaining images will be used as the training sample set. The test sample set consists of visible light images and infrared light images, among which , ;

[0009] (2) Construct an improved YOLOv8 object detection network model:

[0010] The construction includes visible light feature extraction branches arranged in parallel. and infrared light feature extraction branch The YOLOv8 model is a real-time object detection network with a cross-modal feature fusion network cascaded with it as the backbone feature extraction network. The and All include cascaded standard convolutional modules, deformable convolutional modules, and spatial pyramid modules; the cross-modal feature fusion network includes cascaded attention weight learning modules, feature weighting modules, and residual fusion modules, and the input of the attention weight learning module is residually connected to the input of the feature weighting module and the residual module.

[0011] (3) Iteratively train the object detection network model:

[0012] The target detection network model was trained using the sample set. Perform iterative training to obtain a well-trained object detection network model. ;

[0013] (4) Obtain the hot spot detection results of photovoltaic modules:

[0014] The test sample set is used as the trained object detection network model. The input is propagated forward to obtain the hot spot detection results for each test sample.

[0015] Compared with the prior art, the present invention has the following advantages:

[0016] The application replaces the backbone feature extraction network in YOLOv8 with a parallel arrangement of visible light feature extraction branches and infrared light feature extraction branches and a cross-modal feature fusion network cascaded therewith, wherein, in the process of iterative training of the target detection network model and obtaining of the photovoltaic module hot spot detection result, the backbone feature extraction network respectively extracts features from each visible light image and infrared light image, the cross-modal feature fusion network fuses the extracted visible light feature map and infrared light feature map, and then obtains the detection result through the fused features, thereby avoiding the defects of complex detection process and large consumption of computing resources caused by separate detection of visible light images and infrared light images in the prior art, and effectively improving the detection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 The flowchart of the application is shown.

[0018] Figure 2 The overall structure diagram of the detection network model of the application is shown. DETAILED DESCRIPTION

[0019] The application will be described in further detail below with reference to the drawings and specific embodiments.

[0020] Reference Figure 1 A photovoltaic module hot spot detection method, comprising the following steps:

[0021] Step 1) obtaining a training sample set and a test sample set:

[0022] Obtaining a plurality of visible light images and infrared light images containing photovoltaic module hot spots, and labeling the hot spots in the visible light images and infrared light images. The visible light images and infrared light images and their labels are used to form a training sample set, and the remaining visible light images and infrared light images are used to form a test sample set. , ;

[0023] The embodiment of the application , Each image frame has a resolution of , and the images collected are visible light images and infrared light images of different photovoltaic module hot spots, and the labeled results in the images include the bounding box of the hot spot and the category.

[0024] Step 2) constructing an improved YOLOv8 target detection network model, the structure of which is shown in Figure 2

[0025] The construction includes visible light feature extraction branches and infrared light feature extraction branches arranged in parallel ​​​​and infrared light feature extraction branch , and the cross-modal feature fusion network cascaded with it is a real-time target detection network YOLOv8 model with a backbone feature extraction network The output end of the backbone feature extraction network is further cascaded with a neck network And a detection head network .

[0026] The present application replaces the existing backbone feature extraction network in YOLOv8 by arranging and and the cross-modal feature fusion network cascaded with it, to improve YOLOv8, wherein:

[0027] and all include cascaded standard convolution modules, deformable convolution modules and spatial pyramid modules, which contain stacked convolution layers, batch normalization layers and SiLU activation function layers, which are beneficial to the preliminary capture of hot spot details; the deformable convolution module includes stacked convolution layers, offset learning convolution layers and bilinear interpolation layers, which overcome the limitations of fixed shape convolution kernels when processing irregular shape hot spots by adjusting the sampling position and weight of the convolution kernel; the deformable convolution module includes stacked standard convolution layers, offset learning convolution layers and bilinear interpolation layers; the spatial pyramid module includes a first branch composed of stacked convolution layers and pooling layers arranged in parallel, and a second branch composed of convolution layers, and a feature splicing layer cascaded with the two branches;

[0028] The cross-modal feature fusion network includes cascaded attention weight learning modules, feature weighting modules and residual fusion modules, and the input end of the attention weight learning module is connected in residual connection with the input ends of the feature weighting module and the residual module; the attention weight learning module includes stacked adaptive average pooling layers, standard convolution layers, ReLU activation function layers, standard convolution layers and Sigmoid activation function layers; the adaptive average pooling in the attention weight learning module focuses on the hot spot position information, and the weights generated by convolution and activation function can accurately correspond to the hot spot feature area, the feature weighting module can solve the problem of one-sidedness of single modal information, and the residual fusion module can strengthen the fusion effect and avoid information loss and gradient disappearance, which is helpful to improve the detection accuracy.

[0029] The neck network includes cascaded first up-sampling modules, second up-sampling modules, splicing modules, first down-sampling modules, second down-sampling modules and splicing modules;

[0030] The detection head network includes classification convolution modules and prediction frame convolution modules arranged in parallel.

[0031] Step 3) Iterative training of the target detection network model:

[0032] (3a) Initialize the iteration number as , the maximum iteration number as , , the weight and bias parameters of the detection network model in the th iteration as , and let , ; in this embodiment ; ;

[0033] (3b) The visible light feature extraction branch and the infrared light feature extraction branch extract features from the visible light image and the infrared light image in each training sample respectively, and the implementation steps are as follows:

[0034] (3b1) The standard convolution modules in the visible light feature extraction and the infrared light feature extraction perform convolution operations on the visible light image and the infrared light image in each training sample respectively through the convolution layer using a fixed-size 3*3 convolution kernel to obtain an initial feature map. The initial feature map is standardized by the batch normalization layer in terms of mean and variance, and the standardized feature map is nonlinearly mapped by the SiLU activation function layer to obtain a visible light shallow layer feature map and an infrared light shallow layer feature map.

[0035] (3b2) The deformable convolution module performs convolution operations on the extracted visible light shallow layer feature map and infrared light shallow layer feature map respectively through the convolution layer using a 3*3 convolution kernel. The feature map after convolution operation is obtained by the offset learning convolution layer to obtain the feature map of the x / y direction offset with convolution operation position, and then the bilinear interpolation layer is used to perform interpolation based on the x / y direction offset to realize deep feature extraction, thereby obtaining the visible light feature map and the infrared light feature map of each training sample.

[0036] (3b3) The spatial pyramid module performs convolution operations on the visible light feature map and the infrared light feature map respectively through the convolution layer of the first branch using a 3*3 convolution kernel, and then performs maximum pooling through the pooling layer with a preset size of 2*2 pooling window to obtain the first branch feature map with enhanced features. The convolution layer of the second branch uses a 3*3 convolution kernel to perform convolution operation to obtain the second branch feature map with purified features to preserve the spatial details of the original features. The feature maps of the two branches are spliced through the feature splicing layer to obtain the visible light feature map of each training sample. infrared light feature map .

[0037] (3c) Cross-modal feature fusion network for the extracted visible light feature map infrared light feature map Feature fusion is performed by following these steps:

[0038] Attention weight learning module for visible light feature maps infrared light feature map The feature map size is compressed to 1*1 using an adaptive average pooling layer. The 1*1 feature map with C channels is then compressed using a standard convolutional layer to obtain a 1*1 feature map with C / r channels, where r is the channel reduction factor (r=4 in this embodiment). The 1*1 feature map with C / r channels is then introduced with a ReLU activation function layer to maintain the 1*1 size and C / r channel ratio. The feature map is then expanded using a standard convolutional layer to restore the 1*1 feature map with C channels. Finally, the feature values ​​are normalized to the [0,1] interval using a Sigmoid activation function layer to obtain the attention weights corresponding to the visible light feature map. Attention weights corresponding to infrared light feature maps Adaptive average pooling focuses on hot spot location information, and the weights generated by convolution and activation functions can accurately correspond to the hot spot feature regions.

[0039] Feature weighting module through , Post-learning attention weight , Visible light feature map infrared light feature map By performing channel-wise weighted feature aggregation and concatenation, the problem of one-sided information from a single modality can be solved.

[0040] The residual fusion module for visible light feature maps infrared light feature map Perform splicing and analyze the resulting original feature map. Residual fusion is performed with the weighted feature map output by the feature weighting module to obtain a feature map containing visible light and infrared light features. This process enhances the fusion effect while avoiding information loss and gradient vanishing. The fusion process leverages the complementarity of dual-modal data, which helps improve detection accuracy.

[0041] (3d) Neck network Feature map after feature fusion Multiple sampling is performed, and the steps are as follows: neck network The upsampling module and the stitching module in the feature map After multiple upsampling operations, the multiple feature maps obtained from the upsampling are stitched together to obtain the feature map. The downsampling module and the stitching module are... After performing multiple downsampling operations, the multiple feature maps obtained from the downsampling are stitched together to obtain the feature map. .

[0042] (3e) Detection head network For feature maps Perform bounding box and class detection to obtain the bounding box and class of each training sample. The implementation steps are as follows: Detection head network The parallel classification convolutional module and prediction box convolutional module in the middle process the feature map Perform classification convolution and prediction bounding box convolution, and combine the obtained hotspot category information and bounding box information into the bounding box and category of each training sample. .

[0043] (3f) Employs the cross-entropy loss function and uses the label of each training sample. and test results Computational detection network model loss value Then, the stochastic gradient descent method is used to... For weight parameters and bias parameters The detection network model for this iteration is updated. ,in:

[0044] Loss value The calculation formula is:

[0045] ;

[0046] ;

[0047] ;

[0048] in, , These represent the classification loss value and the localization loss value, respectively. , They represent , Weight, This indicates a summation operation.

[0049] For weight parameters and bias parameters The formula for updating is:

[0050] ;

[0051] ;

[0052] in, , They represent the first The weights and bias parameters of the detection network model in the next iteration. Indicates the learning rate. , They represent respectively to and Find the partial derivative.

[0053] (3g) judgment If true, then the trained detection network model is obtained. Otherwise, let , Then proceed with step (3b).

[0054] Step 4) Obtain the hot spot detection results of the photovoltaic module:

[0055] The test sample set is used as the trained detection network model. The input is propagated forward to obtain the hot spot detection results for each test sample.

[0056] A robot for detecting hot spots on photovoltaic modules includes a navigation unit, a moving unit, an image acquisition device, and a detection unit. The navigation unit navigates according to the guidance of a path guidance unit. The moving unit moves the image acquisition device according to the navigation unit's navigation. The image acquisition device acquires visible light images and thermal images of the photovoltaic module during the moving unit's movement or after the moving unit moves to a predetermined position. In this embodiment, the image acquisition device uses a multispectral detector. The detection unit performs hot spot detection on the visible light images and thermal images using the detection method of this invention.

Claims

1. A method for detecting hot spots in photovoltaic modules, characterized in that, Includes the following steps: (1) Obtain the training sample set and the test sample set: Get including Visible light and infrared images containing hot spots from photovoltaic modules were captured, and the images were analyzed. Mark the hot spots in the image, and then... The training sample set consists of visible light images and infrared images along with their labels. The remaining images will be used as the training sample set. The test sample set consists of visible light images and infrared light images, among which , ; (2) Construct an improved YOLOv8 object detection network model: The construction includes visible light feature extraction branches arranged in parallel. and infrared light feature extraction branch The YOLOv8 model is a real-time object detection network with a cross-modal feature fusion network cascaded with it as the backbone feature extraction network. The and All include cascaded standard convolutional modules, deformable convolutional modules, and spatial pyramid modules; the cross-modal feature fusion network includes cascaded attention weight learning modules, feature weighting modules, and residual fusion modules, and the input of the attention weight learning module is residually connected to the input of the feature weighting module and the residual module. (3) Iteratively train the object detection network model: The target detection network model was trained using the sample set. Perform iterative training to obtain a well-trained object detection network model. ; (4) Obtain the hot spot detection results of photovoltaic modules: The test sample set is used as the trained object detection network model. The input is propagated forward to obtain the hot spot detection results for each test sample.

2. The method according to claim 1, characterized in that, The target detection network model described in step (2) It also includes a neck network cascaded with the backbone feature extraction network. and detection head network ,in: The and It includes a standard convolutional module consisting of stacked convolutional layers, batch normalization layers, and SiLU activation function layers; a deformable convolutional module consisting of stacked convolutional layers, offset learning convolutional layers, and bilinear interpolation layers; and a spatial pyramid module consisting of a first branch composed of stacked convolutional layers and pooling layers arranged in parallel, a second branch composed of convolutional layers, and a feature concatenation layer cascaded with the two branches. The cross-modal feature fusion network includes an attention weight learning module comprising stacked adaptive average pooling layers, standard convolutional layers, ReLU activation function layers, and Sigmoid activation function layers. The neck network It includes multiple cascaded upsampling modules, splicing modules, and multiple downsampling and splicing modules; The detection head network It includes parallel-arranged classification convolutional modules and prediction box convolutional modules.

3. The method according to claim 2, characterized in that, The iterative training of the target detection network model described in step (3) is implemented as follows: (3a) Initialize the number of iterations to be The maximum number of iterations is , , No. Sub-iteration detection network model The weights and bias parameters are respectively , and order ; (3b) Visible light feature extraction branch and infrared light feature extraction branch Feature extraction is performed on the visible light and infrared light images of each training sample separately; a cross-modal feature fusion network is used to process each extracted visible light feature image. With each infrared light feature map Feature fusion; neck network Feature map after feature fusion Perform multiple samplings; detect head network Bounding boxes and categories are detected on the sampled feature maps to obtain the bounding box and category for each training sample. ; (3c) Employ the cross-entropy loss function and use the label of each training sample. and Computational detection network model loss value Then, the stochastic gradient descent method is used to... For weight parameters and bias parameters The network model for object detection in this iteration is updated to obtain the model for this iteration. ; (3d) Judgment If true, then a well-trained object detection network model is obtained. Otherwise, let , Then proceed with step (3b).

4. The method according to claim 3, characterized in that, The feature extraction described in step (3b) is implemented as follows: (3b1) Visible light feature extraction Infrared light feature extraction The standard convolutional module in the dataset performs shallow feature extraction on the visible light and infrared light images of each training sample, respectively. (3b2) The deformable convolution module performs deep feature extraction on the extracted visible light shallow feature map and infrared light shallow feature map respectively to obtain the visible light feature map of each training sample. infrared light feature map ; (3b3) Spatial pyramid module for visible light feature map and infrared light feature map Feature enhancement for the first branch and feature refinement for the second branch are performed separately. The feature maps from the enhanced and refined branches are then concatenated to obtain the visible light feature map for each training sample. infrared light feature map .

5. The method according to claim 3, characterized in that, The feature fusion described in step (3b) is implemented as follows: Attention weight learning module for visible light feature maps infrared light feature map Perform weight learning; feature The weighted module passes , Post-learning attention weight , Visible light feature map Infrared light feature map Perform channel-wise weighted feature aggregation and concatenation; The residual fusion module for visible light feature maps infrared light feature map Perform splicing and analyze the resulting original feature map. Residual fusion is performed with the weighted feature map output by the feature weighting module to obtain a feature map with visible light and infrared light characteristics. .

6. The method according to claim 3, characterized in that, The neck network described in step (3b) Feature map after feature fusion To perform multiple samplings, the steps are as follows: Neck network The upsampling module and the stitching module in the feature map After multiple upsampling operations, the multiple feature maps obtained from the upsampling are stitched together to obtain the feature map. The downsampling module and the stitching module are... After performing multiple downsampling operations, the multiple feature maps obtained from the downsampling are stitched together to obtain the feature map. .

7. The method according to claim 6, characterized in that, The detection of bounding boxes and categories described in step (3b) is implemented as follows: Detection head network The parallel classification convolutional module and prediction box convolutional module in the middle process the feature map Separate convolutions are performed for prediction boxes and classification boxes, and the resulting hotspot category and bounding box information are combined to form the bounding box and category of each training sample. .

8. The method according to claim 3, characterized in that, The loss value described in step (3c) The calculation formula is: ; ； ； ； in, , These represent the classification loss value and the localization loss value, respectively. , They represent , Weight, This indicates a summation operation.

9. The method according to claim 3, characterized in that, The weight parameters described in step (3c) and bias parameters The update is performed using the following formula: ； ； in, , They represent the first The weights and bias parameters of the detection network model in the next iteration. Indicates the learning rate. , They represent respectively to and Find the partial derivative.

10. A photovoltaic module hot spot detection robot implementing the method of claim 1, characterized in that, It includes a navigation unit, a moving unit, an image acquisition device, and a detection unit; the navigation unit navigates according to the guidance of the path guidance unit; the moving unit moves the image acquisition device according to the navigation unit's navigation; the image acquisition device acquires visible light images and thermal images of the photovoltaic module during the moving unit's movement or after the moving unit moves to a predetermined position; the detection unit performs hot spot detection using the visible light images and thermal images.

Images