A method for detecting hot spot of photovoltaic module based on unmanned aerial vehicle

Abstract

The application discloses a kind of photovoltaic module hot spot detection methods based on unmanned aerial vehicle, belong to photovoltaic power generation system inspection technical field, comprising: obtaining the infrared image data and label data of photovoltaic hot spot;Image data set is obtained by processing infrared image data and label data;Image data set is input to the semantic segmentation network model trained, and the segmented image is output;The hot spot area in segmented image is extracted, and hot spot proportion data is obtained;Based on hot spot proportion data and the electrical data of photovoltaic system, multi-modal fusion is carried out, the state of photovoltaic system is quantitatively evaluated, and hot spot position detection result and system state evaluation result are obtained.The application can improve the operation efficiency and maintenance effect of photovoltaic system.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of photovoltaic power generation system inspection, and particularly relates to a photovoltaic module hot spot detection method based on an unmanned aerial vehicle. BACKGROUND

[0002] With the global attention and demand for renewable energy, the application range of photovoltaic power generation components is continuously expanding. Photovoltaic components may have hot spot effect during operation, which leads to reduced performance or even damage. Traditional hot spot detection methods rely on ground equipment, and have long detection cycles and are limited by factors such as terrain. In order to improve detection efficiency and reasonably optimize photovoltaic component operation and maintenance, it is of great significance to develop a hot spot detection technology combining sensor data and multi-modal data fusion and analysis.

[0003] There are various quality problems in the use of photovoltaic panels, among which the hot spot problem is relatively common. The so-called hot spot is a local hot area that appears in the photovoltaic panel. It is usually caused by problems in the dry part of the component. Hot spot problems can affect the power generation efficiency of photovoltaic panels and even the safety of photovoltaic panels. Hot spot determination criteria include the following:

[0004] (1) Hot spot temperature is an important indicator for determining hot spot problems. According to relevant national standards, the maximum temperature on the surface of a photovoltaic panel should not exceed 85℃, and the temperature of some high-quality photovoltaic panels can be as low as 60℃ or lower. If the hot spot temperature exceeds this range, further inspection is needed to check whether the photovoltaic panel has a problem.

[0005] (2) Hot spot area is another important indicator for determining photovoltaic panel problems. The larger the hot spot area, the more serious the problem. According to national standards, the area of a hot spot on the surface of a photovoltaic panel should not exceed 5%. If the hot spot area exceeds this range, further inspection is needed to check whether the photovoltaic panel has a problem.

[0006] (3) Hot spot location is one of the important indicators for determining hot spot problems. Severe hot spot problems usually occur in the center of a photovoltaic panel, while minor problems may occur at the corners. If the hot spot location is in the central part of the photovoltaic panel, further inspection is needed to check whether the photovoltaic panel has a problem.

[0007] The hot spot detection method based on image analysis uses infrared images to reflect the temperature distribution of solar cells under different working conditions, and uses machine learning or image processing methods to achieve detection. SUMMARY

[0008] To solve the above technical problems, the present application provides a photovoltaic module hot spot detection method based on a UAV, which segments the infrared image of the photovoltaic module through an MTTR multi-modal semantic segmentation network, detects the hot spot position and area, and quantitatively evaluates the health degree of the photovoltaic system in combination with electrical data and meteorological data.

[0009] Obtain infrared image data and label data of photovoltaic hot spots;

[0010] Process the infrared image data and label data to obtain an image data set;

[0011] Input the image data set into the trained semantic segmentation network model to output a segmented image;

[0012] Feature extraction is performed on the hot spot area in the segmented image to obtain hot spot proportion data;

[0013] Based on the multi-modal fusion of the hot spot proportion data and the electrical data of the photovoltaic system, the state of the photovoltaic system is quantitatively evaluated to obtain hot spot position detection results and system state evaluation results.

[0014] Preferably, the process of obtaining infrared image data and label data of photovoltaic hot spots comprises:

[0015] Collecting original infrared images of photovoltaic modules at multiple scales, performing semantic annotation on the original infrared images, and making label image data;

[0016] Data augmentation is performed on the infrared image data set of the photovoltaic module hot spot, and the Real-ESRGAN super-resolution algorithm is used to perform data denoising operation on the infrared image data and label data to obtain the corresponding data set.

[0017] Preferably, the Real-ESRGAN super-resolution algorithm uses feature matching loss, and the discriminator network uses PatchGAN.

[0018] Preferably, the process of processing the infrared image data and label data to obtain the image data set comprises:

[0019] The infrared image data and label data are processed to make an image data set for the semantic segmentation network, and the image data is batch normalized;

[0020] The image data is processed into a tensor slice to obtain a trainable data set, and the data in the data set is normalized;

[0021] The image data set is divided into a training set and a test set according to a ratio of 7:3.

[0022] Preferably, the process of inputting the image data set into the trained semantic segmentation network model and outputting the segmented image further comprises constructing an improved multi-modal semantic segmentation network model and training the improved multi-modal semantic segmentation network model.

[0023] The improved multi-modal semantic segmentation network comprises an attention mechanism module, a backbone feature extraction network and a multi-modal Transformer feature extraction network.

[0024] Preferably, the process of constructing the improved multi-modal semantic segmentation network model comprises:

[0025] The multi-modal semantic segmentation network is improved by adding a channel attention mechanism module between the migrated VGG network and the input image data.

[0026] The convolutional layer in the multi-modal Transformer feature extraction network uses a dilated convolution.

[0027] The MobilenetV3 feature extraction network is replaced with the original MobilenetV2 feature extraction network by stepwise upsampling.

[0028] Preferably, the process of inputting the image data set into the trained semantic segmentation network model and outputting the segmented image comprises:

[0029] The training sample set of the image data set and the text data set is input into the migrated VGG network.

[0030] The initial image and text features are extracted through the attention mechanism module.

[0031] In the backbone feature extraction network, a channel attention mechanism network is added to allocate information processing resources to the features of the input image through the convolutional attention mechanism network.

[0032] Based on the multi-modal Transformer feature extraction network, a deep space-time encoder is used to extract features from each frame of the sequence V, and a Transformer-based text encoder is used to extract language features from the text query T.

[0033] The space-time and language features are linearly projected to a shared dimension D.

[0034] The features of each frame of interest are flattened and connected with the text embedding respectively to obtain a set of multi-modal sequences, and the sequences are input into the Transformer in parallel.

[0035] In the encoding layer of the Transformer, the text embedding and the visual features of each frame exchange information.

[0036] Each input frame is trained using N object queries by the decoding layer of the Transformer, and the multimodal sequence is queried to obtain information related to the entity and stored in the object query;

[0037] For each output instance sequence, a corresponding mask sequence is generated using a spatial decoder and a dynamically generated conditional convolution kernel;

[0038] Using a text reference score function based on text association, determine the object query sequence most relevant to the described object, and return the segmentation sequence as the prediction result of the model.

[0039] Compared with the prior art, the present application has the following advantages and technical effects:

[0040] The present application is based on original infrared images and labeled data, and the image data is processed by Real-ESRGAN super-resolution algorithm, and the influence of the environment in the implementation is considered to indirectly process the image data. By using the channel attention mechanism in the backbone feature extraction network and improving the backbone feature extraction network using MobilenetV3, the image feature extraction speed is increased, and the accuracy of image segmentation is improved.

[0041] The present application obtains the prediction segmentation graph by semantic segmentation, extracts the area, and quantitatively evaluates the image data and electrical data after fusion, accurately detects the position of the hot spot and can evaluate the state of the photovoltaic system. BRIEF DESCRIPTION OF DRAWINGS

[0042] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated herein for explanation. The schematic embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 The method flowchart of the embodiment of the present application is shown in the figure;

[0044] Figure 2 The multi-modal semantic segmentation network structure of the embodiment of the present application is shown in the figure;

[0045] Figure 3 The mobilenetV3 backbone feature extraction network of the embodiment of the present application is shown in the figure;

[0046] Figure 4 The attention mechanism of the embodiment of the present application is shown in the figure;

[0047] Figure 5 The corresponding relationship between the numerical results and the quantitative rating of the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0048] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0049] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0050] Embodiment one

[0051] As Figure 1 shown, the present embodiment provides a photovoltaic module hot spot detection method based on unmanned aerial vehicle, comprising:

[0052] acquiring infrared image data and label data of photovoltaic hot spot;

[0053] processing the infrared image data and the label data to obtain an image data set;

[0054] inputting the image data set into a trained semantic segmentation network model to output a segmented image;

[0055] extracting features of the hot spot area in the segmented image to obtain hot spot proportion data;

[0056] based on the hot spot proportion data and the electrical data of the photovoltaic system, performing multi-modal fusion to quantitatively evaluate the state of the photovoltaic system, and obtaining hot spot position detection results and system state evaluation results.

[0057] Further, the process of acquiring infrared image data and label data of photovoltaic hot spot includes:

[0058] collecting original infrared images of photovoltaic modules under multiple scales, performing semantic annotation on the original infrared images, and making label image data;

[0059] data augmentation is performed on the infrared image data set of the photovoltaic module hot spot, and Real-ESRGAN super resolution algorithm is used to perform data denoising operation on the infrared image data and the label data to obtain the corresponding data set.

[0060] Further, the Real-ESRGAN super resolution algorithm adopts feature matching loss, and the discriminator network adopts PatchGAN.

[0061] Specifically, meteorological data, voltage and current data, and hot spot images taken by the unmanned aerial vehicle are acquired; the degree and type of hot spots are preliminarily evaluated by analyzing hot spot image data such as temperature distribution and area size; the formation and development of hot spots are evaluated in combination with meteorological data, voltage and current data, especially ambient temperature and irradiance; the voltage and current of the hot spot area of the photovoltaic panel are analyzed to understand the influence of hot spots on the photovoltaic power generation efficiency; based on comprehensive consideration of the above-mentioned various data, an evaluation model of the severity of hot spots is established, and the severity grades of different hot spots are determined. Based on the multi-modal fusion of hot spot proportion data and electrical data of the photovoltaic system, the state of the photovoltaic system is quantitatively evaluated to obtain the hot spot position detection result and the state evaluation result.

[0062] It can be understood that the embodiment collects infrared image data of the photovoltaic module under multiple scales, and then makes a label image.

[0063] Specifically, the infrared imager is used to collect image data of normal photovoltaic modules and modules with hot spots, the Labelme software is used for semantic annotation of the original image, the label image data is made, and the final image data is obtained.

[0064] Further, the embodiment performs data augmentation on the infrared image data set of the photovoltaic module hot spot through the image data enhancement method. According to the characteristics of the infrared image of the photovoltaic module, the resolution of the infrared image is relatively low, and the image collected by the actual photovoltaic system may be blurred due to environmental factors. The Real-ESRGAN super-resolution algorithm is used to clarify the image.

[0065] The Real-ESRGAN super-resolution algorithm of the embodiment performs super-resolution reconstruction on the image. The discriminator network of Real-ESRGAN adopts PatchGAN, which can quickly and effectively distinguish the authenticity and falsity of the boundary of the hot spot, and can distinguish the local image area. In addition, Real-ESRGAN also adopts feature matching loss, which further improves the quality and clarity of the generated photovoltaic hot spot image.

[0066] The discriminant model of Real-ESRGAN is optimized in two steps:

[0067] (1) Optimizing the ability to distinguish the real, that is, constructing a y T all 1, then calculating the BECLoss of D(y i ) and y T , and the formula expression is:

[0068]

[0069] (2) Optimizing the ability to distinguish the false, that is, constructing a y fThen calculate D(y) i ) and y f The formula for BECLoss is:

[0070]

[0071] This embodiment utilizes the Real-ESRGAN super-resolution algorithm to enhance the clarity of thermal infrared images. By increasing the representation of detail information in the image and improving the image resolution, the corresponding data becomes clearer and easier to identify.

[0072] Further processing of infrared image data and tag data to obtain an image dataset includes:

[0073] Infrared image data and label data are processed to create an image dataset for semantic segmentation networks, and the image data is batch normalized.

[0074] Image data is processed into tensor slices to obtain a trainable dataset, and the data in the dataset is normalized.

[0075] The image dataset was divided into a training set and a test set in a 7:3 ratio.

[0076] Specifically, this embodiment processes the extracted infrared image data and label data to create an image dataset for the semantic segmentation network. The dataset is divided into training and testing sets, and the image data undergoes batch normalization, which is beneficial for subsequent image processing by the convolutional neural network. Secondly, the input size of the images is standardized for use as input to the subsequent convolutional network.

[0077] Furthermore, the image data is processed into tensor slices using PyTorch to obtain a trainable dataset. The data in this dataset is then normalized, transforming the data in the [0, 255] interval to the [-1, 1] interval. Simultaneously, the dataset is normalized to obtain data of standard input specifications. The image dataset is then divided into training and testing sets in a 7:3 ratio.

[0078] Furthermore, before inputting the image dataset into the trained semantic segmentation network model and outputting the segmented image, the process also includes constructing an improved multimodal semantic segmentation network model and training the improved multimodal semantic segmentation network model.

[0079] The improved multimodal semantic segmentation network (MTTR) includes an attention mechanism module, a backbone feature extraction network, and a multimodal Transformer feature extraction network.

[0080] Furthermore, such as Figure 2As shown, the process of constructing an improved multimodal semantic segmentation network model includes:

[0081] An improvement to the multimodal semantic segmentation network is achieved by adding a channel attention mechanism module between the VGG network and the input image data to enhance feature extraction, such as... Figure 3 and Figure 4 As shown;

[0082] Dilated convolutions are used in the convolutional layers of the multimodal Transformer feature extraction network to enhance the connections between data contexts.

[0083] In one embodiment, the improved MTTR network parameters corresponding to the photovoltaic panel detection model are as follows: the backbone feature extraction network is a combination of the channel attention mechanism module and the visual and language feature extraction networks, and the multimodal Transformer feature extraction network uses one convolutional layer and one upsampling layer per layer, and is trained in 60 batches.

[0084] The original MobilenetV2 feature extraction network is replaced with the MobilenetV3 feature extraction network through progressive upsampling. The multimodal Transformer feature extraction network MobilenetV2 suffers from slow feature extraction speed. Considering this characteristic, this embodiment uses the MobilenetV3 feature extraction network, which will improve performance. A convolutional attention mechanism is added to the MobilenetV3 feature extraction network within the multimodal Transformer feature extraction network. This helps improve the model's detection performance. Figure 3 As shown.

[0085] Furthermore, during image processing, important information is gathered with high weights and irrelevant information is ignored with low weights. The weights can be continuously adjusted to select important information based on the characteristics of the dataset under different circumstances.

[0086] Furthermore, the channel weights in the SE block can be calculated as follows:

[0087]

[0088] in Hi = 1, j = 1χ ij It is channel-level global average pooling (GAP), where σ is a sigmoid function. Let... Transform into

[0089]

[0090] Here, ReLU represents the linear unit of correction. To avoid high model complexity, the sizes of W1 and W2 are set to... and f {W1，W2} This involves all parameters related to the channel attention block. By projecting channel features into a low-dimensional space, the correspondence between channels and their weights becomes indirect, which reduces the complexity of the model.

[0091] Furthermore, the process of inputting the image dataset into the trained semantic segmentation network model and outputting segmented images includes:

[0092] The training sample sets of the image dataset and the text dataset are input into the transfer VGG network;

[0093] Initial image and text features are extracted using an attention mechanism module;

[0094] In the backbone feature extraction network, a channel attention mechanism network is added to achieve efficient allocation of information processing resources for the features of the input image through the convolutional attention mechanism network.

[0095] To extract and utilize the information, a multimodal Transformer feature extraction network is used to extract features from each frame of sequence V using a deep spatiotemporal encoder, while a Transformer-based text encoder is used to extract language features from text query T.

[0096] Linearly project spatiotemporal and linguistic features onto a shared dimension D;

[0097] In the next step, the features of each frame of interest are flattened and concatenated with the text embedding to obtain a set of multimodal sequences, which are then input into the Transformer in parallel.

[0098] In the encoding layer of the Transformer, text embeddings and visual features of each frame exchange information;

[0099] The Transformer's decoding layer trains each input frame using N object queries, queries multimodal sequences to obtain entity-related information, and stores it in the object queries;

[0100] For each output instance sequence, a corresponding mask sequence is generated using a spatial decoder and dynamically generated conditional convolutional kernels;

[0101] Using a text reference score function based on text association, the most relevant object query sequence to the described object is determined, and the segmented sequence is returned as the model's prediction result.

[0102] Furthermore, feature extraction is performed on the hot spot area of ​​the segmented image to obtain hot spot proportion data. Based on the hot spot proportion data and the electrical data of the photovoltaic system, a multi-modal fusion photovoltaic system state quantification assessment is performed to obtain hot spot location detection results and state assessment results.

[0103] As an example, this embodiment extracts the patch area from the segmentation map obtained by semantic segmentation using OpenCV software, fuses image data and electrical data, establishes a multimodal fusion photovoltaic system status assessment mechanism, and performs quantitative rating on the image and electrical data. The correspondence between the numerical results and the quantitative rating is as follows: Figure 5 As shown.

[0104] Specifically, this embodiment obtains a segmentation map of photovoltaic hot spots through semantic segmentation, extracts the hot spot area from the network using OpenCV, and obtains the proportion of the hot spot area. Corresponding power data and temperature data of the hot spots in the image are extracted experimentally, and the health of the photovoltaic system is evaluated using the following formula:

[0105]

[0106] In the formula, Score1 represents the quantitative score of the hotspot area detected by image semantic segmentation; Score2 represents the quantitative score of the system state through the power value in the electrical data; N is the total number of state points selected in the continuous time period; S T S represents the area of ​​the detected photovoltaic module; i P represents the hotspot area of ​​the photovoltaic module at state i; i P represents the power value of the photovoltaic module at state i; T This indicates the power value under normal conditions during the selected time period.

[0107] Furthermore, by fusing the area score obtained from image semantic segmentation with the score from electrical data quantization, a final quantization score rating is derived.

[0108] In summary, this embodiment extracts infrared image data of the photovoltaic module surface using an infrared thermal imaging thermometer based on the temperature difference between the hot spots of the cell units and the normal cell temperature. Next, based on the features of the extracted image data, the image is segmented using an improved MTTR multimodal image semantic segmentation network. The feature extraction network is improved by incorporating an attention mechanism module to enhance feature extraction. The area of ​​the semantically segmented image is extracted, and electrical data is fused to quantitatively assess the health status of the photovoltaic system.

[0109] This embodiment can fuse image data and electrical data for quantitative evaluation, accurately detect the location of hot spots, and assess the status of the photovoltaic system.

[0110] Example 2

[0111] This embodiment also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in Embodiment 1.

[0112] Example 3

[0113] This embodiment also discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0114] Example 4

[0115] This embodiment also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0116] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting hot spots in photovoltaic modules based on unmanned aerial vehicles (UAVs), characterized in that, include: Acquire infrared image data and tag data of photovoltaic hot spots; The infrared image data and tag data are processed to obtain an image dataset; The image dataset is input into a trained semantic segmentation network model, and the segmented image is output. Feature extraction is performed on the hot spot area in the segmented image to obtain hot spot percentage data; Multimodal fusion is performed based on the hot spot proportion data and the electrical data of the photovoltaic system to quantitatively evaluate the state of the photovoltaic system and obtain the hot spot location detection results and system state evaluation results. The process of inputting the image dataset into a trained semantic segmentation network model and outputting segmented images includes: The training sample sets of the image dataset and the text dataset are input into the transfer VGG network; Initial image and text features are extracted using an attention mechanism module; In the backbone feature extraction network, a channel attention mechanism network is added to allocate information processing resources for the features of the input image through the convolutional attention mechanism network; Based on the multimodal Transformer feature extraction network, a deep spatiotemporal encoder is used to extract features from each frame of sequence V, while a Transformer-based text encoder is used to extract language features from text query T. Linearly project spatiotemporal and linguistic features onto a shared dimension D; The features of each frame of interest are flattened and concatenated with the text embedding to obtain a set of multimodal sequences, which are then input into the Transformer in parallel. In the encoding layer of the Transformer, text embeddings and visual features of each frame exchange information; The Transformer's decoding layer trains each input frame using N object queries, queries multimodal sequences to obtain entity-related information, and stores it in the object queries; For each output instance sequence, a corresponding mask sequence is generated using a spatial decoder and dynamically generated conditional convolutional kernels; Using a text reference score function based on text association, the most relevant object query sequence to the described object is determined, and the segmented sequence is returned as the model's prediction result.

2. The method according to claim 1, characterized in that, The process of acquiring infrared image data and tag data of photovoltaic hot spots includes: Collect raw infrared images of photovoltaic modules at multiple scales, perform semantic annotation on the raw infrared images, and create labeled image data; The infrared image dataset of hot spots of photovoltaic modules is augmented, and the infrared image data and label data are denoised using the Real-ESRGAN super-resolution algorithm to obtain the corresponding dataset.

3. The method according to claim 2, characterized in that, The Real-ESRGAN super-resolution algorithm uses feature matching loss, and the discriminator network uses PatchGAN.

4. The method according to claim 1, characterized in that, The process of processing the infrared image data and tag data to obtain the image dataset includes: The infrared image data and label data are processed to create an image dataset for the semantic segmentation network, and the image data is batch normalized. The image data is processed into tensor slices to obtain a trainable dataset, and the data in the dataset is normalized. The image dataset was divided into a training set and a test set in a 7:3 ratio.

5. The method according to claim 1, characterized in that, Before the image dataset is input into the trained semantic segmentation network model and the segmented image is output, the process also includes constructing an improved multimodal semantic segmentation network model and training the improved multimodal semantic segmentation network model. The improved multimodal semantic segmentation network includes an attention mechanism module, a backbone feature extraction network, and a multimodal Transformer feature extraction network.

6. The method according to claim 5, characterized in that, The process of building an improved multimodal semantic segmentation network model includes: An improvement to the multimodal semantic segmentation network is achieved by adding a channel attention mechanism module between the VGG network and the input image data. Dilated convolutions are used in the convolutional layers of the multimodal Transformer feature extraction network; The original MobilenetV2 feature extraction network was replaced by the MobilenetV3 feature extraction network through progressive upsampling.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-6.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1-6.

Images