A method for detecting hot spots in infrared images of photovoltaic panels based on an improved BETR model

Abstract

The present invention provides a method for detecting hot spots in the infrared image of a photovoltaic panel based on an improved BETR model, which includes the following steps: Step 1, collect the image of the photovoltaic panel and the infrared hot spot image of the photovoltaic panel; Step 2, process the above images, and respectively construct the first and second photovoltaic panel data sets and the photovoltaic panel infrared hot spot data set by using the processed images; Step 3, construct an improved BETR model; Step 4, pre-train the improved BETR model by using the first photovoltaic panel data set, and use the idea of transfer learning to formally train the pre-trained BETR model by using the photovoltaic panel infrared hot spot data set and the second photovoltaic panel data set to obtain an optimized BETR model; Step 5, use the optimized BETR model to identify a new image, classify the sizes of the identified photovoltaic panel infrared hot spots, count the number and position information of each type of hot spot, count the number of photovoltaic panels, and integrate and output the results. The above method has high recognition accuracy for small target objects and can count the number of photovoltaic panels.

Description

Technical Field

[0001] The present invention relates to the technical field of photovoltaic panel infrared image hot spot detection, and in particular to a method for detecting photovoltaic panel infrared image hot spots based on an improved BETR model. Background Art

[0002] Photovoltaic power generation is a new energy power supply method that meets policy and market demands, and has developed rapidly in recent years. However, there are some behaviors of users privately increasing the photovoltaic capacity. From the perspective of individual users, this behavior will cause the voltage at home to exceed the maximum voltage for residential electricity use, resulting in problems such as electrical appliances being burned out and even fires. If users privately increase the capacity, it will also pose a safety hazard to the grid transformers in the area, affecting the grid quality of the entire region. If there is foreign object occlusion on the photovoltaic panel, it will cause the short-circuit current to be less than the current of the component operation, making some photovoltaic panels in a reverse bias state, consuming energy, and thus showing a local high-temperature state, generating hot spots, which are extremely likely to cause safety accidents. Therefore, regularly checking the quantity and status of photovoltaic panels plays an important role in the development of the photovoltaic panel industry.

[0003] Regarding the state detection of photovoltaic panels, there are already some detection methods. For example, the invention patent application with the publication number CN114973032A proposes a method and device for detecting photovoltaic panel hot spots based on a deep convolutional neural network. It uses an improved object detection algorithm Yolov4 to construct a photovoltaic panel recognition model, and identifies and intercepts the photovoltaic panels in the photovoltaic panel infrared image through the photovoltaic panel recognition model; uses an optimized semantic segmentation algorithm DeepLabV3+ to construct a hot spot segmentation model, and detects and segments the hot spots on the intercepted photovoltaic panels through the hot spot segmentation model; however, this method has a poor detection effect for small-sized hot spots and cannot count the number of photovoltaic panels. The invention patent application with the publication number CN114299033A proposes a method and system for detecting photovoltaic panel infrared image hot spots based on YOLOv5. This method constructs an improved YOLOv5 model, uses the trained improved YOLOv5 model to detect the photovoltaic module image to be detected, and after processing the output detection frame, outputs the photovoltaic module category information; however, this method does not improve the recognition of small-sized hot spots, and there is still a problem of poor detection effect for small-sized hot spots. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present application proposes a method for detecting photovoltaic panel infrared image hot spots based on an improved BETR model. The improved BETR model has a faster training speed and recognition speed, high recognition accuracy for small target objects (such as small-sized hot spots), can classify the size of photovoltaic panel infrared hot spots and count the number, record the position information of photovoltaic panel infrared hot spots, and can count the number of photovoltaic panels.

[0005] To achieve the above object, the present application proposes a method for detecting hot spots in infrared images of photovoltaic panels based on an improved BETR model, including the following steps:

[0006] Step 1: Collect photovoltaic panel images and infrared hot spot images of photovoltaic panels through a drone;

[0007] Step 2: Perform data augmentation processing on both the photovoltaic panel images and the infrared hot spot images of photovoltaic panels. Use the processed photovoltaic panel images to construct a first photovoltaic panel dataset for pre-training, and use the processed infrared hot spot images of photovoltaic panels and photovoltaic panel images to construct a photovoltaic panel infrared hot spot dataset and a second photovoltaic panel dataset for formal training respectively;

[0008] Step 3: Construct an improved BETR model: Use a feature pyramid to replace the backbone network for feature extraction in the original BETR model; Improve the position encoder to enhance the ability to record the positions of two-dimensional image features; Improve the FFN feed-forward network to improve the convergence speed of the improved BETR model during training;

[0009] Step 4: Use the first photovoltaic panel dataset obtained in Step 2 to pre-train the improved BETR model. Then, using the idea of transfer learning, use the photovoltaic panel infrared hot spot dataset and the second photovoltaic panel dataset obtained in Step 2 to formally train the pre-trained BETR model to fine-tune the parameters and obtain an optimized BETR model;

[0010] Step 5: Use the BETR model obtained in Step 4 to identify newly collected infrared hot spot images and photovoltaic panel images of photovoltaic panels, classify the sizes of the identified infrared hot spots of photovoltaic panels, count the number and position information of hot spots in each category, and count the number of photovoltaic panels, and integrate and output the results.

[0011] In some embodiments, in the said Step 2, the data augmentation processing process is as follows: Use an image convolution operator Sharpen the image, perform a convolution operation on the image using the image convolution operator, and the calculation process is described as follows:

[0012]

[0013] Where a' is an integer from 0 to 2, b' is an integer from 0 to 2, Kernel(a, b) is the input image convolution operator, and csr(a, b) is the input image, referring to the photovoltaic panel image or the photovoltaic panel infrared hot spot image. The image obtained after the convolution operation is subjected to flipping or size scale transformation processing. Among them, the size scale transformation processing of the image obtained after the convolution operation is to transform the pixel value size of the image to between 240 and 678. Then, a multi-scale sliding window sampling operation is performed on the image obtained after the flipping or size scale transformation processing.

[0014] In some embodiments, in step 3, the ResNet-50 network used for feature extraction in the original BETR model is replaced by a feature pyramid. The feature pyramid is responsible for extracting the features of the image. First, the image features are extracted through 5 convolutional layers Conv1 to Conv5. Then, through upsampling and concat splicing, {Cobv2Conv3, Conv4} are respectively fused with the features obtained by the FFB block. Then, {FFB, FFB} and {FFB2, FFB2} block features are fused. Then, before the feature fusion, an adaptive feature pooling operation is performed on the features extracted by the pyramid layer to unify the features of different dimensions to the same dimension. Then, feature fusion is performed on each of the above features of the same dimension, and finally a feature of D1*H*W1 is obtained, where D1 represents the dimension of the feature, and H and W1 respectively represent the height and width of the feature.

[0015] In some embodiments, in step 3, the position encoder is improved as follows. Each dimension in the feature of d1*HW1 is marked differently. The dimension of the feature is marked with segment embedding ∈ {1, 2, 3...N1}, where the feature of d1*HW1 is obtained by reducing the high-dimensional feature to a smaller dimension d1 using a 1*1 convolutional layer. The high-dimensional feature refers to the feature of D1*H*W1; position embedding is used to mark the position encoding information of each image feature in the same dimension. The position embedding formula is as follows:

[0016]

[0017] Where, P represents the position encoding information, p represents the position of the current feature in the input, 2i is used to represent the even position, 2i + 1 represents the odd position, and d represents the length of the feature vector of the position encoding information; then, the segment embedding and position embedding of each feature are added to obtain the position marking information of the feature. Finally, the position marking information of the feature and the feature value of the feature are added to obtain the input feature of the encoder.

[0018] In some embodiments, in step 3, the improvement to the FFN feed-forward network is as follows: ReLU6 is used as the activation function, the number of hidden dimensions is designed to be d4 dimensions, and finally the features are restored through a linear projection. The softmax function is used in the mapping layer to predict the category of the objective function, and finally the classification result is output; the linear projection formula is as follows: y = xW2 + B, where W2 represents the parameter matrix, B represents the bias matrix, x represents the input, and y represents the output.

[0019] In some embodiments, in step 4, the improved BETR model is pre-trained using the first photovoltaic panel dataset, the initial learning rate is set to 1e-2, and the learning rate is set to decrease as the number of iterations increases; when the improved BETR model converges, the pre-trained BETR model is formally trained using the photovoltaic panel infrared hot spot dataset and the second photovoltaic panel dataset. The training strategy is to first freeze the backbone network, and then unfreeze the backbone network after the improved BETR model converges, and train the BETR model again to fine-tune the parameters of the backbone network of the BETR model to obtain an optimized BETR model;

[0020] Among them, in step 4, during the training process, the overall loss function L used by the BETR model is as follows:

[0021]

[0022] where β, α, K represent three hyperparameters, L iou represents the calculation function of iou, ||.|| represents the norm calculation, a i represents the intersection of the i-th predicted class box and the actual box, a γ(i) represents the union of the i-th predicted class box and the actual box, represents the auxiliary loss function;

[0023] L iou The calculation function expression of is:

[0024]

[0025] where represents the area calculation, B(.) represents the calculation function of max / min;

[0026] Using the auxiliary loss function helps the decoder of the transiormer to correct the output result, and its expression is as follows: where n represents the binary target mask, represents the original mask logical prediction of the BETR model, and relu represents the activation function.

[0027] In some embodiments, in the said step 5, the process of classifying and counting the size of the infrared hot spots on the photovoltaic panel is as follows: Use the FFN feed-forward network to predict the center coordinates of E bounding boxes of a fixed size and E input image frames, which are represented by the set D, and D can be represented as follows:

[0028] D = {b i = (h i , w i ); O i = (O x , O y ), i = 1, 2, 3... N3};

[0029] where b i represents the i-th bounding box, O i represents the center coordinates of the i-th input image frame, (h i , w i ) represents the height and width of the i-th bounding box, (O x , O y ) represents the coordinates of the input image frame on the x-axis and y-axis, N3 is much larger than the number of target objects to be predicted. In addition, a label θ is used to indicate that no object is detected in the detection area; the detected hot spots are divided into two size categories, a threshold for dividing the size of the hot spots is set, and a counter is set inside the prediction loop to count the number of hot spots on the photovoltaic panel in each category.

[0030] The beneficial effect of this solution of the present application lies in the above-mentioned infrared image hot spot detection method for photovoltaic panels based on the improved BETR model, which performs data augmentation on the obtained image data, improves the quality of the data, and the constructed data set is beneficial to improving the robustness and generalization ability of the improved BETR model; this method proposes to use a feature pyramid to replace ResNet-50 in the original BETR model to extract the features of the image, and performs multiple information fusions on the deep and shallow features, and the improved BETR model improves the ability to extract the features of small target objects; this method proposes to re-design the original position encoder of the BETR model, and annotates the image features in each dimension, improving the ability of the BETR model to record the position of two-dimensional image features; this method also re-designs the FFN feed-forward network, increases the number of hidden layers, improves the recognition accuracy, and uses the ReLU6 activation function to ensure the convergence speed of the improved BETR model during training. The training speed and recognition speed of the improved BETR model are faster, and the recognition accuracy for small target objects (such as hot spots with a small area) is high. It can classify and count the size of the infrared hot spots on the photovoltaic panel, record the position information of the infrared hot spots on the photovoltaic panel, and be able to count the number of photovoltaic panels. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Shows the flowchart of the hot spot detection method for the infrared image of the photovoltaic panel based on the improved BETR model in the embodiment.

[0032] Figure 2 Shows the overall structure diagram of the improved BETR model in the embodiment.

[0033] Figure 3 Shows the structure diagram of the feature pyramid used in the embodiment.

[0034] Figure 4 Shows the partial internal block structure diagram of the feature pyramid in the embodiment.

[0035] Figure 5 Shows the design schematic diagram of the position encoder in the embodiment. Detailed implementation manners

[0036] The following further describes the detailed implementation manners of the present application with reference to the accompanying drawings.

[0037] As Figure 1 shown, the hot spot detection method for the infrared image of the photovoltaic panel based on the improved BETR model involved in the present application includes the following steps:

[0038] Step 1, collect the photovoltaic panel image and the infrared hot spot image of the photovoltaic panel by using a drone.

[0039] Step 2, perform data enhancement processing on both the photovoltaic panel image and the infrared hot spot image of the photovoltaic panel, construct the first photovoltaic panel dataset for pre-training by using the processed photovoltaic panel image, and construct the photovoltaic panel infrared hot spot dataset and the second photovoltaic panel dataset for formal training by using the processed infrared hot spot image of the photovoltaic panel and the photovoltaic panel image respectively.

[0040] During the aerial photography of the drone, it is inevitable that some images are not taken clearly. Therefore, in order to make the target object data clearer, in this step, data enhancement processing is performed on both the photovoltaic panel image and the infrared hot spot image of the photovoltaic panel. The specific data enhancement processing process is as follows: use the image convolution operator to perform sharpening processing on the image, perform convolution operation on the image convolution operator and the image, and the calculation process is described as follows:

[0041]

[0042] Where a' is an integer from 0 to 2, b is an integer from 0 to 2, Kernel(a, b) is the input image convolution operator, and cxsr(a, b) is the input image, which refers to the photovoltaic panel image or the photovoltaic panel infrared hot spot image. To increase the number of training samples and improve the robustness and generalization ability of the BETR model, the image obtained after the convolution operation is flipped or subjected to size-scale transformation processing. Among them, the size-scale transformation processing of the image obtained after the convolution operation is to transform the pixel value size of the image to between 240 and 678. After that, a multi-scale sliding window sampling operation is performed on the image obtained after the flipping or size-scale transformation processing.

[0043] The process of constructing the corresponding dataset using the processed photovoltaic panel image and the photovoltaic panel infrared hot spot image is as follows: Use labeIimg to annotate the processed photovoltaic panel image and the photovoltaic panel infrared hot spot image, generate the corresponding xml file, and convert it into the format of the COCO dataset.

[0044] Step 3: Construct an improved BETR model: Use a feature pyramid to replace the backbone network for feature extraction in the original BETR model; Improve the position encoder to enhance the ability to record the positions of two-dimensional image features; Improve the FFN feed-forward network to improve the convergence speed of the improved BETR model during training. The improved BETR model can improve the accuracy of identifying small target objects (such as hot spots with a small area), as Figure 2 shown.

[0045] In this step, use a feature pyramid to replace the ResNet-50 network for feature extraction in the original BETR model. The feature pyramid is responsible for extracting the features of the image, as Figures 3 - 4 shown. First, extract the image features through 5 convolutional layers Conv1 to Conv5. Then, through upsampling and concat splicing, fuse the features obtained from {Conv2, Conv3, Conv4} with the features obtained from the FFB block respectively. After that, fuse the features obtained from {FFB, FFB} with the features obtained from {FFB2, FFB2} blocks to reduce the loss of features. Finally, perform an adaptive feature pooling operation on the features extracted by the pyramid layer before feature fusion to unify features of different dimensions into the same dimension for convenient subsequent processing. After that, perform feature fusion on each of the above-mentioned features of the same dimension, and finally obtain a feature of D1*H*W1, where D1 represents the dimension of the feature, and H and W1 represent the height and width of the feature respectively.

[0046] Before the encoder of the input transformer, a 1×1 convolutional layer is used to reduce the high-dimensional features to a smaller dimension d1, obtaining features of d1×HW1, and then the features are flattened into 1D features as the input to the encoder. After flattening the features into one dimension, the relative positions of the features are lost. To improve the ability of the BETR model to record the positions of two-dimensional image features, the position encoder is thus improved.

[0047] Specifically, the position encoder is improved as follows, as Figure 5 shown: Each dimension in the features of d1×HW1 is marked differently. The dimensions of the features are marked with segment embedding ∈{1, 2, 3...N1}, and the same dimension is marked with the same number; the position encoding information of each image feature on the same dimension is marked using position embedding. The position embedding formula is expressed as follows:

[0048]

[0049] where P represents the position encoding information, p represents the position of the current feature in the input, 2i is used to represent the even positions, 2i + 1 represents the odd positions, and d represents the length of the feature vector of the position encoding information. After that, the segment embedding and position embedding of each feature are added to obtain the position marking information of the feature. Finally, the position marking information of the feature and the feature value of the feature are added to obtain the input features of the encoder.

[0050] The encoder uses the standard architecture of the transformer, which consists of a multi-head self-attention module and an FFN feed-forward network. The input to the self-attention can be expressed as: attn = {X q , X k , N, M}; where X q represents the query sequence of length q, Xk is the key-value query vector of length k, k is the same size as the number of channels d2, N represents the weight tensor of the query and the key, and M represents the projection matrix. Therefore, the multi-head attention mechanism can be expressed as: M = {attn(X q , X k , N); attn(X q , X k , N)...; attn(X q , X k , N)}.

[0051] The decoder uses the multi-head attention mechanism, and the decoding dimension is d3. To accelerate the recognition speed of the BETR model, N2 objects are decoded in parallel, and the position encoding information of the object queries is also input to the decoder.

[0052] Improve the FFN feedforward network to increase the convergence speed of the improved BETR model during training. The specific improvements are as follows: Use ReLU6 as the activation function, design the number of hidden dimensions as d4 dimensions, finally restore the features through a linear projection, use the softmax function in the mapping layer to predict the category of the objective function, and finally output the classification result; the linear projection formula is as follows: y = xW2 + B, where W2 represents the parameter matrix, B represents the bias matrix, x represents the input, and y represents the output.

[0053] Step 4: Use the first photovoltaic panel dataset obtained in Step 2 to pre-train the improved BETR model. Then, using the idea of transfer learning, use the photovoltaic panel infrared hot spot dataset and the second photovoltaic panel dataset obtained in Step 2 to formally train the pre-trained BETR model to fine-tune the parameters and obtain an optimized BETR model.

[0054] In this step, use the first photovoltaic panel dataset to pre-train the improved BETR model, set the initial learning rate to 1e-2, and to improve the convergence speed, set the learning rate to decrease as the number of iterations increases. After the improved BETR model converges, use the photovoltaic panel infrared hot spot dataset and the second photovoltaic panel dataset to formally train the pre-trained BETR model. The training strategy is to first freeze the backbone network, and then unfreeze the backbone network after the improved BETR model converges, and train the BETR model again to fine-tune the parameters of the backbone network of the BETR model to obtain an optimized BETR model.

[0055] In the said Step 4, during the training process, the overall loss function L used by the BETR model is as follows:

[0056]

[0057] where β, α, K represent three hyperparameters, L iou represents the calculation function of iou, ||.|| represents the norm calculation, a i represents the intersection of the i-th predicted class box and the actual box, a γ(i) represents the union of the i-th predicted class box and the actual box, represents the auxiliary loss function.

[0058] L iou The calculation function expression of is:

[0059]

[0060] where represents the area calculation, and B(.) represents the calculation function of max / min.

[0061] Using an auxiliary loss function to help the decoder of the Transformer correct the output result, and its expression is as follows: where n represents the binary target mask, represents the original mask logic prediction of the BETR model, and relu represents the activation function.

[0062] Step 5: Use the BETR model obtained in Step 4 to identify the newly acquired infrared hot spot image of the photovoltaic panel and the photovoltaic panel image, classify the size of the identified infrared hot spot of the photovoltaic panel, count the number and location information of the hot spots in each category, and count the number of photovoltaic panels, and integrate and output the results.

[0063] In this step, in order to give priority to the treatment of large hot spots, it is necessary to classify and count the size of the hot spots. The specific process of classifying and counting the size of the infrared hot spot of the photovoltaic panel is as follows: Use the FFN feed-forward network to predict the center coordinates of E bounding boxes of a fixed size and E input image boxes, which is represented by the set D, and D can be expressed as follows:

[0064] D = {b i = (h i , w i ); O i = (O x , O y ), i = 1, 2, 3... N3};

[0065] where b i represents the i-th bounding box, Oi represents the center coordinates of the i-th input image box, (h i , w i ) represents the height and width of the i-th bounding box, (O x , O y ) represents the coordinates of the input image box on the x-axis and y-axis, N3 is much larger than the number of target objects to be predicted. In addition, a label θ is used to indicate that no object is detected in the detection area.

[0066] The detected hot spots are divided into two categories according to size, a threshold for dividing the size of the hot spots is set, and a counter is set inside the prediction loop to count the number of hot spots of the photovoltaic panel in each category.

[0067] The method for detecting hot spots in infrared images of photovoltaic panels based on the improved BETR model involved in this application performs data enhancement on the obtained image data, improving the data quality. The constructed dataset is conducive to enhancing the robustness and generalization ability of the improved BETR model. This method proposes using a feature pyramid to replace ResNet-50 in the original BETR model for feature extraction of images, and conducts multiple information fusions on deep and shallow features, improving the feature extraction ability of the improved BETR model for small target objects. This method also proposes a redesigned position encoder for the original BETR model, annotating the image features in each dimension to improve the BETR model's ability to record the positions of two-dimensional image features. This method also redesigned the FFN feedforward network, increasing the number of hidden layers to improve the recognition accuracy, and using the ReLU6 activation function to ensure the convergence speed of the improved BETR model during training. The improved BETR model has a faster training speed and recognition speed, has a high recognition accuracy for small target objects (such as hot spots with a small area), can classify the size of infrared hot spots on photovoltaic panels and count the number, record the position information of infrared hot spots on photovoltaic panels, and can also count the number of photovoltaic panels.

[0068] As described above, the above is only a preferred specific embodiment of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application, according to the technical solution and its concept of this application, making equivalent replacements or changes should be covered within the protection scope of this application.

Claims

1. A method for detecting hot spots in infrared images of photovoltaic panels based on an improved BETR model, characterized in that: It includes the following steps: Step 1: Collect the photovoltaic panel images and the infrared hot spot images of the photovoltaic panels by using drones; Step 2: Perform data augmentation processing on both the photovoltaic panel images and the infrared hot spot images of the photovoltaic panels. Use the processed photovoltaic panel images to construct a first photovoltaic panel dataset for pre-training, and use the processed infrared hot spot images and photovoltaic panel images of the photovoltaic panels to construct an infrared hot spot dataset of the photovoltaic panels and a second photovoltaic panel dataset for formal training respectively; Step 3: Construct an improved BETR model: Use a feature pyramid to replace the backbone network for feature extraction in the original BETR model; Improve the position encoder to enhance the ability to record the positions of two-dimensional image features. Among them, the following improvements are made to the position encoder. Different tags are given to each dimension of the feature of d1*HW1. The dimension of the feature is marked with segment embedding∈{1,2,3...N1}, where the feature of d1*HW1 is obtained by using a 1*1 convolutional layer to reduce the high-dimensional feature to a smaller dimension d1. The high-dimensional feature refers to the feature of D1*H*W1; Use position embedding to mark the position encoding information of each image feature on the same dimension. Then, add the segment embedding and position embedding of each feature to obtain the position marking information of the feature. Finally, add the position marking information of the feature and the feature value of the feature to obtain the input feature of the encoder; Improve the FFN feed-forward network to improve the convergence speed of the improved BETR model during training. Among them, the following improvements are made to the FFN feed-forward network: Use ReLU6 as the activation function, design the hidden dimension number to be d4 dimensions, and finally restore the feature through a linear projection. Use the softmax function in the mapping layer to predict the category of the objective function, and finally output the classification result; The linear projection formula is as follows: y = xW2 + B, where W2 represents the parameter matrix, B represents the bias matrix, x represents the input, and y represents the output; Step 4: Use the first photovoltaic panel dataset obtained in Step 2 to pre-train the improved BETR model. Then, using the idea of transfer learning, use the infrared hot spot dataset of the photovoltaic panels and the second photovoltaic panel dataset obtained in Step 2 to formally train the pre-trained BETR model to fine-tune the parameters and obtain an optimized BETR model; Step 5: Use the optimized BETR model obtained in Step 4 to identify the newly collected infrared hot spot images and photovoltaic panel images of the photovoltaic panels, classify the sizes of the identified infrared hot spots of the photovoltaic panels, count the number and position information of the hot spots in each category, and count the number of photovoltaic panels, and integrate and output the results.

2. The method for detecting hot spots in the infrared image of a photovoltaic panel based on the improved BETR model according to claim 1, wherein: In the step 2, the data augmentation process is as follows: Using an image convolution operator to perform sharpening processing on the image, performing a convolution operation on the image convolution operator and the image, and the calculation process is described as follows: Where a' is an integer from 0 to 2, b' is an integer from 0 to 2, Kernel(a, b) is the input image convolution operator, and csr(a, b) is the input image, referring to the photovoltaic panel image or the photovoltaic panel infrared hot spot image. The image obtained after the convolution operation is subjected to flipping or size scale transformation processing. Among them, the size scale transformation processing of the image obtained after the convolution operation is to transform the pixel value size of the image to between 240 and 678. Then, a multi-scale sliding window sampling operation is performed on the image obtained after the flipping or size scale transformation processing.

3. The method for detecting hot spots in the infrared image of a photovoltaic panel based on the improved BETR model according to claim 2, characterized in that: In the step 3, the feature pyramid is used to replace the ResNet-50 network for feature extraction in the original BETR model. The feature pyramid is responsible for extracting the features of the image. First, the image features are extracted through 5 convolutional layers Conv1 to Conv5. Then, through upsampling and concat splicing, the features obtained from {Conv2, Conv3, Conv4} are respectively fused with the features obtained from the FFB block. Then, the features obtained from {FFB, FFB} and {FFB2, FFB2} blocks are fused. Then, before the feature fusion, an adaptive feature pooling operation is performed on the features extracted by the pyramid layer to unify the features of different dimensions to the same dimension. Then, the feature fusion is performed on each of the above features of the same dimension, and finally a feature of D1*H*W1 is obtained, where D1 represents the dimension of the feature, and H and W1 respectively represent the height and width of the feature.

4. The method for detecting hot spots in the infrared image of a photovoltaic panel based on the improved BETR model according to claim 3, characterized in that: In the step 3, the position embedding formula is expressed as follows: Where, P represents the position encoding information, p represents the position of the current feature in the input, 2i is used to represent the even position, 2i + 1 represents the odd position, and d represents the length of the feature vector of the position encoding information.

5. The method for detecting hot spots in the infrared image of a photovoltaic panel based on the improved BETR model according to claim 4, wherein: In the step 4, the improved BETR model is pre-trained using the first photovoltaic panel dataset, the initial learning rate is set to 1e-2, and the learning rate is set to decrease as the number of iterations increases. When the improved BETR model converges, the pre-trained BETR model is formally trained using the photovoltaic panel infrared hot spot dataset and the second photovoltaic panel dataset. The training strategy is to first freeze the backbone network, and then unfreeze the backbone network when the improved BETR model converges, and train the BETR model again to fine-tune the parameters of the backbone network of the BETR model to obtain an optimized BETR model; Where, in the step 4, during the training process, the overall loss function L used by the BETR model is as follows: Among them, β, α, and K represent three hyperparameters, and L iou represents the calculation function of iou, ||.|| represents the norm calculation, and a i represents the intersection of the i-th predicted class box and the actual box, and a γ(i) represents the union of the i-th predicted class box and the actual box, represents the auxiliary loss function; L iou The calculation function expression is: Among them represents area calculation; B(.) represents the calculation function of max / min, where " / " represents "or"; Using an auxiliary loss function to help the decoder of the Transformer correct the output result, and its expression is as follows: where n represents the binary target mask, represents the original mask logic prediction of the BETR model, and relu represents the activation function.

6. The method for detecting hot spots in the infrared image of a photovoltaic panel based on the improved BETR model according to claim 5, characterized in that: In the step 5, the process of classifying and counting the size of the photovoltaic panel infrared hot spot is as follows: Use the FFN feed-forward network to predict the center coordinates of a set of E bounding boxes and E input image boxes with a fixed size, which is represented by the set D. D can be expressed as follows: D = {b i = (h i , w i )); O i = (O x , O y ), i = 1, 2, 3... N3}; where b i represents the i-th bounding box, O i represents the center coordinates of the i-th input image box, (h i , w i ) represents the height and width of the i-th bounding box, (O x , O y ) represents the coordinates of the input image box on the x-axis and y-axis. N3 is much larger than the number of target objects to be predicted. In addition, a label θ is used to indicate that no objects are detected in the detection area. The detected hot spots are divided into two categories according to size. A threshold for dividing the size of the hot spots is set. A counter is set inside the prediction loop to count the number of hot spots of each category of photovoltaic panels.

Images