A photovoltaic module strip and area hot spot detection method and system based on module energy factor

Abstract

The application discloses a kind of photovoltaic module strip and surface hot spot detection method and system based on module energy factor.The method comprises the following steps: firstly, training and optimizing semantic segmentation model U-Net, extracting the photovoltaic array region of input image based on the trained model and performing tilt correction and filtering;Secondly, based on the method of bidirectional local standard deviation, bilateral filtering and morphological filtering, the gap of photovoltaic module is extracted, and the photovoltaic module segmentation is completed;Then, based on the segmented module, the module energy factor is constructed, the hot spot module is identified, and finally the image coordinates of hot spot are inversely transformed to realize the detection of strip and surface hot spot.The application can quickly detect strip and surface hot spot defects of photovoltaic module from unmanned aerial vehicle inspection infrared image, solve the problem of low detection rate of strip and surface hot spot in photovoltaic power station operation and maintenance, and has good detection effect and high detection efficiency for strip and surface hot spot of photovoltaic module in different photovoltaic power station scenes.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a method and system for detecting strip and planar hot spots in photovoltaic modules based on module energy factor. Background Technology

[0002] Photovoltaic power generation, as a sustainable, green, and pollution-free new energy production method, is considered a key technology for solving the global energy crisis. In recent years, China's photovoltaic power generation has experienced rapid growth. As of May 2021, the cumulative installed capacity of photovoltaic power generation accounted for more than 11% of the total installed power capacity nationwide. With the rapid development of photovoltaic power generation and the construction of large-scale photovoltaic power plants, how to quickly troubleshoot photovoltaic module failures and reduce the huge safety hazards and economic losses caused by defects has become a pressing technical challenge for the photovoltaic industry. Photovoltaic module failures will exhibit hot spot effects. Using drones equipped with infrared image acquisition devices to conduct aerial inspections of photovoltaic power plants, and automatically detecting hot spot effects based on image processing and pattern recognition technologies, is a feasible method to solve the operation and maintenance challenges of photovoltaic power plants.

[0003] In infrared images, strip-shaped hot spots on photovoltaic (PV) modules refer to hot spot effects caused by module shading, dirt, diode failure, etc.; planar hot spots are caused by module breakage, multiple diode failures, or module disconnection. Compared to common point-shaped hot spot defects in PV modules, strip-shaped and planar hot spot defects pose a greater threat to PV arrays and a higher level of safety risk. Therefore, research on the detection of strip-shaped and planar hot spots in PV modules based on infrared images has significant practical implications.

[0004] Most existing photovoltaic module hot spot detection methods, both domestically and internationally, are based on electrical characteristics or handheld infrared detection, with little research on strip and area hot spot detection technologies using UAV infrared images. Patent CN112164038A, "A Photovoltaic Hot Spot Detection Method Based on Deep Convolutional Neural Networks," proposes a Faster R-CNN-based hot spot detection method that relies on a large amount of data to learn hot spot features. However, infrared images from different times and different power plants vary significantly, and acquiring a large number of hot spot images is costly, making it difficult to meet practical application needs. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method for detecting strip-shaped and planar hot spots based on module energy factors. In applications where photovoltaic power plant inspections are conducted using UAV aerial photography equipped with infrared thermal imaging, this method improves the detection rate of strip-shaped and planar hot spots on photovoltaic modules while reducing the false alarm rate, thus solving the problem of poor detection performance of strip-shaped and planar hot spots caused by environmental changes and sensor noise.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for detecting strip-shaped and planar hot spots based on modular energy factor includes the following steps:

[0008] Step 1): After preprocessing the original infrared image of the photovoltaic array, the photovoltaic array region of the preprocessed infrared image is extracted using the trained U-Net model, and tilt correction and filtering are performed to obtain a set of photovoltaic array images.

[0009] Step 2): Extract the edges of the photovoltaic modules based on bidirectional local standard deviation, and extract the gaps between the photovoltaic modules based on bilateral filtering and morphological filtering methods. Perform module segmentation of the photovoltaic array image to obtain the image of each photovoltaic module.

[0010] Step 3): Construct module energy factors based on the images of each photovoltaic module, and identify strip and planar hot spots based on the binarization projection method. Construct a hot spot module set, and perform inverse transformation on the image coordinates of the hot spot modules to obtain the positions of strip and planar hot spots in the original infrared image, thus achieving the final detection of strip and planar hot spots.

[0011] Furthermore, in step 1), the preprocessing specifically involves: whitening and normalizing the original infrared image, and then using median filtering to filter the normalized infrared image.

[0012] Furthermore, in step 1), the specific training method of the U-Net model is as follows:

[0013] An infrared image dataset of photovoltaic arrays is constructed, with the infrared images in the dataset serving as samples and the corresponding labels being the coordinates of the quadrilateral vertices of each photovoltaic array region in the infrared image. The U-Net model is iteratively optimized on the infrared image dataset of photovoltaic arrays using the RMSProp method as the optimization strategy, with the optimization objective being to minimize the loss function of the U-Net model.

[0014] The loss function of the U-Net model is:

[0015]

[0016] in, Let represent the loss of the U-Net model, B represent the set of samples in a batch, |B| represent the number of samples in the batch, x represent a sample, y∈{0,1} represent the label corresponding to the sample, and σ(·) represent the sigmoid function.

[0017] Furthermore, in step 1), the specific implementation method of tilt correction and filtering is as follows:

[0018] The minimum bounding rectangle of the extracted photovoltaic array region is calculated, and the minimum bounding rectangle is tilted by perspective transformation. After tilt correction, based on the size parameters of the minimum bounding rectangle, a set of photovoltaic array images with an aspect ratio greater than 2 and an area greater than 2000 is selected.

[0019] Furthermore, in step 2), the specific implementation method for extracting the edges of the photovoltaic module based on bidirectional local standard deviation is as follows:

[0020] The photovoltaic array image is filtered using local standard deviation templates in the horizontal and vertical directions to extract the edges of the photovoltaic modules; the local standard deviation is:

[0021]

[0022]

[0023] Where Ω represents the shape of the local standard deviation template, |Ω| represents the number of pixels within Ω, Var(x) represents the local standard deviation of pixel x, I(x) represents the gray value of pixel x in the current photovoltaic array image, and μ represents the average gray value of pixels within the template.

[0024] Furthermore, in step 2), the specific implementation method for extracting the gaps between photovoltaic modules based on bilateral filtering and morphological filtering methods to perform module segmentation of the photovoltaic array image is as follows:

[0025] The specific steps for repairing the photovoltaic module edges extracted using bilateral filtering based on bidirectional local standard deviation are as follows:

[0026]

[0027]

[0028] Where BF(u) represents the value of pixel u after bilateral filtering, W v Let S represent the accumulated template weights of pixel v, S represent the filter template, u and v represent any pixel in the photovoltaic array image, ∥.∥ represents the L2 norm, exp(.) represents the exponential function, and σ s and σ r Let I(u) represent the spatial standard deviation and grayscale standard deviation of the photovoltaic array image, respectively, and let I(u) represent the grayscale value of pixel u in the current photovoltaic array image.

[0029] Five iterations of erosion were performed on the edge-repaired photovoltaic array image using 1*7 and 7*1 templates respectively to extract the gaps between photovoltaic modules. Based on this, the midline of the extracted module gaps was used as the dividing line to divide the photovoltaic modules, thus realizing the module segmentation of the photovoltaic array image.

[0030] Furthermore, in step 3), the specific implementation method of constructing the module energy factor is as follows:

[0031] Based on the segmented photovoltaic modules, obtain the pixel grayscale matrix of the photovoltaic modules and stretch it into a column vector; calculate the grayscale truncation threshold using the photovoltaic array image where the photovoltaic modules are located, and set the elements not greater than the grayscale truncation threshold to zero; obtain the column vectors of the left and right neighboring modules of the current photovoltaic module, calculate the grayscale energy difference between the current module and the two neighboring modules respectively, and take the minimum value as the module energy factor of the current module.

[0032] The grayscale truncation threshold is:

[0033]

[0034] Where |P| represents the number of pixels in the photovoltaic array image, P represents the set of pixels in the photovoltaic array image, x represents any pixel in the photovoltaic array image, I(.) represents the gray value of the pixel, and ε represents the truncation compensation parameter.

[0035] The grayscale energy difference is:

[0036]

[0037] Where 'a' represents the column vector of the current module, and 'b' represents the column vector of the neighboring modules, a i Let b represent the i-th element of vector a. i Let represent the i-th element of vector b, max(.) means taking the maximum value, and k represents the length of vector a.

[0038] Furthermore, in step 3), the specific implementation of identifying strip-shaped and planar hot spots based on the binarized projection method is as follows:

[0039] A binary image of the photovoltaic module is obtained by binarizing the image using a grayscale truncation threshold. Specifically, the binarization involves setting pixels with grayscale values ​​greater than the grayscale truncation threshold to 1 and other pixels to zero. Based on this, the binarized image is projected horizontally and vertically to obtain the binary projection histogram vectors x and y of the photovoltaic module in the horizontal and vertical directions. Photovoltaic modules with energy factors greater than the photovoltaic module energy threshold are selected. Each selected photovoltaic module is then examined sequentially, and photovoltaic modules with strip-shaped or planar hot spots are selected based on the projection distribution characteristics of the photovoltaic modules.

[0040] The specific methods for projecting in the horizontal and vertical directions are as follows:

[0041]

[0042]

[0043] Where, x i Let x = [x1, ..., x2] represent the i-th element of the binary projection histogram vector of the photovoltaic module in the horizontal direction. n ], y j This represents the j-th element of the binary projection histogram vector of the photovoltaic module in the vertical direction, y = [y1, ..., y2]. m ], m, n represent the length and width of the binary image of the photovoltaic module, and B(i,j) represents the pixel value of the current photovoltaic module at coordinate (i,j) after binarization;

[0044] The specific implementation method for filtering photovoltaic modules with strip-shaped or planar hot spots based on the projection distribution characteristics of photovoltaic modules is as follows: Calculate the area ratio β of pixels with a value of 1 in the binary image of the photovoltaic module in the photovoltaic array image:

[0045]

[0046] Photovoltaic modules that meet the condition β>γ1 are identified as planar hot spots, and photovoltaic modules that meet the condition γ2<β<γ1 are identified as strip hot spots. Here, γ1 is the prior threshold for the area ratio of planar hot spots, and γ2 is the prior threshold for the area ratio of strip hot spots. Photovoltaic modules that are considered to be planar hot spots or strip hot spots constitute a set of hot spot modules.

[0047] Calculate the ratio ρ between the number of pixels with a value of 1 in the binary image of the photovoltaic module and the total number of pixels with a value of 1 in the binary image of the photovoltaic array. Select photovoltaic modules that meet the condition ρ < γ3 and remove them from the hot spot module set, where γ3 is the prior threshold for "reflective" photovoltaic arrays.

[0048] Furthermore, in step 3), the specific implementation method for obtaining the positions of strip-shaped and planar hot spots in the original infrared image by performing an inverse transformation on the image coordinates of the hot spot module is as follows:

[0049] Based on the minimum bounding rectangle parameters of the photovoltaic array region in the pre-correction photovoltaic array image, including the position, size, and rotation angle of the minimum bounding rectangle, the hot spot module set is inversely transformed using a geometric transformation method to obtain the set of position parameters of the hot spot module set in the original infrared image.

[0050] The present invention also proposes a system for implementing the above-described method for detecting strip and planar hot spots based on modular energy factors, the system comprising:

[0051] The photovoltaic array extraction subsystem is used to preprocess the original infrared image, extract the photovoltaic array region, and perform tilt correction and filtering to obtain a photovoltaic array image set.

[0052] The photovoltaic module segmentation subsystem is used to segment the photovoltaic array image into modules, obtaining images of each photovoltaic module.

[0053] The hot spot recognition subsystem is used to identify strip-shaped and planar hot spots in the images of each photovoltaic module, and to perform inverse transformation to obtain the position of the strip-shaped and planar hot spots in the original infrared image, thus achieving the final detection of strip-shaped and planar hot spots.

[0054] In summary, the technical solution conceived in this invention has the following advantages compared with the prior art:

[0055] (1) The photovoltaic array region extraction method based on the U-Net semantic segmentation model proposed in this invention can accurately extract the photovoltaic array region in the infrared image taken by UAV and has robustness for infrared images taken under different lighting conditions and different ground backgrounds.

[0056] (2) The module energy factor metric proposed in this invention fully characterizes the dissimilarity between the photovoltaic module with strip-shaped and planar hot spots and the neighboring module, thereby effectively enhancing the abnormal signal of the hot spot and suppressing background interference.

[0057] (3) The strip and surface hot spot identification method based on binary projection and expert knowledge system proposed in this invention can effectively identify the photovoltaic module where the strip and surface hot spots are located, while effectively suppressing the influence of the "reflective" photovoltaic array on the detection results.

[0058] (4) The photovoltaic module strip and surface hot spot detection method proposed in this invention can quickly detect hot spots from a single infrared image and has high detection accuracy and efficiency. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating the basic steps of an embodiment of the photovoltaic module strip and planar hot spot detection method of the present invention.

[0060] Figure 2 Infrared images taken by a drone for a photovoltaic power station.

[0061] Figure 3 The result is the extraction of the photovoltaic array region after segmentation based on the U-Net model.

[0062] Figure 4 The image shown is of the photovoltaic array region after tilt correction and filtering in the embodiment.

[0063] Figure 5 This is a schematic diagram of the module segmentation process of the present invention, wherein (a) is a photovoltaic array image of a planar hot spot, and (b) is a result diagram of the photovoltaic module segmentation.

[0064] Figure 6 for Figure 5 Numerical curves of the module energy factor for each photovoltaic module.

[0065] Figure 7 The image shows the results of strip and area hot spot recognition based on binarized projection and expert knowledge system.

[0066] Figure 8 The image shows the detection results of an embodiment of the photovoltaic module strip and planar hot spot detection method of the present invention. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this invention clearer, the invention is described in detail below with reference to specific embodiments. Specific embodiments are described below to simplify the invention. However, it should be understood that the invention is not limited to the described embodiments, and various modifications are possible without departing from the basic principles; these equivalent forms also fall within the scope defined by the appended claims.

[0068] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0069] This invention provides a method for detecting strip-shaped and planar hot spots based on module energy factors, such as... Figure 1 As shown, it includes:

[0070] Step 1: Photovoltaic array region extraction: Obtain the UAV infrared image database of photovoltaic power station inspection, construct the infrared image dataset of photovoltaic array, train and optimize the semantic segmentation model U-Net on the dataset, preprocess the original infrared images of photovoltaic array, extract the photovoltaic array region of the preprocessed infrared images based on the trained U-Net model, and perform tilt correction and filtering to obtain the photovoltaic array image set.

[0071] Step 2: Photovoltaic module segmentation: The edges of the photovoltaic modules are extracted based on bidirectional local standard deviation, and the gaps between the photovoltaic modules are extracted based on bilateral filtering and morphological filtering methods to segment the photovoltaic array image into modules;

[0072] Step 3: Detection of strip and planar hot spots: Construct module energy factors based on the segmented photovoltaic modules, and identify strip and planar hot spots based on the photovoltaic array binarized projection and expert knowledge system. Perform inverse transformation on the image coordinates of the hot spots according to the tilt correction parameters of the photovoltaic array to obtain the positions of the strip and planar hot spots in the original infrared image, thus achieving the final detection of strip and planar hot spots.

[0073] Step 1, photovoltaic array region extraction, includes:

[0074] Step 1-1: Construct an infrared image dataset for the photovoltaic array. Using data annotation software, label the coordinates of the quadrilateral vertices of each photovoltaic array region in each acquired infrared image of the photovoltaic power station. Write the coordinate information of all photovoltaic arrays in the same infrared image into the same label file to construct the infrared image dataset for the photovoltaic array.

[0075] Considering the different flight directions and thermal imaging angles of different drone aerial photography flights, the angles of the photovoltaic array arrangement vary. Data augmentation refers to supplementing the original data by performing transformation operations such as flipping and rotating, which is a method that can effectively improve the generalization performance of the model. Specifically, in this embodiment, the infrared image dataset of the photovoltaic array includes original image data and augmented data. The augmented data is constructed by flipping the original images horizontally and vertically, and rotating them clockwise by 90, 180, and 270 degrees, respectively. In this embodiment, the constructed photovoltaic array infrared image dataset contains a total of 2520 images, which are further divided into training and testing datasets at a preset ratio of 9:1.

[0076] Steps 1-2: U-Net Training and Optimization. The U-Net model consists of an encoder and a decoder. The encoder is downsampled four times and then upsampled four times by the decoder, forming a U-shaped structure. The U-Net model is iteratively optimized on the training dataset using the RMSProp method, with the optimization objective being to minimize the loss function of the U-Net model. Specifically, in this embodiment, the RMSProp method with a learning rate of 0.001 and a decay rate of 0.9 is used to optimize the U-Net model.

[0077] The specific implementation of the loss function of the U-Net model is shown in Equation (1).

[0078]

[0079] in, Let represent the loss of the U-Net model, B represent the sample set of a batch, |B| represent the number of samples in the batch, x represent a sample, y∈{0,1} represent the label corresponding to the sample, and σ(·) represent the sigmoid function. In the specific implementation example, the training dataset is randomly divided into 315 batches of sample sets, with each batch containing 8 samples.

[0080] Steps 1-3 involve preprocessing the original infrared image of the photovoltaic array and extracting the photovoltaic array region from the preprocessed input image based on the trained U-Net model.

[0081] The specific implementation method of the preprocessing is as follows:

[0082] First, image whitening and normalization are used to process... Figure 2 The infrared image shown is processed to remove the influence of changes in the lighting environment on the image grayscale features. Then, median filtering is used to filter the normalized infrared image to remove random noise interference in the image.

[0083] Specifically, infrared thermal imaging is affected by multiple factors such as ambient lighting intensity, object reflection, and sensor camera. This leads to significant deviations in the grayscale distribution characteristics of infrared images acquired at different times but in the same scene. Image whitening, through pixel grayscale value standardization, can effectively eliminate the dynamic effects of ambient light changes on the image. Sensor thermal noise interference is another major factor affecting the quality of infrared images. Image whitening and normalization are shown in formulas (2) and (3). The thermal noise is salt-and-pepper noise, and median filtering can effectively suppress the interference of salt-and-pepper noise. In the specific embodiment, a 5*5 template is used to perform median filtering on the whitened and normalized images.

[0084] I1(x,y)=(I0(x,y)-I mean ) / I std (2)

[0085] I2(x,y)=(I1(x,y)-I min ) / (I max -I min (3)

[0086] Where I0(x,y) represents the gray value of pixel (x,y) in the original infrared image, I1(x,y) represents the gray value of the pixel after whitening, and I2(x,y) represents the gray value of the pixel after normalization. mean I represents the mean gray level of the original infrared image. std I represents the standard deviation of gray levels in the original infrared image. max and I min These represent the maximum and minimum grayscale values ​​of the whitened image, respectively.

[0087] Furthermore, the specific implementation method for extracting the photovoltaic array region based on the trained U-Net model is as follows:

[0088] The preprocessed image is input into the trained U-Net model, and the output is a binarized image of the photovoltaic array. The minimum bounding rectangle parameters [x, y, w, h, α] of each connected region in the binarized image are obtained using the minimum bounding rectangle solution method, where x and y represent the coordinates of the top-left vertex of the bounding rectangle, w and h represent the length and width of the bounding rectangle, respectively, and α represents the rotation angle. Figure 3The image shows the result of photovoltaic array region extraction in the embodiment.

[0089] Steps 1-4: Tilt correction and filtering of photovoltaic array images. Tilt correction is performed on the minimum bounding rectangle based on perspective transformation. The rectangle with parameters [x,y,w,h,α] has a rotation angle of 0 after correction. After tilt correction, a set A = {I1,…,I...} of photovoltaic array images that meet the criteria of w / h>2 and w*h>2000 is selected. K}. I i This represents the infrared image of the i-th photovoltaic array that meets the conditions.

[0090] Specifically, due to the camera's shooting angle, photovoltaic arrays in infrared images often exhibit perspective distortion. To facilitate subsequent segmentation of photovoltaic modules and detection of hot spots within the array, it is necessary to correct the tilt of the photovoltaic array. Perspective transformation is an efficient method for tilt correction. In this embodiment, the perspective transformation matrix is ​​calculated based on the minimum bounding rectangle parameters [x,y,w,h,α] and the target rectangle parameters for tilt correction [0,0,w,h,0]. The target rectangle parameter represents the region of the photovoltaic array after tilt correction, extracted as a new image. The perspective transformation matrix is ​​then used to... Figure 2 The infrared image shown is subjected to perspective transformation to obtain the following: Figure 4 The image shows the tilt-corrected infrared images of each photovoltaic array.

[0091] Step 2, module segmentation of the photovoltaic array image includes:

[0092] Step 2-1: Extract the edges of the photovoltaic module based on bidirectional local standard deviation.

[0093] from Figure 4 The photovoltaic array image shown can be seen as consisting of photovoltaic modules arranged in a regular pattern, with narrow gaps between the modules. The grayscale of the gaps differs significantly from that of the photovoltaic modules. By using local standard deviation templates in the horizontal and vertical directions to filter the photovoltaic array image, edge extraction of the photovoltaic modules can be achieved. The specific implementation method of the local standard deviation is shown in formula (4).

[0094]

[0095] Where Ω represents the shape of the local standard deviation template, |Ω| represents the number of pixels within Ω, Var(x) represents the local standard deviation of pixel x, and I(x) represents the gray value of pixel x in the current photovoltaic array image. This represents the average grayscale value of pixels within the template. Specifically, in this embodiment, templates with shapes of 1*3 and 3*1 are used to filter the photovoltaic array image, thereby extracting the horizontal and vertical edges of the photovoltaic array image.

[0096] Step 2-2: Extract the gaps between photovoltaic modules based on bilateral filtering and morphological filtering methods to complete the module segmentation of the photovoltaic array image.

[0097] The edges of the photovoltaic module extracted by bidirectional local standard deviation are repaired using bilateral filtering as shown in formulas (5) and (6).

[0098]

[0099]

[0100] Where BF(u) represents the value of pixel u after bilateral filtering, W V Let S represent the accumulated template weights of pixel v, S represent the filter template, u and v represent any pixel in the photovoltaic array image, ∥.∥ represents the L2 norm, exp(.) represents the exponential function, and σ s and σ r σu and Iu represent the spatial standard deviation and grayscale standard deviation of the photovoltaic array image, respectively, and I(u) represents the grayscale value of pixel u in the current photovoltaic array image. Specifically, in this embodiment, 1*7 and 7*1 templates are used to perform bilateral filtering on the photovoltaic array edge images extracted in the horizontal and vertical directions, respectively, where σu... s and σ r Both are 7.

[0101] Furthermore, 1*7 and 7*1 templates are used respectively to perform 5 iterations of erosion on the edge-repaired photovoltaic array image to extract the gaps between photovoltaic modules. Based on this, the midline of the extracted module gaps is used as the dividing line to segment the photovoltaic modules, thus achieving module segmentation of the photovoltaic array image. Specifically, in the embodiment, for example... Figure 5 The typical photovoltaic array image shown in (a) is segmented into modules to obtain the following results: Figure 5 (b) shows the module segmentation result diagram.

[0102] Furthermore, step 3 includes:

[0103] Step 3-1: Construct the module energy factor.

[0104] The grayscale distribution of photovoltaic modules with strip-shaped and planar hot spots differs from that of their neighboring modules. Considering that the grayscale energy of hot spot modules is significantly higher than that of normal modules, a module energy factor is constructed to measure the energy difference between modules to detect hot spot modules. Based on the segmented photovoltaic modules, the pixel grayscale matrix of the photovoltaic modules is obtained and stretched into a column vector. A grayscale truncation threshold τ is calculated using the photovoltaic array image in which the photovoltaic module is located. Elements in the column vector greater than τ are filtered out, and the remaining elements are set to zero. The column vectors of the left and right neighboring modules of the current photovoltaic module are obtained, and the grayscale energy difference between the current module and its two neighboring modules is calculated as a measure of the local dissimilarity of the photovoltaic modules. The minimum value of the two obtained grayscale energy differences is taken as the module energy factor of the current photovoltaic module.

[0105] Furthermore, the specific implementation method of the grayscale truncation threshold is shown in formula (7).

[0106]

[0107] Where |P| represents the number of pixels in the photovoltaic array image, P represents the set of pixels in the photovoltaic array image, x represents any pixel in the photovoltaic array image, I(.) represents the grayscale value of the pixel, and ε represents the truncation compensation parameter. Specifically, in the embodiment, the grayscale truncation threshold is calculated using formula (7) with ε = 0.15.

[0108] Furthermore, the specific implementation method of the gray-scale energy difference is shown in formula (8).

[0109]

[0110] Where 'a' represents the column vector of the current module, and 'b' represents the column vector of the neighboring modules. i Let b represent the i-th element of the vector. i Let represent the i-th element of vector b, max(.) represents taking the maximum value, and k represents the length of vector a. The specific implementation of the module energy factor is shown in formula (9).

[0111] F(a)=min(E(a,l),E(a,r)) (9)

[0112] Where F(a) represents the module energy factor of the current photovoltaic module, and l and r represent the column vectors of the left and right neighboring modules of the current module, respectively. Specifically, in the embodiment, they are as follows: Figure 5 The photovoltaic module's energy factor calculation module is shown in the figure below, and the numerical curve is as follows: Figure 6 As shown.

[0113] Step 3-2: Identify strip-shaped and planar hot spots based on the binarized projection of the photovoltaic array and the expert knowledge system.

[0114] Using the gray-level truncation threshold of formula (7), the photovoltaic module is binarized to extract abnormal pixels with high gray levels, resulting in a binary image of the photovoltaic module. Specifically, the binarization involves setting pixels with gray values ​​greater than the gray-level truncation threshold to 1 and other pixels to zero. Based on this, binarized projections are performed in the horizontal and vertical directions to obtain the projection histogram vectors x and y in the horizontal and vertical directions, respectively. The module energy factor is constructed for each module in the photovoltaic array image using formulas (8) and (9). Based on the selected photovoltaic module set A = {M1, ..., M...} that satisfies F(a) > ξ, ... t}. Where ξ represents the energy threshold of the photovoltaic module, and A represents the t photovoltaic modules that meet the selection criteria. Specifically, in the embodiment, the module set A is selected based on F(a)>0.1. Each photovoltaic module in set A is examined sequentially. Based on the projection distribution characteristics of the photovoltaic modules and combined with the expert knowledge system of strip and area hot spots, photovoltaic modules that meet the prior knowledge of strip or area hot spots are selected to form a hot spot module set.

[0115] Furthermore, the specific implementation methods for performing binarized projection in the horizontal and vertical directions are shown in formulas (10) and (11).

[0116]

[0117]

[0118] Where x i This represents the i-th element of the binary projection histogram vector of the photovoltaic module in the horizontal direction, y. j Let represent the j-th element of the binary projection histogram vector of the photovoltaic module in the vertical direction, m and n represent the length and width of the photovoltaic array image, and B(i,j) represent the pixel value at coordinates (i,j) in the current binary image of the photovoltaic module.

[0119] Furthermore, the specific implementation method for selecting photovoltaic modules that conform to the prior knowledge of strip or area hot spots based on the projection distribution characteristics of photovoltaic modules and combined with the expert knowledge system of strip and area hot spots is as follows:

[0120] First, obtain the information of a certain photovoltaic module M. i The binary projection histogram vectors x and y in the horizontal and vertical directions are used to calculate M using formula (12). i The area percentage β of pixels with an inner value of 1.

[0121]

[0122] Where m and n represent the size of the photovoltaic array image. Based on the expert prior knowledge of planar hot spots, photovoltaic modules that meet the condition β>γ1 are selected as planar hot spots, where γ1 is the prior threshold for the area ratio of planar hot spots. Based on the expert prior knowledge of strip hot spots, photovoltaic modules that meet the condition γ2<β<γ1 are selected as strip hot spots, where γ2 is the prior threshold for the area ratio of strip hot spots. In order to remove the influence of continuous, large-area high-grayscale pixels in the photovoltaic array infrared image caused by the "reflection" of the photovoltaic array on the detection of strip and planar hot spots, M is calculated using formula (13). i The ratio ρ is the number of pixels with a value of 1 to the number of pixels with a value of 1 in the binary image of the photovoltaic array, where x and y are the projection histogram vectors of the photovoltaic array image in the horizontal and vertical directions. Based on formula (13), the set of module position parameters C = {[x1,y1,w1,h2],…,[x...} is selected to satisfy ρ < γ3. s ,y s ,w s ,h s ]}, and remove it from the hot spot module set, where γ3 is the prior threshold of the "reflective" photovoltaic array, [x i ,y i ,w i ,h i ] represents the minimum bounding rectangle parameter of the module relative to the photovoltaic array image in which it is located, and s represents the number of hot spot modules in the current photovoltaic array image.

[0123]

[0124] Among them, X i Y represents the i-th element of the binary projection histogram vector of the photovoltaic array in the horizontal direction. j This represents the j-th element of the binary projection histogram vector of the photovoltaic array in the vertical direction. The method for solving the binary projection histogram vectors of the photovoltaic array in the horizontal and vertical directions is the same as that for the photovoltaic module.

[0125] In a specific implementation example, the module set A = {M1, ..., M} is checked sequentially. t For each photovoltaic module in the array, the area ratios β and ρ are calculated using formulas (12) and (13). Based on this, planar hotspot modules meeting the criteria of β ≥ 0.5 and ρ > 0.3 are selected, as are strip-shaped hotspot modules meeting the criteria of 0.2 < β < 0.5 and ρ > 0.3. For example... Figure 7 The results shown are the detection and identification results of two typical infrared images of photovoltaic arrays containing planar and strip-shaped hot spots.

[0126] Step 3-3, Inverse transformation of hotspot image coordinates. Based on the minimum bounding rectangle parameters [x,y,w,h,α] of the photovoltaic array region in Step 1-3, the hotspot module set C = {[x1,y1,w1,h2],…,[x s ,y s ,w s ,h s Perform an inverse transform to obtain the set of position parameters C′={[x1′,y′1,w1,h2,θ1],…,[x′} in the original infrared image. s ,y′ s ,w s ,h s ,θ s ]}, where θ i This represents the rotation angle of the minimum bounding rectangle of the i-th hotspot module. The inverse transformation is performed sequentially on all hotspot modules in the photovoltaic array region of the infrared image to achieve the final detection of strip-shaped and planar hotspots on the photovoltaic module. For example... Figure 8 The results shown are from an example.

[0127] Compared with existing technologies, the method of this invention employs a semantic segmentation model to extract the photovoltaic array region from the infrared image, and detects strip-shaped and planar hot spots on the extracted photovoltaic array image, thereby improving detection accuracy. The proposed photovoltaic module segmentation method effectively further segments the photovoltaic array image, which helps to accurately locate the photovoltaic module. The proposed hot spot module recognition method based on binarized projection and expert knowledge system fully considers the grayscale features of strip-shaped and planar hot spot templates, avoiding the influence of "reflective" photovoltaic module arrays on hot spot detection results. The proposed photovoltaic module strip-shaped and planar hot spot detection method can quickly detect hot spot defects in a single frame of infrared image without processing sequential images, thus improving detection efficiency.

[0128] The accompanying drawings illustrating the embodiments of the present invention will make the objectives, technical solutions, and advantages of the present invention clearer. It should be noted that the specific embodiments described herein are merely illustrative and not intended to limit the scope of the invention. All equivalent substitutions and improvements made within the framework of the methods and principles provided by the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting strip-shaped and planar hot spots based on modular energy factor, characterized in that, Includes the following steps: (1) The original infrared image of the photovoltaic array is preprocessed, and the photovoltaic array region of the preprocessed infrared image is extracted using the trained U-Net model. The tilt is corrected and filtered out to obtain a set of photovoltaic array images. (2) The edges of each photovoltaic module in the photovoltaic array are extracted based on the bidirectional local standard deviation, and the gaps between the photovoltaic modules are extracted based on the bilateral filtering and morphological filtering methods. The photovoltaic array image is segmented to obtain the image of each photovoltaic module. (3) Construct module energy factors based on the images of each photovoltaic module, and identify strip and planar hot spots based on the binarization projection method, construct a hot spot module set, and perform inverse transformation on the image coordinates of the hot spot modules to obtain the positions of strip and planar hot spots in the original infrared image, thereby realizing the final detection of strip and planar hot spots; The specific implementation method of the energy factor of the construction module is as follows: Based on the segmented photovoltaic modules, obtain the pixel grayscale matrix of the photovoltaic modules and stretch it into a column vector; calculate the grayscale truncation threshold using the photovoltaic array image where the photovoltaic modules are located, and set the elements not greater than the grayscale truncation threshold to zero; obtain the column vectors of the left and right neighboring modules of the current photovoltaic module, calculate the grayscale energy difference between the current module and the two neighboring modules respectively, and take the minimum value as the module energy factor of the current module. The grayscale truncation threshold is: ； in This indicates the number of pixels in the photovoltaic array image. This represents the set of pixels in a photovoltaic array image. Represents any pixel in a photovoltaic array image. Represents the grayscale value of a pixel. Indicates the truncation compensation parameter; The grayscale energy difference is: ； in This represents the column vector of the current module. The column vector representing the neighborhood module, Representing vectors The One element, Representing vectors The One element, This indicates taking the maximum value. Representing vectors Length; The specific implementation of identifying strip-shaped and planar hot spots and constructing a hot spot module set based on the binarization projection method is as follows: The photovoltaic module is binarized using a grayscale truncation threshold to obtain a binarized image of the photovoltaic module. Specifically, the binarization involves setting pixels with grayscale values ​​greater than the grayscale truncation threshold to 1 and other pixels to zero. The binarized image is then binarized and projected horizontally and vertically to obtain the projection histogram vectors of the photovoltaic module in the horizontal and vertical directions. Photovoltaic modules with module energy factors greater than the photovoltaic module energy threshold are selected. Each selected photovoltaic module is then examined sequentially. Based on the projection distribution characteristics of the photovoltaic modules, photovoltaic modules with strip-shaped or planar hot spots are selected to form a hot spot module set.

2. The method for detecting strip-shaped and planar hot spots based on modular energy factor according to claim 1, characterized in that, The preprocessing specifically involves whitening and normalizing the original infrared image, and then using median filtering to filter the normalized infrared image.

3. The method for detecting strip-shaped and planar hot spots based on modular energy factor according to claim 1, characterized in that, The specific training method for the U-Net model is as follows: An infrared image dataset of photovoltaic arrays is constructed, with the infrared images in the dataset serving as samples and the corresponding labels being the coordinates of the quadrilateral vertices of each photovoltaic array region in the infrared image. The U-Net model is iteratively optimized on the infrared image dataset of photovoltaic arrays using the RMSProp method as the optimization strategy, with the optimization objective being to minimize the loss function of the U-Net model. The loss function of the U-Net model is: ； in, This represents the loss of the U-Net model. This represents a batch of sample sets. Indicates the number of samples in the batch. Indicates a sample, This indicates the label corresponding to the sample. This represents the sigmoid function.

4. The method for detecting strip and planar hot spots based on modular energy factor according to claim 1, characterized in that, The specific implementation method of the tilt correction and filtering is as follows: The minimum bounding rectangle of the extracted photovoltaic array region is calculated, and the minimum bounding rectangle is tilted by perspective transformation. After tilt correction, based on the size parameters of the minimum bounding rectangle, a set of photovoltaic array images with an aspect ratio greater than 2 and an area greater than 2000 is selected.

5. The method for detecting strip-shaped and planar hot spots based on modular energy factor according to claim 1, characterized in that, The specific implementation method for extracting the edges of each photovoltaic module in the photovoltaic array based on bidirectional local standard deviation is as follows: The photovoltaic array image is filtered using local standard deviation templates in the horizontal and vertical directions to extract the edges of the photovoltaic modules; the local standard deviation is: ； ； in, Indicates the shape of the template for local standard deviation. express The number of pixels within, Represents pixels Local standard deviation, Represents the number of pixels in the current photovoltaic array image. grayscale value, This represents the average grayscale value of the pixels within the template.

6. The method for detecting strip and planar hot spots based on modular energy factor according to claim 5, characterized in that, The specific implementation method for extracting the gaps between photovoltaic modules and performing module segmentation of the photovoltaic array image based on bilateral filtering and morphological filtering is as follows: Edge repair of photovoltaic modules extracted by bidirectional local standard deviation is performed using bilateral filtering: ； ； in, Represents pixels The value after bilateral filtering, Represents pixels The accumulated template weights, Indicates the filter template. Represents any pixel in a photovoltaic array image. Represents the L2 norm. Represents an exponential function. and These represent the spatial standard deviation and grayscale standard deviation of the photovoltaic array image, respectively. Represents the number of pixels in the current photovoltaic array image. grayscale value; respectively and The template is used to perform five iterations of erosion on the edge-repaired photovoltaic array image to extract the gaps between photovoltaic modules. Based on this, the midline of the extracted module gaps is used as the dividing line to divide the photovoltaic modules, thereby realizing the module segmentation of the photovoltaic array image.

7. The method for detecting strip and planar hot spots based on modular energy factor according to claim 4, characterized in that, The specific implementation method for obtaining the positions of strip-shaped and planar hot spots in the original infrared image by performing inverse transformation on the image coordinates of the hot spot module is as follows: Based on the minimum bounding rectangle parameters of the photovoltaic array region in the pre-correction photovoltaic array image, including the position, size, and rotation angle of the minimum bounding rectangle, the hot spot module set is inversely transformed using a geometric transformation method to obtain the set of position parameters of the hot spot module set in the original infrared image.

8. A system for implementing the method for detecting strip and planar hot spots based on modular energy factor as described in claim 1, characterized in that, The system includes: The photovoltaic array extraction subsystem is used to preprocess the original infrared image, extract the photovoltaic array region, and perform tilt correction and filtering to obtain a photovoltaic array image set. The photovoltaic module segmentation subsystem is used to segment the photovoltaic array image into modules, obtaining images of each photovoltaic module. The hot spot recognition subsystem is used to identify strip-shaped and planar hot spots in the images of each photovoltaic module, and to perform inverse transformation to obtain the position of the strip-shaped and planar hot spots in the original infrared image, thus realizing the final detection of strip-shaped and planar hot spots; The specific implementation method of the module energy factor is as follows: Based on the segmented photovoltaic modules, obtain the pixel grayscale matrix of the photovoltaic modules and stretch it into a column vector; calculate the grayscale truncation threshold using the photovoltaic array image where the photovoltaic modules are located, and set the elements not greater than the grayscale truncation threshold to zero; obtain the column vectors of the left and right neighboring modules of the current photovoltaic module, calculate the grayscale energy difference between the current module and the two neighboring modules respectively, and take the minimum value as the module energy factor of the current module. The grayscale truncation threshold is: ； in This indicates the number of pixels in the photovoltaic array image. This represents the set of pixels in a photovoltaic array image. Represents any pixel in a photovoltaic array image. Represents the grayscale value of a pixel. Indicates the truncation compensation parameter; The grayscale energy difference is: ； in This represents the column vector of the current module. The column vector representing the neighborhood module, Representing vectors The One element, Representing vectors The One element, This indicates taking the maximum value. Representing vectors Length; The specific implementation method for identifying strip-shaped and planar hot spots is as follows: The photovoltaic module is binarized using a grayscale truncation threshold to obtain a binarized image of the photovoltaic module. Specifically, the binarization involves setting pixels with grayscale values ​​greater than the grayscale truncation threshold to 1 and other pixels to zero. The binarized image is then binarized and projected into the horizontal and vertical directions to obtain the projection histogram vectors of the photovoltaic module in the horizontal and vertical directions. Photovoltaic modules with energy factors greater than the photovoltaic module energy threshold are selected. Each selected photovoltaic module is then examined sequentially. Based on the projection distribution characteristics of the photovoltaic modules, photovoltaic modules with strip-shaped or planar hot spots are selected to form a hot spot module set.

Images