Photovoltaic panel defect detection classification method based on thermal infrared images

Abstract

The application discloses a photovoltaic panel defect detection and classification method based on thermal infrared images, and relates to the technical field of photovoltaic panel thermal infrared detection.The method comprises the following steps: S1: acquiring thermal infrared photos of photovoltaic power stations with longitude, latitude and attitude angle by using a thermal infrared camera carried by a drone; the application acquires photos with longitude, latitude and attitude angle information by using a thermal infrared camera carried by a drone, and generates images in combination with photogrammetry software, thereby getting rid of the dependence of electroluminescence detection on special illumination and external power supply, and enabling detection work to be carried out in a conventional environment, accurately positioning photovoltaic panels, overcoming the positioning difficulty problem caused by the large number of panels and repeated textures, and simultaneously, using a deep learning algorithm, accurately identifying various fault types, integrating the key information of fault panels and outputting the same, thereby providing detailed basis for operation and maintenance personnel, effectively guiding maintenance work, and improving the overall efficiency and accuracy of photovoltaic power station operation and maintenance, thereby guaranteeing the safe and efficient operation of photovoltaic power stations.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic panel thermal infrared detection technology, specifically a photovoltaic panel defect detection and classification method based on thermal infrared images. Background Technology

[0002] With the booming development of the photovoltaic industry and the continuous expansion of photovoltaic power stations, photovoltaic panels inevitably experience various faults during their operation. These faults not only reduce power generation and shorten battery life, but in severe cases, they can even lead to panel burnout, greatly affecting the safe and efficient operation of photovoltaic power stations. Therefore, timely and accurate detection of photovoltaic panel faults is crucial. Currently, photovoltaic panel fault detection methods are mainly divided into two categories: electrical signal-based detection and image-based detection. Image-based detection methods have received widespread attention due to the development of sensor and computer vision technologies. Common methods include detection using electroluminescence, visible light, and thermal infrared images. Electroluminescence detection involves applying a forward bias current to the photovoltaic panel to make it emit light, thereby detecting tiny internal defects. However, this method requires a dark room or nighttime conditions without light and requires an external power supply, which greatly limits its application. Maintenance personnel often use drones to take visible light photos for detection, but this method requires extremely high image resolution, which is not ideal for rapid inspection.

[0003] While existing thermal image-based detection methods can isolate faulty panels from thermal images, they have several shortcomings. On the one hand, they cannot effectively distinguish the defect types of faulty panels. On the other hand, even if the fault type is distinguished, it is often not clear enough to reflect the cause of the fault and cannot provide specific and effective solutions for operation and maintenance personnel. In addition, due to the large number of photovoltaic panels in photovoltaic power plants and their similar textures, it is extremely difficult to obtain the precise location of faulty panels from panoramic views or photos of the electric field, which brings great challenges to the operation and maintenance of photovoltaic power plants. Therefore, developing a photovoltaic panel defect detection and classification method based on thermal infrared images is of great significance. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a photovoltaic panel defect detection and classification method based on thermal infrared images. It can use a drone equipped with a thermal infrared camera to acquire photos with longitude, latitude, and attitude angle information and generate images. It can carry out inspection work in normal environments, accurately locate photovoltaic panels, and overcome the problem of difficult positioning caused by a large number of panels and repetitive textures. At the same time, it can accurately identify multiple fault types and integrate the key information output of faulty panels to provide detailed basis for operation and maintenance personnel and effectively guide maintenance work.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a photovoltaic panel defect detection and classification method based on thermal infrared images, the method comprising the following steps:

[0006] S1: Use a thermal infrared camera mounted on a drone to acquire thermal infrared photos of the photovoltaic power station with longitude, latitude, and attitude angle;

[0007] S2: Use photogrammetry software to process thermal infrared photos with longitude, latitude, and attitude angle to generate a thermal infrared orthophoto (TIRDOM) of the photovoltaic power station;

[0008] S3: Use a trained segmentation neural network to identify pixels in the TIRDOM that are photovoltaic panel arrays and generate a binarized image;

[0009] S4: Based on the GDAL module, the photovoltaic panels in TIRDOM are divided into individual photovoltaic panels, saved as image format and their geographic coordinates are stored;

[0010] S5: Use a trained classification neural network to detect faults in individual photovoltaic panels. If a fault exists, identify its type; otherwise, classify it as a normal working panel.

[0011] S6: Integrate fault panel information and output fault type and geographic coordinate information.

[0012] Furthermore, step S1 specifically includes the following steps:

[0013] S1.1: The weather was clear and cloudless when the thermal infrared image was taken, and the batteries in the photovoltaic power station were working normally and outputting current.

[0014] S1.2: The photos of the photovoltaic power station taken by the drone must have an overlap of no less than 60% in the flight direction and no less than 30% in the side direction, and the photos must be taken perpendicular to the ground.

[0015] Furthermore, in step S3, a trained segmentation neural network is used to identify pixels in the TIRDOM that are photovoltaic panel arrays to generate a binarized image. Specifically, a semantic segmentation network is used to remove the background information of the TIRDOM and segment out the photovoltaic panels. The specific method is as follows:

[0016] S3.1: Select some images from different photovoltaic power station images as training data, generate corresponding binary masks by manually vectorizing the boundaries of photovoltaic panels, and crop the original images to correspond with the masks to images of the same size with a certain degree of overlap as training data.

[0017] S3.2: Randomly input small batches of training data into the semantic segmentation neural network model for training. Given that panel segmentation is a binary classification semantic segmentation task, the loss function during network training is set as follows:

[0018] ,in The network's predicted value. For tags, This is a smoothing term used to prevent the denominator from being zero;

[0019] S3.3: Input the TIRDOM of the photovoltaic power station into the trained semantic segmentation neural network model. The model divides the pixels into two categories: photovoltaic panels and background environment, where the value of photovoltaic panels is 1 and the value of background environment is 0.

[0020] Furthermore, in step S4, the photovoltaic panels in TIRDOM are segmented into individual photovoltaic panels based on the GDAL module, stored as image formats, and their geographic coordinates are stored. The specific steps of the single-panel image segmentation and coordinate extraction algorithm based on the GDAL library are as follows:

[0021] S4.1: Convert the binarized image predicted by the neural network in step S3.3 into a vector graphic of the photovoltaic panel;

[0022] S4.2: An algorithm based on the GDAL library is written to input vectorized graphics of photovoltaic panels and TIRDOM of photovoltaic power plants into the program, and output a single photovoltaic panel image with geographic coordinates;

[0023] S4.3: An algorithm based on the GDAL library is used to extract the center geographic coordinates of a single photovoltaic panel, realizing the panel positioning function. The formula for calculating the center coordinates of the panel is: , ,in The coordinates of the center point of the photovoltaic panel image. Image of a photovoltaic panel Minimum value in coordinate direction Image of a photovoltaic panel Maximum value in coordinate direction Image of a photovoltaic panel Minimum value in coordinate direction Image of a photovoltaic panel Maximum value in coordinate direction.

[0024] Furthermore, the algorithm steps for classifying photovoltaic panel faults using a classification neural network in step S5 are as follows:

[0025] S5.1: Create a classification dataset, and label the segmented individual battery panels as 6 defect types and 1 normal type, including junction box overheating, sub-string open circuit, debris obstruction, panel breakage, dust, ordinary hot spot, and normal panel. Set the image size of the photovoltaic panel to 64×64 pixels.

[0026] S5.2: During model training, the classification dataset is input into the neural network to train the model, and the cross-entropy function, which is used in multi-class classification tasks, is selected as the loss function.

[0027] The formula is: ,in Indicates the number of categories. For the sign function, the sample The true category equals The value is 1 if it is true, and 0 otherwise. For the sample Category The probability is determined by a learning rate decay strategy during training, which gradually reduces the learning rate as the number of training rounds increases.

[0028] S5.3: During model prediction, a single photovoltaic panel image is input into the trained classification neural network model to determine the working status of the photovoltaic panel, which is divided into normal operation and various fault types.

[0029] Furthermore, step S6 integrates faulty photovoltaic panel information, including photovoltaic panel serial number, geographical coordinates, normal working status, faulty working status, fault type, and fault area mask.

[0030] Furthermore, based on the above method, a photovoltaic panel fault location and detection system based on thermal infrared images is adopted. This system includes:

[0031] The image acquisition module is used to acquire thermal infrared images of photovoltaic power plants with longitude, latitude, and attitude angle.

[0032] The photogrammetry module is used to perform aerial triangulation on thermal infrared photographs of photovoltaic power plants with longitude, latitude, and attitude angle to generate TIRDOMs of photovoltaic power plants;

[0033] The panel recognition module is used by the semantic segmentation neural network to identify pixels belonging to photovoltaic panels in the TIRDOM of a photovoltaic power station, and to separate the photovoltaic panels from the background.

[0034] The single panel segmentation and positioning module is used to segment the photovoltaic panel portion of the entire photovoltaic power plant's TIRDOM into individual panel images and locate the geographic coordinates of each individual photovoltaic panel.

[0035] The fault detection and classification module is used to classify individual photovoltaic panel images using a classification neural network to identify their panel operating status and fault type;

[0036] The data integration module is used to integrate the output results, combining photovoltaic panel serial numbers, geographic coordinates, and fault type information.

[0037] Furthermore, the image acquisition module includes an image storage unit for storing thermal infrared photos of the photovoltaic power station with longitude, latitude, and attitude angle acquired by the thermal infrared camera mounted on the UAV in a local storage device and a cloud storage server. The stored image data is encrypted using an encryption algorithm. The image acquisition module has an automatic calibration function to automatically calibrate the parameters of the thermal infrared camera before shooting.

[0038] Furthermore, when performing aerial triangulation, the photogrammetry module automatically adjusts the measurement parameters, including adjusting the distribution and number of image control points, based on the different terrains of the photovoltaic power station and the layout characteristics of the photovoltaic panels.

[0039] Compared with existing technologies, this photovoltaic panel defect detection and classification method based on thermal infrared images has the following advantages:

[0040] I. This invention utilizes a drone equipped with a thermal infrared camera to acquire photos containing longitude, latitude, and attitude angle information, and combines this with photogrammetry software to generate images. This eliminates the dependence of electroluminescence detection on special lighting and external power supplies, allowing detection work to be carried out in normal environments. Employing deep learning algorithms, it accurately locates photovoltaic panels, overcoming the difficulty in locating panels due to their large number and repetitive textures. Furthermore, it can accurately identify various fault types and integrate key information from faulty panels, providing detailed guidance for maintenance personnel and effectively directing repair work. This improves the overall efficiency and accuracy of photovoltaic power plant operation and maintenance, ensuring the safe and efficient operation of photovoltaic power plants.

[0041] Second, this invention identifies photovoltaic panel pixels in TIRDOM by training a segmentation neural network, segments the panels and extracts their geographic coordinates, achieving accurate positioning of numerous photovoltaic panels with repetitive textures. This solves the problem of difficulty in obtaining the specific location of faulty panels from panoramic images or photos. By training a classification neural network to detect faults and identify their types, and integrating information such as the faulty panel's serial number, coordinates, and fault type, the invention accurately distinguishes multiple fault types, providing detailed fault information for maintenance personnel, effectively guiding the formulation of maintenance measures, and improving the efficiency and accuracy of photovoltaic power plant operation and maintenance.

[0042] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0044] Figure 1 This is a flowchart of a photovoltaic panel defect detection and classification method based on thermal infrared images;

[0045] Figure 2 This is a schematic diagram of a photovoltaic panel defect detection and classification system based on thermal infrared images. Detailed Implementation

[0046] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0047] Example 1

[0048] In a large photovoltaic power station located on the edge of a desert, frequent photovoltaic panel failures occur due to long-term wind and sand erosion and high-temperature environment. To promptly detect and address these failures and ensure the stable power generation efficiency of the station, see [reference needed]. Figure 1 and Figure 2 The power station has decided to adopt a photovoltaic panel defect detection and classification method based on thermal infrared images for daily inspections. The power station covers a large area, with a large number of photovoltaic panels and a complex layout. Traditional manual inspection methods are inefficient and difficult to accurately locate faults. Therefore, there is an urgent need for an efficient and accurate detection technology.

[0049] A professional drone equipped with a high-precision thermal infrared camera was selected for filming. On clear, cloudless days, the normal operation and current output of the cells in the photovoltaic power station were ensured to obtain thermal infrared images that accurately reflect the working status of the panels. According to the filming requirements, the drone's forward overlap was set to 65% and the lateral overlap to 35%. During the flight, the camera was kept perpendicular to the ground. During the flight, the thermal infrared camera on the drone continuously acquired thermal infrared photos with longitude, latitude, and attitude angle. The photos were stored on local storage devices and cloud storage servers using encryption algorithms through the image storage unit in the image acquisition module for subsequent processing and analysis. Before each filming, the automatic calibration function of the image acquisition module automatically calibrated the parameters of the thermal infrared camera to ensure that the quality of the captured thermal infrared photos met the standards.

[0050] Photogrammetric software was used to process the acquired thermal infrared images with longitude, latitude, and attitude angle. During aerial triangulation, the photogrammetric module automatically adjusted the measurement parameters based on the desert terrain and irregular layout of the photovoltaic power station panels, reasonably increasing the number of control points and optimizing their distribution. This improved the accuracy and quality of the generated thermal infrared orthophoto map (TIRDOM). After processing, a clear and accurate TIRDOM for the photovoltaic power station was successfully generated, laying the foundation for subsequent panel identification and fault detection.

[0051] To ensure the training data is widely representative, training data was selected from images of photovoltaic power plants from various terrains (including mountains, plains, and hills), different climate conditions (high temperature, low temperature, high humidity, and low humidity), and different manufacturers and models. During the data selection process, the impact of different environmental factors on the thermal infrared imaging of photovoltaic panels was fully considered to ensure that all possible situations were covered. For example, the heating characteristics of photovoltaic panels may differ from those in normal temperature environments, and photovoltaic panels produced by different manufacturers also have different thermal infrared radiation characteristics due to differences in materials and processes.

[0052] For the selected images, the boundaries of the photovoltaic panels are carefully vectorized manually using professional image editing software to generate corresponding binary masks. Then, the original images are precisely matched with the masks, and the images are cropped into images of 256×256 pixels each with an overlap of 40% as training data. This cropping method can ensure the diversity of training data and ensure that each image contains sufficient panel feature information, which is beneficial to improving the training effect of the neural network model.

[0053] The training data is randomly input into a small batch to train the semantic segmentation neural network model. Since panel segmentation is a binary classification semantic segmentation task, the loss function is set as follows during network training:

[0054] ,in This is the network's predicted value (output value). For tags, To smooth out the terms and prevent the denominator from being zero, a dynamic learning rate adjustment strategy was adopted during training. The learning rate was automatically adjusted based on the number of training rounds and the model's convergence. Initially, a larger learning rate was set to accelerate the model's convergence speed. As training progressed and the model's convergence speed gradually slowed down, the learning rate was gradually reduced to prevent the model from oscillating around the optimal solution, thereby improving the model's training accuracy. After hundreds of rounds of training and optimization, the model's performance gradually stabilized, and the accuracy of identifying photovoltaic panels and the background environment reached a high level.

[0055] After training, the TIRDOM of the photovoltaic power station is input into the trained semantic segmentation neural network model. The model analyzes and judges each pixel in the image, accurately classifying it into two categories: photovoltaic panel (value 1) and background environment (value 0), thus obtaining a clear binarized image. Through this process, the photovoltaic panel and the background are accurately separated, providing a clear image foundation for subsequent panel segmentation and fault detection.

[0056] The photovoltaic power station's TIRDOM is input into a trained semantic segmentation neural network model. The model divides the pixels into two categories: photovoltaic panels (value 1) and background environment (value 0), successfully generating a binarized image and achieving effective separation between the photovoltaic panels and the background environment.

[0057] Image processing algorithms are used to convert the binary image predicted by the neural network into a vector graphic of the photovoltaic panel. This conversion process uses edge detection, contour extraction and other techniques to convert the boundaries of the photovoltaic panel in the binary image into vector form, which facilitates subsequent accurate processing and analysis.

[0058] Leveraging the powerful geospatial data processing capabilities of the GDAL library, a specialized algorithm was developed. This algorithm simultaneously inputs the generated vectorized graphics of photovoltaic panels and the TIRDOM (Telephoto Mapping Rendering) of the photovoltaic power station into the program. After a series of calculations and processing steps, it outputs individual photovoltaic panel images with precise geographic coordinates. These images not only contain the shape and texture information of the photovoltaic panels but also accurately label their location coordinates in actual geographic space.

[0059] To achieve precise panel positioning, an algorithm was rewritten based on the GDAL library to extract the center geographic coordinates of a single photovoltaic panel, according to the formula... , ,in The coordinates of the center point of the photovoltaic panel image. Image of a photovoltaic panel Minimum value in coordinate direction Image of a photovoltaic panel Maximum value in coordinate direction Image of a photovoltaic panel Minimum value in coordinate direction Image of a photovoltaic panel The maximum value in the coordinate direction is used to accurately calculate the center coordinates of each photovoltaic panel, thus completing the panel positioning work.

[0060] A classification dataset was created, and the segmented individual battery panels were labeled as six defect types and one normal type: junction box overheating, sub-string open circuit, debris obstruction, panel breakage, dust, ordinary hot spot, and normal panel. Considering that most photovoltaic panels after TIRDOM segmentation in photovoltaic power plants have a resolution of no more than 70 pixels, the image size of the photovoltaic panels was uniformly set to 64×64 pixels.

[0061] During model training, the classification dataset is input into the neural network to train the model. The loss function used is the cross-entropy function from multi-class classification tasks, and the formula is: ,in Indicates the number of categories. For the sign function, the sample The true category equals The value is 1 if it is true, and 0 otherwise. For the sample Category The probability is determined by the learning rate decay strategy during training. As the number of training rounds increases, the learning rate is gradually reduced to improve the convergence speed and stability of the model.

[0062] During model prediction, a single photovoltaic panel image is input into a trained classification neural network model to determine the working status of the photovoltaic panel and accurately classify it into normal operation and various fault types.

[0063] The data integration module integrates the detected faulty panel information, including the photovoltaic panel serial number, geographical coordinates, normal working status, faulty working status, fault type, and fault area mask. This information is then compiled into a detailed table, clearly presenting the specific situation of each faulty panel, making it convenient for maintenance personnel to view and use.

[0064] In summary, this implementation successfully applied a photovoltaic panel defect detection and classification method based on thermal infrared imaging to this large-scale desert photovoltaic power station. During the actual inspection process, faulty photovoltaic panels were quickly and accurately located with a high level of accuracy, effectively solving the location challenges caused by the large number and complex layout of photovoltaic panels. Furthermore, it accurately identified various fault types, such as junction box overheating and panel breakage, providing maintenance personnel with detailed fault information to help them develop targeted repair plans in a timely manner. This significantly improved the operation and maintenance efficiency of the photovoltaic power station. Compared with traditional detection methods, it saved substantial manpower and time costs, ensured stable power generation, and improved the economic benefits and reliability of the power station.

[0065] Example 2

[0066] There is a photovoltaic power station in the mountains, built on a terrain with significant undulations. Due to the complex terrain, manual inspection is not only difficult but also inefficient. Coupled with variable weather conditions and frequent natural disasters, the risk of photovoltaic panel failure is high. To ensure the stable operation of the power station, see [reference needed]. Figure 1 and Figure 2 Therefore, it was decided to adopt a photovoltaic panel defect detection and classification method based on thermal infrared images for monitoring.

[0067] A drone adapted to complex environments was selected, equipped with a high-resolution thermal infrared camera. The shooting was conducted on a clear, cloudless day when the photovoltaic power station batteries were operating normally and outputting current. Based on the characteristics of the mountainous terrain, the drone's forward overlap was set to 70%, and the lateral overlap was set to 32% to ensure that the photos could completely cover the photovoltaic area. During shooting, the camera was kept perpendicular to the ground, and the flight altitude was adjusted according to changes in the terrain. After the drone acquired thermal infrared photos with longitude, latitude, and attitude angle information, it used the image storage unit of the image acquisition module to store the photos on local storage devices and cloud storage servers using an encryption algorithm. Before shooting, the automatic calibration function of the image acquisition module calibrated the parameters of the thermal infrared camera to ensure the quality of the shooting.

[0068] Professional photogrammetry software was used to process thermal infrared photos with longitude, latitude, and attitude angle. Due to the complex mountainous terrain, the photogrammetry module intelligently adjusted the measurement parameters based on the terrain undulations and photovoltaic panel distribution during aerial triangulation. It also increased the density of image control points in areas with large terrain changes and increased the number of image control points to generate high-precision thermal infrared orthophotos (TIRDOM), providing accurate basic data for subsequent analysis.

[0069] Training data was selected from images of photovoltaic power stations in various terrains such as mountains and plains, taking into account images under different lighting, temperature and humidity conditions, as well as images of photovoltaic panels from different manufacturers and models. The boundaries of the photovoltaic panels were manually vectorized to generate a binary mask. The original images were matched with the mask and cropped into 256×256 pixel images with an overlap of 40% as training data.

[0070] The training data is randomly input into a small batch to train the semantic segmentation neural network model, and the loss function is set to... After multiple rounds of training and optimization, the model performance is stable. When the TIRDOM of the photovoltaic power station is input into the trained model, the pixels are divided into two categories: photovoltaic panels (value 1) and background environment (value 0), resulting in a clear binary image and achieving accurate separation of photovoltaic panels from the background.

[0071] The binary image predicted by the neural network is converted into a vector graphic of the photovoltaic panel. An algorithm based on the GDAL library is written to input the vector graphics and TIRDOM into the program, and output a single photovoltaic panel image with geographic coordinates. The algorithm is then used according to the formula... , The center coordinates of the panel are calculated to achieve precise panel positioning, overcoming the difficulty of positioning photovoltaic panels in mountainous areas.

[0072] Create a classification dataset and label the segmented individual battery panels as seven types: junction box overheating, sub-string open circuit, debris obstruction, panel breakage, dust, ordinary hot spot, and normal panel. Set the image size to 64×64 pixels.

[0073] During model training, the dataset is input into the neural network, and the cross-entropy function is selected. As the loss function, a learning rate decay strategy is adopted during training. After the model training is completed, a single photovoltaic panel image is input to determine whether its working status is normal or faulty, and to determine the fault type.

[0074] The data integration module collects information such as the serial number, geographical coordinates, working status, fault type, and fault area mask of the fault panel, and outputs it in tabular form. Maintenance personnel can quickly obtain detailed information about the fault panel through the table, providing strong support for maintenance decisions.

[0075] In summary, after implementing this method at the mountain photovoltaic power station, the inspection challenges caused by the complex terrain were successfully solved. The faulty photovoltaic panels were quickly located with positioning errors within acceptable limits, significantly improving positioning efficiency. Various faults were accurately identified with high classification accuracy, enabling maintenance personnel to quickly develop repair plans, reducing the impact of faults on power generation. Overall, this reduced maintenance costs, improved the power generation stability and economic benefits of the photovoltaic power station, and effectively ensured the reliable operation of the mountain photovoltaic power station.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. Photovoltaic panel defect detection classification method based on thermal infrared images, characterized in that, The method comprises the following steps: S1: acquiring a thermal infrared photo of a photovoltaic power station with longitude, latitude and attitude angle by using a thermal infrared camera carried by a UAV; S2: processing the thermal infrared photo with longitude, latitude and attitude angle by using photogrammetry software to generate a thermal infrared orthographic image TIRDOM of the photovoltaic power station; S3: identifying the pixels in the TIRDOM as photovoltaic panel arrays by using a segmented neural network trained, and dividing the pixels into two categories of photovoltaic panels and background environment, wherein the value of the photovoltaic panel is 1 and the value of the background environment is 0, and generating a binary image; S4: segmenting the photovoltaic panels in the TIRDOM into single photovoltaic panels based on a GDAL module, storing the single photovoltaic panels in an image format and storing the geographic coordinates thereof; S5: detecting faults in the single photovoltaic panels by using a classification neural network trained, and identifying the fault types if there are faults, or classifying the single photovoltaic panels as normal panels if there are no faults; S6: integrating the fault panel information and outputting the fault types and geographic coordinate information.

2. The method for photovoltaic panel defect detection classification based on thermal infrared images according to claim 1, characterized in that, The step S1 specifically comprises the following steps: S1.1: the thermal infrared image is taken when the weather is sunny and cloudless, and the cells in the photovoltaic power station are normally working and outputting current; S1.2: the taken photo of the photovoltaic power station is taken vertically to the ground with an overlap degree of no less than 60% in the UAV heading direction and no less than 30% in the lateral direction.

3. The method for photovoltaic panel defect detection classification based on thermal infrared images according to claim 1, characterized in that, In the step S3, the segmented neural network trained is used to identify the pixels in the TIRDOM as photovoltaic panel arrays to generate a binary image, specifically, a semantic segmentation network is used to remove the background information of the TIRDOM and segment out the photovoltaic panels, and the specific method is as follows: S3.1: selecting part of the images from different photovoltaic power station images as training data, generating corresponding binary masks by artificially vectorizing the boundaries of the photovoltaic panels, corresponding the original images with the masks, and cutting the images into images with consistent size and a certain overlap degree as training data; S3.2: Randomly input the training data in small batches into the semantic segmentation neural network model for training. Since the panel segmentation belongs to the binary classification semantic segmentation task, the loss function is set as follows during network training: wherein is the predicted value of the network, is the label, is a smoothing term for preventing the denominator from being 0; S3.3: inputting the TIRDOM of the photovoltaic power station into the trained semantic segmentation neural network model, and the model divides the pixels into two categories of photovoltaic panels and background environment, wherein the value of the photovoltaic panel is 1 and the value of the background environment is 0.

4. The method for photovoltaic panel defect detection classification based on thermal infrared images according to claim 3, characterized in that, In the step S4, the photovoltaic panels in the TIRDOM are segmented into single photovoltaic panels based on the GDAL module, and the single panel images are stored in an image format and the geographic coordinates thereof are stored, and the specific algorithm steps based on the GDAL library for single panel image segmentation and coordinate extraction are as follows: S4.1: converting the binary image predicted by the neural network in step S3.3 into a photovoltaic panel vector graphic; S4.2: based on the GDAL library, writing an algorithm, inputting the photovoltaic panel vector graphic and the TIRDOM of the photovoltaic power station into the program, and outputting single photovoltaic panel images with geographic coordinates; S4.3: Based on GDAL library algorithm, the center geographic coordinates of a single photovoltaic panel are extracted, the panel positioning function is realized, and the formula for calculating the panel center coordinates is: , , wherein is the center point coordinate of the photovoltaic panel image, is the minimum value of the coordinate direction of the photovoltaic panel image , is the maximum value of the coordinate direction of the photovoltaic panel image , is the minimum value of the coordinate direction of the photovoltaic panel image , is the maximum value of the coordinate direction of the photovoltaic panel image .

5. The method for photovoltaic panel defect detection classification based on thermal infrared images according to claim 1, characterized in that, In the step S5, the algorithm steps for classifying the photovoltaic panel faults by using the classification neural network are as follows: S5.1: making a classification data set, marking the segmented single cell panel as 6 defect types of junction box heating, sub-string open circuit, foreign matter shielding, panel breakage, dust, ordinary hot spot and 1 normal type, and setting the image size of the photovoltaic panel to 64x64 pixels; S5.2: During model training, the classification data set is input into the neural network training model, and the cross-entropy function in the multi-classification task is selected as the loss function, and the formula is: wherein represents the number of categories, is a symbol function, and the true category of sample is equal to , 1 otherwise 0, is the probability that sample belongs to category , and the training process adopts a learning rate decay strategy, and the learning rate is gradually reduced with the increase of the training round number; S5.3: When the model is predicted, the single photovoltaic panel image is input into the trained classification neural network model to determine the working condition of the photovoltaic panel, and is divided into normal working and various fault types.

6. The thermal infrared image based photovoltaic panel defect detection classification method of claim 1, wherein, The step S6 integrates the fault photovoltaic panel information, including the photovoltaic panel serial number, geographic coordinates, normal working state, fault working state, fault type and fault area mask.

7. The method of claim 1-6, a photovoltaic panel fault localization and detection system based on thermal infrared images is used, characterized in that, The system comprises: An image acquisition module for acquiring a photovoltaic power station thermal infrared photo with longitude, latitude and attitude angle; A photogrammetry module for performing aerial triangulation on the photovoltaic power station thermal infrared photo with longitude, latitude and attitude angle to generate a photovoltaic power station TIRDOM; A panel recognition module for a semantic segmentation neural network to recognize pixels belonging to photovoltaic panels in the photovoltaic power station TIRDOM, separating photovoltaic panels from the background; A single panel segmentation and positioning module for segmenting the photovoltaic panel part in the TIRDOM of the entire photovoltaic power station into single panel images and positioning the geographic coordinates of the single photovoltaic panels; A fault detection and classification module for a classification neural network to classify single photovoltaic panel images to identify their panel working state and fault type; A data integration module for integrating output results, integrating photovoltaic panel serial numbers, geographic coordinates and fault type information.

8. The method for photovoltaic panel defect detection classification based on thermal infrared images according to claim 7, characterized in that, The image acquisition module comprises an image storage unit for storing photovoltaic power station thermal infrared photos with longitude, latitude and attitude angle acquired by a thermal infrared camera carried by a UAV in a local storage device and a cloud storage server, and the stored image data is encrypted using an encryption algorithm, and the image acquisition module has an automatic calibration function to automatically calibrate the parameters of the thermal infrared camera before shooting.

9. The method for photovoltaic panel defect detection classification based on thermal infrared images according to claim 7, characterized in that, When performing aerial triangulation, the photogrammetry module automatically adjusts the measurement parameters according to the different terrain of the photovoltaic power station and the layout characteristics of the photovoltaic panels, including adjusting the distribution and number of image control points.

Images