Digital method for unmanned aerial vehicle automatic inspection and positioning of photovoltaic panel defects

Abstract

This invention provides a digital method for automatic inspection and location of defects in photovoltaic (PV) panels using unmanned aerial vehicles (UAVs). The method includes: determining the UAV inspection shooting point and the geographical location index of each PV panel within the shooting area of ​​the shooting point on a preset PV digital map; acquiring UAV inspection infrared images based on the shooting point; processing the UAV inspection infrared images using a target defect detection algorithm and infrared relative temperature difference method to determine the PV area status and infrared thermal defect type within the shooting area corresponding to the shooting point; performing image segmentation processing on the UAV inspection infrared images to determine the mapping from image pixel coordinates to geographical latitude and longitude coordinates for each PV panel; mapping the geographical latitude and longitude coordinates of each PV panel onto the preset PV digital map, thereby marking the operating status of each PV panel on the PV digital map. Through the above technical solution, digital management of PV power plants is achieved, and defective PV panels can be accurately located.

Description

Technical Field

[0001] This invention relates to the field of drone inspection of photovoltaic power plants, and more specifically, to a digital method for automatic drone inspection and location of defects in photovoltaic panels. Background Technology

[0002] In recent years, the number of newly installed photovoltaic (PV) capacity has been increasing. Common PV power plants include mountain and hilly PV power plants, desert and Gobi PV power plants, solar-integrated photovoltaic (SPV), agricultural-solar-integrated photovoltaic (ASPV), hydro-solar-integrated photovoltaic (HPV) power plants, distributed PV power plants, and offshore PV. For the daily maintenance of PV power plants, it is difficult for PV maintenance personnel to reach them, resulting in low maintenance efficiency and high costs.

[0003] With the maturity of drone technology, photovoltaic inspection is no longer limited to manual operation. However, in reality, there is no standardized inspection plan for photovoltaic power stations. Trees in the environment and high-voltage transmission lines passing over photovoltaic power stations can affect the flight of drones. In addition, the area that drones can inspect is limited, which affects the inspection efficiency. As the service life of photovoltaics increases, there is an increasingly urgent need for the diagnosis and management of defects in photovoltaic panels. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a digital method for automatic inspection and location of defects in photovoltaic panels by unmanned aerial vehicles (UAVs).

[0005] According to one aspect of the present invention, a digital method for automatic inspection and location of defects in photovoltaic panels using unmanned aerial vehicles (UAVs) is provided.

[0006] Optionally, on a preset photovoltaic digital map, based on a preset drone inspection altitude, the drone inspection shooting point and the geographical location index of each photovoltaic panel within the shooting area where the shooting point is located are determined;

[0007] Based on the shooting points of the drone inspection, obtain the drone inspection infrared images;

[0008] The infrared images of the UAV inspection are processed according to the target defect detection algorithm and the infrared relative temperature difference method to determine the photovoltaic area status and infrared thermal defect type within the shooting area corresponding to the shooting point.

[0009] The infrared images from the UAV inspection are segmented to determine the mapping of image pixel coordinates to geographical latitude and longitude coordinates for each photovoltaic panel.

[0010] The geographical location coordinates of each photovoltaic panel are mapped to the preset photovoltaic digital map to locate photovoltaic panels with infrared thermal defects, determine the operating status of each photovoltaic panel, and then mark the operating status of each photovoltaic panel on the photovoltaic digital map for digital management of each photovoltaic panel.

[0011] Optionally, determining the drone inspection shooting point and the geographical location index of each photovoltaic panel within the shooting area of ​​the preset drone inspection point on the preset photovoltaic digital map, based on the preset drone inspection altitude, includes:

[0012] By continuously capturing images of the target photovoltaic area using drones, an orthophoto map of the target photovoltaic area is determined. Then, using map creation tools, a photovoltaic digital map of the target photovoltaic area is created based on the orthophoto map, and a geographical location index is established for each photovoltaic panel in the photovoltaic digital map.

[0013] Based on the preset drone inspection height, determine the drone's shooting area under orthographic projection;

[0014] Based on the length, width, and tilt angle of the photovoltaic panel, determine the row and column spacing between the shooting points and the shooting points within the shooting area during the drone inspection.

[0015] The photovoltaic digital map is traversed according to the row interval and the column interval to determine the latitude and longitude coordinates of all shooting points inspected by the drone.

[0016] Based on all the shooting points inspected by the drone and the photovoltaic digital map, the geographical location index corresponding to each photovoltaic panel within the shooting area where the shooting point is located is determined.

[0017] Optionally, determining the drone's shooting area under orthographic projection based on the preset drone inspection height includes:

[0018]

[0019]

[0020] Where F represents the camera's focal length, x c y represents the length of the camera sensor pixel. c H represents the width of the camera sensor pixel, H represents the preset drone inspection height, shotwidth represents the length of the shooting area, and shotheight represents the width of the shooting area.

[0021] Optionally, determining the row and column spacing between the shooting points and shooting points within the shooting area during the drone inspection, based on the length, width, and tilt angle of the photovoltaic panel, includes:

[0022] projheight=pvheight×cosθ

[0023] interheight=interesting(2projheight+hallway)

[0024]

[0025]

[0026] Where pvheight represents the length of the photovoltaic panel, pvwidth represents the width of the photovoltaic panel, θ represents the tilt angle of the photovoltaic panel installation, projheight represents the tilted projection length of the photovoltaic panel, hallway represents the channel spacing between rows in the photovoltaic panel array, interheight represents the row spacing where the tilted projection length of the photovoltaic panel and the channel spacing overlap, laprate represents the overlap rate of the target photovoltaic area being photographed, and interval... row This indicates the row interval for the drone to inspect and photograph the target photovoltaic area; interval col This indicates the column spacing of the target photovoltaic area inspected and photographed by the drone.

[0027] Optionally, the step of processing the UAV inspection infrared image according to the target defect detection algorithm and the infrared relative temperature difference method to determine the photovoltaic area status and infrared thermal defect type within the shooting area corresponding to the shooting point includes:

[0028] Acquire training images of UAV inspection at various altitudes, establish labels according to the types of infrared thermal defects, train the target defect detection algorithm, and determine the infrared photovoltaic defect prediction model. The types of infrared thermal defects include blocky hot spots, linear hot spots, and short circuits.

[0029] The infrared images from the UAV inspection are input into the infrared photovoltaic defect prediction model to determine the predicted photovoltaic defect area and the infrared thermal defect type corresponding to the predicted photovoltaic defect area.

[0030] The predicted photovoltaic defect area is detected by the infrared relative temperature difference method, and the state of the predicted photovoltaic defect area is determined. The state of the predicted photovoltaic defect area includes photovoltaic defect state, photovoltaic early warning state, and photovoltaic normal state.

[0031] Optionally, the step of detecting the predicted photovoltaic defect region based on the infrared relative temperature difference method and determining the state of the predicted photovoltaic defect region includes:

[0032] According to the OTSU (Taiwanese Federal Law), the predicted photovoltaic defect region is divided into inner and outer regions of the binarized temperature region.

[0033] The highest infrared temperature of the inner region of the binarized temperature region, the average infrared temperature of the inner region of the binarized temperature region, and the average infrared temperature of the outer region of the binarized temperature region are obtained.

[0034] If the highest infrared temperature in the inner region of the binarized temperature region is greater than the first preset temperature, the state of the predicted photovoltaic defect region is determined to be a photovoltaic defect state.

[0035] If the temperature difference is greater than the third preset temperature and less than the second preset temperature, the state of the predicted photovoltaic defect area is determined to be a photovoltaic early warning state.

[0036] If the temperature difference is less than the third preset temperature, the state of the predicted photovoltaic defect area is determined to be the normal photovoltaic state.

[0037] Optionally, the image pixel coordinates of the photovoltaic panel include the image center point coordinates of the photovoltaic panel and the latitude and longitude coordinates of the image position of the photovoltaic panel;

[0038] The step of performing image segmentation processing on the infrared images inspected by the UAV to determine the mapping from the image pixel coordinates of each photovoltaic panel to the latitude and longitude coordinates of its geographical location includes:

[0039] The infrared images inspected by the UAV are processed by image segmentation to determine the area where each photovoltaic panel is located, the outline of the area where each photovoltaic panel is located is extracted, and the coordinates of the image center point of each photovoltaic panel are determined based on the outline of the area where each photovoltaic panel is located.

[0040] The geographical location coordinates of each photovoltaic panel are determined based on the image center point coordinates of each photovoltaic panel and the image location latitude and longitude coordinates of each photovoltaic panel carried on the UAV inspection infrared image.

[0041] Optionally, determining the geographical location coordinates of each photovoltaic panel based on the image center point coordinates of each photovoltaic panel and the image location coordinates of each photovoltaic panel carried on the UAV inspection infrared image includes:

[0042] Lat convert =0.000008×x c / imagewidth×H / F

[0043] Lon convert =0.000009×y c / imageheight×H / F

[0044]

[0045]

[0046] Where imagewidth represents the pixel width of the infrared image inspected by the drone, imageheight represents the pixel height of the infrared image inspected by the drone, and Lat convert The longitude coefficient representing the latitude and longitude coordinates of a geographical location. convert The values ​​represent the latitude coefficients of the geographical location's latitude and longitude coordinates, ImageLatitude represents the longitude of the photovoltaic panel's image location's latitude and longitude coordinates, ImageLongitude represents the latitude of the photovoltaic panel's image location's latitude and longitude coordinates, xcenter represents the x-coordinate of the center point of the outer rectangle of the photovoltaic panel, ycenter represents the y-coordinate of the center point of the outer rectangle of the photovoltaic panel, panelLatitude represents the longitude value of the photovoltaic panel's geographical location's latitude and longitude coordinates, and panelLongitude represents the latitude value of the photovoltaic panel's geographical location's latitude and longitude coordinates.

[0047] Optionally, determining the operating status of each photovoltaic panel includes:

[0048] If the overlap area between the photovoltaic area within the shooting area corresponding to the shooting point and each photovoltaic panel in the UAV inspection infrared image after image segmentation is greater than a preset area, the photovoltaic area state of the overlap area represents the state of the photovoltaic panel within the overlap area, and the infrared thermal defect type of the overlap area represents the infrared thermal defect type of the photovoltaic panel within the overlap area.

[0049] Optionally, mapping the geographical location coordinates of each photovoltaic panel to the preset photovoltaic digital map, simultaneously locating photovoltaic panels with infrared thermal defects, determining the operating status of each photovoltaic panel, and then marking the operating status of each photovoltaic panel on the photovoltaic digital map for digital management of each photovoltaic panel, includes:

[0050] Traverse the UAV inspection infrared images, and search for the minimum distance based on the spatial latitude and longitude information of each photovoltaic panel segmented from the UAV inspection infrared images and the latitude and longitude coordinates of the image center point of each photovoltaic panel on the photovoltaic digital map;

[0051] If the distance is less than or equal to a preset distance, the latitude and longitude coordinates of the geographical location of the photovoltaic panel corresponding to the UAV inspection infrared image and the photovoltaic digital map are mapped onto the preset photovoltaic digital map, and the operating status of each photovoltaic panel is marked on the photovoltaic digital map for digital management.

[0052] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:

[0053] The above technical solutions can effectively control the flight altitude of drones and automatically generate autonomous inspection trajectories. A deep learning-based target defect detection algorithm combined with the infrared relative temperature difference method can reliably detect infrared thermal defects in photovoltaic panels. This automatically segments the infrared images inspected by the drone, determining the mapping from pixel coordinates of each photovoltaic panel image to its geographical location's latitude and longitude coordinates. Furthermore, it maps the geographical location's latitude and longitude coordinates of each photovoltaic panel in the drone's infrared inspection images onto a digital photovoltaic map, thereby achieving precise location of photovoltaic panels with infrared thermal defects. The diagnostic operating status of each photovoltaic panel is then marked on the photovoltaic digital map, enabling digital management of the photovoltaic power station. Attached Figure Description

[0054] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0055] Figure 1 This is a flowchart illustrating a digital method for automatically inspecting and locating defects in photovoltaic panels using a drone, according to an exemplary embodiment.

[0056] Figure 2 This is a flowchart illustrating a method for determining the shooting point of an unmanned aerial vehicle (UAV) inspection and the geographic location index of each photovoltaic panel within the shooting area where the shooting point is located, according to an exemplary embodiment.

[0057] Figure 3 This is a flowchart illustrating a method for determining the state of a photovoltaic region and the type of infrared thermal defects within a shooting area corresponding to a shooting point, according to an exemplary embodiment.

[0058] Figure 4 This is a flowchart illustrating a method for determining the state of the predicted photovoltaic defect region according to an exemplary embodiment. Detailed Implementation

[0059] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0060] In this disclosure, a DJI Mavic 2 Enterprise Edition drone is used. The infrared camera lens has a focal length of approximately F=9mm, an equivalent focal length of approximately 38mm, an image size of 640×512, and a camera sensor surface size of x. c ,y c The dimensions are 7.68mm × 6.144mm. The method disclosed herein is implemented in a floating photovoltaic power station in Zhejiang.

[0061] This disclosure uses Python software for verification and employs the Python SQLModel database management tool. According to photovoltaic power station construction rules: photovoltaic panels face due east, where the sun rises, with rows of panels connected together in a north-south direction; two rows of panels form a single row, with channels between rows.

[0062] Figure 1 This is a flowchart illustrating a digital method for automatically inspecting and locating defects in photovoltaic panels using a drone, according to an exemplary embodiment.

[0063] like Figure 1 As shown, a digital method for automatic inspection and location of defects in photovoltaic panels using unmanned aerial vehicles (UAVs) includes steps S11 to S15:

[0064] S11, on the preset photovoltaic digital map, based on the preset drone inspection altitude, determine the drone inspection shooting point and the geographical location index of each photovoltaic panel within the shooting area where the shooting point is located;

[0065] S12, based on the shooting points of the drone inspection, acquire the drone inspection infrared image;

[0066] S13, Process the UAV inspection infrared image according to the target defect detection algorithm and infrared relative temperature difference method to determine the photovoltaic area status and infrared thermal defect type in the shooting area corresponding to the shooting point;

[0067] S14, Perform image segmentation processing on the infrared images inspected by the drone to determine the mapping of image pixel coordinates to geographical latitude and longitude coordinates for each photovoltaic panel;

[0068] S15 maps the geographical location latitude and longitude coordinates of each photovoltaic panel to a preset photovoltaic digital map, locates photovoltaic panels with infrared thermal defects, determines the operating status of each photovoltaic panel, and then marks the operating status of each photovoltaic panel on the photovoltaic digital map to achieve digital management of each photovoltaic panel.

[0069] The above technical solutions can effectively control the flight altitude of drones and automatically generate autonomous inspection trajectories. Based on deep learning-based target defect detection algorithms and infrared relative temperature difference methods, infrared thermal defects of photovoltaic panels can be reliably detected. The infrared images of drone inspections can be automatically segmented, and the mapping of pixel coordinates of each photovoltaic panel image to geographical latitude and longitude coordinates can be determined. Furthermore, the geographical latitude and longitude coordinates of each photovoltaic panel on the infrared images of drone inspections can be mapped onto a digital photovoltaic map, thereby achieving precise positioning of photovoltaic panels with infrared thermal defects and marking the operating status of each photovoltaic panel on the photovoltaic digital map, realizing digital management of photovoltaic power plants.

[0070] S11, on the preset photovoltaic digital map, based on the preset drone inspection altitude, determine the drone inspection shooting point and the geographical location index of each photovoltaic panel within the shooting area where the shooting point is located.

[0071] Figure 2 This is a flowchart illustrating a method for determining the shooting point of an unmanned aerial vehicle (UAV) inspection and the geographic location index of each photovoltaic panel within the shooting area where the shooting point is located, according to an exemplary embodiment.

[0072] In some possible embodiments, such as Figure 2 As shown, S11 includes S21 to S25.

[0073] S21, continuously capture images of the target photovoltaic area using a drone, determine the orthophoto map of the target photovoltaic area, and use a map creation tool to create a photovoltaic digital map of the target photovoltaic area based on the orthophoto map, and establish a geographical location index for each photovoltaic panel in the photovoltaic digital map.

[0074] The pre-set photovoltaic digital map can be used in multiple inspections. A unique ID index is established for each photovoltaic panel on the photovoltaic digital map, which serves as the geographical location index for each photovoltaic panel.

[0075] A drone was used to model the photovoltaic area. DJI Pilot modeling software was used on a DJI drone to determine the digital map of the photovoltaic area, using the following method:

[0076] Identify the photovoltaic area to be modeled and generate the mapping area.

[0077] Obtain the modeling image:

[0078] In a flat-ground photovoltaic power station, 2D modeling is performed using drones. By capturing continuous images of the photovoltaic area, the continuous images are stitched together to obtain an orthophoto map.

[0079] In mountain photovoltaic or distributed photovoltaic power stations, 3D modeling is performed using drones. By controlling the drone to adjust the camera angle, repeated flight photography is carried out along the same flight path to determine oblique photography data. Oblique photography can preserve the depth information of the object being measured, and the height of the model can be measured after modeling.

[0080] Generate a panoramic photovoltaic image:

[0081] Using DJI's mapping software or the open-source Pix4Dmapper, a panoramic image of the photovoltaic area can be generated based on the acquired modeling image.

[0082] As an example, in UAV 2D modeling, images of the photovoltaic area are continuously captured and stitched together to obtain an orthophoto map. Then, QGIS mapping software is used to create a photovoltaic digital map from the orthophoto map.

[0083] One method is to use the line drawing tool in QGIS software to generate a photovoltaic digital map, with a unit of one photovoltaic panel and an accuracy of one centimeter.

[0084] A vector file (SHP) for a photovoltaic digital map is created using QGIS software. A single Shapefile includes: a main file (storing the geometry of geographic features); an index file (spatial data index file, storing the index of geometric features of the geographic data); a DBASE file (storing attribute data, also known as a table file or dbf file); a spatial projection file (storing spatial reference, i.e., the projection method file); and a geographic data index file (storing the index of geographic features).

[0085] Spatial projection files and geographic data index files exist when performing spatial relationship queries between topics, spatial joins between topics, and indexing the shape field.

[0086] The information of each photovoltaic panel in the photovoltaic digital map, including its ID index, row number, column number, and the coordinates of its left vertex, right vertex, top vertex, bottom vertex, and center point, is stored in a database named PVSdatabase.

[0087] The structure information of the PVSdatabase file is as follows:

[0088] id: Geographic location index of the photovoltaic panel;

[0089] row: Photovoltaic panel row number;

[0090] col: Column number of the photovoltaic panel;

[0091] left_lon: Longitude of the left vertex of the photovoltaic panel;

[0092] left_lat: Latitude of the left vertex of the photovoltaic panel;

[0093] top_lon: Longitude of the vertex on the photovoltaic panel;

[0094] top_lat: The latitude of the vertex on the photovoltaic panel;

[0095] right_lon: Longitude of the right vertex of the photovoltaic panel;

[0096] right_lat: Latitude of the right vertex of the photovoltaic panel;

[0097] bottom_lon: Longitude of the lower vertex of the photovoltaic panel;

[0098] bottom_lat: Latitude of the lower vertex of the photovoltaic panel;

[0099] center_lon: Longitude of the center point of the photovoltaic panel;

[0100] center_lat: Latitude of the center point of the photovoltaic panel.

[0101] By setting up a photovoltaic digital map, it is possible to associate the photovoltaic panels in the infrared images inspected by drones with the geographical location index of the photovoltaic digital map.

[0102] S22 determines the shooting area of ​​the drone under orthographic projection based on the preset drone inspection altitude.

[0103] When the drone is projected vertically downwards (with the gimbal pointing vertically downwards), the shooting area is determined using the following method:

[0104]

[0105]

[0106] Where F represents the camera's focal length, x c y represents the length of the camera sensor pixel. c H represents the width of the camera sensor pixel, H represents the preset drone inspection height, shotwidth represents the length of the shooting area, and shotheight represents the width of the shooting area.

[0107] As one example, the drone inspection height is 30 meters, and the shooting area is 25.6m × 20.48m. As another example, the drone inspection height is 15 meters, and the shooting area is 12.8m × 10.24m.

[0108] S23. Based on the length, width, and tilt angle of the photovoltaic panel, determine the row and column spacing between the shooting points and the shooting points within the shooting area during drone inspection.

[0109] In one possible embodiment, the row and column spacing between shooting points during inspection is determined in the following way:

[0110] projheight=pvheight×cosθ

[0111] interheight=interesting(2projheight+hallway)

[0112]

[0113]

[0114] Where pvheight represents the length of the photovoltaic panel, pvwidth represents the width of the photovoltaic panel, θ represents the tilt angle of the photovoltaic panel installation, projheight represents the tilted projection length of the photovoltaic panel, hallway represents the channel spacing between rows in the photovoltaic panel array, interheight represents the row spacing where the tilted projection length of the photovoltaic panel and the channel spacing overlap, laprate represents the overlap rate of the target photovoltaic area, and interval... row Indicates the line spacing between shooting points; interval col This indicates the column spacing between shooting points.

[0115] S24: Traverse the photovoltaic digital map according to the row and column intervals to determine the latitude and longitude coordinates of all shooting points inspected by the drone.

[0116] As an example, the photovoltaic panel has a length of 1.640m, a width of 0.992m, an installation tilt angle of 30°, a channel spacing of 1.720m between the photovoltaic panels, and a tilted projection length of 1.42m. After determining the row and column spacing between shooting points according to the above formula, the PVSdatabase is traversed to determine the number of rows and columns of photovoltaic panels in the photovoltaic area inspected by the drone. The number of rows is denoted as total_row_num, and the number of columns is denoted as eachrow_colnum.

[0117] Based on the photovoltaic digital map, the row and column intervals of the shooting points for drone inspection are determined. The center point coordinates of each photovoltaic panel are extracted from the PVSdatabase. The longitude of the shooting point is the average of the longitude of the center point of the first photovoltaic panel in each row, and the latitude of the shooting point is the average of the latitudes of the center points of the photovoltaic panels in the longest row.

[0118] As another example, set the starting row number for the inspection to row. start The end row number is row end Then, the inspection and shooting points are generated by traversing in the following manner.

[0119] The first step is to determine the coordinates of the center point of the photovoltaic panel at each shooting point based on the row and column spacing between shooting points.

[0120] for row = row start ,row start +interval row ,…,row end do

[0121] Here, row represents the row number of the photovoltaic panel.

[0122] The row range for photographing the photovoltaic area is from row to row + interval. row .

[0123] If row + interval row If the size is greater than total_row_num, reduce the shooting area to row to row+row. end .

[0124] Determine the location from row to row + interval in the photovoltaic area being photographed. row Within a row range, search for the longest column length maxcolnum. row .

[0125] The second step is to expand the row range of the photovoltaic area from row to row + interval. row Row-interval loop, using the longest column number maxcolnum row The column loop is performed to determine the latitude and longitude coordinates of the shooting point in the photovoltaic digital map.

[0126] for col = 1, 1 + interval col ,…,maxcolnum row do

[0127] Here, col represents the column number of the photovoltaic panel.

[0128] S25, based on all the shooting points inspected by the drone and the photovoltaic digital map, determine the geographical location index of each photovoltaic panel within the shooting area where the shooting point is located.

[0129] Following the example above, the third step is to store the geographic location index of the photovoltaic panels in the photovoltaic digital map in the dictionary of the shooting points.

[0130] rowcount = 0

[0131] for select col in range(col,col+interval col)

[0132] for select row in range(row,row+interval row )

[0133] if select row existing:

[0134] rowcount = rowcount + 1

[0135] templong+=select_database(engine,select row ,select col [0]

[0136] template+=select_database(engine,select row ,select col [1])

[0137] shooted_id.append([select row ,select col ])

[0138] Store the geographic location index of the photovoltaic panel in the `shooted_id` list to determine the shooting point:

[0139] sh oot long =temp long / interval col

[0140] sh oot lat =temp lat / rowcount

[0141] Finally, all the identified shooting points are linked together to generate a KML (markup language for XML) trajectory flight file, which is then imported into the drone to achieve drone inspection and shooting.

[0142] By using the above technical solution, the drone inspection shooting point is determined by setting the drone inspection altitude and generating a flight trajectory, multiple inspections can be carried out, ensuring the consistency of the drone inspection infrared images and improving the image quality of the drone inspection infrared images.

[0143] S12: Obtain infrared images of the drone inspection based on the shooting points of the drone inspection.

[0144] Following the example above, the drone is controlled to perform inspections along a flight path at a preset altitude, and infrared images of the drone inspection are captured at the determined shooting points.

[0145] S13. Based on the target defect detection algorithm and the infrared relative temperature difference method, process the infrared image of the UAV inspection to determine the state of the photovoltaic area and the type of infrared thermal defect in the shooting area corresponding to the shooting point.

[0146] The target defect detection algorithm uses the YOLOX network, a type of target detection network without prior knowledge. It is unaffected by pre-defined knowledge, possesses a flexible solution space, and can adapt to complex detection environments with large target scale variations. The target YOLOX network is used to determine and predict photovoltaic defect regions and detect the types of infrared thermal defects within those regions.

[0147] The YOLOX network is a single-stage detection network that integrates an anchorless detector and employs a SimOTA label assignment training strategy. The YOLOX network consists of a backbone network CSPDarkent, a PANet network, and a YOLOHead network. The CSPDarkent backbone network is used for feature extraction, the PANet network for feature integration, and the YOLOHead network for converting feature information into the final detection result.

[0148] The Focus network in the backbone network CSPDarkent expands the height and width information of the input image into the channel dimension to improve the efficiency of image feature extraction; the SPP network fuses information from different receptive domains by performing pooling operations of different sizes on the convolutional information and then stacking them.

[0149] The PANet network first upsamples the features, then downsamples them, and finally integrates the features to fully fuse features from different levels.

[0150] The YOLOHead network is a detection tap that includes a classifier and a regressor to perform convolutional computations on the integrated information to determine the category and location parameters of the detected object.

[0151] Infrared thermometry is used to determine the state of predicted photovoltaic (PV) defect regions. Specifically, it measures the internal and external temperatures of the predicted PV defect region with infrared thermal defects, and determines the state of the predicted PV defect region detected by the target YOLOX network based on the internal and external temperature difference. The predicted PV defect region states include PV defect state, PV early warning state, and PV normal state.

[0152] Figure 3This is a flowchart illustrating a method for determining the state of a photovoltaic region and the type of infrared thermal defects within a shooting area corresponding to a shooting point, according to an exemplary embodiment.

[0153] like Figure 3 As shown, S13 includes S31 to S33.

[0154] S31: Acquire training images of UAV inspections at various altitudes, establish labels based on the type of infrared thermal defects, train the target defect detection algorithm, and determine the infrared photovoltaic defect prediction model.

[0155] The types of infrared thermal defects include blocky hot spots, linear hot spots, and short circuits. These three types of hot spots—blocky hot spots, linear hot spots, and short-circuit hot spots—are labeled on UAV inspection training images at various altitudes. The labeled UAV inspection training images at various altitudes are then used to train a deep learning-based YOLOX network algorithm, enabling the determined infrared photovoltaic defect prediction model to identify blocky hot spots, linear hot spots, and short-circuit hot spots as infrared thermal defects.

[0156] As an example, 525 drone inspection training images at various altitudes were collected, including 912 hot spots and 50 bypass diode short-circuit fault targets, to train the YOLOX network.

[0157] S32, input the UAV inspection infrared image into the infrared photovoltaic defect prediction model to determine the predicted photovoltaic defect area and the infrared thermal defect type corresponding to the predicted photovoltaic defect area.

[0158] Specifically, regions with infrared defects identified by the infrared photovoltaic defect prediction model are designated as predicted photovoltaic defect regions, and these predicted photovoltaic defect regions are designated as suspected photovoltaic defect regions.

[0159] S33, based on the infrared relative temperature difference method, detect and predict the photovoltaic defect area, and determine the state of the predicted photovoltaic defect area.

[0160] The predicted state of photovoltaic defect areas includes photovoltaic defect state, photovoltaic early warning state, and photovoltaic normal state.

[0161] The above technical solution further verifies the state of the photovoltaic region with infrared defects predicted by the target defect detection algorithm through the infrared relative temperature difference method, determines the degree of photovoltaic defects, and prevents misjudgment.

[0162] Figure 4 This is a flowchart illustrating a method for determining the state of the predicted photovoltaic defect region according to an exemplary embodiment.

[0163] In one possible embodiment, such as Figure 4 As shown, S33 includes S41 to S44.

[0164] S41. Divide and process the predicted photovoltaic defect area according to the Otsu method of the great law, and determine the inner area of the binary temperature area and the outer area of the binary temperature area.

[0165] Obtain the temperature within the predicted photovoltaic defect area, quantize the infrared temperature within the predicted photovoltaic defect area into an integer, and perform a binary temperature area division process on the predicted photovoltaic defect area through the globally adaptive threshold determined by the Otsu method of the great law to determine the inner area of the binary temperature area and the outer area of the binary temperature area.

[0166] S42. Determine the highest temperature of the inner area of the binary temperature area, the average temperature of the inner area of the binary temperature area, and the average temperature of the outer area of the binary temperature area.

[0167] Among them, the highest temperature of the inner area of the binary temperature area can be represented by vagtemp, the average temperature of the inner area of the binary temperature area can be represented by intemp, and the average temperature of the outer area of the binary temperature area can be represented by outtemp.

[0168] S43. If the highest temperature of the inner area of the binary temperature area is greater than the first preset temperature, determine the state of the predicted photovoltaic defect area as the photovoltaic defect state.

[0169] As an example, the first preset temperature is 40°, and the second preset temperature is 20°.

[0170] That is, when vagtemp≥40° and abs(intemp - outtemp)>20°, determine the state of the predicted photovoltaic defect area as the photovoltaic defect state, that is, the photovoltaic defect area.

[0171] S44. If the temperature difference is greater than the third preset temperature and less than the second preset temperature, determine the state of the predicted photovoltaic defect area as the photovoltaic warning state.

[0172] Continuing with the above example, the third preset temperature is 10°.

[0173] That is, when 10° < abs(intemp - outtemp) < 20°, determine the state of the predicted photovoltaic defect area as the photovoltaic warning state, that is, the photovoltaic warning area.

[0174] S45. If the temperature difference is less than the third preset temperature, determine the state of the predicted photovoltaic defect area as the photovoltaic normal state.

[0175] That is, when abs(intemp - outtemp) < 10°, determine the state of the predicted photovoltaic defect area as the photovoltaic normal state, that is, the photovoltaic normal area.

[0176] In another possible embodiment, the highest temperature of the inner region of the binarized temperature region, the median temperature of the inner region of the binarized temperature region, and the median temperature of the outer region of the binarized temperature region are determined. Based on the temperature difference between the highest temperature of the inner region of the binarized temperature region, the median temperature of the inner region of the binarized temperature region, and the median temperature of the outer region of the binarized temperature region, the state of the predicted photovoltaic defect region is determined.

[0177] The method for determining the state of a photovoltaic defect region based on the temperature difference between the highest temperature in the inner region of the binarized temperature region, the median temperature in the inner region of the binarized temperature region, and the median temperature in the outer region of the binarized temperature region is the same as the method described above, which is based on the temperature difference between the highest temperature in the inner region of the binarized temperature region, the average temperature in the inner region of the binarized temperature region, and the average temperature in the outer region of the binarized temperature region. Therefore, it will not be repeated here.

[0178] By collecting the median temperature, we can prevent the irregular hot spot shape from causing the inner region of the binarized temperature area to have a lower temperature, and we can also solve the problem of temperature misjudgment caused by thermal noise, thus improving the accuracy of the temperature difference between the inside and outside.

[0179] S14, perform image segmentation processing on the infrared images inspected by the drone to determine the mapping from the image pixel coordinates of each photovoltaic panel to the latitude and longitude coordinates of its geographical location.

[0180] In some possible embodiments, the infrared images inspected by the drone are processed by image segmentation to determine the area where each photovoltaic panel is located, the outline of the area where each photovoltaic panel is located is extracted, and the coordinates of the center point of the outer rectangle of each photovoltaic panel are determined, that is, the center point coordinates of each photovoltaic panel.

[0181] The process involves converting the UAV inspection infrared images from infrared to HSV color space images, and then performing image segmentation according to the following rules:

[0182] 0≤H≤180, 0≤S≤255, 226≤V≤255

[0183] Where H represents the hue value, S represents the saturation value, and V represents the brightness value.

[0184] The infrared image of the drone inspection is divided into multiple photovoltaic panel regions, with each photovoltaic panel as a unit. The outline of each photovoltaic panel region is extracted, and the center point coordinates of the outer rectangle of each photovoltaic panel are generated, i.e., the center point coordinates of each photovoltaic panel, denoted as (xcenter, ycenter).

[0185] The geographical location coordinates of each photovoltaic panel are determined based on the center point coordinates of each photovoltaic panel and the latitude and longitude coordinates of each photovoltaic panel in the infrared image of each photovoltaic panel carried by the drone inspection.

[0186] Lat convert =0.000008×x c / imagewidth×H / F

[0187] Lon convert =0.000009×y c / imageheight×H / F

[0188]

[0189]

[0190] Where imagewidth represents the pixel width of the infrared image inspected by the drone, imageheight represents the pixel height of the infrared image inspected by the drone, and Lat convert The longitude coefficient representing the latitude and longitude coordinates of a geographical location. convert The values ​​represent the latitude coefficients of the geographical location coordinates, ImageLatitude represents the longitude of the photovoltaic panel's image location coordinates, ImageLongitude represents the latitude of the photovoltaic panel's image location coordinates, xcenter represents the abscissa of the photovoltaic panel's center point, ycenter represents the ordinate of the photovoltaic panel's center point, panelLatitude represents the longitude of the photovoltaic panel's geographical location coordinates, and panelLongitude represents the latitude of the photovoltaic panel's geographical location coordinates.

[0191] The image pixel coordinates of the photovoltaic panel are mapped one-to-one with the latitude and longitude coordinates of the photovoltaic panel's geographical location.

[0192] S15 maps the geographical location latitude and longitude coordinates of each photovoltaic panel to a preset photovoltaic digital map, locates photovoltaic panels with infrared thermal defects, determines the operating status of each photovoltaic panel, and then marks the operating status of each photovoltaic panel on the photovoltaic digital map to achieve digital management of each photovoltaic panel.

[0193] In one possible embodiment, determining the operating status of each photovoltaic panel includes:

[0194] If the overlap area between the photovoltaic area within the shooting area corresponding to the shooting point and each photovoltaic panel in the UAV inspection infrared image after image segmentation processing is greater than a preset area, the photovoltaic area state of the overlapping area represents the state of the photovoltaic panels within the overlapping area, and the infrared thermal defect type of the overlapping area represents the infrared thermal defect type of the photovoltaic panels within the overlapping area. In this disclosure, the preset area can be 50%.

[0195] Specifically, when the overlap area between a photovoltaic area in a photovoltaic defect state and a photovoltaic panel is greater than 50%, the photovoltaic panel is considered to be in a photovoltaic defect state, and the photovoltaic defect type corresponding to the photovoltaic panel is the same as the photovoltaic defect type of the overlapping area; when the overlap area between a photovoltaic area in a photovoltaic warning state and a photovoltaic panel is greater than 50%, the photovoltaic panel is considered to be in a photovoltaic warning state; when the overlap area between a photovoltaic area in a normal photovoltaic state and a photovoltaic panel is greater than 50%, the photovoltaic panel is considered to be in a normal photovoltaic state.

[0196] In one possible embodiment, the UAV inspection infrared images are traversed to determine the diagnostic and positioning information of each photovoltaic panel on the UAV inspection infrared images. The diagnostic and positioning information includes: the geographical location index of the photovoltaic panel, the center point coordinates of the photovoltaic panel image, the geographical location latitude and longitude coordinates of the photovoltaic panel, the infrared thermal defect type of the photovoltaic panel, the status of the photovoltaic panel, and the highest temperature of the photovoltaic panel.

[0197] Among them, the infrared thermal defect types of photovoltaic panels include: blocky hot spots, linear hot spots, and short circuits; the status of photovoltaic panels includes photovoltaic defect status, photovoltaic warning status, and photovoltaic normal status.

[0198] In this disclosure, the diagnostic and positioning information of the photovoltaic (PV) panel is stored in a PV panel location information database. The average latitude of each row of the PV panel is pre-stored in the location information database. The latitude of the PV panel after image segmentation is matched with the average latitude of each row of the PV panel in the location information database to determine the row number of the PV panel. The longitude of each PV panel in that row is read to determine the column number of the PV panel. The PV panel information (average temperature of the PV panel, state of the PV panel, infrared thermal defect type of the PV panel, image center point coordinates of the PV panel, and geographical location latitude and longitude coordinates of the PV panel) with a geographic location index is stored in the database according to the geographic location index.

[0199] The above technical solution maps the diagnostic and positioning information of each photovoltaic panel generated from the infrared images inspected by the drone onto a photovoltaic digital map. The geographical location information of the photovoltaic panel is associated with the photovoltaic digital map. The number of rows in the photovoltaic digital map is determined by the longitude information of the photovoltaic panel, and the number of columns in the photovoltaic digital map is determined by the latitude information of the photovoltaic panel, so as to achieve accurate positioning of photovoltaic panels with infrared thermal defects.

[0200] In some possible embodiments, the minimum distance is searched by traversing the UAV inspection infrared images and segmenting the spatial latitude and longitude coordinates of each photovoltaic panel from the UAV inspection infrared images and the latitude and longitude coordinates of the image center point of each photovoltaic panel on the photovoltaic digital map.

[0201]

[0202] Where distance represents distance, the spatial latitude and longitude coordinates of the photovoltaic panel are (panelLat, panelLon), and the latitude and longitude coordinates of the center point of the photovoltaic panel image are (mapLat, mapLon).

[0203] If the distance is less than or equal to the preset distance, the geographical location latitude and longitude coordinates of the photovoltaic panels corresponding to the UAV inspection infrared image and the photovoltaic digital map are mapped onto the preset photovoltaic digital map, and the operating status of the photovoltaic panels corresponding to the UAV inspection infrared image and the photovoltaic digital map is marked on the photovoltaic digital map for digital management.

[0204] The preset distance can be set to 1.5m. When the distance is less than or equal to 1.5m, the photovoltaic panel in the unmanned inspection infrared image will be matched with the photovoltaic panel in the digital photovoltaic map, and the geographical location latitude and longitude coordinates and the operating status of the photovoltaic panel will be marked on the digital photovoltaic map to realize the digital management of the photovoltaic panel.

[0205] In one possible embodiment, the diagnostic results of the drone inspection are mapped onto the photovoltaic digital map. They can also be mapped onto the digital photovoltaic map simultaneously with the previous inspection results to form a trend of changes in the status of the photovoltaic panels, which facilitates management and analysis by staff.

[0206] In some possible embodiments, the generated KML trajectory flight file is downloaded to the flight mobile phone, the flight file is confirmed, the drone is controlled to perform inspection, the photovoltaic digital map diagnosed with infrared thermal defects is downloaded to the mobile phone, the photovoltaic panel with infrared thermal defects is clicked, and the distance between the mobile phone positioning and the photovoltaic panel positioning is used to guide the maintenance personnel to the precise location of the faulty component in a walking navigation manner.

[0207] The above technical solutions enable digital management of photovoltaic power plants, locate photovoltaic panels with infrared thermal defects, and improve the positioning accuracy of faulty photovoltaic panels.

[0208] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. A digital method for automatic inspection and location of defects in photovoltaic panels using unmanned aerial vehicles (UAVs), characterized in that, include: On the preset photovoltaic digital map, based on the preset drone inspection altitude, the shooting point of the drone inspection and the geographical location index of each photovoltaic panel in the shooting area where the shooting point is located are determined; Based on the shooting points of the drone inspection, obtain the drone inspection infrared images; The infrared images of the UAV inspection are processed according to the target defect detection algorithm and the infrared relative temperature difference method to determine the photovoltaic area status and infrared thermal defect type within the shooting area corresponding to the shooting point. The infrared images from the UAV inspection are segmented to determine the mapping of image pixel coordinates to geographical latitude and longitude coordinates for each photovoltaic panel. The geographical location coordinates of each photovoltaic panel are mapped to the preset photovoltaic digital map to locate photovoltaic panels with infrared thermal defects, determine the operating status of each photovoltaic panel, and then mark the operating status of each photovoltaic panel on the photovoltaic digital map for digital management of each photovoltaic panel. The step of determining the drone inspection shooting point and the geographical location index of each photovoltaic panel within the shooting area where the shooting point is located, based on the preset drone inspection altitude, includes: By continuously capturing images of the target photovoltaic area using drones, an orthophoto map of the target photovoltaic area is determined. Then, using map creation tools, a photovoltaic digital map of the target photovoltaic area is created based on the orthophoto map, and a geographical location index is established for each photovoltaic panel in the photovoltaic digital map. Based on the preset drone inspection height, determine the drone's shooting area under orthographic projection; The row and column spacing between adjacent shooting points during the UAV inspection are determined based on the length, width, and tilt angle of the photovoltaic panel. The photovoltaic digital map is traversed according to the row interval and the column interval to determine the latitude and longitude coordinates of all shooting points inspected by the drone. Based on all the shooting points inspected by the drone and the photovoltaic digital map, the geographical location index corresponding to each photovoltaic panel within the shooting area where the shooting point is located is determined.

2. The method according to claim 1, characterized in that, The step of determining the shooting area of ​​the drone under orthographic projection based on the preset drone inspection height includes: ； ； Where F represents the camera's focal length. Indicates the length of the camera sensor pixels. H represents the width of the camera sensor pixel, and H represents the preset drone inspection height. This indicates the length of the shooting area. This indicates the width of the shooting area.

3. The method according to claim 1, characterized in that, The step of determining the row and column spacing between adjacent shooting points during UAV inspection based on the length, width, and tilt angle of the photovoltaic panel includes: = ； ； ； ； in, This indicates the length of the photovoltaic panel. This indicates the width of the photovoltaic panel. This indicates the tilt angle at which the photovoltaic panel is installed. This indicates the tilted projection length of the photovoltaic panel. This indicates the channel spacing between rows in the photovoltaic panel array. This indicates that the tilted projection length of the photovoltaic panel and the channel spacing form a repeating row spacing. This indicates the overlap rate of the captured target photovoltaic area. This indicates the row spacing between adjacent shooting points; This indicates the column spacing between adjacent shooting points.

4. The method according to claim 1, characterized in that, The process of processing the UAV inspection infrared image based on the target defect detection algorithm and the infrared relative temperature difference method to determine the photovoltaic area status and infrared thermal defect type within the shooting area corresponding to the shooting point includes: Acquire training images of UAV inspection at various altitudes, establish labels according to the types of infrared thermal defects, train the target defect detection algorithm, and determine the infrared photovoltaic defect prediction model. The types of infrared thermal defects include blocky hot spots, linear hot spots, and short circuits. The infrared images from the UAV inspection are input into the infrared photovoltaic defect prediction model to determine the predicted photovoltaic defect area and the infrared thermal defect type corresponding to the predicted photovoltaic defect area. The predicted photovoltaic defect area is detected by the infrared relative temperature difference method, and the state of the predicted photovoltaic defect area is determined. The state of the predicted photovoltaic defect area includes photovoltaic defect state, photovoltaic early warning state, and photovoltaic normal state.

5. The method according to claim 4, characterized in that, The step of detecting the predicted photovoltaic defect region based on the infrared relative temperature difference method and determining the state of the predicted photovoltaic defect region includes: According to the OTSU (Taiwanese Observatory for the Study of Photovoltaic Defects), the predicted photovoltaic defect region is divided into inner and outer regions of the binarized temperature region. Determine the highest temperature of the inner region of the binarized temperature region, the average temperature of the inner region of the binarized temperature region, and the average temperature of the outer region of the binarized temperature region. If the highest temperature in the inner region of the binarized temperature region is greater than the first preset temperature, and the temperature difference is greater than the second preset temperature, the state of the predicted photovoltaic defect region is determined to be a photovoltaic defect state. If the temperature difference is greater than the third preset temperature and less than the second preset temperature, the state of the predicted photovoltaic defect area is determined to be a photovoltaic early warning state. If the temperature difference is less than the third preset temperature, the state of the predicted photovoltaic defect area is determined to be the normal photovoltaic state.

6. The method according to claim 1, characterized in that, The image pixel coordinates of the photovoltaic panel include the coordinates of the center point of the photovoltaic panel and the latitude and longitude coordinates of the photovoltaic panel's image position; The step of performing image segmentation processing on the infrared images inspected by the UAV to determine the mapping from the image pixel coordinates of each photovoltaic panel to the geographical location latitude and longitude coordinates includes: The infrared images inspected by the UAV are processed by image segmentation to determine the area where each photovoltaic panel is located, the outline of the area where each photovoltaic panel is located is extracted, and the coordinates of the image center point of each photovoltaic panel are determined based on the outline of the area where each photovoltaic panel is located. The geographical location coordinates of each photovoltaic panel are determined based on the image center point coordinates of each photovoltaic panel and the image location latitude and longitude coordinates of each photovoltaic panel carried on the UAV inspection infrared image.

7. The method according to claim 6, characterized in that, The step of determining the geographical location coordinates of each photovoltaic panel based on the image center point coordinates of each photovoltaic panel and the image location coordinates of each photovoltaic panel carried on the UAV inspection infrared image includes: ； ； ； ； in, This indicates the pixel width of the infrared image inspected by the drone. This indicates the pixel height of the infrared image inspected by the drone. Longitude coefficients represent the latitude and longitude coordinates of a geographical location. Latitude coefficients represent the latitude and longitude coordinates of a geographical location. The image location of the photovoltaic panel is indicated by its longitude coordinates. The image location of the photovoltaic panel is indicated by its latitude and longitude coordinates. This represents the x-coordinate of the center point of the photovoltaic panel. This represents the ordinate of the center point of the photovoltaic panel. The longitude represents the geographical location coordinates of the photovoltaic panel. The latitude of the photovoltaic panel represents its geographical location.

8. The method according to claim 1, wherein determining the operating status of each photovoltaic panel includes: If the overlap area between the photovoltaic area within the shooting area corresponding to the shooting point and each photovoltaic panel in the UAV inspection infrared image after image segmentation is greater than a preset area, the photovoltaic area state of the overlap area represents the state of the photovoltaic panel within the overlap area, and the infrared thermal defect type of the overlap area represents the infrared thermal defect type of the photovoltaic panel within the overlap area.

9. The method according to claim 1, characterized in that, The process of mapping the geographical location (latitude and longitude coordinates) of each photovoltaic panel to the preset photovoltaic digital map, simultaneously locating photovoltaic panels with infrared thermal defects, determining the operating status of each photovoltaic panel, and then marking the operating status of each photovoltaic panel on the photovoltaic digital map for digital management of each photovoltaic panel includes: Traverse the UAV inspection infrared images, and search for the minimum distance based on the spatial latitude and longitude coordinates of each photovoltaic panel segmented from the UAV inspection infrared images and the latitude and longitude coordinates of the image center point of each photovoltaic panel on the photovoltaic digital map; If the distance is less than or equal to a preset distance, the latitude and longitude coordinates of the geographical location of the photovoltaic panel corresponding to the UAV inspection infrared image and the photovoltaic digital map are mapped onto the preset photovoltaic digital map, and the operating status of each photovoltaic panel is marked on the photovoltaic digital map for digital management.

Images