Unmanned aerial vehicle inspection AI auxiliary photographing method and system for photovoltaic power station

Abstract

The invention relates to the technical field of photovoltaic system monitoring, in particular to an unmanned aerial vehicle inspection AI auxiliary photographing method and system for a photovoltaic power station. Comprising the following steps: S1, planning an inspection path; s2, collecting an inspection image; s3, processing an inspection image; s4, constructing a target feature matrix set; s5, positioning and photographing; s6, training a convolutional neural network model; s7, defect identification; and S8, defect repair. By collecting video data and obtaining a positioning frame image, whether the inspection image is an abnormal image is rapidly judged, if the inspection image is an abnormal image, accurate positioning and photographing are conducted, the defect type is recognized, the inspection efficiency can be improved through preliminary judgment and accurate recognition, and the method is particularly suitable for a large photovoltaic power station.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic system monitoring technology, specifically to an AI-assisted photography method and system for drone inspection of photovoltaic power plants. Background Technology

[0002] A photovoltaic (PV) power station is a system that uses solar energy to generate electricity. It converts sunlight into electrical energy through photovoltaic panels and can convert the generated direct current (DC) into alternating current (AC) through an inverter for use by the power grid or directly. The main structure of a PV power station includes PV modules, a controller, and an inverter. PV modules, also known as solar panels, are the core component of a PV power station. They are responsible for absorbing sunlight and converting it into direct current. These modules consist of multiple solar cells connected in series and parallel to achieve the required voltage and current specifications.

[0003] Photovoltaic power stations require ample sunlight and are generally located in areas with minimal obstructions. During operation, the solar panels are often obscured by dust, dirt, bird droppings, and other grime, which affects their power generation efficiency. Therefore, regular cleaning and inspection of the solar panels are necessary to maintain the optimal operating condition of the photovoltaic power station.

[0004] Patent CN117713684A discloses a method for inspecting photovoltaic power plants for shading based on drones and deep learning image recognition. The method includes a drone with shooting capabilities and a host computer that interacts with the drone. The host computer issues commands to the drone based on the presence or absence of shading areas in the acquired images. When there are no shading areas, the drone performs normal inspection; when there are shading areas, the drone rotates at an angle appropriate to the direction of the light until the acquired image shows no shading, at which point it stops rotating and maintains that angle for image acquisition and inspection. This method effectively eliminates the interference of shading on the images captured by the drone, improves the quality of the images, and thus increases the reliability and stability of drone-based photovoltaic inspections, achieving the goal of shading inspection.

[0005] Patent CN112633535A discloses an intelligent inspection method and system for photovoltaic power plants based on UAV images. The system includes a UAV data import module, an inspection data parsing module, a deep learning algorithm module, a digital photovoltaic power plant module, an inspection result management module, and an inspection playback module. By introducing deep learning algorithms into multimodal inspection image data, more accurate photovoltaic defect detection is achieved. By establishing a digital photovoltaic power plant, the overall layout of the photovoltaic power plant can be expressed and the inspection process can be played back. By registering virtual scene images with real-world acquired images, the location and number of photovoltaic panels in the real scene can be obtained. The playback function intuitively displays the inspection process and results of a single batch, achieving accurate detection of photovoltaic panel defects. The inspection playback function visualizes the inspection process, facilitating intuitive debugging and inspection management by maintenance personnel, significantly improving the level of automated operation and maintenance of photovoltaic power plants, and effectively increasing the inspection efficiency of photovoltaic power plants.

[0006] However, the aforementioned existing technologies still have the following problems:

[0007] (1) Large photovoltaic power stations are located in areas with few obstructions and are often accompanied by strong winds, making it difficult for drones to take fixed-point photos during drone inspections, which affects inspection efficiency.

[0008] (2) At present, the flight time of drones is relatively short and they cannot be inspected for a long time. If the angle of each solar panel is adjusted to identify defects, the daily inspection task will be difficult to complete and the inspection efficiency will be low. In order to ensure the flight time of drones, the investment cost will also increase. Summary of the Invention

[0009] To achieve the objectives of this invention, the invention is implemented through the following technical solution: an AI-assisted photography method for drone inspection of photovoltaic power plants, characterized by comprising the following steps:

[0010] S1. Inspection route planning: Based on the arrangement of solar panels, the solar panels are divided into zones and numbered, and an inspection route is generated based on the solar panel numbers.

[0011] S2. Collect inspection images; Set the drone inspection speed, collect video data according to the inspection path, extract frame images from the video data at preset time intervals, and obtain inspection images.

[0012] S3. Inspection Image Processing: Preprocess the inspection images, extract the features of the inspection images through a convolutional neural network model, and generate the inspection image feature matrix.

[0013] S4. Construct a target feature matrix set; Collect images of defect-free solar panels under different lighting conditions, extract the features of the defect-free solar panel images through a convolutional neural network, and construct a target feature matrix set;

[0014] S5. Positioning and taking pictures; After the single-area inspection is completed, the feature matrix is ​​compared. If the feature matrix of the inspected image does not belong to the target feature matrix set, the inspected image is marked as an abnormal image and added to the abnormal dataset. The drone is then controlled to go to the abnormal image positioning point to take pictures and obtain the image to be classified.

[0015] S6. Train the convolutional neural network model; collect multiple defect images, divide them into training and test sets according to proportions, and label them according to the type of defect. Complete the training of the convolutional neural network model through continuous iterative optimization.

[0016] S7. Defect identification: Input the image to be classified into a trained convolutional neural network model, and identify the defect type through the model;

[0017] S8. Defect Reporting: Based on the image to be classified, the solar panel number, and the identified defect type, an inspection report is generated, and the defect is processed through a preset defect processing strategy.

[0018] Furthermore, the specific steps for collecting video data according to the inspection path include:

[0019] S21. Collect the first video data according to the inspection path and store the first video data in the database;

[0020] S22. Collect the second video data according to the reverse inspection path and store the second video data in the database.

[0021] Furthermore, the step of extracting frame images from video data at preset time intervals includes the following specific steps:

[0022] S23. Determine the starting point of the frame extraction time; if the data is the first video data, the starting point is when the drone reaches the left edge of the solar panel; if the data is the second video data, the starting point is when the drone reaches the right edge of the solar panel.

[0023] S24. Determine the time interval; the formula for calculating the preset time interval is as follows:

[0024]

[0025] Where, Δt i For the i-th time interval, d i v0 is the distance between the center points of adjacent solar panels, i = n k -1, n k Let k be the number of solar panels in region k.

[0026] S25. Two sets of frame images are extracted based on different video data.

[0027] Furthermore, the preprocessing of the inspection images includes cropping, rotation, contrast enhancement, and image stitching; the specific steps of image stitching include:

[0028] S31. Obtain the frames corresponding to the same solar panel from the two sets of frame images respectively;

[0029] S32. Evaluate the area of ​​overexposed regions in the frame images. Use the frame images with smaller overexposed regions as the main images and remove the overexposed regions. Use the frame images with larger overexposed regions as the sub-images.

[0030] S33. The main image and the sub-image are stitched together using a feature point matching algorithm.

[0031] Furthermore, the target feature matrix set is as follows:

[0032] M = {S j}={S1,S2,…,S n}

[0033] Where M is the target feature matrix set, S j Let be the j-th feature matrix, and n be the number of defect-free solar panel images under different illumination conditions, calculated by the following formula:

[0034]

[0035] Among them, T x Let t0 be the time interval for drone inspection, and n be the time interval for changes in illumination. c The number of similar solar panel images is calculated using a clustering algorithm.

[0036] Furthermore, the specific steps for comparing the feature matrices include:

[0037] S51. Calculate the difference matrix:

[0038]

[0039] Where B is the inspection image feature matrix, c xy The element that differs in the x-th row and y-th column;

[0040] S52. Modify the difference matrix so that the difference elements satisfy:

[0041]

[0042] Where ε is the difference coefficient, which controls the sensitivity to judging abnormal images;

[0043] S53. Determine the anomaly matrix: If ∑c xyIf <Y, then the feature matrix of the inspected image is a normal matrix, and if ∑c xy If ≥Y, then the feature matrix of the inspection image is an anomaly matrix.

[0044] Furthermore, the specific steps for controlling the drone to travel to the abnormal image location point to take pictures include:

[0045] (1) Establish a regional coordinate system, calculate the coordinates of the center point of the solar panel, and associate the coordinates with the number;

[0046] (2) Construct a coordinate transformation matrix to convert the coordinates of the center point of the solar panel into latitude and longitude using the original latitude and longitude of the coordinate system;

[0047] (3) The UAV receives the solar panel number corresponding to the abnormal image, identifies the corresponding coordinates, and goes there.

[0048] Furthermore, in step S3, multiple different convolutional kernels are used to extract inspection image features through a convolutional neural network, resulting in multiple different feature maps; in step S4, the convolutional kernel used to extract defect-free solar panel image features through a convolutional neural network is the same as that used in step S3.

[0049] Furthermore, the defect types include: dust, dirt, heat spots, aging, bubbles, and breakage.

[0050] This invention also includes an AI-assisted photography system for drone inspection of photovoltaic power plants, comprising:

[0051] The path planning module is used to partition and number the solar panels according to their arrangement, and generate inspection paths based on the solar panel numbers.

[0052] The image acquisition module is used to acquire video data according to the inspection path, extract frame images from the video data at preset time intervals, and obtain inspection images.

[0053] The image processing module is used for image preprocessing and extracting the feature matrix of the acquired images;

[0054] The drone control module is used to control the drone to plan its flight path and travel to fixed coordinates to take pictures;

[0055] The defect identification module is used to train the convolutional neural network model and identify defect types.

[0056] The report generation module is used to store defect handling strategies and generate inspection reports;

[0057] A database is used to store the acquired video and image data.

[0058] The beneficial effects of this invention are as follows:

[0059] 1. By collecting video data and obtaining positioning frames, it can quickly determine whether the inspection image is abnormal. If it is an abnormal image, it can accurately locate and take pictures to identify the defect type. Through preliminary judgment and accurate identification, the efficiency of inspection can be improved, which is especially suitable for large photovoltaic power plants.

[0060] 2. By capturing image data from dual angles and stitching the images together during image preprocessing, the reflection defects are overcome, and the angle does not need to be adjusted multiple times, thus improving inspection efficiency.

[0061] 3. By extracting image features through different convolution kernels, multiple feature matrices are generated, thereby improving the robustness of feature comparison. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the method flow of the present invention; Detailed Implementation

[0063] To enhance understanding of the present invention, the present invention will be further described in detail below with reference to embodiments. These embodiments are only used to explain the present invention and do not constitute a limitation on the scope of protection of the present invention.

[0064] Example 1

[0065] according to Figure 1 As shown, this embodiment provides an AI-assisted photography method for drone inspection of photovoltaic power plants, characterized by the following steps:

[0066] S1. Inspection route planning: The solar panels are divided into zones and numbered according to their layout. Inspection routes are generated based on the solar panel numbers. Inspection routes can be generated from the numbers in ascending order or from the smallest to the largest.

[0067] S2. Collect inspection images; Set the drone inspection speed, collect video data according to the inspection path, extract frame images from the video data at preset time intervals, and obtain inspection images.

[0068] S3. Inspection Image Processing: Preprocess the inspection images, extract the features of the inspection images through a convolutional neural network model, and generate the inspection image feature matrix.

[0069] S4. Construct a target feature matrix set; Collect images of defect-free solar panels under different lighting conditions, extract the features of the defect-free solar panel images through a convolutional neural network, and construct a target feature matrix set;

[0070] S5. Positioning and taking pictures; After the single-area inspection is completed, the feature matrix is ​​compared. If the feature matrix of the inspected image does not belong to the target feature matrix set, the inspected image is marked as an abnormal image and added to the abnormal dataset. The drone is then controlled to go to the abnormal image positioning point to take pictures and obtain the image to be classified.

[0071] S6. Train the convolutional neural network model; collect multiple defect images, divide them into training and test sets according to proportions, and label them according to the type of defect. Complete the training of the convolutional neural network model through continuous iterative optimization.

[0072] S7. Defect identification: Input the image to be classified into a trained convolutional neural network model, and identify the defect type through the model;

[0073] S8. Defect Reporting: Based on the image to be classified, the solar panel number, and the identified defect type, an inspection report is generated, and the defect is processed through a preset defect processing strategy.

[0074] More specifically, the steps for collecting video data according to the inspection path include:

[0075] S21. Collect the first video data according to the inspection path and store the first video data in the database;

[0076] S22. Collect the second video data according to the reverse inspection path and store the second video data in the database.

[0077] More specifically, the step of extracting frame images from video data at preset time intervals includes:

[0078] S23. Determine the starting point of the frame extraction time; if the data is the first video data, the starting point is when the drone reaches the left edge of the solar panel; if the data is the second video data, the starting point is when the drone reaches the right edge of the solar panel.

[0079] S24. Determine the time interval; the formula for calculating the preset time interval is as follows:

[0080]

[0081] Where, Δt i For the i-th time interval, d i v0 is the distance between the center points of adjacent solar panels, i = n k -1, n k Let k be the number of solar panels in region k.

[0082] S25. Two sets of frame images are extracted based on different video data.

[0083] More specifically, the preprocessing of the inspection images includes cropping, rotation, contrast enhancement, and image stitching; the specific steps of image stitching include:

[0084] S31. Obtain the frames corresponding to the same solar panel from the two sets of frame images respectively;

[0085] S32. Evaluate the area of ​​overexposed regions in the frame images. Use the frame images with smaller overexposed regions as the main images and remove the overexposed regions. Use the frame images with larger overexposed regions as the sub-images.

[0086] S33. The main image and the sub-image are stitched together using a feature point matching algorithm.

[0087] More specifically, the target feature matrix set is:

[0088] M = {S j}={S1,S2,…,S n}

[0089] Where M is the target feature matrix set, S j Let be the j-th feature matrix, and n be the number of defect-free solar panel images under different illumination conditions, calculated by the following formula:

[0090]

[0091] Among them, T x Let t0 be the time interval for drone inspection, and n be the time interval for changes in illumination. c The number of similar solar panel images is calculated using a clustering algorithm.

[0092] More specifically, the specific steps for comparing the feature matrices include:

[0093] S51. Calculate the difference matrix:

[0094]

[0095] Where B is the inspection image feature matrix, c xy The element that differs in the x-th row and y-th column;

[0096] S52. Modify the difference matrix so that the difference elements satisfy:

[0097]

[0098] Where ε is the difference coefficient, which controls the sensitivity to judging abnormal images;

[0099] S53. Determine the anomaly matrix: If ∑c xy If <Y, then the feature matrix of the inspected image is a normal matrix, and if ∑c xyIf ≥Y, then the feature matrix of the inspection image is an anomaly matrix.

[0100] More specifically, the specific steps for controlling the drone to travel to the abnormal image location point to take pictures include:

[0101] (1) Establish a regional coordinate system, calculate the coordinates of the center point of the solar panel, and associate the coordinates with the number;

[0102] (2) Construct a coordinate transformation matrix to convert the coordinates of the center point of the solar panel into latitude and longitude using the original latitude and longitude of the coordinate system;

[0103] (3) The UAV receives the solar panel number corresponding to the abnormal image, identifies the corresponding coordinates, and goes there.

[0104] More specifically, in S3, multiple different convolutional kernels are used to extract inspection image features through a convolutional neural network, resulting in multiple different feature maps; in S4, the convolutional kernel used to extract defect-free solar panel image features through a convolutional neural network is the same as that used in S3.

[0105] More specifically, the defect types include: dust, dirt, heat spots, aging, bubbles, and breakage.

[0106] Example 2

[0107] An AI-assisted photography system for drone inspection of photovoltaic power plants includes:

[0108] The path planning module is used to partition and number the solar panels according to their arrangement, and generate inspection paths based on the solar panel numbers.

[0109] The image acquisition module is used to acquire video data according to the inspection path, extract frame images from the video data at preset time intervals, and obtain inspection images.

[0110] The image processing module is used for image preprocessing and extracting the feature matrix of the acquired images;

[0111] The drone control module is used to control the drone to plan its flight path and travel to fixed coordinates to take pictures;

[0112] The defect identification module is used to train the convolutional neural network model and identify defect types.

[0113] The report generation module is used to store defect handling strategies and generate inspection reports;

[0114] A database is used to store the acquired video and image data.

[0115] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A drone inspection AI-assisted photography method for photovoltaic power plants, characterized in that, Includes the following steps: S1. Inspection route planning: Based on the arrangement of solar panels, the solar panels are divided into zones and numbered, and an inspection route is generated based on the solar panel numbers. S2. Collect inspection images; Set the drone inspection speed, collect video data according to the inspection path, extract frame images from the video data at preset time intervals, and obtain inspection images. S3, Inspection Image Processing; The inspection images are preprocessed, and the features of the inspection images are extracted through a convolutional neural network model to generate an inspection image feature matrix; S4. Construct the target feature matrix set; Collect images of defect-free solar panels under different lighting conditions, extract the features of the defect-free solar panel images through a convolutional neural network, and construct the target feature matrix set; S5, Location-based photography; After a single-area inspection is completed, a feature matrix comparison is performed. If the feature matrix of the inspected image does not belong to the target feature matrix set, the inspected image is marked as an abnormal image and added to the abnormal dataset. The drone is then controlled to go to the abnormal image location point to take a picture, thus obtaining the image to be classified. S6. Train the convolutional neural network model; collect multiple defect images, divide them into training and test sets according to proportions, and label them according to the type of defect. Complete the training of the convolutional neural network model through continuous iterative optimization. S7. Defect identification: Input the image to be classified into a trained convolutional neural network model, and identify the defect type through the model; S8. Defect Reporting: Based on the image to be classified, the solar panel number, and the identified defect type, an inspection report is generated, and the defect is processed through a preset defect processing strategy.

2. The method for AI-assisted photography during drone inspection of photovoltaic power plants according to claim 1, characterized in that, The specific steps for collecting video data according to the inspection path include: S21. Collect the first video data according to the inspection path and store the first video data in the database; S22. Collect the second video data according to the reverse inspection path and store the second video data in the database.

3. The method for AI-assisted photography during drone inspection of photovoltaic power plants according to claim 2, characterized in that, The specific steps for extracting frame images from video data at preset time intervals include: S23. Determine the starting point of the frame extraction time; if the video data is the first video data, the starting point is when the drone reaches the left edge of the solar panel; if the video data is the second video data, the starting point is when the drone reaches the right edge of the solar panel. S24. Determine the time interval; the formula for calculating the preset time interval is as follows: Where, Δt i For the i-th time interval, d i v0 is the distance between the center points of adjacent solar panels, i = n k -1, n k Let k be the number of solar panels in region k. S25. Two sets of frame images are extracted based on different video data.

4. The AI-assisted photography method for drone inspection of photovoltaic power plants according to claim 3, characterized in that, The preprocessing of the inspection images includes cropping, rotation, contrast enhancement, and image stitching; the specific steps of image stitching include: S31. Obtain the frames corresponding to the same solar panel from the two sets of frame images respectively; S32. Evaluate the area of ​​overexposed regions in the frame images. Use the frame images with smaller overexposed regions as the main images and remove the overexposed regions. Use the frame images with larger overexposed regions as the sub-images. S33. The main image and the sub-image are stitched together using a feature point matching algorithm.

5. The AI-assisted photography method for drone inspection of photovoltaic power plants according to claim 1, characterized in that, The target feature matrix set is: M={S j }={S1,S2,…,S n } Where M is the target feature matrix set, S j Let be the j-th feature matrix, and n be the number of defect-free solar panel images under different illumination conditions, calculated by the following formula: Among them, T x Let t0 be the time interval for drone inspection, and n be the time interval for changes in illumination. c The number of similar solar panel images is calculated using a clustering algorithm.

6. The AI-assisted photography method for drone inspection of photovoltaic power plants according to claim 5, characterized in that, The specific steps for comparing the feature matrices include: S51. Calculate the difference matrix: Where B is the inspection image feature matrix, c xy The element that differs in the x-th row and y-th column; S52. Modify the difference matrix so that the difference elements satisfy: Where ε is the difference coefficient, which controls the sensitivity to judging abnormal images; S53. Determine the anomaly matrix: If ∑c xy If <Y, then the feature matrix of the inspected image is a normal matrix, and if ∑c xy If ≥Y, then the feature matrix of the inspection image is an anomaly matrix.

7. The AI-assisted photography method for drone inspection of photovoltaic power plants according to claim 1, characterized in that, The specific steps for controlling the drone to travel to the abnormal image location point to take pictures include: (1) Establish a regional coordinate system, calculate the coordinates of the center point of the solar panel, and associate the coordinates with the number; (2) Construct a coordinate transformation matrix to convert the coordinates of the center point of the solar panel into latitude and longitude using the original latitude and longitude of the coordinate system; (3) The UAV receives the solar panel number corresponding to the abnormal image, identifies the corresponding coordinates, and goes there.

8. The AI-assisted photography method for drone inspection of photovoltaic power plants according to claim 7, characterized in that, In step S3, multiple different convolution kernels are used to extract inspection image features through a convolutional neural network, resulting in multiple different feature maps. In step S4, the same convolution kernel is used as in step S3 when extracting image features of defect-free solar panels through a convolutional neural network.

9. The AI-assisted photography method for drone inspection of photovoltaic power plants according to claim 1, characterized in that, The defect types include: dust, dirt, heat spots, aging, bubbles, and breakage.

10. An AI-assisted photography system for drone inspection of photovoltaic power plants, used to implement the method as described in any one of claims 1-9, characterized in that, include: The path planning module divides and numbers the solar panels according to their layout, and generates inspection paths based on the solar panel numbers. The image acquisition module is used to acquire video data according to the inspection path, extract frame images from the video data at preset time intervals, and obtain inspection images. The image processing module is used for image preprocessing and extracting the feature matrix of the acquired images; The drone control module is used to control the drone to plan its flight path and travel to fixed coordinates to take pictures; The defect identification module is used to train the convolutional neural network model and identify defect types. The report generation module is used to store defect handling strategies and generate inspection reports; A database is used to store the acquired video and image data.

Images