Photovoltaic station capital construction progress detection method and device for unmanned aerial vehicle inspection and computer readable storage medium

Abstract

The invention belongs to the technical field of deep learning, and more specifically relates to a photovoltaic station infrastructure progress detection method and device for unmanned aerial vehicle patrol, and a computer readable storage medium. The method comprises the step of carrying out two-dimensional plane data acquisition on a photovoltaic field station. And converting the acquired data into an orthographic image file of the whole site through a three-dimensional reconstruction technology. And inputting the orthographic image file into an analysis host, and performing intelligent analysis on the image by using a deep learning algorithm. And the analysis host outputs an installation progress report of the photovoltaic panel according to an analysis result of the deep learning algorithm. The unmanned aerial vehicle inspection technology and the deep learning technology are combined, a YOLOV8 detection model and an SAHI slice reasoning method are adopted, and calculation of the number of photovoltaic panels, the number of stand columns and the installation angles of the photovoltaic panels is achieved in combination with a related post-processing method. According to the invention, the problems of low efficiency, small coverage range and difficulty in identification of installation defects and abnormities in traditional inspection detection are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning. More specifically, it relates to a method, device and computer-readable storage medium for detecting the infrastructure progress of a photovoltaic power station by drone inspection. Background Art

[0002] Detecting the infrastructure progress of a photovoltaic power station by drone inspection is an efficient method for inspecting and evaluating the installation progress of photovoltaic panels by means of drone technology. By combining the high-definition camera carried by the drone with deep learning technology, it is possible to analyze and evaluate the installation progress, installation quality and potential problems by obtaining image data of the photovoltaic panel installation site in real time. In terms of the current technology status, the drone photovoltaic panel installation progress inspection system integrates advanced remote sensing technology, high-definition image acquisition technology and artificial intelligence technology, which can efficiently and accurately cover each installation area of the photovoltaic power station and record the installation position and status of each photovoltaic panel in detail.

[0003] Chinese invention patent CN118918479A discloses a method for detecting individual trees in visible light images of drones applicable to complex forest environments. Based on a drone observation platform and a visible light sensor, visible light images of the upper canopy of the forest are collected in the individual tree detection area, and image processing including image alignment and dense point cloud generation steps is carried out to generate a forest digital orthophoto map. The forest digital orthophoto data is sliced to generate image blocks of N×M pixel size, and an object detection instance annotation tool is used to annotate the instances of the individual tree crowns. After all the crowns are annotated, a dataset for individual tree detection is constructed. According to the training set data, local correction of the model weights is carried out, and based on the validation set data, verification feedback of the model training performance is carried out, and hyperparameters are fine-tuned to train the optimal model weights.

[0004] In summary, the traditional inspection cycle cannot ensure timely grasp of the installation progress in large-scale photovoltaic power stations, and the current infrastructure technology for inspecting photovoltaic power stations has problems such as low efficiency, small coverage, and difficulty in identifying installation defects and abnormalities. Summary of the Invention

[0005] The present invention aims to overcome at least one defect of the above-mentioned prior art, and provides a method for detecting the infrastructure progress of a photovoltaic power station by drone inspection, which is used to ensure the efficient and high-quality construction of the photovoltaic power station and is transmitted to the ground station or cloud server in real time through a stable wireless transmission system for in-depth analysis and processing. In this process, the artificial intelligence algorithm plays a core role. Through the trained deep learning model, the system can automatically identify the installation progress of the photovoltaic panels, including information such as the number of installed photovoltaic panels, position distribution and installation quality.

[0006] In addition, it also includes a device for implementing the method for detecting the infrastructure progress of a photovoltaic power station by drone inspection.

[0007] In another aspect of the present invention, a computer-readable storage medium is also provided, which stores executable instructions for executing a method for detecting the construction progress of a photovoltaic power station by means of drone patrol.

[0008] The detailed technical solution of the present invention is as follows:

[0009] A method for detecting the construction progress of a photovoltaic power station by means of drone patrol, the method comprising:

[0010] S1. Obtain the materials required for three-dimensional reconstruction by means of drone inspection, convert the inspection pictures into tif format data through three-dimensional reconstruction technology, and synthesize the orthophoto of the power station as the image data to be detected;

[0011] S2. Input the image data to be detected into the infrastructure detection model, which is used to detect the photovoltaic panels and columns in the photovoltaic power station. The corresponding detection results are obtained by performing SAHI slicing inference on the image data to be detected, and the detection results include defect categories and category confidence levels;

[0012] S3. Post-process the infrastructure target detection results of the image data to be detected to count the number of photovoltaic panels, the number of columns, and the installation angle of the photovoltaic panels;

[0013] S4. Process the filtered detection results of the photovoltaic panels to obtain a more accurate detection frame for the photovoltaic panels, which is used to calculate the installation angle of the photovoltaic panels.

[0014] Further, S1 further includes preprocessing the image data to be detected:

[0015] First, perform format conversion through the PIL library to convert the tif format data into jpg format;

[0016] Then perform equal ratio scaling, and the scaling factor is one-fourth of the original. Then slice according to the training data format, the slice size is 1280*1280, and the overlap rate is 20%. The overlap rate calculation formula is:

[0017] y_overlap = int(overlap_height_ratio * slice_height) (1);

[0018] x_overlap = int(overlap_width_ratio * slice_width) (2);

[0019] In formulas (1) to (2), x_overlap is the overlapping height in the horizontal direction, y_overlap is the overlapping height in the vertical direction, overlap_height_ratio refers to the slice height overlapping ratio, slice_height refers to the slice height, overlap_width_ratio refers to the slice width overlapping ratio, and slice_width refers to the slice width.

[0020] Finally, the sliced image is input into the infrastructure detection model for SAHI slice inference.

[0021] Furthermore, the infrastructure detection model is improved based on the YOLOv8 model, including:

[0022] Embed the Coordinate Attention (CA) unit after the C2F module in the YOLOv8 backbone network, specifically located between the second C2F module and the fourth C2F module structure. Its optimization steps include:

[0023] 1) Spatial dimension aggregation: The original input feature map first undergoes global average pooling along the width and height axes respectively after passing through the residual structure, generating two orthogonal dimension feature compression maps of W×1 and 1×H. H is the height of the image, and W is the width of the image. This operation can effectively capture long-range spatial dependencies;

[0024] 2) Cross-dimensional feature fusion: Concatenate the above two spatial compression features in the channel dimension to construct a joint representation matrix of H+W composite dimensions, realizing the cross-fusion of spatial information;

[0025] 3) Channel dimension compression: Apply a non-linear transformation to the joint representation matrix through a 1×1 convolutional kernel to compress the channel dimension to C / r*1*(W+H). C refers to the number of channels, and r is the dimensionality reduction multiple. Here, C / r represents the dimensionality reduction multiple of the feature map in the C direction of the number of channels. This design significantly reduces the computational complexity while maintaining information integrity;

[0026] 4) Spatial attention decoupling: After batch normalization and non-linear activation, the two compressed representation matrices are separated into two independent branches. Each branch uses a 1×1 convolution for dimension restoration and applies the Sigmoid function to generate attention probability distribution maps in the H and W directions;

[0027] 5) Dynamic feature calibration: Multiply the decoupled spatial attention weights with the original input feature map channel by channel to achieve feature recalibration based on spatial importance. This adaptive adjustment mechanism can strengthen the feature response in key regions and effectively improve the model's perception accuracy of the target spatial position.

[0028] Furthermore, the SAHI slicing inference of the image data to be detected specifically includes:

[0029] S21. Image slicing before inputting the infrastructure detection model: The original image is divided into multiple overlapping blocks of size M×N. These slices cover the entire image, and there is a certain overlapping area between adjacent slices;

[0030] S22. Independent inference: Each slice is used to perform inference using a specified infrastructure detection model to obtain the object detection results in that slice;

[0031] S23. Result merging: The prediction results of all slices are merged together to generate a detection result list for the entire image.

[0032] Among them, the target detection boxes are fused by NMM, and the calculation formula of the intersection over union (IoU) is:

[0033]

[0034] In formula (3), Area of Intersection represents the area of the intersection of two detection boxes, Area of Union represents the area of the union of two detection boxes, and the value range of IoU is [0, 1]. The larger the value, the higher the overlapping degree of the two detection boxes.

[0035] For overlapping detection boxes, NMM uses a weighted average method for merging. There are two overlapping detection boxes B1 and B2, with their coordinates being (x1, y1, w1, h1) and (x2, y2, w2, h2) respectively, and the confidence levels being s1 and s2. The coordinates x merged , y merged and the confidence level w merged are calculated through the following formulas:

[0036] Coordinate weighted average:

[0037]

[0038] This step usually involves post-processing algorithms such as target detection box fusion to eliminate duplicate or overlapping detection boxes and generate the final detection results.

[0039] Furthermore, post-processing the infrastructure target detection results of the image data to be detected includes:

[0040] For the detection results, first perform confidence filtering;

[0041] For the columns, the number can be directly counted for the filtered results;

[0042] For the photovoltaic panel, outlier filtering is performed on it. The specific method is to calculate the width of the photovoltaic panel after confidence filtering, and the interquartile range method is used to filter outliers. The calculation formula of the IQR is:

[0043] IQR = Q3 - Q1, where Q3 is the third quartile of the data set and Q1 is the first quartile;

[0044] The outlier judgment formula is:

[0045] Lower limit = Q1 - 1.5 × IQR (5);

[0046] Upper limit = Q3 + 1.5 × IQR (6);

[0047] Any data point below the lower limit or above the upper limit is considered an outlier.

[0048] Furthermore, to obtain a more accurate detection frame for the photovoltaic panel, it specifically includes:

[0049] (1) Crop the photovoltaic panel through the detection frame;

[0050] (2) Obtain the binary image of each photovoltaic panel;

[0051] (3) Perform morphological processing on the binary image, first dilate and then erode;

[0052] (4) Obtain the contour of the photovoltaic panel string image and draw the contour on the original image;

[0053] (5) Obtain the contour with the largest area;

[0054] (6) Compare the obtained contour with the area of the detection frame, set a threshold. When it is greater than the set threshold, take the width and height of the contour as the required result. If it is less than the set threshold, take the width and height of the detection frame, corresponding to w and H in the formula;

[0055] (7) Calculate the pixel distance in the projection through the actual width of the photovoltaic panel, that is Calculate the height of the projection through the pixel distance as Finally, obtain the corresponding angle by calculating the cosine value of the actual height of the photovoltaic panel, that is

[0056]

[0057] (8) Finally, compare the number of detected components and the number of columns with the number planned to be installed at the station. At the same time, each photovoltaic panel outputs its respective angle.

[0058] In another aspect of the present invention, a device for detecting the infrastructure progress of a photovoltaic power station by drone patrol is provided. The device includes:

[0059] At least one processor; and

[0060] A memory that stores instructions which, when executed by the at least one processor, cause the at least one processor to execute a method for detecting the infrastructure construction progress of a photovoltaic power station by drone inspection as described above.

[0061] In another aspect of the present invention, there is also provided a computer-readable storage medium that stores executable instructions which, when executed, cause the machine to execute a method for detecting the infrastructure construction progress of a photovoltaic power station by drone inspection as described above.

[0062] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0063] The method, device and computer-readable storage medium for detecting the infrastructure construction progress of a photovoltaic power station by drone inspection provided by the present invention. Drone inspection can quickly cross a vast photovoltaic array to achieve full-range and non-missing monitoring, greatly shortening the inspection cycle and ensuring that the installation progress can be timely grasped even in large-scale photovoltaic power stations. At the same time, through high-precision imaging equipment combined with an optimized deep learning algorithm, it can automatically identify problems such as the number of installed components and non-standard installation, providing a scientific basis for timely correcting errors, ensuring the construction quality of the power station, and accurately identifying installation defects and abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flowchart of the method for detecting the infrastructure construction progress of a photovoltaic power station by drone inspection according to the present invention.

[0065] Figure 2 is a schematic diagram of the network structure of the infrastructure detection model in Embodiment 1 of the present invention.

[0066] Figure 3 is a schematic diagram of the recognition logic of the infrastructure detection model and the post-processing flow of the detection results in Embodiment 1 of the present invention.

[0067] Figure 4 is a schematic diagram of photovoltaic panel angle calculation in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0068] The present invention will be further described below in conjunction with the drawings and embodiments.

[0069] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0070] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they specify the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0071] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0072] Embodiment 1

[0073] Refer Figure 1 , this embodiment provides a method for detecting the infrastructure progress of a photovoltaic power station by drone inspection. The method includes:

[0074] S1. The drone conducts inspection and takes pictures and synthesizes the orthophoto of the power station;

[0075] The drone conducts inspection and takes the materials required for 3D reconstruction. The inspection pictures are converted into tif format data through 3D reconstruction technology. After synthesizing the orthophoto of the power station, it is used as the image data to be detected.

[0076] In this embodiment, the model of the inspection drone is DJI Mavic 3T, and its parameter configuration includes: wide-angle: equivalent focal length 24 mm, 48 million pixels, telephoto: equivalent focal length 162 mm, 12 million pixels, 56x hybrid zoom, thermal imaging: DFOV: 61°, equivalent focal length 40 mm, resolution 640*512. Preferably, the 3D reconstruction method uses the open-source algorithm OpenDroneMap or DJI mapping. The orthophoto image of the power station synthesized through 3D reconstruction is the image data to be detected.

[0077] Specifically, the drone conducts inspection and takes the materials required for 3D reconstruction. The requirements for material collection are as follows:

[0078] 1. Shoot the target area from multiple angles and positions to ensure that all details of the area can be captured; 2. Keep the camera stable during shooting to avoid image blurring caused by shaking and movement; 3. There should be a certain overlap between adjacent photos so that the images can be accurately matched and stitched during the subsequent 3D reconstruction process; 4. The resolution of the images should be high enough to capture sufficient details and meet the accuracy requirements of 3D reconstruction; 5. The shooting should be completed in the shortest possible time to reduce image differences caused by weather changes, light changes, or object movement.

[0079] Preprocess the image data to be detected: First, perform format conversion through the PIL library to convert tif format data to jpg format. Then, perform proportional scaling. Currently, the scaling factor is one-fourth of the original. Next, slice according to the training data format, with a slice size of 1280*1280 and an overlap rate of 20%. The overlap rate calculation formula is:

[0080] y_overlap = int(overlap_height_ratio * slice_height) (1);

[0081] x_overlap = int(overlap_width_ratio * slice_width) (2);

[0082] In formulas (1) to (2), x_overlap is the overlapping height in the horizontal direction, y_overlap is the overlapping height in the vertical direction, overlap_height_ratio refers to the slice height overlap ratio, slice_height refers to the slice height, overlap_width_ratio refers to the slice width overlap ratio, and slice_width refers to the slice width.

[0083] Input the sliced image into the infrastructure detection model for SAHI sliced inference.

[0084] S2. Input the image data to be detected into the infrastructure detection model. The infrastructure detection model is used to detect photovoltaic panels and columns in a photovoltaic power station. By performing sliced inference on the image data to be detected, the corresponding detection results are obtained. The detection results include defect categories and category confidence levels.

[0085] In this embodiment, the infrastructure detection model uses the open-source algorithm Yolov8l-dec-1280. Since the photovoltaic power station is large and the photovoltaic panels are extremely small targets compared to the orthophoto of the entire power station, the detection effect of conventional detection models for small targets is relatively poor. Therefore, a coordinate attention optimization detection algorithm is adopted. The optimization method based on the coordinate attention mechanism can significantly improve the model's perception ability of spatial position features and optimize the calculation efficiency while enhancing the feature representation performance.

[0086] See Figure 2 As shown, the infrastructure detection model is improved based on the YOLOv8 model, including: embedding a coordinate attention CA unit after the C2F module in the YOLOv8 backbone network, specifically located between the second C2F module and the fourth C2F module structure. This mechanism enhances spatial perception through multi-dimensional feature interaction.

[0087] Optimization steps of the embedded coordinate attention CA unit, specifically including five core stages:

[0088] 1) Spatial dimension aggregation: After passing through the residual structure, the original input feature map first performs global average pooling along the width and height axes respectively, namely w average pooling and H average pooling, to generate two feature compression maps with orthogonal dimensions of W×1 and 1×H. H is the height of the image, and W is the width of the image. This operation can effectively capture long-range spatial dependencies.

[0089] 2) Cross-dimensional feature fusion: The above two spatial compression features are concatenated in channels, that is, feature fusion, to construct a joint representation matrix with an H+W composite dimension, realizing cross-fusion of spatial information.

[0090] 3) Channel dimension compression: The joint representation matrix is subjected to a non-linear transformation through a 1×1 convolution kernel, and the channel dimension is compressed to C / r*1*(W+H). C refers to the number of channels, and r is the dimensionality reduction multiple. Among them, C / r represents the dimensionality reduction multiple of the feature map in the direction of the number of channels C. This design significantly reduces the computational complexity while maintaining information integrity.

[0091] 4) Spatial attention decoupling: After batch normalization and non-linear activation, the two compressed representation matrices are separated into two independent branches by features. Each branch respectively uses a 1×1 convolution for dimension recovery, and the Sigmoid function is applied to generate attention probability distribution maps, that is, weights, in the H and W biaxial directions.

[0092] 5) Dynamic feature calibration: The decoupled spatial attention weights are multiplied channel by channel with the original input feature map, that is, Re-weight, to achieve feature recalibration based on spatial importance. This adaptive adjustment mechanism can strengthen the feature response in key regions and effectively improve the model's perception accuracy of the target spatial position.

[0093] Through the orthogonal spatial decomposition and dynamic weight allocation strategy, this innovative structure introduces the analysis of spatial dimension correlation on the basis of channel attention, enabling the network to capture the spatial distribution characteristics of the target more accurately. At the same time, thanks to the dimension compression strategy, a good balance is achieved between computational resource consumption and model performance.

[0094] After the above model is constructed, it is necessary to pre-train the constructed model first. The data acquisition method used for training is the same as that of the data to be measured at the station. The size of the image cropping in this experiment is 1280*1280, and the overlap rate is 20%. Since the specifications of the photovoltaic panels at different stations are similar, the data of 5 stations are used as training data in this experiment. A total of 5300 training sets after slicing can achieve a very high generalization.

[0095] Since random cropping is performed according to the image size during slicing, there will be cases where the same photovoltaic panel is cropped into different images. For such photovoltaic panels, they need to be fully labeled during annotation so that the subsequent SAHI slice inference can better fuse the detection results.

[0096] After the model training is completed, the infrastructure detection of the station to be detected can be carried out. Figure 3 shows the recognition logic of the infrastructure detection model in this embodiment.

[0097] Input the sliced images into the infrastructure detection model for SAHI slice inference. The basic idea of SAHI slice inference is to divide the large image into multiple overlapping slices or patches, and perform independent object detection inference on each slice, thereby improving the detection ability of small objects. The specific steps include:

[0098] 1. Image slicing: The original image is divided into multiple slices of size M×N. These slices cover the entire image, and there is a certain overlapping area between adjacent slices.

[0099] 2. Independent inference: Use the specified object detection model to perform inference on each slice to obtain the object detection results in that slice.

[0100] 3. Result merging: Merge the prediction results of all slices together to generate a detection result list for the entire image. This step usually involves post-processing algorithms such as target detection box fusion to eliminate duplicate or overlapping detection boxes and generate the final detection result.

[0101] Among them, for target detection box fusion, by comparing the three fusion methods of NMS, NMM, and GREEDYNMM, the best-performing NMM is selected. The formula for the intersection over union is:

[0102]

[0103] In formula (3), Area of Intersection represents the area of the intersection of two detection boxes, and Area of Union represents the area of the union of two detection boxes. The value range of IoU is [0,1], and the larger the value, the higher the overlap degree of the two detection boxes.

[0104] For overlapping detection boxes, NMM uses a weighted average method for merging. There are two overlapping detection boxes B1 and B2, with coordinates (x1,y1,w1,h1) and (x2,y2,w2,h2) respectively, and confidence levels s1 and s2. The coordinates and confidence level of the merged detection box can be calculated by the following formula:

[0105] Coordinate weighted average:

[0106]

[0107] The coordinate weighted average method in formula (4) needs to be adjusted according to specific circumstances in practical applications.

[0108] Preferably, when directly performing weighted averaging on coordinates does not conform to the geometric meaning, further, the center points of the detection frames can be weighted averaged while keeping the width and height unchanged; or, adjust according to the size of the merged frame.

[0109] In NMM, the conditions for merging detection frames are usually based on the IoU value. That is, when the IoU value of two detection frames exceeds a preset threshold, it is considered that they overlap and need to be merged. In the present invention, the IoU is set to 0.5. After slicing inference, the detection frames of the photovoltaic panels and the columns and their corresponding confidence levels can be obtained.

[0110] S3. Post-process the infrastructure target detection results of the to-be-detected image data to count the number of photovoltaic panels, the number of columns, and the installation angles of the photovoltaic panels.

[0111] In this embodiment, first, it is set that the confidence level of the photovoltaic panel is 0.6 and the confidence level of the column is 0.4. After obtaining the detection results, confidence level filtering is first performed;

[0112] For the columns, the number can be directly counted for the filtered results;

[0113] For the photovoltaic panels, outlier filtering needs to be performed on them again. The specific method is to calculate the width of the photovoltaic panels after confidence level filtering and use the interquartile range IQR method to filter out outliers. The calculation formula of IOR is:

[0114] IQR = Q3 - Q1, where Q3 is the third quartile of the data set and Q1 is the first quartile.

[0115] The outlier judgment formula is:

[0116] Lower limit = Q1 - 1.5 × IQR (5);

[0117] Upper limit = Q3 + 1.5 × IQR (6);

[0118] Any data point below the lower limit or above the upper limit is regarded as an outlier. Since in the same photovoltaic power station in the plain area, the specifications of the photovoltaic panels are basically the same, some extremely detected results can be filtered out through the above method.

[0119] S4. Process the filtered photovoltaic panel detection results to obtain more accurate photovoltaic panel detection frames for calculating the installation angles of the photovoltaic panels.

[0120] In this embodiment, referFigure 4 The width and height specifications of the photovoltaic panel when laid flat are shown, as well as the width and height specifications of the photovoltaic panel after being installed at a certain angle.

[0121] As can be seen from Figure 4 , the included angle between the photovoltaic panel and the ground can be calculated by calculating the cosine value through the actual height of the photovoltaic panel and its projection length H on the ground. The effect presented in the orthographic view is the projection of the photovoltaic panel on the ground. In the projection, the width w of the photovoltaic panel is not affected by the installation angle, and the height h becomes H in the projection. The actual width and height specifications of the photovoltaic panel are known. Assuming the actual width and height of the photovoltaic panel are width and height respectively, the included angle

[0122] To obtain the accurate outline of the photovoltaic panel, the following method is adopted:

[0123] (1) Crop the photovoltaic panel through a detection frame;

[0124] (2) Obtain the binary image of each photovoltaic panel;

[0125] (3) Perform morphological processing on the binary image, first dilate and then erode;

[0126] (4) Obtain the outline of the photovoltaic panel string picture and draw the outline on the original image;

[0127] (5) Obtain the outline with the largest area;

[0128] (6) Compare the area of the obtained outline with the area of the detection frame, set a threshold. When it is greater than the set threshold, take the width and height of the outline as the required result. If it is less than the set threshold, take the width w and height H of the detection frame;

[0129] (7) Calculate the pixel distance in the projection through the actual width of the photovoltaic panel, that is Calculate the height of the projection through the pixel distance as Finally, calculate the corresponding angle by calculating the cosine value with the actual height of the photovoltaic panel, that is

[0130]

[0131] (8) The detection frame detection includes components and columns. Finally, compare the number of detected components and columns with the number planned to be installed at the station. At the same time, each photovoltaic panel outputs its own angle.

[0132] Embodiment 2

[0133] This embodiment provides a device for implementing a method for detecting the infrastructure progress of a photovoltaic power station for drone patrol. The device includes:

[0134] At least one processor; and

[0135] A memory that stores instructions which, when executed by the at least one processor, cause the at least one processor to execute a method for detecting the infrastructure progress of a photovoltaic power station by means of drone inspection as described above.

[0136] In this embodiment, the electronic device includes but is not limited to: personal computers, server computers, workstations, desktop computers, laptop computers, notebook computers, mobile computing devices, smart phones, tablet computers, cellular phones, personal digital assistants (PDAs), handheld devices, messaging devices, wearable computing devices, consumer electronic devices, etc.

[0137] Embodiment 3

[0138] This embodiment also provides a computer-readable storage medium storing executable instructions that, when executed, cause the machine to execute a method for detecting the infrastructure progress of a photovoltaic power station by means of drone inspection as described above.

[0139] Specifically, a system or device equipped with a readable storage medium can be provided, on which software program code for implementing the functions of any one of the above embodiments is stored, and the computer or processor of the system or device is caused to read and execute the instructions stored in the readable storage medium.

[0140] In this case, the program code read from the readable medium itself can implement the functions of any one of the above embodiments, so the computer-readable code and the readable storage medium storing the computer-readable code constitute a part of this specification.

[0141] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer or a cloud via a communication network.

[0142] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0143] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce a means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or a means for implementing the functions specified in multiple blocks.

[0144] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction means that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or the functions specified in multiple blocks.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or the functions specified in multiple blocks.

[0146] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solutions of the present invention, rather than limitations on the specific implementation manners of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the claims of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A method for detecting the infrastructure progress of a photovoltaic power station during drone patrol, characterized in that, The method includes: S1. Shoot the materials required for 3D reconstruction through drone inspection, convert the inspection pictures into tif-format data through 3D reconstruction technology, and synthesize the orthophoto of the station as the image data to be detected; S2. Input the image data to be detected into the infrastructure detection model. The infrastructure detection model is used to detect the photovoltaic panels and columns in the photovoltaic power station. The corresponding detection results are obtained by performing SAHI slicing inference on the image data to be detected. The detection results include the defect category and the category confidence level; S3. Post-process the infrastructure target detection results of the image data to be detected, and count the number of photovoltaic panels, the number of columns, and the installation angle of the photovoltaic panels; S4. Process the filtered photovoltaic panel detection results to obtain a more accurate detection frame for the photovoltaic panels, which is used to calculate the installation angle of the photovoltaic panels.

2. The method for detecting the infrastructure progress of a photovoltaic power station inspected by a drone according to claim 1, characterized in that, S1 also includes preprocessing the image data to be detected: First, perform format conversion through the PIL library to convert the tif-format data into jpg format; Then perform equi-ratio scaling, and the scaling factor is one-fourth of the original. Then slice it according to the training data format, with the slice size of 1280*1280 and the overlap rate of 20%. The overlap rate calculation formula is: y_overlap = int(overlap_height_ratio * slice_height) (1); x_overlap = int(overlap_width_ratio * slice_width) (2); In formulas (1) to (2), x_overlap is the overlapping height in the horizontal direction, y_overlap is the overlapping height in the vertical direction, overlap_height_ratio refers to the slice height overlap ratio, slice_height refers to the slice height, overlap_width_ratio refers to the slice width overlap ratio, and slice_width refers to the slice width.

3. The method for detecting the construction progress of a photovoltaic power station inspected by a drone according to claim 1, wherein The infrastructure detection model is improved based on the YOLOv8 model, including: Embed the coordinate attention CA unit after the C2F module of the YOLOv8 backbone network, specifically located between the second C2F module and the fourth C2F module structure. Its optimization steps include: 1) Spatial dimension aggregation: The original input feature map is first subjected to global average pooling along the width and height axes respectively to generate two orthogonal dimension feature compression maps of W×1 and 1×H. H is the height of the image, and W is the width of the image; 2) Cross-dimensional feature fusion: Concatenate the channels of the above two spatial compression features to construct a joint representation matrix of H+W composite dimensions, and realize the cross-fusion of spatial information; 3) Channel dimension compression: Perform non-linear transformation on the joint representation matrix through a 1×1 convolution kernel to compress the channel dimension to C / r*1*(W+H). C refers to the number of channels, and r is the dimensionality reduction multiple, where C / r represents the dimensionality reduction multiple of the feature map in the C direction of the number of channels; 4) Spatial attention decoupling: After batch normalization and non-linear activation, the two compressed feature matrices are separated into two independent branches by feature separation. Each branch uses a 1×1 convolution to restore the dimension and applies the Sigmoid activation function to generate the attention probability distribution maps in the H and W axes; 5) Dynamic feature calibration: Multiply the decoupled spatial attention weights with the original input feature map channel by channel to achieve feature recalibration based on spatial importance.

4. A method for detecting the construction progress of a photovoltaic power station during drone patrol according to claim 3, characterized in that, The SAHI slicing inference of the image data to be detected specifically includes: S21. Image slicing before inputting into the infrastructure detection model: The original image is divided into multiple overlapping blocks of size M×N. These slices cover the entire image, and there is a certain overlapping area between adjacent slices; S22. Independent inference: Use the specified infrastructure detection model to perform inference on each slice to obtain the object detection results in that slice; S23. Result merging: Merge the prediction results of all slices together to generate a detection result list for the entire image; Among them, the target detection boxes are fused by NMM, and the formula for calculating the intersection over union (IoU) is: In formula (3), Area of Intersection represents the area of the intersection of two detection boxes, Area of Union represents the area of the union of two detection boxes, and the value range of IoU is [0,1]. The larger the value, the higher the overlapping degree of the two detection boxes; For overlapping detection boxes, NMM merges them in a weighted average manner. There are two overlapping detection boxes B1 and B2, with coordinates (x1, y1, w1, h1) and (x2, y2, w2, h2) respectively, and confidence levels s1 and s2 respectively. The coordinates and confidence level of the merged detection box are calculated by the following formula: Coordinate weighted average:

5. A method for detecting the infrastructure progress of a photovoltaic power station during unmanned aerial vehicle inspection according to claim 4, characterized in that, Post-process the infrastructure target detection results of the image data to be detected, including: For the detection results, first perform confidence filtering; For the columns, the number can be directly counted for the filtered results; For the photovoltaic panels, perform outlier filtering on them. The specific method is to calculate the width of the photovoltaic panels after confidence filtering and use the interquartile range method to filter out the outliers. The formula for calculating the interquartile range (IOR) is: IQR = Q3 - Q1, where Q3 is the third quartile of the dataset and Q1 is the first quartile; The outlier judgment formula is: Lower limit = Q1 - 1.5×IQR(5); Upper limit = Q3 + 1.5×IQR(6); Any data point below the lower limit or above the upper limit is considered an outlier.

6. The method for detecting the infrastructure progress of a photovoltaic power station during drone patrol according to claim 5, wherein Obtain a more accurate detection box for the photovoltaic panels, specifically including: (1) Crop the photovoltaic panels through the detection box; (2) Obtain the binary image of each photovoltaic panel; (3) Perform morphological processing on the binary image, first dilate and then erode; (4) Obtain the contour of the photovoltaic panel string image and draw the contour on the original image; (5) Obtain the contour with the largest area; (6) Compare the obtained contour with the area of the detection box, set a threshold. When it is greater than the set threshold, take the width and height of the contour as the required results. If it is less than the set threshold, take the width and height of the detection box, corresponding to w and H in the formula. (7) Calculate the pixel distance in the projection through the actual width of the photovoltaic panel, that is Calculate the height of the projection through the pixel distance as Finally, obtain the corresponding angle by calculating the cosine value of the actual height of the photovoltaic panel, that is (8) Finally, compare the number of detected components and the number of columns with the number planned to be installed at the station. At the same time, each photovoltaic panel outputs its respective angle.

7. An apparatus for a method of detecting the construction progress of a photovoltaic power station during drone patrol, characterized in that, The device includes: A processor; A memory having stored thereon a computer program that can run on the processor; Wherein, when the computer program is executed by the processor, it implements the steps of a method for detecting the construction progress of a photovoltaic power station during unmanned aerial vehicle inspection as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images