Unmanned aerial vehicle photovoltaic diagnosis method and device, storage medium and unmanned aerial vehicle

Abstract

The invention discloses an unmanned aerial vehicle photovoltaic diagnosis method and device, a storage medium and an unmanned aerial vehicle, and the method comprises the steps: carrying out the segmentation processing of a photovoltaic image collected by the unmanned aerial vehicle, and obtaining at least one photovoltaic sub-image; performing evaluation processing on each photovoltaic sub-graph by using a pre-trained image evaluation classification model to obtain an evaluation grade corresponding to each photovoltaic sub-graph; determining a target super-resolution algorithm according to the evaluation grade, and carrying out super-resolution restoration processing on the corresponding photovoltaic sub-graph by adopting the target super-resolution algorithm to obtain each photovoltaic restoration image; and diagnosing each photovoltaic restoration image to obtain a fault diagnosis result of the photovoltaic image. According to the unmanned aerial vehicle photovoltaic diagnosis method, the calculated amount of the super-resolution algorithm is reduced while the quality of the photovoltaic restoration image is ensured, the real-time performance of fault diagnosis is improved, and the method has low requirements on unmanned aerial vehicle hardware configuration and can be deployed on an edge calculation device of the unmanned aerial vehicle.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic diagnostic technology, and in particular to a photovoltaic diagnostic method, device and storage medium for unmanned aerial vehicles (UAVs), and the UAV itself. Background Technology

[0002] This solution for automated inspection and diagnosis of photovoltaic modules based on drones utilizes drones to collect image data of photovoltaic modules and uses AI-based image recognition algorithms to automatically diagnose faults in the collected photovoltaic module image data.

[0003] However, the resolution of cameras on drones is limited, and they may be affected by factors such as lighting and weather during flight, resulting in low-resolution and blurry images of photovoltaic panels. To improve the image quality of photovoltaic modules acquired by drones, image super-resolution algorithms have been introduced. However, in drone-based photovoltaic inspection scenarios, these image super-resolution algorithms suffer from high computational complexity and require high-end hardware, limiting their practical application. Summary of the Invention

[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, one objective of this invention is to propose a photovoltaic diagnostic method for UAVs that reduces the computational load of super-resolution algorithms while ensuring the quality of photovoltaic restoration images and improving the real-time performance of fault diagnosis. Furthermore, it has low hardware requirements for UAVs and can be deployed on the edge computing devices of UAVs.

[0005] The second objective of this invention is to provide a photovoltaic diagnostic device for unmanned aerial vehicles (UAVs).

[0006] A third objective of this invention is to provide a computer-readable storage medium.

[0007] The fourth objective of this invention is to provide an electronic device.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a photovoltaic (PV) diagnostic method for unmanned aerial vehicles (UAVs). The method includes: segmenting PV images acquired by the UAV to obtain at least one PV sub-image; evaluating each PV sub-image using a pre-trained image evaluation and classification model to obtain an evaluation level corresponding to each PV sub-image; determining a target super-resolution algorithm based on the evaluation level, and using the target super-resolution algorithm to perform super-resolution repair processing on the corresponding PV sub-image to obtain repaired PV images; and diagnosing each repaired PV image to obtain a fault diagnosis result for the PV image.

[0009] According to the UAV photovoltaic diagnostic method of the present invention, the photovoltaic image acquired by the UAV is segmented, and a photovoltaic sub-image containing only the photovoltaic module portion is retained. The quality of each photovoltaic sub-image is evaluated, and a target super-resolution algorithm is used to perform super-resolution repair processing on the corresponding photovoltaic sub-image based on the evaluation results. The photovoltaic repaired image obtained after repair processing is then diagnosed. This method reduces the computational load of the super-resolution algorithm while ensuring the quality of the photovoltaic repaired image and improves the real-time performance of fault diagnosis. Furthermore, the method has low requirements for UAV hardware configuration and can be deployed on the edge computing device of the UAV.

[0010] In addition, the UAV photovoltaic diagnostic method proposed in the above embodiments of the present invention may also have the following additional technical features: According to an embodiment of the present invention, the training process of the image evaluation classification model includes: acquiring a training dataset, the training dataset including low-resolution images collected by a drone and their corresponding true rating categories; inputting the low-resolution images in the training dataset into the image evaluation classification model to obtain predicted rating categories; constructing a loss function based on the true rating categories and predicted rating categories of the low-resolution images; and training the image evaluation classification model based on the loss function to obtain the trained image evaluation classification model.

[0011] According to one embodiment of the present invention, the image evaluation classification model includes a first convolutional layer, a first ReLU activation layer, a second convolutional layer, a second activation layer, a third convolutional layer, a third activation layer, a fourth convolutional layer, an average pooling layer, a fully connected layer, and a Softmax layer connected in sequence.

[0012] According to one embodiment of the present invention, obtaining the training dataset includes: using a preset super-resolution algorithm to perform restoration processing on the low-resolution image to obtain a low-resolution restored image; calculating the mean square error between the low-resolution image and its corresponding low-resolution restored image, and calculating the peak signal-to-noise ratio (PSNR) of the low-resolution image and its corresponding low-resolution restored image based on the mean square error; and determining the true class category of the low-resolution image based on the PSNR of the low-resolution image and its corresponding low-resolution restored image.

[0013] According to one embodiment of the present invention, the peak signal-to-noise ratio is negatively correlated with the true class level.

[0014] According to one embodiment of the present invention, the evaluation level includes a first level, a second level, a third level, and a fourth level, and the super-resolution algorithm includes the FSRCNN super-resolution algorithm, the CARN super-resolution algorithm, the SRResNet super-resolution algorithm, and the RCAN super-resolution algorithm. Determining the target super-resolution algorithm based on the evaluation level includes: if the evaluation level is the first level, then the target super-resolution algorithm is the FSRCNN super-resolution algorithm; if the evaluation level is the second level, then the target super-resolution algorithm is the CARN super-resolution algorithm; if the evaluation level is the third level, then the target super-resolution algorithm is the SRResNet super-resolution algorithm; and if the evaluation level is the fourth level, then the target super-resolution algorithm is the RCAN super-resolution algorithm.

[0015] According to one embodiment of the present invention, the method includes: segmenting photovoltaic images acquired by a UAV using a pre-trained UNet segmentation model.

[0016] To achieve the above objectives, a second aspect of the present invention provides a photovoltaic diagnostic device for unmanned aerial vehicles (UAVs). The device includes: a segmentation module for segmenting photovoltaic images acquired by the UAV to obtain at least one photovoltaic sub-image; an evaluation module for evaluating each photovoltaic sub-image using a pre-trained image evaluation classification model to obtain an evaluation level corresponding to each photovoltaic sub-image; a repair module for determining a target super-resolution algorithm based on the evaluation level and using the target super-resolution algorithm to perform super-resolution repair processing on the corresponding photovoltaic sub-image to obtain repaired photovoltaic images; and a diagnosis module for diagnosing each repaired photovoltaic image to obtain a fault diagnosis result for the photovoltaic image.

[0017] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described photovoltaic diagnostic method for unmanned aerial vehicles.

[0018] To achieve the above objectives, a fourth aspect of the present invention provides a drone, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the drone photovoltaic diagnostic method as described above.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] Figure 1(a) is a schematic diagram of a photovoltaic image captured by a high-resolution camera mounted on a drone; Figure 1(b) is a schematic diagram of a photovoltaic image captured by a low-resolution camera mounted on a drone; Figure 2(a) is a schematic diagram of photovoltaic images collected by the UAV at a low flight altitude; Figure 2(b) is a schematic diagram of photovoltaic images collected by the UAV at a higher flight altitude; Figure 3 This is a flowchart of a photovoltaic diagnostic method for unmanned aerial vehicles according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a photovoltaic image captured by a drone according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the network structure of an image evaluation and classification model according to an embodiment of the present invention; Figure 6 This is a flowchart of the training process of an image evaluation and classification model according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating the acquisition of a training dataset according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the network structure of a UNet segmentation model according to an embodiment of the present invention; Figure 9(a) is a schematic diagram of a photovoltaic image before segmentation according to an embodiment of the present invention; Figure 9(b) is a schematic diagram of photovoltaic image segmentation according to an embodiment of the present invention; Figure 10 This is a schematic diagram of a UAV photovoltaic diagnostic device according to an embodiment of the present invention; Figure 11 This is a structural block diagram of the drone according to an embodiment of the present invention. Detailed Implementation

[0021] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0022] With the rapid development of the photovoltaic power generation industry, the scale of photovoltaic power plants is constantly expanding. Ensuring the stable and efficient operation of photovoltaic power plants is crucial for improving power generation efficiency. Currently, photovoltaic power plant inspections mainly rely on two methods: manual inspection and drone inspection.

[0023] Manual inspections suffer from low efficiency, high labor intensity, and geographical limitations, making it difficult to meet the inspection needs of large-scale photovoltaic power plants. While drone inspections can overcome the drawbacks of manual inspections to some extent, in practical applications, the limited resolution of drone-mounted cameras and the potential impact of factors such as lighting and weather during flight often result in low-resolution, blurry images of photovoltaic panels. This poses significant challenges to subsequent fault identification and analysis. For example, subtle cracks, stains, and hot spots are difficult to accurately identify from low-resolution images, leading to missed or misjudged faults and affecting the normal operation and maintenance of photovoltaic power plants. Figures 1(a) and 1(b) show images of photovoltaic panels in the same area collected by the same drone equipped with both high-resolution and low-resolution cameras. Furthermore, to ensure flight safety, drones need to increase their flight altitude when there are obstacles near the photovoltaic power plant, such as transmission lines and towers. This higher altitude also reduces image resolution. Figures 2(a) and 2(b) show images of photovoltaic panels in the same area collected by the same drone at different flight altitudes.

[0024] To improve the image quality of photovoltaic panels acquired by drones, image super-resolution algorithms are introduced. Super-resolution (SR) is an image processing technique whose main purpose is to convert low-resolution (LR) images into high-resolution (HR) images using complex algorithms and advanced techniques.

[0025] Mainstream image super-resolution algorithms mainly include traditional image super-resolution algorithms and deep learning-based super-resolution algorithms. Traditional image super-resolution algorithms include interpolation methods and sparse representation-based algorithms; their main drawbacks are poor generalization ability and poor adaptability to complex textures. Deep learning-based super-resolution algorithms include CNN-based algorithms, algorithms based on residual networks and optimized structures, algorithms based on generative adversarial networks, and algorithms based on Transformers. Their advantages are stronger detail reconstruction capabilities and better generalization ability.

[0026] In the field of photovoltaic inspection, super-resolution algorithms based on deep learning (such as SRResNet) are more commonly used due to their high recognition accuracy and good generalization ability. However, these algorithms have high hardware requirements for the inference model and long processing time, which increases implementation costs and makes it difficult to deploy edge computing devices on drones for real-time diagnosis.

[0027] To address the aforementioned problems, embodiments of the present invention provide a photovoltaic diagnostic method, apparatus, storage medium, and drone for unmanned aerial vehicles (UAVs). The following detailed description, in conjunction with the accompanying drawings and specific embodiments, provides a detailed explanation of the photovoltaic diagnostic method, apparatus, storage medium, and drone for unmanned aerial vehicles (UAVs) according to embodiments of the present invention.

[0028] Figure 3 This is a flowchart of a photovoltaic diagnostic method for unmanned aerial vehicles (UAVs) according to an embodiment of the present invention. Figure 3 As shown, a photovoltaic diagnostic method for unmanned aerial vehicles (UAVs) may include: S101, segment the photovoltaic images collected by the drone to obtain at least one photovoltaic sub-image.

[0029] Specifically, the drone is controlled to inspect photovoltaic (PV) modules in a target area at a preset altitude, acquiring images of the PV modules in the target area (PV images). Since PV modules typically only occupy 50% of the PV image area acquired by the drone, see [link to relevant documentation]. Figure 4 .

[0030] To reduce the computational load of the super-resolution algorithm carried on the UAV, this embodiment of the invention performs segmentation processing on the photovoltaic images collected by the UAV, that is, segments the photovoltaic module part and the background part in the photovoltaic image, and retains only the image of the photovoltaic module part to obtain at least one photovoltaic sub-image.

[0031] S102, using a pre-trained image evaluation and classification model, evaluate each photovoltaic sub-image to obtain the evaluation level corresponding to each photovoltaic sub-image.

[0032] Because the imaging quality of photovoltaic modules varies depending on the shooting altitude and environment during drone inspection, the resulting images also differ. For images with high quality, a less complex super-resolution algorithm can be used; for images with low quality, a more complex super-resolution algorithm is required to ensure the quality of the repaired image. Therefore, to further reduce the computational load of the super-resolution algorithm mounted on the drone, this embodiment of the invention evaluates each photovoltaic sub-image to determine the target super-resolution algorithm corresponding to each sub-image based on the evaluation results.

[0033] In practice, each photovoltaic sub-image is input into a pre-trained image evaluation and classification model. The pre-trained image evaluation and classification model evaluates each photovoltaic sub-image and outputs its predicted evaluation level for each photovoltaic sub-image. Based on the evaluation level of each photovoltaic sub-image, the target super-resolution algorithm corresponding to each photovoltaic sub-image is determined.

[0034] S103. Determine the target super-resolution algorithm based on the evaluation level, and use the target super-resolution algorithm to perform super-resolution repair processing on the corresponding photovoltaic sub-image to obtain each photovoltaic repair image.

[0035] Specifically, the target super-resolution algorithm for each photovoltaic sub-image is determined based on its evaluation level. The target super-resolution algorithm is then used to perform super-resolution inpainting on the corresponding photovoltaic sub-image, thereby ensuring the quality of the photovoltaic inpainted image while reducing the computational load of the super-resolution algorithm.

[0036] S104, perform diagnosis on each photovoltaic repair image to obtain the fault diagnosis results of the photovoltaic image.

[0037] Specifically, by using a pre-trained fault diagnosis model to diagnose each photovoltaic repair image, fault diagnosis results of the photovoltaic images collected by the UAV can be obtained.

[0038] The photovoltaic diagnostic method for drones in this invention segmentes photovoltaic images acquired by the drone, retains photovoltaic sub-images containing only the photovoltaic modules, evaluates the quality of each photovoltaic sub-image, performs super-resolution repair processing on the corresponding photovoltaic sub-images based on the evaluation results, and diagnoses the repaired photovoltaic images. This method reduces the computational load of the super-resolution algorithm while ensuring the quality of the repaired photovoltaic images and improves the real-time performance of fault diagnosis. Furthermore, this method has low requirements for drone hardware configuration and can be deployed on the edge computing device of the drone.

[0039] To reduce resource consumption, this embodiment of the invention performs a quality assessment on the segmented photovoltaic subgraphs to match each subgraph with a suitable super-resolution algorithm. Specifically, higher-quality photovoltaic subgraphs are matched with super-resolution algorithms that consume less computational resources, while lower-quality subgraphs are matched with super-resolution algorithms that consume more computational resources.

[0040] The image evaluation classification model in this embodiment of the invention uses the Image Evaluation Classification Network (IECN) to evaluate the photovoltaic sub-image. In one embodiment of the present invention, such as Figure 5 As shown, the image evaluation classification model may include a first convolutional layer, a first ReLU activation layer, a second convolutional layer, a second activation layer, a third convolutional layer, a third activation layer, a fourth convolutional layer, an average pooling layer, a fully connected layer, and a softmax layer connected in sequence.

[0041] Specifically, the image evaluation classification model uses a lightweight classification network to evaluate the input image and determine its super-resolution difficulty.

[0042] See Figure 5The image evaluation and classification network extracts features using a small number of convolutional layers, and then outputs the probability results, corresponding to the M-class probabilities, using an average pooling layer and a fully connected layer. In this embodiment of the invention, the image evaluation and classification network has a simple structure and consumes very few computational resources.

[0043] In one embodiment of the present invention, such as Figure 6 As shown, the training process of an image evaluation classification model may include: S201, Obtain the training dataset, which may include low-resolution images collected by the drone and their corresponding true rank categories; S202, input the low-resolution images from the training dataset into the image evaluation classification model to obtain the predicted grade category; S203, construct a loss function based on the true and predicted class categories of the low-resolution image; S204. Train the image evaluation classification model according to the loss function to obtain the trained image evaluation classification model.

[0044] Specifically, a large number of low-resolution images collected by drones are acquired, and the corresponding grade category of each low-resolution image is determined. The determined grade category is recorded as the true grade category of the low-resolution image.

[0045] In one specific embodiment, the true classification level can be preset to 4, that is, the classification level corresponding to the low-resolution image is divided into four levels, namely the first level, the second level, the third level, and the fourth level. It should be noted that the higher the level of the low-resolution image, the higher the complexity of the super-resolution algorithm required.

[0046] Low-resolution images from the training dataset are input into an image rating classification model. The model evaluates these images and outputs their predicted rating categories. A loss function is constructed based on the true and predicted rating categories of the low-resolution images. The model is then trained using this loss function. A well-trained image rating classification model is obtained when the loss function converges to its minimum value.

[0047] In one specific embodiment, the image evaluation and classification model of this invention uses cross-entropy as the loss function. Among them, the loss function The corresponding expression is:

[0048] Where y is the probability distribution of the true rank category (usually one-hot encoded). It is the probability distribution of the predicted grade categories, where m is the preset total number of grade categories.

[0049] In one embodiment of the present invention, such as Figure 7 As shown, obtaining the training dataset can include: S301, using a preset super-resolution algorithm, performs restoration processing on the low-resolution image to obtain a low-resolution restored image; S302, calculate the mean square error between the low-resolution image and its corresponding low-resolution restored image, and calculate the peak signal-to-noise ratio between the low-resolution image and its corresponding low-resolution restored image based on the mean square error. S303, determine the true grade category of the low-resolution image based on the peak signal-to-noise ratio of the low-resolution image and its corresponding low-resolution restored image.

[0050] Specifically, when determining the grade category corresponding to each low-resolution image in the training dataset, a preset super-resolution algorithm can be used to repair the low-resolution images.

[0051] It should be noted that when determining the preset super-resolution algorithm, considering that the training process of image evaluation and classification models is usually completed on a server, the preset super-resolution algorithm can be the RCAN super-resolution algorithm, which has a large computational resource consumption but good performance (other super-resolution algorithms with similar performance can also be used). This embodiment of the invention does not limit the preset super-resolution algorithm.

[0052] A low-resolution restored image is generated for each low-resolution image in the training dataset using a pre-defined super-resolution algorithm, resulting in a low-resolution image-low-resolution restored image pairing set (LR-SR pairing set).

[0053] Calculate the mean squared error (MSE) between the low-resolution image and its corresponding low-resolution restored image. The expression for the mean squared error (MSE) is as follows:

[0054] in, The mean squared error is represented by M, the image height by N, and the image width by I. I represents a low-resolution image. This indicates a low-resolution image being repaired.

[0055] The Peak Signal-to-Noise Ratio (PSNR) of the low-resolution image and its corresponding low-resolution restored image is calculated based on the mean squared error. PSNR measures the similarity between the restored image and the low-resolution image; a higher PSNR indicates better reconstruction. The expression for PSNR is as follows:

[0056] in, Indicates peak signal-to-noise ratio. This represents the mean square error. This represents the maximum possible pixel value for a low-resolution image. For example, for an 8-bit format low-resolution image, The value is set to 255.

[0057] The classification of a low-resolution image is determined based on the Peak Signal-to-Noise Ratio (PSNR) of the low-resolution image and its corresponding low-resolution restored image. The PSNR interval (1 / M) is determined based on the preset total number of classification categories M. For example, when the preset total number of classification categories M is 4, the PSNR is divided into four categories at 1 / 4 intervals: Simple (the highest 1 / 4 of the images), Moderate (PSNR between 1 / 4 and 1 / 2), Difficult (PSNR between 1 / 2 and 3 / 4), and Very Difficult (the lowest 1 / 4 of the images).

[0058] In one embodiment of the present invention, the peak signal-to-noise ratio is negatively correlated with the true class level.

[0059] Specifically, a higher peak signal-to-noise ratio (PSNR) indicates a better reconstruction effect, a lower complexity super-resolution algorithm, and a lower corresponding classification level; conversely, a lower PSNR indicates a worse reconstruction effect, a higher complexity super-resolution algorithm, and a higher corresponding classification level.

[0060] In this embodiment of the invention, the super-resolution algorithms, from low to high complexity, are: FSRCNN super-resolution algorithm, CARN super-resolution algorithm, SRResNet super-resolution algorithm, and RCAN super-resolution algorithm.

[0061] In one specific embodiment of the present invention, the evaluation level may include a first level, a second level, a third level, and a fourth level; the super-resolution algorithm may include the FSRCNN super-resolution algorithm, the CARN super-resolution algorithm, the SRResNet super-resolution algorithm, and the RCAN super-resolution algorithm; determining the target super-resolution algorithm based on the evaluation level may include: If the evaluation level is Level 1, then the target super-resolution algorithm is the FSRCNN super-resolution algorithm; If the evaluation level is level two, then the target super-resolution algorithm is the CARN super-resolution algorithm; If the evaluation level is level three, then the target super-resolution algorithm is the SRResNet super-resolution algorithm; If the evaluation level is level four, then the target super-resolution algorithm is the RCAN super-resolution algorithm.

[0062] Specifically, when the preset total number of level categories M is 4, the actual evaluation levels include the first level, the second level, the third level, and the fourth level. Super-resolution algorithms, ranked from lowest to highest complexity, include FSRCNN, CARN, SRResNet, and RCAN.

[0063] Therefore, when the photovoltaic sub-image output by the pre-trained image evaluation and classification model has an evaluation level of Level 1, the FSRCNN super-resolution algorithm is used for super-resolution restoration. When the evaluation level is Level 2, the CARN super-resolution algorithm is used. When the evaluation level is Level 3, the SRResNet super-resolution algorithm is used. When the evaluation level is Level 4, the RCAN super-resolution algorithm is used.

[0064] In one embodiment of the present invention, the UAV photovoltaic diagnostic method may include: segmenting photovoltaic images acquired by the UAV using a pre-trained UNet segmentation model.

[0065] In practice, a pre-trained UNet segmentation model can be used to segment photovoltaic images acquired by a drone. It should be noted that the UNet segmentation model is a deep learning model for image segmentation tasks, named for its U-shaped network structure.

[0066] The UNet segmentation model in this embodiment of the invention uses an encoder and decoder structure. The encoder consists of a series of convolutional and pooling layers, which extract and compress features from the input image. The decoder then uses upsampling and deconvolutional layers to expand the low-resolution feature maps output by the encoder back to the original resolution, thereby obtaining the segmentation result.

[0067] The UNet segmentation model in this embodiment employs a skip connection structure. Skip connections link certain layers in the encoder with corresponding layers in the decoder, allowing the decoder to utilize more low-level features for segmentation. This skip connection structure effectively addresses the problems of information loss and inaccurate segmentation during the process.

[0068] See the network structure of the UNet segmentation model. Figure 8The system mainly consists of: Input layer: receives the input image. Encoder: consists of convolutional and pooling layers, used to progressively reduce the image size and extract features. Decoder: consists of convolutional and upsampling layers, used to progressively restore the image size and combine the features extracted from the encoder with the corresponding layers of the decoder. Skip connections: combine the features from the encoder with the corresponding layers of the decoder to restore the image resolution. Output layer: outputs a segmented image with the same size as the input image, where each pixel is labeled as one of the segmentation categories.

[0069] Specifically, when using the UNet segmentation model to segment photovoltaic images, the photovoltaic image is input into the input layer of the UNet segmentation model. While receiving the photovoltaic image, the input layer can also perform preprocessing on the input image. For example, operations such as horizontal flipping and pixel mirroring can be performed on the input image to increase data diversity and allow edge pixels to obtain similar contextual information. Furthermore, the large image is divided into multiple small patches with overlapping areas, facilitating subsequent processing.

[0070] The encoder downsamples the input image: the image is processed through a series of convolutional layers for feature extraction. Each step typically involves two 3×3 convolutional operations (without padding), followed by a ReLU activation function. Next, a 2×2 max-pooling operation with a stride of 2 is performed for downsampling, halving the feature map size while doubling the number of feature channels. Through multiple such downsampling steps, the image size is gradually reduced, and higher-level semantic information is extracted, allowing the network to capture a wider range of contextual information.

[0071] The decoder upsamples the input image: each step first upsamples the feature map, which can be achieved through transposed convolution (deconvolution) or upsampling layers (such as bilinear interpolation, nearest neighbor interpolation, etc.) to enlarge the feature map size. Since convolution operations cause the loss of boundary pixels, the feature map of the corresponding layer in the encoder needs to be cropped to match the size of the upsampled feature map. Then, the two are concatenated to fuse the high-resolution information from the encoder. Two more 3×3 convolution operations are then performed, each followed by a ReLU activation function for further feature map processing.

[0072] Output layer processing: In the last layer of the decoder, a 1×1 convolution operation is used to map each feature vector to the required number of categories, resulting in a segmentation image of the same size as the input image. Each pixel is then classified to determine its segmentation category. See Figures 9(a) and 9(b) for the before and after segmentation of photovoltaic images acquired by a UAV using the UNet segmentation model.

[0073] The UAV photovoltaic diagnostic method of this invention enables low-cost image super-resolution processing during UAV photovoltaic diagnostics. This method employs image segmentation and peak signal-to-noise ratio (PSNR)-based image evaluation techniques, optimizing the super-resolution algorithm from global image processing and single-algorithm processing to a mode of partial image processing and targeted algorithm selection based on image quality. This solves the problem of high computational resource requirements of traditional super-resolution algorithms, significantly reducing the hardware configuration cost of UAV photovoltaic inspections. Furthermore, the image processing module can be pre-positioned on the edge computing device on the UAV to achieve real-time fault diagnosis. This technology provides strong technical support for the intelligent and unmanned operation and maintenance of photovoltaic power plants.

[0074] The UAV photovoltaic diagnostic method of this invention segments photovoltaic modules from photovoltaic images using image segmentation algorithms (such as the U-Net segmentation model). Super-resolution processing is performed only on the photovoltaic sub-image containing the photovoltaic modules, reducing the computational load of the super-resolution algorithm. To further reduce the computational complexity of super-resolution, the image quality of the retained photovoltaic sub-images after segmentation is evaluated. Images with better quality are processed using low-computation super-resolution algorithms (such as FSRCNN), while images with poorer quality are processed using high-computation super-resolution algorithms (such as RCAN), thus meeting the requirements of low-configuration computing resources.

[0075] The UAV photovoltaic diagnostic method of this invention has low hardware configuration requirements for the UAV, can quickly achieve super-resolution image processing, and realize real-time fault diagnosis of photovoltaic modules.

[0076] This invention provides a photovoltaic diagnostic device for unmanned aerial vehicles (UAVs).

[0077] Figure 10 This is a schematic diagram of a drone photovoltaic diagnostic device according to an embodiment of the present invention. Figure 10 As shown, the UAV photovoltaic diagnostic device 100 may include a segmentation module 10, an evaluation module 20, a repair module 30, and a diagnostic module 40.

[0078] The segmentation module 10 is used to segment the photovoltaic images collected by the UAV to obtain at least one photovoltaic sub-image; the evaluation module 20 is used to evaluate each photovoltaic sub-image using a pre-trained image evaluation classification model to obtain the evaluation level corresponding to each photovoltaic sub-image; the repair module 30 is used to determine the target super-resolution algorithm based on the evaluation level and use the target super-resolution algorithm to perform super-resolution repair processing on the corresponding photovoltaic sub-image to obtain each photovoltaic repaired image; the diagnosis module 40 is used to diagnose each photovoltaic repaired image to obtain the fault diagnosis result of the photovoltaic image.

[0079] It should be noted that other specific embodiments of the UAV photovoltaic diagnostic device provided in the embodiments of the present invention can be found in other specific embodiments of the UAV photovoltaic diagnostic method of the above embodiments of the present invention.

[0080] The UAV photovoltaic diagnostic device of this invention enables low-cost image super-resolution processing during UAV photovoltaic diagnostics. This method employs image segmentation and peak signal-to-noise ratio (PSNR)-based image evaluation techniques, optimizing the super-resolution algorithm from global image processing and single-algorithm processing to a mode of partial image processing and targeted algorithm selection based on image quality. This solves the problem of high computational resource requirements of traditional super-resolution algorithms, significantly reducing the hardware configuration cost of UAV photovoltaic inspections. Furthermore, the image processing module can be pre-positioned on the edge computing device on the UAV to achieve real-time fault diagnosis. This technology provides strong technical support for the intelligent and unmanned operation and maintenance of photovoltaic power plants.

[0081] This invention provides a computer-readable storage medium.

[0082] In this embodiment, a computer program is stored on a computer-readable storage medium. When the computer program is executed by a processor, it can implement the UAV photovoltaic diagnostic method described above.

[0083] This invention provides a drone.

[0084] In this embodiment, the drone may include a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, it implements the drone photovoltaic diagnostic method described above.

[0085] Figure 11 This is a structural block diagram of the drone according to an embodiment of the present invention.

[0086] like Figure 11 As shown, the drone 500 includes a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the drone 500 may also include a transceiver 504. It should be noted that in practical applications, the transceiver 504 is not limited to one type, and the structure of the drone 500 does not constitute a limitation on the embodiments of the present invention.

[0087] Processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 501 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0088] Bus 502 may include a pathway for transmitting information between the aforementioned components. Bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 502 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0089] The memory 503 stores a computer program corresponding to the UAV photovoltaic diagnostic method of the above embodiments of the present invention. This computer program is controlled and executed by the processor 501. The processor 501 executes the computer program stored in the memory 503 to implement the content shown in the foregoing method embodiments. Figure 11 The drone 500 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0090] The computer-readable storage medium and the drone of the present invention, according to the embodiments of the present invention, use the above-described drone photovoltaic diagnostic method to perform real-time diagnostics on the photovoltaic images collected by the drone.

[0091] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0092] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0093] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0094] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0095] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0096] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0097] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0098] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A photovoltaic diagnostic method for unmanned aerial vehicles (UAVs), characterized in that, The method includes: The photovoltaic images collected by the drone are segmented to obtain at least one photovoltaic sub-image; Using a pre-trained image evaluation and classification model, each photovoltaic sub-image is evaluated to obtain the evaluation level corresponding to each photovoltaic sub-image. The target super-resolution algorithm is determined based on the evaluation level, and the target super-resolution algorithm is used to perform super-resolution repair processing on the corresponding photovoltaic sub-image to obtain each photovoltaic repair image; Each of the photovoltaic repair images is diagnosed to obtain the fault diagnosis results of the photovoltaic images.

2. The UAV photovoltaic diagnostic method according to claim 1, characterized in that, The training process of the image evaluation and classification model includes: Obtain a training dataset, which includes low-resolution images collected by the UAV and their corresponding true rank categories; The low-resolution images in the training dataset are input into the image evaluation and classification model to obtain the predicted grade category. A loss function is constructed based on the true and predicted class categories of the low-resolution image; The image evaluation classification model is trained according to the loss function to obtain the trained image evaluation classification model.

3. The UAV photovoltaic diagnostic method according to claim 2, characterized in that, The image evaluation and classification model includes a first convolutional layer, a first ReLU activation layer, a second convolutional layer, a second activation layer, a third convolutional layer, a third activation layer, a fourth convolutional layer, an average pooling layer, a fully connected layer, and a softmax layer, connected in sequence.

4. The UAV photovoltaic diagnostic method according to claim 2, characterized in that, The acquisition of the training dataset includes: The low-resolution image is repaired using a preset super-resolution algorithm to obtain a low-resolution repaired image. Calculate the mean square error between the low-resolution image and its corresponding low-resolution restored image, and calculate the peak signal-to-noise ratio between the low-resolution image and its corresponding low-resolution restored image based on the mean square error; The true classification of the low-resolution image is determined based on the peak signal-to-noise ratio of the low-resolution image and its corresponding low-resolution restored image.

5. The UAV photovoltaic diagnostic method according to claim 4, characterized in that, The peak signal-to-noise ratio is negatively correlated with the true class level.

6. The UAV photovoltaic diagnostic method according to claim 4, characterized in that, The evaluation levels include a first level, a second level, a third level, and a fourth level. The super-resolution algorithms include FSRCNN super-resolution algorithm, CARN super-resolution algorithm, SRResNet super-resolution algorithm, and RCAN super-resolution algorithm. Determining the target super-resolution algorithm based on the evaluation levels includes: If the evaluation level is the first level, then the target super-resolution algorithm is the FSRCNN super-resolution algorithm; If the evaluation level is the second level, then the target super-resolution algorithm is the CARN super-resolution algorithm; If the evaluation level is level three, then the target super-resolution algorithm is the SRResNet super-resolution algorithm; If the evaluation level is level four, then the target super-resolution algorithm is the RCAN super-resolution algorithm.

7. The photovoltaic diagnostic method for unmanned aerial vehicles according to claim 1, characterized in that, The method includes: segmenting photovoltaic images collected by a drone using a pre-trained UNet segmentation model.

8. A photovoltaic diagnostic device for unmanned aerial vehicles, characterized in that, The device includes: The segmentation module is used to segment the photovoltaic images collected by the UAV to obtain at least one photovoltaic sub-image; The evaluation module is used to evaluate each photovoltaic sub-image using a pre-trained image evaluation and classification model to obtain the evaluation level corresponding to each photovoltaic sub-image. The repair module is used to determine the target super-resolution algorithm according to the evaluation level, and use the target super-resolution algorithm to perform super-resolution repair processing on the corresponding photovoltaic sub-image to obtain each photovoltaic repair image; The diagnostic module is used to diagnose each of the photovoltaic repair images and obtain the fault diagnosis results of the photovoltaic images.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the UAV photovoltaic diagnostic method as described in any one of claims 1-7.

10. A drone, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the computer program is executed by the processor, it implements the UAV photovoltaic diagnostic method as described in any one of claims 1-7.

Images