An efficient method for positioning photovoltaic inspection results using drones

Abstract

This invention discloses an efficient method for locating the results of unmanned aerial vehicle (UAV) photovoltaic inspections, relating to the fields of UAV applications and photovoltaic power station detection technology. The method comprises: optimizing the real-time position of the UAV using a fusion algorithm combined with multi-source sensor data; enhancing the low-resolution portions of visible light images and temperature distribution images using a super-resolution reconstruction algorithm; matching the pre-processed and enhanced visible light and temperature distribution images with the UAV's positioning information; and extracting key features of the photovoltaic components from the matched visible light and temperature distribution images. The invention combines a particle filter algorithm with deep learning. The particle filter is used to predict and update the UAV's position status, while deep learning is used to process multi-source sensor data, enhancing the ability to perceive complex environmental features. This method allows for more accurate estimation of the UAV's position, addressing the issue of decreased positioning accuracy in complex environments caused by traditional triangulation.

Description

Technical Field

[0001] The present invention relates to the field of drone application and photovoltaic power station detection technology, and in particular to an efficient method for positioning photovoltaic inspection results using a drone. Background Art

[0002] With the rapid development of the photovoltaic industry, the scale of photovoltaic power plants continues to expand. Especially with the increasing market demand for photovoltaic power generation, the operation, maintenance, and management of photovoltaic power plants have become increasingly important. Traditional manual photovoltaic inspections require significant manpower and material resources, making them inefficient and costly. Furthermore, manual inspections can be challenging to detect hidden faults, often failing to promptly identify potential issues with photovoltaic modules, such as hot spots, cracks, and contamination. This can lead to reduced energy efficiency and even prolonged downtime for photovoltaic power plants. As an efficient and flexible solution, drone-based photovoltaic inspections are becoming a mainstream method for inspecting photovoltaic power plants. Equipped with high-definition cameras and infrared thermal imagers, drones can quickly capture high-resolution image data, enabling the timely identification of photovoltaic module faults, significantly improving inspection efficiency and reducing labor costs. Traditional manual photovoltaic inspections are inefficient, costly, and difficult to detect, while drone-based inspections offer advantages such as efficiency and flexibility.

[0003] However, current drone-based PV inspections suffer from low positioning accuracy and time-consuming positioning. For example, in complex terrain and weather conditions, the image data captured by drones can exhibit deviations, leading to inaccurate positioning and affecting the efficiency and accuracy of subsequent PV module fault handling.

[0004] Currently, no effective solutions have been proposed for the problems in related technologies. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention proposes an efficient method for locating photovoltaic inspection results using drones. This method addresses the issues raised in the aforementioned background art, such as low accuracy and time-consuming positioning. For example, in complex terrain and weather conditions, image data acquired by drones can exhibit deviations, leading to inaccurate positioning and affecting the efficiency and accuracy of subsequent handling of photovoltaic module failures.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] An efficient method for positioning photovoltaic inspection results using a drone, comprising:

[0008] Collect visible light images and temperature distribution images of photovoltaic modules in real time, and use fusion algorithms to combine multi-source sensor data to optimize the real-time position of the drone;

[0009] Preprocess the visible light image and temperature distribution image, and use super-resolution reconstruction algorithm to enhance the low-resolution parts of the visible light image and temperature distribution image;

[0010] According to the information of the UAV waypoints and image acquisition timestamp, the pre-processed and enhanced visible light image and temperature distribution image are matched with the UAV positioning information;

[0011] Using generative adversarial networks and semantic segmentation techniques, key features of photovoltaic modules are extracted from the matched visible light image and temperature distribution image.

[0012] Through geographic information system technology, the positioning results and fault status of photovoltaic modules are displayed in real time.

[0013] Furthermore, real-time collection of visible light images and temperature distribution images of photovoltaic modules is performed, and a fusion algorithm is used to combine multi-source sensor data to optimize the real-time position of the drone, including:

[0014] The UAV collects real-time visible light images and temperature distribution images of photovoltaic modules by integrating a satellite positioning system, an inertial measurement unit, a high-resolution frequency-modulated continuous-wave laser sensor, and a visual sensor. At the same time, it obtains motion information from the inertial measurement unit and distance information from the high-resolution frequency-modulated continuous-wave laser sensor.

[0015] The collected visible light images are processed using a convolutional neural network. The motion information is used as a particle motion model through a particle filter algorithm to make a preliminary prediction of the drone's position. The particle state is updated based on the distance information. Based on the extracted environmental features and distance information, the matching degree between the particles and the observed data is calculated, and the particle weights are updated.

[0016] A collaborative positioning network of multiple drones is constructed. Each drone generates particles based on its own sensor data. Through information sharing and collaboration, the particles are updated and fused to obtain a global position estimate. Tasks are assigned and coordinated among drones to ensure the efficient execution of photovoltaic module inspection tasks.

[0017] Furthermore, a convolutional neural network is used to process the collected visible light images. The motion information is used as a particle motion model through a particle filter algorithm to make a preliminary prediction of the drone's position. The particle state is updated in combination with the distance information. Based on the extracted environmental features and distance information, the matching degree between the particles and the observed data is calculated. The particle weights are updated in the following ways:

[0018] Build a deep learning model based on convolutional neural networks and use preset images of different environments to train the deep learning model to identify environmental features related to the drone's location;

[0019] At the initial moment, a set of particles is generated based on prior information, each particle represents the position state of the drone, and an initial weight is assigned to each particle;

[0020] According to the UAV's motion model, the particle weights are modified using motion information and distance information. Combined with environmental characteristics, the final particle weights are calculated to predict the position and velocity of each particle at the next moment.

[0021] Taking environmental features and satellite positioning system signals as observations, the likelihood between each particle and the observation is calculated, and the particle weight is updated according to the likelihood;

[0022] Based on the particle weights, the particles are resampled, the particles with the lowest weight are discarded, the particles with the highest weight are copied, and a new set of particles is generated;

[0023] Based on the new particle set, the estimated position of the UAV is calculated through methods such as weighted averaging to achieve real-time positioning.

[0024] Furthermore, a collaborative positioning network of multiple drones is constructed. Each drone generates particles based on its own sensor data. Through information sharing and collaboration, the particles are updated and integrated to obtain a global position estimate. Tasks are then assigned and coordinated among the drones to ensure the efficient execution of PV panel inspection tasks. This includes:

[0025] Select drone models suitable for collaborative positioning tasks and equip each drone with communication equipment to enhance real-time communication between drones;

[0026] Based on the collaborative positioning algorithm, and combined with the particles generated by each drone based on its own sensor data, the particles are fused and updated through information sharing and interaction to obtain the global positioning result;

[0027] Assign and coordinate UAV tasks based on their location, sensor type, and mission requirements;

[0028] Based on the trajectory optimization algorithm, the flight trajectory of the drone is optimized to ensure that the drone can avoid obstacles and complete the task with the optimal path when performing inspection tasks.

[0029] Furthermore, based on the collaborative positioning algorithm and combining the particles generated by each drone based on its own sensor data, the particles are fused and updated through information sharing and interaction, and the global positioning results include:

[0030] Define the state vector and measurement vector of each UAV, and establish the state transition model and measurement model;

[0031] Initialize the state vector and covariance matrix of each UAV, broadcast the state estimation and covariance matrix to other UAVs through information sharing and interaction, and receive the state estimation information sent by other UAVs;

[0032] According to the state transition model, predict the state of each drone at the next moment;

[0033] The measurement values ​​of each drone are fused with the information of other drones. Based on the measurement model, the actual measurement values ​​of each drone are compared with the predicted state, and the measurement residual is calculated. The drone state prediction and data fusion are repeated until the preset termination conditions are met to obtain the global positioning result.

[0034] Furthermore, based on the trajectory optimization algorithm, the flight trajectory of the drone is optimized to ensure that the drone can avoid obstacles and complete the task with the optimal path when performing inspection tasks, including:

[0035] Model the environment in which the drone flies and divide the environment into a grid structure;

[0036] Define the coordinates for each grid cell, set the starting point and target point, use the A algorithm to search for a path, and obtain the path from the starting point to the target point;

[0037] Based on the path from the starting point to the target point, a graph structure is constructed and the Dijkstra algorithm is used to optimize the path of the graph structure to ensure that the drone can avoid obstacles and complete the task with the optimal path.

[0038] Furthermore, the coordinates of each grid cell are defined, the starting point and the target point are set, and the path search is performed using the A algorithm. The path from the starting point to the target point is obtained by:

[0039] Define the coordinates for each cell in the grid structure, set the starting point and target point, create an open list for storing the nodes to be evaluated and a closed list for the evaluated nodes, add the starting point to the open list to start the search;

[0040] Calculate the evaluation value of each node in the open list, combining the actual cost from the starting point and the estimated cost to the goal point;

[0041] Select the node with the smallest evaluation value from the open list as the current processing node, remove it from the open list and add it to the closed list;

[0042] Check the adjacent nodes of the current node. If the adjacent node is not in the open list or the closed list, add it to the open list and set the parent node of the adjacent node as the current node. If the adjacent node is already in the open list, check whether the actual cost of reaching the adjacent node through the current path is the minimum. If so, update the parent node of the adjacent node as the current node.

[0043] The process of evaluating nodes, selecting nodes, and expanding nodes is repeated until the target point is processed, and finally the path from the starting point to the target point is obtained by backtracking the parent node.

[0044] Furthermore, based on the path from the starting point to the target point, a graph structure is constructed and the Dijkstra algorithm is used to optimize the path of the graph structure to ensure that the drone can avoid obstacles and complete the mission with the optimal path. The following steps are included:

[0045] Based on the path from the starting point to the target point, a graph structure is constructed and the grid cells on the path are connected with the adjacent passable cells;

[0046] Set an initial distance value for each node in the graph, preset the distance of the starting node, and set the distance values ​​of other nodes to infinity. Create a priority queue to store the nodes to be processed, and add the starting node to the priority queue;

[0047] The node with the smallest distance is taken from the priority queue as the current processing node, and a new distance value is calculated for its adjacent nodes. If the new distance value is less than the current distance value of the adjacent node, the distance value and parent node of the adjacent node are updated, and the updated adjacent node is added to the priority queue;

[0048] Continue processing nodes until all nodes are processed, and then find the path that avoids obstacles and has the best cost by backtracking to the parent node of the target node.

[0049] The drone flies along the optimized path and obtains location information in real time. If an obstacle is detected, the obstacle avoidance mechanism is triggered and the flight path is adjusted to avoid collision.

[0050] When an obstacle is detected blocking the path, the local obstacle avoidance algorithm is used to adjust the path to avoid the obstacle and replan the path from the current position to the target point to ensure that the drone can avoid the obstacle and complete the mission with the optimal path.

[0051] Furthermore, the visible light image and the temperature distribution image are preprocessed, and the low-resolution parts of the visible light image and the temperature distribution image are enhanced using a super-resolution reconstruction algorithm, including:

[0052] Collect temperature distribution images and visible light images of photovoltaic modules and perform pixel-level annotation to obtain the accurate temperature value of each pixel;

[0053] A deep neural network model is constructed based on convolutional neural networks. Small convolution kernels are used to enhance the ability to capture small targets. Residual blocks and skip connections are combined to optimize the structure of the deep neural network model.

[0054] The network parameters are set according to the characteristics of the dataset and computing resources, and the dataset is divided into training set, validation set and test set. The mean square error is used as the loss function and adaptive moment estimation is used to train the deep neural network model.

[0055] The trained deep neural network model is evaluated using the test set. The quality of the generated temperature distribution images and visible light images is calculated, and evaluation metrics are used to measure the degree of distortion and structural similarity of the temperature distribution images and visible light images.

[0056] Construct a generative adversarial network model, and use it to generate enhanced and denoised high-resolution temperature distribution images and visible light images from low-resolution temperature distribution images and visible light images;

[0057] A recurrent neural network is used to extract the temporal features of temperature distribution images and visible light image sequences, and the temporal features are integrated into the generative adversarial network model for training. The trained generative adversarial network is then used to enhance low-resolution visible light images and temperature distribution images.

[0058] Furthermore, using generative adversarial networks and semantic segmentation technology, the key features of photovoltaic modules are extracted from the matched visible light image and temperature distribution image, including:

[0059] Based on the acquired visible light image and temperature distribution image, a coordinate matching relationship between the image and the map is established. By annotating the locations of photovoltaic modules in the visible light image and temperature distribution image, the semantic segmentation training data and coordinate transformation model are constructed.

[0060] Based on the image features of different types of photovoltaic modules, a multi-scale generator and discriminator structure is established, and a generative adversarial network framework is designed through deconvolution layers and convolution layers.

[0061] The generator and discriminator are trained using a semantic segmentation training dataset. The discriminant results are used to guide the generator to continuously update its parameters, thus establishing a generative adversarial network model that simulates real photovoltaic module images.

[0062] Build a semantic segmentation network model based on the preset semantic segmentation network framework, and input the enhanced visible light image, temperature distribution image and annotation information to train the semantic segmentation network model; form semantic recognition capabilities;

[0063] A high-resolution frequency-modulated continuous-wave laser sensor is used to acquire point cloud data of photovoltaic power plants. This point cloud data is then fused with visible light images to assist a semantic segmentation network model in extracting spatial structural information of photovoltaic modules.

[0064] Based on the photovoltaic module image position output by the generative adversarial network model and the semantic segmentation network model, the photovoltaic module position information in the visible light image and the temperature distribution image is mapped to the map through the coordinate transformation model to complete the spatial positioning of the photovoltaic module on the map.

[0065] The beneficial effects of the present invention are:

[0066] 1. The present invention combines the particle filter algorithm with deep learning. The particle filter is used to predict and update the position status of the UAV, and deep learning is used to process multi-source sensor data to improve the perception of complex environmental features, thereby more accurately estimating the UAV position and solving the problem of reduced positioning accuracy of traditional triangulation positioning in complex environments. The deep learning model can learn changes in environmental features in real time. When the environment changes during the flight of the UAV (such as entering a built-up area from an open area), it can quickly adapt to environmental changes and adjust the positioning strategy to ensure that the UAV can achieve accurate real-time positioning in different environments, build a multi-UAV collaborative positioning network, and improve the overall positioning accuracy and coverage through information sharing and collaboration between multiple UAVs.

[0067] 2. The present invention expands the coverage of positioning through multi-UAV collaborative positioning, and also supplements the UAV real-time positioning algorithm based on the fusion of particle filtering and deep learning with multiple sensors, thereby improving the accuracy and reliability of positioning and enhancing the fault tolerance of the system.

[0068] 3. The GAN-based image enhancement and denoising algorithm of the present invention has significant advantages over traditional histogram equalization and Gaussian filtering methods. Traditional algorithm parameters are fixed and it is difficult to adapt to the image diversity of different scenes. The GAN algorithm can automatically perform adaptive enhancement and denoising according to the characteristics of the input image. Especially in drone image processing, the algorithm can adjust the degree of enhancement and denoising according to different lighting conditions, so that the image can present good visual effects in various scenes. By utilizing the temporal information of the image sequence, GAN further improves the quality of the image, making the generated image more natural and realistic. It not only removes noise and enhances contrast, but also repairs blurred areas. The processed image is bright in color and rich in details, providing a better visual experience. At the same time, the GAN algorithm can efficiently remove complex noise and retain useful information by learning noisy image samples, providing reliable data support for subsequent image analysis.

[0069] 4. The photovoltaic module positioning algorithm based on GAN and semantic segmentation in the present invention can enhance image features and pixel-level classification, and can accurately identify photovoltaic modules in environments such as lighting changes and occlusion, and can accurately locate them even when they are blocked by shadows. Secondly, the algorithm utilizes the parallel computing power of deep learning. Compared with the traditional nearest neighbor matching method, it can quickly process large amounts of image data and improve positioning efficiency. It is particularly suitable for inspection work in large-scale photovoltaic power stations. Finally, through the multi-scale generative adversarial network, the algorithm can adapt to different types of photovoltaic modules, regardless of shape, color or specification, thereby avoiding the dependence of traditional algorithms on specific types of modules and improving the universality and accuracy of positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 This is a flow chart of an efficient method for locating photovoltaic inspection results using a drone according to an embodiment of the present invention;

[0072] Figure 2 This is an architectural diagram of a UAV photovoltaic inspection system according to an embodiment of the present invention;

[0073] Figure 3 2 is an example diagram of photovoltaic module image feature extraction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0075] In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting relative importance.

[0076] According to an embodiment of the present invention, an efficient method for positioning photovoltaic inspection results using a drone is provided.

[0077] The present invention will now be further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, the efficient UAV photovoltaic inspection result positioning method according to an embodiment of the present invention includes:

[0078] S1, real-time acquisition of visible light images and temperature distribution images of photovoltaic modules, and optimization of the real-time position of the UAV by combining multi-source sensor data with fusion algorithms;

[0079] Specifically, drones equipped with various sensors, such as high-resolution cameras and infrared thermal imagers, conduct inspection flights over photovoltaic power plants along pre-set routes. During flight, the high-resolution camera utilizes optical imaging principles to convert visible light reflection information from photovoltaic modules into digital image signals. The distribution and color values ​​of its pixels capture the module's appearance details. The infrared thermal imager, based on the principle of infrared radiation, detects infrared radiation energy emitted from the surface of photovoltaic modules and converts it into a temperature distribution image. Different temperature zones correspond to different operating conditions of the modules. The drone's positioning is optimized during data collection to ensure maximum accuracy during the data collection phase.

[0080] S2. Preprocessing the visible light image and the temperature distribution image, and enhancing the low-resolution parts of the visible light image and the temperature distribution image using a super-resolution reconstruction algorithm;

[0081] Specifically, the resolution of traditional temperature distribution images generated based on the principle of infrared radiation may be limited. By utilizing a deep learning super-resolution reconstruction algorithm, a deep neural network for photovoltaic scenes is designed and trained to learn the mapping relationship between low-resolution temperature distribution images and high-resolution images. A low-resolution temperature distribution image obtained based on the principle of infrared radiation is input, and after processing by the network model, a high-resolution temperature distribution image is output, achieving super-resolution reconstruction of the image. This not only makes the temperature distribution image visually clearer, but also provides richer detailed information for subsequent temperature analysis, resolving the technical problem of temperature features being difficult to accurately identify due to low resolution. For example, when detecting temperature changes in small areas on the surface of a device, a high-resolution temperature distribution image can more accurately locate abnormal areas.

[0082] S3, matching the pre-processed and enhanced visible light image and temperature distribution image with the UAV's positioning information based on the UAV's waypoints and image acquisition timestamp information;

[0083] Specifically, the positioning of the collected images is optimized: the images taken at the established waypoints in the route are classified and sorted, including:

[0084] 1) Extracting the information associated with waypoints and images:

[0085] 1.1) Obtain route information from the drone flight system or related record files, which clearly includes the coordinates (GPS information) of each established waypoint and the corresponding shooting timestamp and other key data.

[0086] 1.2) At the same time, organize all images taken by the drone and ensure that each image contains the shooting time information. For some image formats that do not directly record the shooting time, the shooting time can be obtained by reading the image file metadata (such as EXIF ​​information).

[0087] 2) Establish the corresponding relationship between images and waypoints:

[0088] 2.1) Match the image's capture time with the waypoint's elapsed time. Since the drone triggers a capture action upon reaching each waypoint during flight, the capture time and the waypoint's elapsed time should be very close. Set a reasonable time threshold (e.g., plus or minus 1 second). If the image's capture time falls within the threshold for a waypoint's elapsed time, the image is considered to have been taken at that waypoint.

[0089] 2.2) For ambiguous matches between multiple waypoint times and the image capture time, further information such as the drone's flight speed and direction, as well as geographic features in the image, can be used to make a more precise judgment. For example, if the geographic features shown in the image match a landmark near a waypoint, and the time is also close, then the image is confirmed to correspond to the waypoint.

[0090] 3) Classification and sorting operations:

[0091] 3.1) Classification: Classify images by waypoint number or sequence. For example, images taken at the first waypoint would be classified as category one, images taken at the second waypoint as category two, and so on. This categorized storage can be achieved by creating folders or using classification fields in the database.

[0092] 3.2) Sorting: Within each waypoint's image category, sort the images by their capture date. This ensures that images taken at the same waypoint are arranged in chronological order, facilitating subsequent review and analysis. For images with the same capture date (which may occur in rare cases), sort them based on other unique identifiers, such as file name or file size.

[0093] 4) Get the original image GPS information:

[0094] 4.1) Read the metadata for each image and extract the GPS coordinates contained therein. Most images taken with digital devices include the device's GPS location at the time of capture in their EXIF ​​metadata. This information can be easily retrieved using an appropriate image processing library (such as the Python Pillow library combined with the Exif Read plugin).

[0095] 5) Analyze the differences between waypoint GPS and image GPS:

[0096] 5.1) Compare the GPS coordinates of each waypoint in each image with the original GPS coordinates in the image itself. Calculate the distance difference (using a latitude and longitude distance formula, such as the Haversine formula) and the directional deviation (by calculating the azimuth of the two coordinate points).

[0097] 5.2) Analyze the distribution of these differences. For example, calculate the average and maximum differences between the image GPS and waypoint GPS for different flight segments, altitudes, and environmental conditions. This helps understand the factors that cause GPS deviations, such as signal interference and device accuracy.

[0098] 6) Optimization algorithm design:

[0099] 6.1) Statistical Model-Based Optimization: Build a statistical model based on the difference distribution patterns obtained from the above analysis. For example, if a linear relationship is found between the GPS deviations of images and waypoints within a certain area, a linear regression model can be used to predict the GPS deviations of other images in the same area and perform corrections to improve the accuracy of GPS deviation predictions.

[0100] 6.2) Fusion of Other Sensor Data: Drones may also be equipped with other sensors, such as an inertial measurement unit (IMU) and a high-resolution frequency-modulated continuous-wave laser sensor, which can provide information such as the drone's attitude and acceleration. By fusing IMU data with waypoint GPS information, we can more accurately estimate the drone's actual position at the time the image was taken.

[0101] 6.3) Integrate information from other drones in the multi-UAV collaborative positioning network: According to the shooting time in the image, the data obtained by the GPS and body sensors of other drones in the multi-UAV collaborative positioning network at that time are correlated and calibrated.

[0102] 6.4) Introducing environmental features as auxiliary variables: Environmental characteristics of the image area, such as terrain, weather, and lighting, may affect GPS bias. Introducing these environmental features as auxiliary variables allows the model to consider more factors and improve prediction accuracy.

[0103] 7) Implement optimization operations:

[0104] 7.1) Data Partitioning and Training: The image dataset with known GPS deviations is divided into a training set and a validation set. The linear regression model is trained using the training set and the validation set.

[0105] 7.2) Obtain sensor information, flight time, and trajectory of all drones in the multi-UAV collaborative network: According to the shooting time in the image, the data obtained by the GPS and body sensors of other drones in the multi-UAV collaborative positioning network at that time are correlated and calibrated.

[0106] 7.3) Environmental feature collection: Collect environmental information when the image was taken, such as terrain (mountainous, plain, urban, etc.), weather conditions (sunny, cloudy, rainy), light intensity, etc.

[0107] 7.4) Feature encoding and integration: Encode environmental features, convert them into numerical data, and integrate them with the original features of the image (such as color, texture, etc.).

[0108] 7.5) Improved Linear Regression Model: Using the integrated features as input, an extended linear regression model is constructed. The influence coefficient of each feature on GPS deviation is determined through training data.

[0109] 7.6) Correction Application: Use the calibrated prediction results to correct the GPS deviation of other images in the area.

[0110] 7.7) During the optimization process, the results need to be verified. This can be done by selecting a representative sample of images and, through field inspection or comparison with high-precision map data, verifying that the optimized GPS coordinates accurately reflect the actual location where the images were taken. If necessary, the algorithm parameters can be adjusted and optimized to improve the overall optimization effect.

[0111] S4. Use generative adversarial networks and semantic segmentation technology to extract key features of photovoltaic modules from the matched visible light image and temperature distribution image;

[0112] S5. Through geographic information system technology, the positioning results and fault status of photovoltaic modules are displayed in real time.

[0113] It should be explained that, in order to facilitate understanding of the above technical solutions of the present invention, the following describes the photovoltaic inspection by drones in the actual process of the present invention.

[0114] 1. Hardware preparation:

[0115] 1) Select a suitable UAV platform that has stable flight performance, long flight time, and strong payload capacity to carry a variety of sensors.

[0116] 2) Equipped with a high-resolution camera with a resolution of no less than 48 million pixels to ensure clear capture of details of photovoltaic modules; the temperature resolution of the infrared thermal imager reaches 0.1°C, which can accurately detect temperature anomalies of photovoltaic modules.

[0117] 3) Install a high-precision GPS module and differential GPS device to obtain accurate location information.

[0118] 2. Software Settings:

[0119] 1) Develop image acquisition and transmission software to achieve real-time transmission of image data between the UAV and the ground control center.

[0120] 2) If Figure 3 As shown in FIG, existing image processing libraries, such as OpenCV, are used to develop software modules for image preprocessing, feature extraction, and matching.

[0121] 3) Write positioning optimization algorithm and result output software to accurately display the positioning results on the monitoring platform.

[0122] 3. Inspection process, such as Figure 2 As shown (in the figure, 1 represents the satellite, 2 represents the ground station, 3 represents the ground control center, and 4 represents the drone carrying the sensor):

[0123] 1) Before the drone takes off, plan the flight route according to the layout and terrain of the photovoltaic power station, and input the route information into the drone's flight control system.

[0124] 2) The drone flies along a preset route. During the flight, the sensors collect data in real time and transmit the data to the ground control center.

[0125] 3) The computer at the ground control center processes the received data and performs image preprocessing, feature extraction and matching, positioning optimization and other operations according to the steps in the above technical solution, and finally obtains and outputs accurate positioning results.

[0126] Specifically, the final positioning results, including the PV module's location coordinates, fault type, and other information, are visually output to the ground control center's monitoring platform. Using Geographic Information System (GIS) technology, the location coordinates are mapped onto an electronic map, visually displaying the PV module's position within the power plant. Fault type information is displayed on the monitoring platform using a combination of text and icons, using a pre-set correspondence between fault codes and fault names. This allows staff to intuitively understand the location and condition of the PV module fault and promptly initiate repairs.

[0127] In this optional embodiment, real-time acquisition of visible light images and temperature distribution images of photovoltaic modules and optimization of the real-time position of the drone using a fusion algorithm (i.e., a drone real-time positioning algorithm based on the fusion of particle filtering and deep learning) combined with multi-source sensor data include:

[0128] The UAV collects real-time visible light images and temperature distribution images of photovoltaic modules through an integrated satellite positioning system (GPS), an inertial measurement unit (IMU), a high-resolution frequency modulated continuous wave laser sensor, and a visual sensor. At the same time, it obtains motion information from the IMU and distance information from the high-resolution frequency modulated continuous wave laser sensor.

[0129] The collected visible light images are processed using a convolutional neural network. The motion information is used as a particle motion model through a particle filter algorithm to make a preliminary prediction of the drone's position. The particle state is updated based on the distance information. Based on the extracted environmental features and distance information, the matching degree between the particles and the observed data is calculated, and the particle weights are updated.

[0130] A collaborative positioning network of multiple drones is constructed. Each drone generates particles based on its own sensor data. Through information sharing and collaboration, the particles are updated and fused to obtain a global position estimate. Tasks are assigned and coordinated among drones to ensure the efficient execution of photovoltaic module inspection tasks.

[0131] Specifically, two algorithms for assisting drone positioning are described below:

[0132] The first method uses a real-time positioning algorithm for the drone, combining satellite signals received by the drone's GPS module with a particle filter, deep learning, and multiple sensors. This algorithm, unlike traditional triangulation positioning, relies solely on limited measured angle information to determine position, making it susceptible to interference in complex environments and offering limited positioning accuracy. This algorithm combines a particle filter with deep learning. The particle filter is used to predict and update the drone's position state, while deep learning processes multi-source sensor data, improving its ability to perceive complex environmental features and thereby more accurately estimating the drone's position. Its innovation is that it adds a high-resolution frequency-modulated continuous-wave laser sensor to the existing real-time positioning algorithm. Relying solely on visual information for positioning may have limitations in certain scenarios, such as lighting changes and occlusions. Incorporating data from other sensors (such as an inertial measurement unit (IMU) and a high-resolution frequency-modulated continuous-wave laser sensor) and fusing it with the visual data processed by the particle filter and deep learning algorithm can improve positioning accuracy. During the particle filter process, the motion information provided by the IMU is used as a priori for the particle motion model, providing a preliminary prediction of the particle state. At the same time, the distance information of the high-resolution frequency-modulated continuous-wave laser sensor is used to correct the weight of the particle, and the features extracted from the visual image by deep learning are combined to comprehensively calculate the final weight of the particle.

[0133] 1. Implementation steps of the UAV real-time positioning algorithm based on the fusion of particle filtering and deep learning:

[0134] 1) Multi-source sensor data acquisition: In addition to traditional signal sources used for triangulation (such as GPS and base station signals), an inertial measurement unit (IMU), a high-resolution frequency-modulated continuous-wave laser sensor, and a visual sensor (camera) are also added. The IMU can measure the drone's acceleration and angular velocity in real time, providing dynamic information for position prediction. The visual sensor captures environmental features by capturing images of the surrounding environment, and the high-resolution frequency-modulated continuous-wave laser sensor obtains three-dimensional distance information of the surrounding environment.

[0135] 2) Deep Learning Environmental Feature Extraction: A deep learning model based on a convolutional neural network (CNN) is constructed to process image data collected by the visual sensor. After being trained on a large number of images of diverse environments, the model can identify features related to the drone's location, such as landmarks and specific landforms. These extracted features are then fused with IMU data and traditional positioning signal data. For example, the CNN model can identify the outline of a mountain in an image as a landmark feature and, combined with flight direction and speed information from the IMU data, assist in positioning.

[0136] 3) Particle filter initialization: At the initial moment, a set of particles is generated based on prior information (such as the drone's takeoff position). Each particle represents a possible position state of the drone. Information such as the particle's position and velocity is generated through random sampling, and each particle is assigned an initial weight, which is generally equal.

[0137] 4) Particle Filter Prediction Step: Based on the drone's motion model (e.g., dynamic equations), the particle weights are modified using acceleration and angular velocity information measured by the IMU and distance information from a high-resolution frequency-modulated continuous-wave laser sensor. Combined with features extracted from visual images using deep learning, the final particle weights are calculated to predict the position and velocity of each particle at the next moment. The motion model takes into account factors such as the drone's flight attitude and air resistance, ensuring that the predictions are more consistent with actual flight conditions.

[0138] 5) Particle filter update step: The environmental features extracted by deep learning and traditional positioning signals (such as GPS signals) are used as observations. The likelihood between each particle and the observation is calculated, and the particle weight is updated based on the likelihood. For example, if the environmental features surrounding the predicted location of a particle closely match the features collected by the visual sensor and high-resolution FMCW laser sensor, and are close to the location calculated by the GPS signal, the weight of the particle is increased.

[0139] 6) Resampling: Based on the particle weights, particles are resampled, low-weight particles are discarded, and high-weight particles are copied to generate a new particle set. The new particle set is more concentrated in the possible location area, improving the accuracy of positioning.

[0140] 7) Position Estimation: Based on the resampled particle set, the estimated position of the drone is calculated through methods such as weighted averaging to achieve real-time positioning. For example, the particle positions are used as samples, and the particle weights are used as the weights of the corresponding samples. The weighted average is calculated to obtain the final position estimate of the drone.

[0141] The second is the optimization algorithm for assisting drone positioning: building a multi-drone collaborative positioning network.

[0142] A single drone may be limited by its own sensor accuracy and field of view during positioning. Based on the first drone positioning optimization algorithm, a multi-drone collaborative positioning network is constructed. Through information sharing and collaboration among multiple drones, the overall positioning accuracy and coverage can be improved. This includes:

[0143] 1) UAV selection:

[0144] Choose an appropriate drone model. Consider the drone's flight performance (e.g., flight time greater than 30 minutes, flight speed 5-10 meters per second, flight altitude no less than 200 meters), sensor configuration (e.g., GPS module, inertial measurement unit, visual sensor, high-resolution frequency-modulated continuous wave laser sensor), and communication capabilities (e.g., communication distance greater than 20 km, data transmission rate greater than 100 Mbps). Consider expanding the capabilities of the communication equipment to ensure the drone's performance meets the requirements for collaborative positioning.

[0145] 2) Communication and information sharing:

[0146] Each drone is equipped with communication equipment to enhance real-time communication between drones. Drones share their positioning information, visual images, and sensor data with other drones and ground control systems.

[0147] 3) Collaborative positioning algorithm:

[0148] A collaborative localization algorithm was designed to leverage information from multiple drones for joint localization. This algorithm combines the particles generated by each drone based on its own local information in the first localization algorithm. Through information sharing and interaction, these particles are fused and updated to achieve a more accurate global localization result.

[0149] 4) Task allocation and collaboration:

[0150] Based on the location, sensor type, and mission requirements of the drones, reasonable task allocation and collaboration can be carried out. For example, some drones are responsible for acquiring visual information of a specific area, while others are responsible for providing motion information or performing data fusion.

[0151] 5) Introducing trajectory optimization algorithm to enhance positioning effect:

[0152] Based on the positioning algorithm, a hybrid trajectory optimization method combining the A algorithm and the Dijkstra algorithm is employed. The A algorithm quickly searches for an approximate path from the starting point to the target point, while the Dijkstra algorithm precisely optimizes the path, ensuring that the drones can complete their inspection missions along the optimal path while avoiding obstacles in complex environments. This hybrid algorithm optimizes the drones' flight trajectories, reducing energy consumption and time wasted during flight. The optimized flight trajectories directly enhance the collaborative positioning capabilities of the drones.

[0153] In this optional embodiment, a convolutional neural network is used to process the collected visible light images. The motion information is used as a motion model for the particles through a particle filter algorithm to make a preliminary prediction of the drone's position. The particle state is updated in combination with the distance information. Based on the extracted environmental features and distance information, the matching degree between the particles and the observed data is calculated. Updating the particle weights includes:

[0154] Build a deep learning model based on convolutional neural networks and use preset images of different environments to train the deep learning model to identify environmental features related to the drone's location;

[0155] At the initial moment, a set of particles is generated based on prior information, each particle represents the position state of the drone, and an initial weight is assigned to each particle;

[0156] According to the UAV's motion model, the particle weights are modified using motion information and distance information. Combined with environmental characteristics, the final particle weights are calculated to predict the position and velocity of each particle at the next moment.

[0157] Taking environmental features and satellite positioning system signals as observations, the likelihood between each particle and the observation is calculated, and the particle weight is updated according to the likelihood;

[0158] Based on the particle weights, the particles are resampled, the particles with the lowest weight are discarded, the particles with the highest weight are copied, and a new set of particles is generated;

[0159] Based on the new particle set, the estimated position of the UAV is calculated through methods such as weighted average to achieve real-time positioning.

[0160] In this optional embodiment, a collaborative positioning network of multiple drones is constructed. Each drone generates particles based on its own sensor data. Through information sharing and collaboration, the particles are updated and integrated to obtain a global position estimate. Tasks are then assigned and coordinated among the drones to ensure efficient execution of PV panel inspection tasks. This includes:

[0161] Select drone models suitable for collaborative positioning tasks and equip each drone with communication equipment to enhance real-time communication between drones;

[0162] Based on the collaborative positioning algorithm, and combined with the particles generated by each drone based on its own sensor data, the particles are fused and updated through information sharing and interaction to obtain the global positioning result;

[0163] Assign and coordinate UAV tasks based on their location, sensor type, and mission requirements;

[0164] Based on the trajectory optimization algorithm, the flight trajectory of the drone is optimized to ensure that the drone can avoid obstacles and complete the task with the optimal path when performing inspection tasks.

[0165] In this optional embodiment, based on the collaborative positioning algorithm and in combination with the particles generated by each drone based on its own sensor data, the particles are fused and updated through information sharing and interaction to obtain the global positioning results including:

[0166] Define the state vector and measurement vector of each UAV, and establish the state transition model and measurement model;

[0167] Initialize the state vector and covariance matrix of each UAV, broadcast the state estimation and covariance matrix to other UAVs through information sharing and interaction, and receive the state estimation information sent by other UAVs;

[0168] According to the state transition model, predict the state of each drone at the next moment;

[0169] The measurement values ​​of each drone are fused with the information of other drones. Based on the measurement model, the actual measurement values ​​of each drone are compared with the predicted state, and the measurement residual is calculated. The drone state prediction and data fusion are repeated until the preset termination conditions are met to obtain the global positioning result.

[0170] Specifically, the collaborative positioning algorithm includes:

[0171] 1) System modeling:

[0172] State definition: Define the state vector X of each drone i , usually including information such as position and speed. For a drone on a two-dimensional plane, the state vector can be expressed as:

[0173] X i =[X i , Y i , VX i , VY i ]T.

[0174] Among them, X i and Y i is the position coordinate of drone i, VX i and VY i is the velocity component of UAV i.

[0175] State transition model: Establish a model for the transition of the drone state over time, which can usually be expressed as:

[0176] X i，k +1=F k X i，k +B k u i，k +W i，k .

[0177] Among them, F k is the state transition matrix, B k is the control input matrix, u i，k is the control input vector, W i，k is the process noise, which obeys the Gaussian distribution N(0, Q k ).

[0178] Measurement model: defines the measurement vector Z of each drone i，k , such as GPS measurements, relative distance measurements between drones, etc. The measurement model can be expressed as:

[0179] Z i，k =H k X i，k +V i，k .

[0180] Among them, H k is the measurement matrix, V i，k is the measurement noise, which obeys the Gaussian distribution N(0, R k ).

[0181] 2) Initialization:

[0182] is the state vector X of each drone i，0 and the covariance matrix P i，0 Assign initial values. The initial state can be estimated based on the initial position and velocity of the UAV, and the initial covariance matrix represents the uncertainty of the initial state estimate.

[0183] 3) Information interaction:

[0184] Each drone estimates its own state and the covariance matrix Broadcast to other drones. At the same time, receive state estimation information sent by other drones.

[0185] 4) State prediction:

[0186] According to the state transition model, the state of each drone is predicted:

[0187] Prediction status:

[0188]

[0189] Forecast covariance matrix:

[0190]

[0191] in, is the predicted value of the state of drone i at time k based on the information at time k-1. The k on the left of the vertical line "I" here represents the time to be predicted, and the k-1 on the right represents the information used for prediction from time k-1. k is the state transition matrix, which describes how the UAV state evolves over time; for example, in a simple uniform motion model, F k The position and speed information of the previous moment can be mapped to the predicted position and speed of the current moment; is the estimated state value of UAV i at time k-1; B k is the control input matrix, which controls the input u k，i Transformed into the effect on the state; u k，i It is the control input vector, which represents the external control effect on the drone state, such as the acceleration command of the drone; It is the predicted value of the covariance of the state estimate of UAV i at time k based on the information at time k-1. The covariance matrix describes the uncertainty of the state estimate. The elements in the matrix represent the correlation between the state variables and their respective variances. is the state estimation covariance matrix of UAV i at time k-1; is the state transition matrix F k The transpose of The covariance matrix of the previous moment can be mapped to the current moment, taking into account the propagation of uncertainty during the state transition process; Q k is the process noise covariance matrix, which represents the uncertainty introduced by factors such as the inaccuracy of the system model and external interference.

[0192] 5) Data Fusion

[0193] For each drone, its own measurement value is fused with the information of other drones. Take the fusion of relative distance measurement value as an example:

[0194] Calculate the measurement residuals:

[0195]

[0196] in, is the measurement residual vector, which represents the difference between the actual measurement value and the theoretical measurement value calculated based on the predicted state, Z k，i is the actual measurement value of UAV i at time k, such as the position measured by GPS, the relative distance measurement between UAVs, etc.; H k is the measurement matrix, which maps the state vector to the measurement space, that is, calculates the theoretical measurement value according to the state vector; Based on the predicted state Calculated theoretical measurement value.

[0197] Compute the covariance of the measurement residuals:

[0198]

[0199] Among them, S i，k is the covariance matrix of the measurement residuals, which describes the uncertainty of the measurement residuals; The predicted state covariance matrix is Mapping to the measurement space, taking into account the impact of state prediction uncertainty on measurement residuals; R k is the measurement noise covariance matrix, which represents the uncertainty introduced in the measurement process, such as the measurement error of the sensor;

[0200] Calculate the gain:

[0201]

[0202] Among them, K i，k is the Kalman gain matrix, which determines how much weight should be given to the measured and predicted values ​​during the data fusion process; It is the part that maps the predicted state covariance matrix to the measurement space; It is the inverse of the measurement residual covariance matrix. The role of the Kalman gain is to make a trade-off between the measured value and the predicted value. When the measurement noise is small, the Kalman gain is large, which means that the measured value is more trusted. When the state prediction uncertainty is small, the Kalman gain is small, which means that the predicted value is more trusted.

[0203] Update the state estimate:

[0204]

[0205] in, is the updated state estimate of UAV i at time k; is the predicted state at time k; This is the part that corrects the predicted state based on the measurement residual and the Kalman gain. By adding this part of the correction, the updated state estimate is closer to the actual state.

[0206] Update the covariance matrix:

[0207]

[0208] in, is the updated state estimation covariance matrix of UAV i at time k; I is the identity matrix; K i，k H k It reflects the impact on the covariance matrix when the state estimate is corrected based on the measurement value. By updating the covariance matrix, the uncertainty of the state estimate can be continuously adjusted to make the estimation result more reliable.

[0209] 6) Repeat the information exchange, state prediction, and data fusion process until the termination condition is met (such as reaching a predetermined number of iterations or the positioning error is less than a certain threshold). This process continuously performs state prediction and data fusion, allowing the drone's positioning estimate to gradually approach its true position while continuously updating the uncertainty of the state estimate.

[0210] In this optional embodiment, optimizing the flight trajectory of the drone based on a trajectory optimization algorithm to ensure that the drone can avoid obstacles and complete the task along the optimal path when performing an inspection mission includes:

[0211] Model the environment in which the drone flies and divide the environment into a grid structure;

[0212] Define the coordinates for each grid cell, set the starting point and target point, use the A algorithm to search for a path, and obtain the path from the starting point to the target point;

[0213] Based on the path from the starting point to the target point, a graph structure is constructed, and the Dijkstra algorithm (i.e., Dijkstra algorithm) is used to optimize the path of the graph structure to ensure that the drone can avoid obstacles and complete the task with the optimal path.

[0214] In this optional embodiment, coordinates are defined for each grid cell, a starting point and a target point are set, and a path search is performed using the A algorithm to obtain a path from the starting point to the target point. The path includes:

[0215] Define the coordinates for each cell in the grid structure, set the starting point and target point, create an open list for storing the nodes to be evaluated and a closed list for the evaluated nodes, add the starting point to the open list to start the search;

[0216] Calculate the evaluation value of each node in the open list, combining the actual cost from the starting point and the estimated cost to the goal point;

[0217] Select the node with the smallest evaluation value from the open list as the current processing node, remove it from the open list and add it to the closed list;

[0218] Check the adjacent nodes of the current node. If the adjacent node is not in the open list or the closed list, add it to the open list and set the parent node of the adjacent node as the current node. If the adjacent node is already in the open list, check whether the actual cost of reaching the adjacent node through the current path is the minimum. If so, update the parent node of the adjacent node as the current node.

[0219] The process of evaluating nodes, selecting nodes, and expanding nodes is repeated until the target point is processed, and finally the path from the starting point to the target point is obtained by backtracking the parent node.

[0220] Specifically, the path planning implemented by combining the A algorithm with the Dijkstra algorithm includes:

[0221] 1) Environmental modeling:

[0222] 1.1) First, the complex environment in which the drone flies is modeled. The environment is divided into a grid structure, and each grid cell is assigned specific attributes, such as whether it is occupied by an obstacle and whether it is passable.

[0223] 1.2) Define coordinates for each grid cell so that subsequent algorithms can perform location identification and path search.

[0224] 2) A algorithm preliminary path search:

[0225] 2.1) Initialization: Set the starting point and the target point, create an open list (for storing nodes to be evaluated) and a closed list (for storing evaluated nodes). Add the starting point to the open list.

[0226] 2.2) Evaluate nodes: For each node in the open list, calculate its evaluation function value f(n) = g(n) + h(n), where g(n) is the actual cost from the starting point to the current node n (such as the distance moved or the energy consumed), and h(n) is the estimated cost from the current node n to the target point (usually calculated using heuristic functions such as Manhattan distance or Euclidean distance).

[0227] 2.3) Select node: Select the node with the smallest f(n) value from the open list as the current processing node, remove it from the open list and add it to the closed list.

[0228] 2.4) Expanding Nodes: Check the current node's neighboring nodes (grid cells in the upper, lower, left, right, and diagonal directions, ensuring they are within the environment boundary and traversable). For each neighboring node, if it is not already in the open list or closed list, add it to the open list and set its parent node to the current node. If it is already in the open list, check whether the g(n) value of the current path to that node is smaller. If so, update the node's parent node to the current node and its g(n) value.

[0229] 2.5) Repeated Search: Repeat the above process of evaluating nodes, selecting nodes, and expanding nodes until the target node is added to the closed list or the open list is empty (indicating that no path can be found). At this point, by backtracking to the target node's parent node, an approximate path from the starting point to the target node can be obtained.

[0230] 3) Dijkstra algorithm precise path optimization:

[0231] 3.1) Initialization: Based on the approximate path obtained by Algorithm A, a graph is constructed. The nodes in the graph are the grid cells on the approximate path and their adjacent traversable grid cells. Each edge in the graph is assigned a weight. The weight can be determined based on factors such as travel distance and environmental complexity. For example, higher weights can be assigned to areas with complex terrain or close to obstacles.

[0232] 3.2) Distance Initialization: Set a distance value for each node in the graph. Set the distance value of the starting node to 0 and the distance values ​​of other nodes to infinity. Create a priority queue to store the nodes to be processed and add the starting node to the priority queue.

[0233] 3.3) Node Processing: Remove the node with the smallest distance from the priority queue as the current node. For each neighboring node of the current node, calculate the new distance from the current node to the neighboring node (the current node's distance plus the edge weight). If the new distance is less than the neighboring node's current distance, update the neighboring node's distance value and the predecessor node (i.e., parent node). Add the updated neighboring node to the priority queue (if it is not already in the queue).

[0234] 3.4) Path Generation: Repeat the above node processing process until all nodes are processed or the distance value of the target node is no longer updated. At this point, by backtracking to the target node's predecessor nodes, a precise path that avoids obstacles and minimizes cost in a complex environment can be obtained.

[0235] 4) UAV path execution and obstacle avoidance:

[0236] 4.1) Path Tracking: The drone flies according to a precise path obtained by the Dijkstra algorithm. During flight, it obtains its own position information in real time (e.g., through GPS, inertial navigation, etc.) and compares it with the planned path.

[0237] 4.2) Obstacle Detection: The drone uses sensors (such as lidar and cameras) to detect obstacles in the surrounding environment in real time. When an obstacle is detected, the obstacle avoidance mechanism is triggered.

[0238] 4.3) Local Path Adjustment: If an obstacle is detected on the planned path, the drone will make local adjustments to its current path based on a local obstacle avoidance algorithm (e.g., an artificial potential field method or a dynamic window method) to avoid the obstacle. After avoiding the obstacle, the drone will replan the path from its current location to the target point (again, using a combination of the A-algorithm and the Dijkstra algorithm), ensuring that the drone can continue to complete the inspection mission along the optimal path.

[0239] In this optional embodiment, a graph structure is constructed based on the path from the starting point to the target point, and the Dijkstra algorithm is used to optimize the path of the graph structure to ensure that the drone can avoid obstacles and complete the task along the optimal path. The following steps are included:

[0240] Based on the path from the starting point to the target point, a graph structure is constructed and the grid cells on the path are connected with the adjacent passable cells;

[0241] Set an initial distance value for each node in the graph, preset the distance of the starting node, and set the distance values ​​of other nodes to infinity. Create a priority queue to store the nodes to be processed, and add the starting node to the priority queue;

[0242] The node with the smallest distance is taken from the priority queue as the current processing node, and a new distance value is calculated for its adjacent nodes. If the new distance value is less than the current distance value of the adjacent node, the distance value and parent node of the adjacent node are updated, and the updated adjacent node is added to the priority queue;

[0243] Continue processing nodes until all nodes are processed, and then find the path that avoids obstacles and has the best cost by backtracking to the parent node of the target node.

[0244] The drone flies along the optimized path and obtains location information in real time. If an obstacle is detected, the obstacle avoidance mechanism is triggered and the flight path is adjusted to avoid collision.

[0245] When an obstacle is detected blocking the path, the local obstacle avoidance algorithm is used to adjust the path to avoid the obstacle and replan the path from the current position to the target point to ensure that the drone can avoid the obstacle and complete the mission with the optimal path.

[0246] In this optional embodiment, preprocessing the visible light image and the temperature distribution image, and enhancing the low-resolution portion of the visible light image and the temperature distribution image using a super-resolution reconstruction algorithm includes:

[0247] Collect temperature distribution images and visible light images of photovoltaic modules and perform pixel-level annotation to obtain the accurate temperature value of each pixel;

[0248] A deep neural network model is constructed based on convolutional neural networks. Small convolution kernels are used to enhance the ability to capture small targets. Residual blocks and skip connections are combined to optimize the structure of the deep neural network model.

[0249] The network parameters are set according to the characteristics of the dataset and computing resources, and the dataset is divided into training set, validation set and test set. The mean square error is used as the loss function and adaptive moment estimation is used to train the deep neural network model.

[0250] The trained deep neural network model is evaluated using the test set. The quality of the generated temperature distribution images and visible light images is calculated, and evaluation metrics are used to measure the degree of distortion and structural similarity of the temperature distribution images and visible light images.

[0251] Construct a generative adversarial network model, and use it to generate enhanced and denoised high-resolution temperature distribution images and visible light images from low-resolution temperature distribution images and visible light images;

[0252] A recurrent neural network is used to extract the temporal features of temperature distribution images and visible light image sequences, and the temporal features are integrated into the generative adversarial network model for training. The trained generative adversarial network is then used to enhance low-resolution visible light images and temperature distribution images.

[0253] Specifically, the temperature distribution image super-resolution reconstruction algorithm based on deep learning includes:

[0254] 1) Data collection and preprocessing:

[0255] 1.1) Collect a large number of temperature distribution image samples: Obtain temperature distribution images from various infrared thermal imaging devices in different scenarios and temperature ranges, covering image data under both normal and abnormal operating conditions to ensure sample diversity. For example, collect temperature distribution images of different parts of industrial equipment during operation, including images of high-temperature areas, low-temperature areas, and areas with gradual temperature changes.

[0256] 1.2) Image Annotation: Annotate the collected images to clearly identify the actual temperature value corresponding to each pixel in the image, providing accurate supervision information for subsequent training. The annotation process must strictly follow unified standards to ensure accuracy and consistency.

[0257] 1.3) Data Preprocessing: Cropping and normalization are performed on the original image. Redundant portions of the image not related to the temperature distribution are cropped, and the image size is uniformly adjusted to a suitable input for the network. Normalization maps image pixel values ​​to the range [0, 1] or [-1, 1], accelerating model training convergence. Data augmentation operations such as random flipping and rotation are also performed on the image to expand the dataset and improve the model's generalization capabilities.

[0258] 2) Build a deep neural network model for photovoltaic scenarios:

[0259] 2.1) Network Architecture Selection: A convolutional neural network (CNN)-based super-resolution reconstruction network architecture, such as SRResNet (super-resolution residual network) or EDSR (enhanced deep super-resolution network), is employed. Innovations are introduced based on this architecture: In PV inspections conducted by drones, small objects such as panels, weeds, and power lines often need to be inspected. Small objects occupy fewer pixels in an image, and small convolution kernels (such as 1×1 and 3×3) can capture local details and better extract edge and texture information of small objects. In the shallow layers of the network, two 3×3 convolution kernels are stacked instead of a single 5×5 convolution kernel to increase the network's nonlinear representation capability while reducing the number of parameters. This innovative network architecture, through a combination of multiple convolutional and deconvolution layers, effectively learns the mapping relationship from low-resolution images to high-resolution images. It comprises multiple residual blocks, each consisting of two convolutional layers and a skip connection. The skip connection helps the network better learn image details.

[0260] 2.2) Determine network parameters: Based on the characteristics of the dataset and computing resources, appropriately set network parameters such as the number of layers, convolution kernel size, and number of channels. For our current application scenario, we set the network to 16 layers, a 3×3 convolution kernel size, and 64 channels. These parameters require multiple experimental verifications to achieve optimal reconstruction results.

[0261] 2.3) Deep Neural Network Model Training for Photovoltaic Scenario:

[0262] 2.3.1) Dataset Partitioning: The preprocessed dataset is divided into a training set, a validation set, and a test set, with a 70% / 15% / 15% split. The training set is used to train the model, while the validation set is used to evaluate model performance during training, adjust model parameters, and prevent overfitting. The test set is used to evaluate the performance of the trained model on unseen data.

[0263] 2.3.2) Selecting a Loss Function and Optimizer: The loss function uses either the mean squared error (MSE) or a perceptual loss function to measure the difference between the generated high-resolution image and the ground-truth high-resolution image. For the optimizer, choose an adaptive learning rate optimizer such as Adam or Adagrad. These optimizers automatically adjust the learning rate based on the training process, accelerating model convergence. For example, set the initial learning rate to 0.001 and dynamically adjust it during training based on performance on the validation set.

[0264] 2.3.3) Start training: Input the low-resolution temperature distribution images from the training set into the constructed network model. A high-resolution image is generated through forward propagation. The error between the generated image and the true high-resolution image is then calculated using the loss function. The network parameters are then updated through backpropagation. During training, the model's performance is regularly evaluated on the validation set. If the loss on the validation set stops decreasing or overfitting occurs, adjust the learning rate or employ regularization methods (such as L1 or L2 regularization) to prevent overfitting.

[0265] 2.4) Model testing and evaluation:

[0266] 2.4.1) Testing the model: Use the test set to test the trained model. Input the low-resolution temperature distribution image of the test set into the model to obtain the reconstructed high-resolution temperature distribution image.

[0267] 2.4.2) Evaluation Metrics: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are used to evaluate the quality of reconstructed images. PSNR measures the peak signal-to-noise ratio between the reconstructed image and the ground truth, reflecting the degree of image distortion. SSIM measures the structural similarity between the reconstructed image and the ground truth, which is more consistent with human visual characteristics. The performance of different models is evaluated by comparing their PSNR and SSIM values ​​on the test set.

[0268] 2.5) Practical Application:

[0269] 2.5.1) Model Deployment: Deploy the trained and evaluated model to an actual temperature monitoring system. In practice, when a low-resolution temperature distribution image generated based on infrared radiation is obtained, it is input into the deployed model, which then quickly outputs a high-resolution temperature distribution image.

[0270] 2.5.2) Temperature Analysis: Using reconstructed high-resolution temperature distribution images for temperature analysis can more accurately identify abnormal temperature areas on equipment surfaces and analyze temperature trends, providing a more reliable basis for equipment fault diagnosis and maintenance. For example, high-resolution images can clearly reveal tiny hot spots on equipment surfaces, allowing for the timely detection of potential fault hazards.

[0271] 3) Photovoltaic scene image processing algorithms based on the generative adversarial network (GAN) image enhancement and denoising algorithm combined with the deep learning-based enhanced super-resolution generative adversarial network include:

[0272] 3.1) Principle Overview:

[0273] Based on the characteristics of photovoltaic scenarios (including standard photovoltaic module models and the heating characteristics of faulty modules), a generative adversarial network (GAN) consists of a generator and a discriminator. The generator is responsible for producing enhanced and denoised images, while the discriminator determines whether the generated image is real or a fake one created by the generator. The two compete with each other and continuously optimize, making the images produced by the generator increasingly similar to real, clear images in terms of visual quality and features, thus achieving the purpose of image preprocessing. In UAV photovoltaic scenario applications, a series of continuous visible light and thermal infrared images are typically acquired. Leveraging the temporal information in the image sequence can further improve the image enhancement and denoising effects.

[0274] 3.2) Implementation steps:

[0275] 3.2.1) Dataset Preparation: Collect a large number of clear, high-quality images and their corresponding noisy, low-contrast images as training data. For drone image preprocessing, collect clear drone aerial images of various scenarios (such as mountain photovoltaic power plants, urban rooftop photovoltaic power plants, and open-field centralized photovoltaic power plants). Simultaneously, generate corresponding low-quality image samples by artificially adding noise and reducing contrast to construct the training dataset.

[0276] 3.2.2) Build a Generative Adversarial Network Model: Design the network architecture of the generator and discriminator. The generator typically uses a deconvolutional architecture based on a convolutional neural network (CNN), gradually converting low-quality images into high-quality images through multiple layers of deconvolution operations. The discriminator uses a standard CNN architecture to determine whether the input image is a real, clear image or an image generated by the generator. For example, the generator can consist of multiple deconvolutional layers and ReLU activation functions, while the discriminator can consist of multiple convolutional layers and Sigmoid activation functions.

[0277] 3.2.3) Sequence Feature Extraction from Inspection Data: Use a recurrent neural network (RNN) or long short-term memory (LSTM) network to extract features from image sequences. By feeding the image at each time step into the network, the network learns the temporal dependencies between images.

[0278] 3.2.4) Improved Generative Adversarial Network Architecture: Incorporating the extracted temporal features into the GAN generator and discriminator. An additional input layer is added to the generator to receive temporal features, allowing the generator to take into account the historical information of the image.

[0279] 3.2.5) Temporal Consistency Constraint: A temporal consistency constraint is added to the loss function to ensure that the images of adjacent frames have reasonable changes after processing. The difference between adjacent images is calculated and added as a loss term to the generator's loss function.

[0280] 3.2.6) Innovative effect of adding sequence feature extraction: Image sequence preprocessing based on temporal information can reduce flicker and jitter in the image and improve image stability and continuity.

[0281] 3.2.7) Model Training: Low-quality images from the training dataset are fed into the generator, which then outputs enhanced and denoised images. The discriminator then distinguishes between the generated images and real, clear images. Based on the discriminant results, the generator and discriminator each update their network parameters. The generator aims to generate images that deceive the discriminator, while the discriminator aims to accurately distinguish between real and generated images. Through continuous iterative training, the quality of the images generated by the generator continues to improve.

[0282] 3.2.8) Image Preprocessing Application: After training the GAN model, the drone imagery that requires preprocessing is fed into the generator, which then outputs an enhanced and denoised image. The resulting image exhibits significant improvements in contrast, clarity, and noise levels.

[0283] In this optional embodiment, using a generative adversarial network and semantic segmentation technology, key features of the photovoltaic module are extracted from the matched visible light image and temperature distribution image, including:

[0284] Based on the acquired visible light image and temperature distribution image, a coordinate matching relationship between the image and the map is established. By annotating the locations of photovoltaic modules in the visible light image and temperature distribution image, the semantic segmentation training data and coordinate transformation model are constructed.

[0285] Based on the image features of different types of photovoltaic modules, a multi-scale generator and discriminator structure is established, and a generative adversarial network framework is designed through deconvolution layers and convolution layers.

[0286] The generator and discriminator are trained using a semantic segmentation training dataset. The discriminant results are used to guide the generator to continuously update its parameters, thus establishing a generative adversarial network model that simulates real photovoltaic module images.

[0287] Build a semantic segmentation network model based on the preset semantic segmentation network framework, and input the enhanced visible light image, temperature distribution image and annotation information to train the semantic segmentation network model; form semantic recognition capabilities;

[0288] A high-resolution frequency-modulated continuous-wave laser sensor is used to acquire point cloud data of photovoltaic power plants. This point cloud data is then fused with visible light images to assist a semantic segmentation network model in extracting spatial structural information of photovoltaic modules.

[0289] Based on the photovoltaic module image position output by the generative adversarial network model and the semantic segmentation network model, the photovoltaic module position information in the visible light image and the temperature distribution image is mapped to the map through the coordinate transformation model to complete the spatial positioning of the photovoltaic module on the map.

[0290] Specifically, in the preprocessed image, feature points of photovoltaic modules, such as the edges and corners of the modules, are extracted. The improvements to the photovoltaic module positioning algorithm based on generative adversarial networks and semantic segmentation include:

[0291] 1) Overview of the improved algorithm principle:

[0292] A generative adversarial network (GAN) consists of a generator and a discriminator. The generator aims to generate images that resemble real photovoltaic panel images, while the discriminator determines whether the generated images are realistic. Semantic segmentation can classify different object categories in an image at the pixel level. A GAN is first used to enhance and generate features for the photovoltaic panel images. Semantic segmentation is then used to precisely identify the panels' locations within the images. Compared to traditional SIFT and nearest neighbor matching algorithms, this approach is more accurate and adaptable to localization in complex environments. A multi-scale GAN architecture is introduced, in which the generator generates and enhances features of the photovoltaic panel images at different scales. A generator with multiple convolution kernel sizes and strides is designed to capture the overall layout features of the photovoltaic panels at a coarse scale and focus on detailed texture features at a fine scale. A large learning rate is used in the early stages of training to accelerate convergence, and the learning rate is gradually reduced as the optimal solution is approached to improve accuracy. Point cloud data of the photovoltaic power station is acquired using a high-resolution frequency-modulated continuous-wave laser sensor onboard a drone and then fused with visible light images. First, the image and point cloud data are registered to align their coordinate systems. Then, in the process of semantic segmentation and feature point extraction, the three-dimensional information provided by the point cloud data, such as the height and slope of the components, is combined to assist in determining the position and characteristics of the photovoltaic components.

[0293] 2) Implementation steps:

[0294] 2.1) Data Collection and Preprocessing: Collect a large number of field images containing PV panels and their corresponding high-precision map data. Label the images to clearly define the locations and boundaries of the PV panels, forming a training dataset for semantic segmentation. Simultaneously, calibrate the map data and convert its format to match the image data. For example, establish a correspondence between the map's geographic coordinate system and the image's pixel coordinate system.

[0295] 2.2) Constructing the innovative multi-scale generative adversarial network architecture: The generator and discriminator network structures are designed based on different PV module models. The generator uses a deconvolutional architecture based on a convolutional neural network (CNN). Through multiple layers of deconvolution operations, random noise vectors are converted into images with PV module characteristics. The discriminator uses a conventional CNN architecture to determine whether the input image is a real PV module image or an image generated by the generator. For example, the generator consists of multiple deconvolutional layers and ReLU activation functions, while the discriminator consists of multiple convolutional layers and Sigmoid activation functions.

[0296] 2.3) The resulting multi-scale generative adversarial network structure after training innovations: PV panel images from the training dataset are fed into the generator, which then outputs simulated PV panel images. The discriminator then distinguishes between the generated images and real PV panel images. Based on the discrimination results, the generator and discriminator each update their network parameters. Through continuous iterative training, the images generated by the generator become increasingly similar in visual quality and features to real PV panel images, enhancing the image's expressiveness and providing a better data foundation for subsequent semantic segmentation.

[0297] 2.4) Build a semantic segmentation network model: Select a suitable semantic segmentation network architecture, such as U-Net. This network, through an encoder-decoder structure, can classify the input image at the pixel level and identify PV panels, background, and other objects. The GAN-enhanced PV panel image is fed into the semantic segmentation network for training. Using the labeled data, the network learns the characteristics of the PV panels, enabling it to accurately segment their locations within the image.

[0298] 2.5) Using a high-resolution frequency-modulated continuous-wave laser sensor mounted on a drone, we acquire point cloud data of the PV power plant and fuse it with the visible light image. First, we register the image and point cloud data to align their coordinate systems. Then, during semantic segmentation and feature point extraction, we combine the 3D information provided by the point cloud data, such as module height and slope, to assist in determining the location and characteristics of the PV modules.

[0299] 2.6) Determining the Location of PV Panels on the Map: The field-captured image to be located is processed sequentially through a trained GAN model and a semantic segmentation model (this network, through an encoder-decoder structure, can classify the input image at the pixel level and identify PV panels, background, and other objects. The GAN-enhanced PV panel image is then fed into a semantic segmentation network for training. The labeled data is used to guide the network in learning the features of the PV panels, enabling it to accurately segment the PV panels within the image). This results in the precise location of the PV panels within the image. Then, based on the previously established coordinate correspondence between the image and the map, the PV panel positions in the image are mapped onto the map, thereby determining the PV panel positions within the map.

[0300] 2.7) Based on the matching results, preliminarily determine the location of the PV panels on the map.

[0301] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An efficient method for positioning photovoltaic inspection results using drones, characterized in that: include: Collect visible light images and temperature distribution images of photovoltaic modules in real time, and use fusion algorithms to combine multi-source sensor data to optimize the real-time position of the drone; Preprocess the visible light image and temperature distribution image, and use super-resolution reconstruction algorithm to enhance the low-resolution parts of the visible light image and temperature distribution image; According to the information of the UAV waypoints and image acquisition timestamp, the pre-processed and enhanced visible light image and temperature distribution image are matched with the UAV positioning information; Using generative adversarial networks and semantic segmentation techniques, key features of photovoltaic modules are extracted from the matched visible light image and temperature distribution image. Through geographic information system technology, the positioning results and fault status of photovoltaic modules are displayed in real time; The real-time acquisition of visible light images and temperature distribution images of photovoltaic modules and the optimization of the real-time position of the UAV by combining multi-source sensor data with a fusion algorithm include: The UAV collects real-time visible light images and temperature distribution images of photovoltaic modules by integrating a satellite positioning system, an inertial measurement unit, a high-resolution frequency-modulated continuous-wave laser sensor, and a visual sensor. At the same time, it obtains motion information from the inertial measurement unit and distance information from the high-resolution frequency-modulated continuous-wave laser sensor. The collected visible light images are processed using a convolutional neural network. The motion information is used as a particle motion model through a particle filter algorithm to make a preliminary prediction of the drone's position. The particle state is updated based on the distance information. Based on the extracted environmental features and distance information, the matching degree between the particles and the observed data is calculated, and the particle weights are updated. A collaborative positioning network of multiple drones is constructed. Each drone generates particles based on its own sensor data. Through information sharing and collaboration, the particles are updated and fused to obtain a global position estimate. Tasks are assigned and coordinated among drones to ensure the efficient execution of photovoltaic module inspection tasks.

2. The efficient UAV photovoltaic inspection result positioning method according to claim 1 is characterized in that: The convolutional neural network is used to process the collected visible light images, and the motion information is used as the motion model of the particles through the particle filter algorithm to make a preliminary prediction of the position of the drone, and the state of the particles is updated in combination with the distance information. Based on the extracted environmental features and distance information, the matching degree between the particles and the observation data is calculated, and the particle weight is updated. Build a deep learning model based on convolutional neural networks and use preset images of different environments to train the deep learning model to identify environmental features related to the drone's location; At the initial moment, a set of particles is generated based on prior information, each particle represents the position state of the drone, and an initial weight is assigned to each particle; According to the UAV's motion model, the particle weights are modified using motion information and distance information. Combined with environmental characteristics, the final particle weights are calculated to predict the position and velocity of each particle at the next moment. Taking environmental features and satellite positioning system signals as observations, the likelihood between each particle and the observation is calculated, and the particle weight is updated according to the likelihood; Based on the particle weights, the particles are resampled, the particles with the lowest weight are discarded, the particles with the highest weight are copied, and a new set of particles is generated; Based on the new particle set, the estimated position of the UAV is calculated through methods such as weighted averaging to achieve real-time positioning.

3. The efficient UAV photovoltaic inspection result positioning method according to claim 1 is characterized in that: The above-mentioned collaborative positioning network of multiple drones is constructed. Each drone generates particles based on its own sensor data. Through information sharing and collaboration, the particles are updated and integrated to obtain a global position estimate. The drones are assigned tasks and collaborated to ensure the efficient execution of photovoltaic module inspection tasks. The following are included: Select drone models suitable for collaborative positioning tasks and equip each drone with communication equipment to enhance real-time communication between drones; Based on the collaborative positioning algorithm, and combined with the particles generated by each drone based on its own sensor data, the particles are fused and updated through information sharing and interaction to obtain the global positioning result; Assign and coordinate UAV tasks based on their location, sensor type, and mission requirements; Based on the trajectory optimization algorithm, the flight trajectory of the drone is optimized to ensure that the drone can avoid obstacles and complete the task with the optimal path when performing inspection tasks.

4. The efficient UAV photovoltaic inspection result positioning method according to claim 3 is characterized in that: Based on the collaborative positioning algorithm, the particles generated by each drone based on its own sensor data are combined and updated through information sharing and interaction to obtain the global positioning results including: Define the state vector and measurement vector of each UAV, and establish the state transition model and measurement model; Initialize the state vector and covariance matrix of each UAV, broadcast the state estimation and covariance matrix to other UAVs through information sharing and interaction, and receive the state estimation information sent by other UAVs; According to the state transition model, predict the state of each drone at the next moment; The measurement values ​​of each drone are fused with the information of other drones. Based on the measurement model, the actual measurement values ​​of each drone are compared with the predicted state, and the measurement residual is calculated. The drone state prediction and data fusion are repeated until the preset termination conditions are met to obtain the global positioning result.

5. The efficient UAV photovoltaic inspection result positioning method according to claim 3 is characterized in that: The trajectory optimization algorithm is used to optimize the flight trajectory of the UAV to ensure that the UAV can avoid obstacles and complete the task along the optimal path when performing inspection tasks. The following steps are involved: Model the environment in which the drone flies and divide the environment into a grid structure; Define the coordinates for each grid cell, set the starting point and target point, use the A* algorithm to search for a path, and obtain the path from the starting point to the target point; Based on the path from the starting point to the target point, a graph structure is constructed and the Dijkstra algorithm is used to optimize the path of the graph structure to ensure that the drone can avoid obstacles and complete the task with the optimal path.

6. The efficient UAV photovoltaic inspection result positioning method according to claim 5 is characterized in that: The method of defining coordinates for each grid cell, setting a starting point and a target point, and using the A* algorithm to search for a path to obtain a path from the starting point to the target point includes: Define the coordinates for each cell in the grid structure, set the starting point and target point, create an open list for storing the nodes to be evaluated and a closed list for the evaluated nodes, add the starting point to the open list to start the search; Calculate the evaluation value of each node in the open list, combining the actual cost from the starting point and the estimated cost to the goal point; Select the node with the smallest evaluation value from the open list as the current processing node, remove it from the open list and add it to the closed list; Check the adjacent nodes of the current node. If the adjacent node is not in the open list or the closed list, add it to the open list and set the parent node of the adjacent node as the current node. If the adjacent node is already in the open list, check whether the actual cost of reaching the adjacent node through the current path is the minimum. If so, update the parent node of the adjacent node as the current node. The process of evaluating nodes, selecting nodes, and expanding nodes is repeated until the target point is processed, and finally the path from the starting point to the target point is obtained by backtracking the parent node.

7. The efficient UAV photovoltaic inspection result positioning method according to claim 5 is characterized in that: The method of constructing a graph structure based on the path from the starting point to the target point and using the Dijkstra algorithm to optimize the path of the graph structure to ensure that the drone can avoid obstacles and complete the mission along the optimal path includes: Based on the path from the starting point to the target point, a graph structure is constructed and the grid cells on the path are connected with the adjacent passable cells; Set an initial distance value for each node in the graph, preset the distance of the starting node, and set the distance values ​​of other nodes to infinity. Create a priority queue to store the nodes to be processed, and add the starting node to the priority queue; The node with the smallest distance is taken from the priority queue as the current processing node, and a new distance value is calculated for its adjacent nodes. If the new distance value is less than the current distance value of the adjacent node, the distance value and parent node of the adjacent node are updated, and the updated adjacent node is added to the priority queue; Continue processing nodes until all nodes are processed, and then find the path that avoids obstacles and has the best cost by backtracking to the parent node of the target node. The drone flies along the optimized path and obtains location information in real time. If an obstacle is detected, the obstacle avoidance mechanism is triggered and the flight path is adjusted to avoid collision. When an obstacle is detected blocking the path, the local obstacle avoidance algorithm is used to adjust the path to avoid the obstacle and replan the path from the current position to the target point to ensure that the drone can avoid the obstacle and complete the mission with the optimal path.

8. The efficient UAV photovoltaic inspection result positioning method according to claim 1 is characterized in that: The preprocessing of the visible light image and the temperature distribution image and the enhancement of the low-resolution parts of the visible light image and the temperature distribution image using a super-resolution reconstruction algorithm include: Collect temperature distribution images and visible light images of photovoltaic modules and perform pixel-level annotation to obtain the accurate temperature value of each pixel; A deep neural network model is constructed based on convolutional neural networks. Small convolution kernels are used to enhance the ability to capture small targets. Residual blocks and skip connections are combined to optimize the structure of the deep neural network model. The network parameters are set according to the characteristics of the dataset and computing resources, and the dataset is divided into training set, validation set and test set. The mean square error is used as the loss function and adaptive moment estimation is used to train the deep neural network model. The trained deep neural network model is evaluated using the test set. The quality of the generated temperature distribution images and visible light images is calculated, and evaluation metrics are used to measure the degree of distortion and structural similarity of the temperature distribution images and visible light images. Construct a generative adversarial network model, and use it to generate enhanced and denoised high-resolution temperature distribution images and visible light images from low-resolution temperature distribution images and visible light images; A recurrent neural network is used to extract the temporal features of temperature distribution images and visible light image sequences, and the temporal features are integrated into the generative adversarial network model for training. The trained generative adversarial network is then used to enhance low-resolution visible light images and temperature distribution images.

9. The efficient UAV photovoltaic inspection result positioning method according to claim 1 is characterized in that: The key features of photovoltaic modules extracted from the matched visible light image and temperature distribution image using the generative adversarial network and semantic segmentation technology include: Based on the acquired visible light image and temperature distribution image, a coordinate matching relationship between the image and the map is established. By annotating the locations of photovoltaic modules in the visible light image and temperature distribution image, the semantic segmentation training data and coordinate transformation model are constructed. Based on the image features of different types of photovoltaic modules, a multi-scale generator and discriminator structure is established, and a generative adversarial network framework is designed through deconvolution layers and convolution layers. The generator and discriminator are trained using a semantic segmentation training dataset. The discriminant results are used to guide the generator to continuously update its parameters, thus establishing a generative adversarial network model that simulates real photovoltaic module images. Build a semantic segmentation network model based on the preset semantic segmentation network architecture, and input the enhanced visible light image, temperature distribution image and annotation information to train the semantic segmentation network model; form semantic recognition capabilities; A high-resolution frequency-modulated continuous-wave laser sensor is used to acquire point cloud data of photovoltaic power plants. This point cloud data is then fused with visible light images to assist a semantic segmentation network model in extracting spatial structural information of photovoltaic modules. Based on the photovoltaic module image position output by the generative adversarial network model and the semantic segmentation network model, the photovoltaic module position information in the visible light image and the temperature distribution image is mapped to the map through the coordinate transformation model to complete the spatial positioning of the photovoltaic module on the map.

Images