Unmanned aerial vehicle photovoltaic inspection real-time path planning method and system

Abstract

The invention discloses an unmanned aerial vehicle photovoltaic inspection real-time path planning method and system, and relates to the technical field of control in a specific dimension. Comprising the steps of planning an initial route and setting inspection routes in different areas; an image is collected by adjusting the camera pitch angle of the second holder, and the distance is measured in real time through the laser range finder; uploading an image shot by the second pan-tilt camera, identifying the photovoltaic string, establishing a string position correction model, and calculating the three-dimensional coordinate of the center point of the string; implementing local route optimization according to the string center point coordinate data set and the initial route based on an ant colony algorithm; the unmanned aerial vehicle flies to a set waypoint along a flight route, the first pan-tilt camera shoots an image, identifies a picture center photovoltaic string, calculates a center point coordinate difference, and the unmanned aerial vehicle corrects the position and executes an inspection image acquisition task. The problems of insufficient real-time performance, poor adaptability to complex terrains and unstable image acquisition quality are solved, and the purposes of strong real-time performance, strong adaptability, high image quality, high recognition capability and high precision are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of control in a specific dimension, and in particular to three-dimensional position or route control. Background Art

[0002] With the exponential growth of the photovoltaic industry and the large-scale deployment of photovoltaic power generation equipment, the traditional manual operation and maintenance of photovoltaic power stations has gradually been replaced by an operation and maintenance system dominated by drones. Using drones or drone charging and swapping hangars deployed at power stations can achieve efficient inspections of photovoltaic power generation equipment throughout the station. However, the current drone operation and maintenance system still has the following problems: (1) The drone inspection route is usually initially framed by humans on the remote control or hangar platform. Centralized photovoltaic power stations are generally located in complex terrain areas such as hillsides, river valleys, and Gobi Deserts far away from urban areas. The diverse layout and wide distribution of photovoltaic strings in the area pose a great challenge to drone route planning; (2) The traditional inspection strategy sets the drone to perform inspections along a preset route at a fixed flight altitude. The layout and laying characteristics of the strings in complex terrain power stations will also cause the collected photovoltaic module images to have height anomalies, low clarity, abnormal string orientation or partial missing in the image, and other image quality issues that affect the accuracy of subsequent photovoltaic fault diagnosis.

[0003] For example, Chinese patent publication number CN118732695A discloses a drone trajectory planning method and storage medium for automatic inspection of photovoltaic power stations, and provides the following technical solution. The present invention relates to the field of drone path planning, specifically to a drone trajectory planning method and storage medium for automatic inspection of photovoltaic power stations; a first image is taken when the drone is located at a first shooting point, and the edge contour line of the feature in the binary image is obtained after binarization processing, and then the noise feature is eliminated by calculating the contour ratio to obtain the edge contour line of the photovoltaic component, and then the edge contour line of the photovoltaic component is adjusted to the middle position of the second image and maintained within a certain size range through the second image, and then a reference line is extracted through the second image, and the movement direction of the drone is controlled based on the first boundary distance, so as to ensure that the drone can directly capture the image of the photovoltaic component after reaching the second shooting point; it solves the problem in the prior art that when the drone uses a camera to capture the image of the photovoltaic component, the quality of the captured photovoltaic component image is sometimes poor, which affects the accuracy of judging whether the photovoltaic component is faulty based on the image.

[0004] However, the above-mentioned drone trajectory planning method and storage medium for automatic inspection of photovoltaic power stations are based on image processing methods such as binarization to identify the contours of photovoltaic modules in the collected images in real time. This image processing method is not robust to the recognition of photovoltaic module contours. Common inspection situations such as insufficient contrast between the image background and the photovoltaic modules and partial occlusion of the photovoltaic modules by vegetation will cause the technical route of correcting the drone's planned route based on identifying the edge contours of the photovoltaic modules to fail to achieve the expected results. Summary of the Invention

[0005] The present invention solves the problems in the prior art of insufficient real-time performance, poor adaptability to complex terrain, and unstable image acquisition quality in drone photovoltaic inspection path planning, and proposes a real-time path planning method and system for drone photovoltaic inspection, achieving the goals of strong real-time performance, strong adaptability, high image quality, high recognition ability, and high accuracy.

[0006] To achieve the above object, the present invention adopts the following technical solutions: A real-time path planning method for photovoltaic inspection by a UAV includes the following steps: S1: Plan the initial route and set up inspection routes in different areas; S2: Capture images by adjusting the pitch angle of the camera on the second gimbal and measure the distance in real time using a laser rangefinder; S3: Upload the image captured by the second gimbal camera, identify the photovoltaic strings, establish a string position correction model, and calculate the three-dimensional coordinates of the string center point; S4: Based on the ant colony algorithm, local route optimization is implemented according to the cluster center point coordinate data set and the initial route; S5: The UAV flies along the route to the set waypoint. The first gimbal camera captures an image, identifies the photovoltaic string in the center of the image, calculates the center point coordinate difference, and the UAV corrects its position and performs the inspection image acquisition task.

[0007] It has realized a complete closed-loop process from global path planning to local dynamic optimization. The combination of edge computing and ant colony algorithm has significantly improved the real-time path adjustment capability under complex terrain. At the same time, the dual gimbal collaborative mechanism has ensured the image acquisition quality and coordinate positioning accuracy.

[0008] A real-time path planning system for unmanned aerial vehicle (UAV) photovoltaic inspections includes: a UAV equipped with a first pan-tilt platform and a second pan-tilt platform, and a UAV charging and swapping motor depot equipped with an edge computing unit. The first pan-tilt platform is equipped with an infrared thermal imaging sensor and a visible light lens, and the second pan-tilt platform is equipped with a laser rangefinder and a visible light lens. The UAV charging and swapping motor depot is also equipped with a data storage unit.

[0009] Preferably, in step S1, a number of inspection zones of the photovoltaic power station are divided according to the configuration of the photovoltaic power station and the distance between the take-off point and the zone, and irregular boundary adjustments are performed on each inspection zone of the photovoltaic power station taking into account the zone boundaries and obstacle distribution.

[0010] Through dynamic partitioning strategy and task type classification mechanism, the one-size-fits-all planning defects of traditional methods in complex terrain scenarios are solved. Irregular boundary adjustment enhances terrain adaptability, and the yaw angle and string azimuth angle alignment design effectively reduces the risk of image distortion.

[0011] Preferably, the refined inspection tasks specifically include: S1.1: Obtain the target area's terrain data based on satellite maps, combine it with no-fly zones and obstacle distribution information to output a set of prohibited routes for drones. Combined with the power plant's installed capacity data and string distribution, a list of takeoff and stopover points is set within the set of route areas. S1.2: Initialize the route waypoint set, the shortest path, the distance from the takeoff point to any stopover point, and the priority queue, placing the takeoff point in the priority queue. S1.3: If the priority queue is empty, output the route set and the shortest path, and proceed to S2; otherwise, extract the node in the priority queue as the current path point and add it to the route set, and proceed to S1.4; S1.4: If the route set already contains all the stopover points, output the result and enter S2. Otherwise, traverse the adjacent points of the current node and calculate the shortest path from the stopover point to the adjacent points. S1.5: Determine whether the obtained shortest path from the takeoff point to the stopover point to the optional adjacent point passes through the prohibited path set T. If so, remove the shortest path of the current path point, set the corresponding adjacent point as unavailable, and return to step 1.4. If not, place the adjacent points of the shortest path of the current path point in the priority queue, update the shortest path, and return to step S1.4.

[0012] An algorithm is used to achieve refined path planning. By prohibiting the set of routes and the minimum turning radius constraint, the algorithm convergence speed is improved while ensuring path safety. The priority queue mechanism significantly reduces the computational complexity in complex terrain.

[0013] Preferably, in step S1, the inspection task is divided into a quick inspection task and a refined inspection task.

[0014] Preferably, step S3 specifically includes: the second gimbal camera maintains a pitch angle to collect visible light images of the photovoltaic power generation equipment ahead of the route and measures the distance, and uses a semantic segmentation neural network model, and the edge computing unit identifies the photovoltaic strings in the visible light image and obtains the coordinates of the four corner points of the outline.

[0015] The use of private links ensures the security of data transmission. Real-time image processing based on semantic segmentation neural networks breaks through the limitations of traditional binarization methods and significantly improves the accuracy of string contour recognition in complex backgrounds.

[0016] Preferably, the semantic segmentation neural network model specifically includes: S3.1: Build and train a semantic segmentation neural network model, perform normalization on the input visible light image, perform inference on the image input model, and output a single-channel binary image of the identified PV string. S3.2: Extract the binary image contour, obtain the pixel coordinates of the four corners of each photovoltaic string, correct the image distortion based on perspective transformation, and update the string center coordinates; S3.3: If the distance between the cluster corner point and the image boundary or the contour area is less than the set threshold, the cluster is determined to be incomplete and is removed; S3.4: Establish coordinate transformation models from the world coordinate system to the drone camera coordinate system, from the camera coordinate system to the image coordinate system, and from the pixel coordinate system to the image coordinate system, obtain the PV string coordinate position correction model, and obtain the corrected longitude and latitude coordinates.

[0017] Through the multi-level spatial mapping of perspective correction and coordinate transformation models, the coordinate deviation problem caused by the drone's perspective distortion is solved. The dual screening mechanism of area threshold and edge distance is combined to effectively eliminate invalid group data and improve the reliability of three-dimensional coordinate calculation.

[0018] Preferably, the construction and training of the semantic segmentation neural network model specifically includes: using a drone to collect visible light images of photovoltaic station strings, dividing the images into different data sets according to the string laying method and photovoltaic station type, manually annotating the photovoltaic strings for each data set using segmentation and annotation software, and dividing them into training set, test set and validation set according to a certain proportion.

[0019] Preferably, the step S4 specifically includes: S4.1: Construct an initial path model and use the ant colony algorithm to calculate the total path length of each individual and update the total pheromone concentration of the path; S4.2: Iterate S4.1, set the number of iterations, and the algorithm terminates after the number of iterations is reached, and outputs the route planning result; S4.3: The path planning algorithm update frequency is determined by the route speed, flight altitude, and PV string arrangement. The hangar edge computing unit executes the path planning algorithm at a specific frequency to update the route path output by steps S1 and S4.2. S4.4: When the total area of the PV strings in the second PTZ image is lower than the threshold, adjust the pitch angle; if it still does not meet the threshold or the center point of the string exceeds the inspection area, turn according to the preset route in S1.

[0020] The dynamic pheromone update mechanism and frequency adaptation strategy are adopted to solve the problem of slow convergence of traditional ant colony algorithm. Combined with the boundary recognition method of pitch angle linkage adjustment, intelligent perception of inspection area boundaries and smooth steering control are realized.

[0021] Preferably, the step S5 specifically includes: S5.1: The UAV flies to the set waypoint along the optimal route output in step S4; S5.2: The aircraft slows down and hovers before the waypoint. The first gimbal camera collects dual-light images and uploads them to the hangar. The edge computing unit identifies the PV strings and calculates their center coordinates, selecting the target PV string located at the center of the image. S5.3: If the deviation between the center coordinates of the target string and the drone positioning exceeds the threshold, the hangar controls the drone to fine-tune to the target point, identifies the long side of the string according to the pixel value coordinates of the four corner points of the target string, and rotates the fuselage so that the long side of the string is parallel to the long side of the image; S5.4: After completing image acquisition, fly to the next waypoint, and repeat S5.2-S5.3 until all waypoints are completed, and return to complete the mission.

[0022] A high positioning accuracy is achieved through a multi-level closed-loop correction mechanism. Combined with a dual-mode image acquisition strategy, it significantly improves the image quality consistency required for defect diagnosis while ensuring inspection efficiency.

[0023] Compared with the prior art, the present invention has the following beneficial effects.

[0024] 1. At the algorithm level, the present invention combines the improved ant colony optimization algorithm with the hierarchical path planning mechanism of the Dijkstra algorithm, and solves the problems of slow convergence and high computational redundancy of traditional path planning algorithms in complex terrains through a dynamic pheromone update strategy and priority queue design. At the same time, the collaborative application of the U-net semantic segmentation neural network model and the coordinate correction model based on edge computing breaks through the limitations of traditional image processing methods that are sensitive to illumination and occlusion, and achieves high-precision recognition of photovoltaic string contours and dynamic mapping of three-dimensional coordinates. In addition, the heterogeneous data acquisition architecture and pitch angle adaptive adjustment mechanism of the dual-pan-tilt camera, through the fusion of laser ranging and visual perception, builds real-time spatial perception capabilities, providing an accurate spatiotemporal data foundation for the dynamic correction of path planning.

[0025] 2. At the hardware level, the present invention uses a dual gimbal camera equipped on the drone to respectively perform infrared thermal imaging diagnosis and real-time visual positioning functions, and combines it with a laser ranging module to form a multimodal perception system to ensure the comprehensiveness and reliability of data collection. The edge computing unit integrates a hardware accelerator to achieve localized processing of high-computational-density tasks such as semantic segmentation, coordinate conversion, and path planning, avoiding the transmission delay problem of traditional cloud computing. At the software level, it supports seamless switching between rapid inspections and refined inspection tasks, and enhances the system's adaptability to complex terrains such as mountains and river valleys through mechanisms such as dynamic adjustment of inspection partitions and set filtering of no-fly zones.

[0026] 3. This invention improves inspection quality by leveraging the collaborative operation of dual gimbal cameras and precise positioning using a coordinate correction model. This solves the image distortion and string loss issues inherent in traditional methods due to flight altitude deviations, ensuring that the captured infrared and visible light images meet the millimeter-level accuracy requirements for photovoltaic defect diagnosis. Furthermore, real-time path optimization based on edge computing significantly reduces dynamic adjustment response time, and through coordinated pitch angle adjustment and boundary steering strategies, high inspection coverage is achieved in complex terrain. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is an overall flow chart of a real-time path planning method and system for UAV photovoltaic inspections according to the present invention.

[0028] Figure 2 This is a flowchart of the refined inspection initial route of a real-time path planning method and system for UAV photovoltaic inspection in the present invention. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the present disclosure more apparent, embodiments of the present disclosure are described in further detail below with reference to the accompanying drawings. The proportions of the components herein are not drawn to scale, and the proportions and dimensions shown in the accompanying drawings are not intended to limit the essential technical solutions of the present disclosure. These embodiments do not describe all details in detail, nor do they limit the present disclosure to the specific embodiments described.

[0030] See also Figure 1-2 As shown, a method for optical fiber relay communication includes the following steps: A real-time path planning method for photovoltaic inspection by a UAV includes the following steps: S1: Plan the initial route and set up inspection routes in different areas; S2: Capture images by adjusting the pitch angle of the camera on the second gimbal and measure the distance in real time using a laser rangefinder; S3: Upload the image captured by the second gimbal camera, identify the photovoltaic strings, establish a string position correction model, and calculate the three-dimensional coordinates of the string center point; S4: Based on the ant colony algorithm, local route optimization is implemented according to the cluster center point coordinate data set and the initial route; S5: The UAV flies along the route to the set waypoint. The first gimbal camera captures an image, identifies the photovoltaic string in the center of the image, calculates the center point coordinate difference, and the UAV corrects its position and performs the inspection image acquisition task.

[0031] A real-time path planning system for unmanned aerial vehicle (UAV) photovoltaic inspections comprises: a UAV equipped with a first gimbal and a second gimbal, and a UAV charging and swapping motor depot equipped with an edge computing unit, wherein the first gimbal is equipped with an infrared thermal imaging sensor and a visible light lens, and the second gimbal is equipped with a laser rangefinder and a visible light lens. The edge computing unit comprises at least one processor configured with a number of hardware accelerators to execute programs. When the programs are executed by the processors of the edge computing unit, the UAV charging and swapping motor depot equipped with the edge computing unit implements real-time path planning for UAV photovoltaic power station inspections. The UAV charging and swapping motor depot is further equipped with a data storage unit comprising at least one storage device for storing processing programs and processing program execution results as well as real-time collected images uploaded by the UAV to the UAV charging and swapping motor depot.

[0032] like Figure 1 In one embodiment shown, Figure 1 This is an overall flow chart of a real-time path planning method and system for UAV photovoltaic inspections according to the present invention. The details are as follows: S1: Plan the initial route and perform inspection tasks. The specific method is: S1-1: PV power station area division. Specifically, divide the PV power station into several inspection zones based on the installed capacity, module model, string layout, and the distance between the takeoff point and the zone. Ensure that the drone can cover one inspection zone on a single charge. Optimally, consider zone boundaries and obstacle distribution, and adjust the irregular boundaries of each PV power station inspection zone.

[0033] S1-2: Zoning inspection route setting. Specifically, for fast inspection tasks, the inspection route is quickly generated by remote control presets, and the route parameters are set to ensure that the initial route yaw angle of the drone is consistent with the azimuth angle of the photovoltaic string at the initial waypoint; for refined inspection tasks, the inspection route can be generated by common path planning algorithms such as Dijkstra algorithm, Astar algorithm, and RRT algorithm. The fast inspection tasks and refined inspection tasks described in the present invention are defined as: 1) Rapid inspection mission: UAVs collect images of photovoltaic power generation equipment throughout the station and are often used for daily inspection missions in photovoltaic power plants.

[0034] 2) Detailed inspection tasks: The drone collects images of the photovoltaic power generation equipment in the entire station, and after performing deceleration or hovering operations, conducts inspection operations directly above the photovoltaic power generation equipment. It is often used in photovoltaic power stations with poor power generation efficiency that urgently need defect diagnosis or carry out detailed inspections on a regular basis.

[0035] S1-3: Generate the initial route for refined inspection.

[0036] S2: The second gimbal sets up a second gimbal camera equipped with a laser rangefinder and a visible light lens, controls the pitch angle α of the second lens, and uses the laser rangefinder to detect the distance between the drone and the target point. The specific method is: S2-1: The present invention provides a drone equipped with two gimbals, wherein the first gimbal is equipped with a dual-light (infrared thermal imaging sensor + visible light lens) camera to collect image data of photovoltaic modules. During the inspection process, the camera maintains a constant pitch angle of -90°, and the digital zoom factor of the infrared thermal imaging sensor is set so that the infrared thermal imaging sensor and the visible light lens have the same field of view. The second gimbal is equipped with a camera equipped with a laser rangefinder and a visible light lens. The visible light lens supports the collection of target group image data, and the laser rangefinder supports the measurement of the straight-line distance between the camera and the target object at the center point of the image. The camera maintains a pitch angle of α during the inspection process.

[0037] S2-2: Setting the pitch angle α of the second gimbal camera. Specifically, the pitch angle α is determined by the string arrangement, the drone's height from the strings, and the drone's speed. Setting the pitch angle to α ensures that the drone uses the second gimbal camera to capture a complete image of the PV strings at a specific distance d ahead of the route. This specific distance d is determined by the processor performance of the drone's hangar. This specific distance d is set to ensure that the drone captures images from the second gimbal camera, the hangar processor performs edge computing, and then uploads the optimally planned route, and that the drone successfully executes the route to the center point of the PV strings in the image captured by the second gimbal camera.

[0038] S2-3: The present invention also provides a drone charging and swapping motor library configured with an edge computing unit, which supports controlling drones to perform inspection tasks.

[0039] S2-4: The present invention provides an edge computing unit comprising one or more processors configured with multiple hardware accelerators (e.g., GPUs, TPUs, etc.) to execute the procedures described in the present invention, such as initial route planning, photovoltaic string identification, and optimal inspection path planning. When executed by the one or more processors of the edge computing unit, the drone charging and swapping power bank configured with the edge computing unit implements real-time route planning for drone inspections of photovoltaic power plants.

[0040] S2-5: The present invention provides a data storage unit configured in a drone charging and swapping motor depot, comprising one or more storage devices for storing the processing program mentioned in S2-4, the processing program execution results, and the real-time collected images uploaded by the drone to the drone charging and swapping motor depot, so that the drone charging and swapping motor depot equipped with the data storage unit can realize real-time path planning for drone photovoltaic power station inspections.

[0041] S3: Upload the image captured by the second gimbal camera to the drone hangar, identify the photovoltaic strings, establish a photovoltaic string position correction model, and calculate the three-dimensional coordinates of the string center point. The specific method is: S3-1: The UAV conducts an inspection along the preset route generated by S1. The second gimbal camera maintains a pitch angle α to collect visible light images of the photovoltaic power generation equipment in front of the route and measures the distance. The collected visible light images and ranging data of the power generation equipment are uploaded to the data storage unit of the UAV charging and swapping station using the UAV private link.

[0042] S3-2: The edge computing unit identifies the photovoltaic string in the visible light image and obtains the coordinates of the four corner points of the outline. Specifically, taking the U-net semantic segmentation neural network as an example: First, normalization is performed on the input visible light image, and the calculation formula is as follows: where x i Represents the initial pixel value of any point i in the image, max(x) and min(x) represent the maximum and minimum pixel values in the image, respectively.

[0043] The normalized image is input into the model for inference, and a single-channel binary image of the identified photovoltaic string is output.

[0044] The contour of the single-channel binary image of photovoltaic strings is extracted, and the pixel value coordinates of each string are output to identify them.

[0045] S3-3: Optional: Build and train the U-net semantic segmentation neural network model. The specific method is: 1) Use drones to collect visible light images of photovoltaic strings at a photovoltaic station. These images are divided into different datasets based on string installation method and photovoltaic station type. Each dataset is manually annotated with the photovoltaic strings using segmentation and annotation software to form a dataset. The datasets are then divided into training, test, and validation sets based on a specific ratio. Preferably, semi-automatic annotation tools such as ISAT and SAM are used to annotate the datasets to improve annotation efficiency.

[0046] 2) Perform data augmentation operations on the labeled dataset, including random scaling, rotation, flipping, blackening, brightness adjustment, etc., and set the probability of triggering any data augmentation operation.

[0047] 3) Build the U-net semantic segmentation neural network framework and set the parameters as follows: The number of input image features = 3, the encoding network feature dimension = 64, the activation function is set to ReLU, dropout = 0.1, and the loss function is CrossEntropyLoss, whose formula is: where x i is the true label element i, y i is the probability that x belongs to the i-th category.

[0048] Set the training optimizer to SGD or Adam. Take SGD as an example: lr = 0.01, momentum = 0.9, weight_decay = 4*10 -4 ,Set the training rounds according to the sample size, set the early stopping patience parameter patience=20, and complete the training after the validation set loss does not update the lowest value in 20 iterations.

[0049] S3-4: Optionally, perform perspective correction on the photovoltaic string output by the U-net neural network. The specific method is: obtain the pixel value coordinates of the four corner points of the photovoltaic string output by the neural network Use the warpPerspective function to perform perspective correction on the source image photovoltaic strings. The formula is: dst=src×M Where dst represents the processed image, src represents the input source image, M is the perspective transformation equation, and the perspective correction formula after expansion is: Where T 11 , T 12 , T 31 , T 13 , T 23 , T 12 , T 22 , T 32 Represents the perspective transformation coefficient, (x, y) and (u, v) represent the coordinates of the corner points of the source image and the image after perspective transformation, respectively.

[0050] The longest side is obtained according to the coordinates of the corner pixel values. The transformed corner coordinates are set according to the pan-tilt angle α and the inclination angle of the photovoltaic string, and then substituted into the corner pixels of the source image to complete the perspective correction process.

[0051] Update the target photovoltaic string center point coordinates according to the processing results of step 3).

[0052] S3-5: Preferably, incomplete strings in the image captured by the second PTZ camera are removed. The specific method is: 1) Obtain the pixel coordinates of the corner points of all photovoltaic strings in the image. When the distance d from the corner point to the image edge is less than the set threshold d0, the string corresponding to the corner point is determined to be incomplete.

[0053] 2) Calculate the identified contour area of each photovoltaic string using the following formula: Where (x n ,y n ) is the pixel value coordinate of the nth point in the quadrilateral.

[0054] When S is less than the set area threshold S0, the string corresponding to the corner point is determined to be an incomplete string.

[0055] 3) If there is only one PV string that meets 1) or 2), the data of this string will be eliminated.

[0056] S3-6: Establish a PV string coordinate position correction model and calculate the three-dimensional coordinates of the string center point. The specific method is: establish a coordinate conversion model from the world coordinate system to the drone camera coordinate system, and the formula is: Where (x c ,y c , z c ) represents the coordinates in the camera coordinate system, (X w , Y w , Z w ) represents the coordinates in the world coordinate system, and Represent the rotation matrix and translation matrix respectively, and their formulas are: in, Represents the pitch, roll, and yaw angles of the second gimbal camera. (T x , T y , T z ) represents the camera's translation in the x, y, and z axes.

[0057] Establish a coordinate transformation model from the camera coordinate system to the image coordinate system, and its formula is: where Z c represents the scale factor, (x i ,y i , 1) represents the coordinates in the image coordinate system, and f represents the focal length of the second gimbal camera.

[0058] Establish a coordinate transformation model from pixel coordinate system to image coordinate system, the formula is: Where (u, v, 1) represents the pixel value coordinates within the pixel coordinates, and dx and dy represent the horizontal and vertical lengths of a single pixel on the camera's photosensitive plate.

[0059] Finally, the photovoltaic string coordinate position correction model is obtained by converting the pixel coordinate system coordinates to the world coordinate system coordinates, and the input (x m ,y m , 1) = (u, v, 1), and obtain the corrected longitude and latitude coordinates of the target photovoltaic string (X w , Y w , Z w ), where (x m ,y m ) represents the pixel value coordinates of the target photovoltaic string center point after processing by S3-2 and S3-5. The final coordinate correction model formula is: S4: Based on the PV string center point coordinate dataset, a path planning algorithm is used to generate the optimal inspection path. The specific method is as follows: S4-1: Use a path planning algorithm to find the optimal solution for the inspection path for the multiple string center point coordinates output by S3. Commonly used path planning algorithms include the RRT algorithm, Dijkstra algorithm, Astar algorithm, and ant colony optimization algorithm. Taking the ant colony optimization algorithm as an example, the specific method is as follows: S4-2: Construct the initial path model. Specifically, in the ant colony algorithm, for any individual k, the probability of starting from node i and going to node j is expressed as follows: Where τ(i, j) represents the pheromone concentration of path (i, j), η(i, j) = 1 / d(i, j) represents the expected value of individual k from node i to node j, d(i, j) represents the spatial distance between nodes i and j, α and β represent the weight parameters that control the effects of heuristic information and pheromone concentration, respectively. Optionally, set α = 1, β = 3, S k represents the set of nodes in the area passed by step S1 and not visited by individual k, T k Represents the prohibited area in step S1 or the set of nodes visited by individual k.

[0060] S4-3: Initialize the individual array and pheromone τ0. Optionally, the setting formula of τ0 is as follows, where m represents the total number of individuals k, set m = 10, C m Represents the total length of the greedy algorithm path planning when α=0.

[0061] S4-4: Select a different starting node i for each individual k S4-5: According to P k(i, j) selects the next node j for each individual k to go to.

[0062] S4-6: Repeat the operations of S4-5 until each individual k traverses all nodes in the given set.

[0063] S4-7: Pheromone update. Calculate the total path length of each individual k and update the total pheromone concentration of path (i, j). The formula is as follows: Where (1-ρ) represents the residual coefficient after each round of pheromone evaporation. Optionally, ρ=0.4 is set.

[0064] S4-8: Iterate the operations from S4-5 to S4-7, set the number of iterations itermax = 100, and the algorithm terminates after the number of iterations is reached, and outputs the route planning result.

[0065] S4-9: Route Update Strategy. Specifically, set the update frequency f of the path planning algorithm in S4-1. f is determined by the route speed, flight altitude, and the approximate arrangement of the PV strings. The hangar edge computing unit executes the path planning algorithm at this specific frequency f to update the route path output by S1 and S4-8.

[0066] S4-9: Inspection area boundary steering strategy. Specifically, when the total area value of the PV strings (including complete and incomplete strings) in the image captured by the second gimbal camera is less than a preset threshold, the drone hangar controls the pitch angle of the drone's second gimbal camera to increase by a certain value. After the increase, if the total area value of the PV strings in the image captured by the second gimbal camera is still less than the preset threshold or the coordinates of the identified complete PV strings' center points are outside the set S of the drone's passing area, the drone is considered to be about to reach the inspection area boundary and the drone is steered according to the preset route generated in S1.

[0067] S5: The drone flies along the route and arrives at the set waypoint. The first gimbal camera takes an image and transmits it back to the hangar. The PV string in the center of the image is identified and the center point coordinate difference is calculated. The hangar controls the drone to correct the position of the fuselage, and the drone performs the PV inspection image acquisition task. The specific method is as follows: S5-1: The drone flies to the set waypoint along the optimal route output by S4.

[0068] S5-2: When the drone reaches the designated waypoint during a rapid inspection, it controls the first gimbal camera to capture visible light and infrared images. When the drone reaches the designated waypoint during a refined inspection, it decelerates before reaching the waypoint and hovers directly above it, capturing visible light and infrared images.

[0069] S5-3: When this round is a refined inspection task: the visible light image and infrared image data collected by the first gimbal camera are uploaded to the drone hangar. The edge computing unit uses the U-net semantic segmentation neural network to perform photovoltaic string recognition operations on the uploaded visible light images, obtains the pixel value coordinates of the four corner points of all identified complete photovoltaic strings, and calculates the pixel value coordinates of the center point of the complete photovoltaic string. The execution process of the edge computing unit is consistent with the S3 step.

[0070] S5-4: Filter the target photovoltaic string located at the exact center of the image from the photovoltaic string center point pixel value coordinates obtained in S5-3, and obtain the longitude and latitude coordinates of the center point of the target photovoltaic string based on the photovoltaic string position correction model established in S3-6.

[0071] S5-5: When the difference between the longitude and latitude coordinates of the center point of the target string and the longitude and latitude coordinates of the first gimbal camera body is greater than a preset threshold, the hangar fine-tunes the drone body with the longitude and latitude coordinates of the center point of the target string as the target point.

[0072] S5-6: Identify the long side of the string based on the pixel coordinates of the four corner points of the target string. The hangar controls the drone to rotate so that the long side of the target string is parallel to the long side of the first gimbal camera's image. That is, the absolute value of the target string's azimuth angle is consistent with the absolute value of the first gimbal camera's yaw angle.

[0073] S5-7: The UAV controls the first gimbal camera to collect visible light images and infrared images of the target string.

[0074] S5-8: The drone flies to the next preset waypoint and repeats steps S5-2 to S5-7 until it reaches the last stop on the preset route generated in S1. The drone then flies to the takeoff point, and the inspection mission ends.

[0075] like Figure 2 In one embodiment shown, Figure 2 This is a flowchart of the refined inspection initial route of a real-time path planning method and system for UAV photovoltaic inspection of the present invention. Refined inspection initial route generation. Specifically, taking the Dijkstra algorithm as an example: based on satellite maps, the terrain data of the target area is obtained, and the UAV prohibited route set T is output in combination with the no-fly zone and obstacle distribution information, where T = {T1+T2+T3+…+T n}.

[0076] Based on the inspection zones of the photovoltaic power station divided by S1-1, the set of drone passing areas S is output in combination with the prohibited passing set T and the minimum turning radius limit of the drone. The detailed data of the power station installed capacity and the string distribution data are obtained, and the take-off point s and the stop point list P are set in the passing area set S, where P = {P1, P2, P3, ..., P n}.

[0077] Initialize the route waypoint set B, the shortest path d, the distance d(Pn) from the take-off point to any stopover point Pn, and the priority queue A, where A={}, B={}, d=0, and d(Pn)=+∞.

[0078] Place the takeoff point into the priority queue A, A = {s}.

[0079] Determine whether the priority queue is an empty set: If the answer is yes, output the route point set B and the shortest path d from the departure point s to all the stopover points, and end the program; If the answer is no, take out the node Pn in the priority queue as the current path point and put it into the route point set B, B = {s}, A = {}, and proceed to step 6).

[0080] Determine whether the route point set B contains all the stopover point set P: If the answer is yes, output the route point set B and the shortest path d from the departure point s to all the stopover points, and end the program; If the answer is no, traverse all optional adjacent points Pn' of the current path point, calculate the path from the take-off point via Pn to Pn', and obtain the shortest path d(Pn') = min{d(Pn) + W(Pn, Pn')} from the take-off point via Pn to Pn', where W(p n ,p n’ ) represents p n to p n’ distance, proceed to step 7).

[0081] Determine whether the shortest path d(Pn') obtained in step 6) passes through the prohibited path set T: If the answer is yes, remove the shortest path of the current path point, set the corresponding adjacent point Pn' as unavailable, and return to step 6); If the answer is no, place the adjacent point Pn' of the shortest path of the current path point into the priority queue, update the shortest path, A = {Pn'}, d = d(Pn'), and proceed to step 8).

[0082] Repeat steps 5) to 7) until step 5) or step 7) is judged to be yes, output the route point set B and the shortest path d from the departure point s to all stopover points, and end the program. At this time, the route point set B = {s, P}.

[0083] In summary, the present invention, using the Dijkstra algorithm and the ant colony optimization algorithm as examples, proposes a combined path planning strategy that combines preset path planning with a real-time path planning algorithm based on an edge computing module. This strategy significantly reduces the data acquisition and preprocessing workload before PV power plant inspections and improves the path planning algorithm's robustness to actual string layouts. Furthermore, by incorporating the ant colony algorithm into the preset path, the strategy addresses the low optimization level of path planning algorithms such as Dijkstra and Astar, while also addressing the problem of a single ant colony algorithm easily falling into local optimal solutions. Using the U-net semantic segmentation neural network as an example, the present invention proposes a real-time PV string recognition technology based on an edge computing module. This technology utilizes a semantic segmentation neural network to accurately identify PV strings. Compared to traditional image processing techniques that lack prior knowledge, this technology effectively overcomes the low PV string segmentation accuracy caused by factors such as vegetation occlusion and background interference. Furthermore, the U-net network has a simple structure, high inference efficiency, and is easily customizable and extensible, enabling efficient deployment on edge computing units. The present invention proposes a PV string position correction model that accurately locates target PV strings, improving the robustness and accuracy of the path planning algorithm for actual string layouts. This paper proposes a photovoltaic inspection strategy that fully considers and overcomes the impact of factors such as site obstacles, terrain undulations, string layout, and inspection area edges on inspections. It proposes two inspection strategies: rapid inspection tasks and refined inspection tasks, and proposes optimization strategies for different inspection task requirements. In refined inspection tasks, real-time photovoltaic string identification technology and a photovoltaic string position correction model are utilized to further improve the accuracy of the real-time path planning algorithm. This paper proposes a fuselage fine-tuning strategy for image acquisition to ensure that the photovoltaic strings captured by the drone are complete, clear, and meet the requirements of photovoltaic defect diagnosis.

[0084] The present invention is not limited to the above-mentioned embodiments. Regardless of any changes in shape or material composition, any structural design provided by the present invention is a variation of the present invention and should be considered within the scope of protection of the present invention.

Claims

1. A real-time path planning method for UAV photovoltaic inspection, characterized in that: The following steps are involved: S1: Plan the initial route and set up inspection routes in different areas; S2: Capture images by adjusting the pitch angle of the camera on the second gimbal and measure the distance in real time using a laser rangefinder; S3: Upload the image captured by the second gimbal camera, identify the photovoltaic strings, establish a string position correction model, and calculate the three-dimensional coordinates of the string center point; S4: Based on the ant colony algorithm, local route optimization is implemented according to the cluster center point coordinate data set and the initial route; S5: The UAV flies along the route to the set waypoint. The first gimbal camera captures an image, identifies the photovoltaic string in the center of the image, calculates the center point coordinate difference, and the UAV corrects its position and performs the inspection image acquisition task.

2. A real-time path planning method for UAV photovoltaic inspection according to claim 1, characterized in that: In step S1, a number of inspection zones of the photovoltaic power station are divided according to the configuration of the photovoltaic power station and the distance between the take-off point and the zone. The irregular boundary adjustment is performed on each inspection zone of the photovoltaic power station in consideration of the zone boundary and the distribution of obstacles.

3. A real-time path planning method for UAV photovoltaic inspection according to claim 1, characterized in that: In step S1, the inspection task is divided into a fast inspection task and a refined inspection task.

4. A real-time path planning method for UAV photovoltaic inspection according to claim 2, characterized in that: The detailed inspection tasks specifically include: S1.1: Obtain the target area's terrain data based on satellite maps, combine it with no-fly zones and obstacle distribution information to output a set of prohibited routes for drones. Combined with the power plant's installed capacity data and string distribution, a list of takeoff and stopover points is set within the set of route areas. S1.2: Initialize the route waypoint set, the shortest path, the distance from the takeoff point to any stopover point, and the priority queue, placing the takeoff point in the priority queue. S1.3: If the priority queue is empty, output the route set and the shortest path, and proceed to S2; otherwise, extract the node in the priority queue as the current path point and add it to the route set, and proceed to S1.4; S1.4: If the route set already contains all the stopover points, output the result and enter S2. Otherwise, traverse the adjacent points of the current node and calculate the shortest path from the stopover point to the adjacent points. S1.5: Determine whether the obtained shortest path from the takeoff point to the stopover point to the optional adjacent point passes through the prohibited path set T. If so, remove the shortest path of the current path point, set the corresponding adjacent point as unavailable, and return to step 1.

4. If not, place the adjacent points of the shortest path of the current path point in the priority queue, update the shortest path, and return to step S1.

4.

5. A real-time path planning method for UAV photovoltaic inspection according to claim 4, characterized in that: The step S3 specifically includes: the second gimbal camera maintains a pitch angle to collect visible light images of the photovoltaic power generation equipment ahead of the route and measures the distance, and uses a semantic segmentation neural network model, and the edge computing unit identifies the photovoltaic strings in the visible light image and obtains the coordinates of the four corner points of the outline.

6. A real-time path planning method for UAV photovoltaic inspection according to claim 5, characterized in that: The semantic segmentation neural network model specifically includes: S3.1: Build and train a semantic segmentation neural network model, perform normalization on the input visible light image, perform inference on the image input model, and output a single-channel binary image of the identified PV string. S3.2: Extract the binary image contour, obtain the pixel coordinates of the four corners of each photovoltaic string, correct the image distortion based on perspective transformation, and update the string center coordinates; S3.3: If the distance between the cluster corner point and the image boundary or the contour area is less than the set threshold, the cluster is determined to be incomplete and is removed; S3.4: Establish coordinate transformation models from the world coordinate system to the drone camera coordinate system, from the camera coordinate system to the image coordinate system, and from the pixel coordinate system to the image coordinate system, obtain the PV string coordinate position correction model, and obtain the corrected longitude and latitude coordinates.

7. A real-time path planning method for UAV photovoltaic inspection according to claim 5 or 6, characterized in that: The construction and training of the semantic segmentation neural network model specifically include: using drones to collect visible light images of photovoltaic station strings, dividing the images into different data sets based on the string laying method and photovoltaic station type, manually annotating the photovoltaic strings in each data set using segmentation and annotation software, and dividing the data set into training, test, and validation sets according to a certain ratio.

8. A real-time path planning method for UAV photovoltaic inspection according to claim 7, characterized in that: The step S4 specifically includes: S4.1: Construct an initial path model and use the ant colony algorithm to calculate the total path length of each individual and update the total pheromone concentration of the path; S4.2: Iterate S4.1, set the number of iterations, and the algorithm terminates after the number of iterations is reached, and outputs the route planning result; S4.3: The path planning algorithm update frequency is determined by the route speed, flight altitude, and PV string arrangement. The hangar edge computing unit executes the path planning algorithm at a specific frequency to update the route path output by steps S1 and S4.

2. S4.4: When the total area of the PV strings in the second PTZ image is lower than the threshold, adjust the pitch angle; if it still does not meet the threshold or the center point of the string exceeds the inspection area, turn according to the preset route in S1.

9. A real-time path planning method for UAV photovoltaic inspection according to claim 8, characterized in that: The step S5 specifically includes: S5.1: The UAV flies to the set waypoint along the optimal route output in step S4; S5.2: The aircraft slows down and hovers before the waypoint. The first gimbal camera collects dual-light images and uploads them to the hangar. The edge computing unit identifies the PV strings and calculates their center coordinates, selecting the target PV string located at the center of the image. S5.3: If the deviation between the center coordinates of the target string and the drone positioning exceeds the threshold, the hangar controls the drone to fine-tune to the target point, identifies the long side of the string according to the pixel value coordinates of the four corner points of the target string, and rotates the fuselage so that the long side of the string is parallel to the long side of the image; S5.4: After completing image acquisition, fly to the next waypoint, and repeat S5.2-S5.3 until all waypoints are completed, and return to complete the mission.

10. A real-time path planning system for photovoltaic inspection by an unmanned aerial vehicle, adopting a real-time path planning method for photovoltaic inspection by an unmanned aerial vehicle according to any one of claims 1 to 9, characterized in that: include: The drone is equipped with a first gimbal and a second gimbal, and a drone charging and swapping motor library is equipped with an edge computing unit. The first gimbal is equipped with an infrared thermal imaging sensor and a visible light lens, and the second gimbal is equipped with a laser rangefinder and a visible light lens. The drone charging and swapping motor library is also equipped with a data storage unit.

Images